Slashdot Mirror
Working With 2 ISPs For Home Networking?
An anonymous reader writes "This is, I think, a simple question — but one which I can't get the answer to.
As a typical, but perhaps high-demand home user I would like to use 2 separate ISPs. ADSL is pretty cheap nowadays, and 2 x ADSL seems a better value than one fast one — especially in terms of reliability.
If one breaks, at least the other will work.
Using an old box as a router/firewall, how can I configure a system to use two completely separate ISPs in a sensible manner?
Ideally, I'd like the load of my browsing to be balanced, but at the minimum, I'd want some kind of 'fail-over.' If I leave torrents running over night, I'd like the router to use whichever connection doesn't block the traffic — and preferably for it to reset the errant connection.
Ideas?"
ADSL is pretty cheap nowadays, and 2 x ADSL seems a better value than one fast one â" especially in terms of reliability. If one breaks, at least the other will work.
When your DSL is down, it's likely that your neighbor's DSL is down too. Consider cable + DSL, not cable + cable or DSL + DSL.
You can get a "Firebox" VPN/Firewall/Router pretty cheap on ebay. They are running about $75.00US for the Firebox 1200/2. The "/2" part means it has 2 WAN ports and you can load balance across both, it is setup to be redundant, so if one goes down, it moves all traffic to the other automagically. I use one and it works like a champ. There are more expensive solutions, and probably "Roll your own" solutions, but as most of us know, that can provide months and months of aggravation!
"My immediate reaction is "WTF? What kind of moron doesn't make things 64-bit safe to begin with?" Linus
There are little Linux distributions like Brazilfw which run on old hardware and work out of the box with features like QOS, load-balancing, port forwarding, etc. Maybe that's what you need.
pfSense can handle the load balance and failover for you. Then you just need to get two ISPs. Preferably one cable + one DSL but if you can get the two DSL lines on separate circuits, that would work well.
Isn't a dual-WAN router the simplest/cheapest method, whatever you are planning to put downstream of it? http://www.networkworld.com/reviews/2004/0913rev.html
called Clarkconncect (http://www.clarkconnect.com/)
It's basically a CentOs (aka free Red Hat) wich can do multi-Wan. It has a nice web interface fir Firewall, ftp, web and mail server, shell..
No idea if it can reset errant connections, but it can do anything you can on redhat, including using two Wans simultaneously. (chek Clarkconnects forums for multi wan)
up and running within 30 minutes, mine has reached 165 days uptime (Bi-P3 GHz, 2 Go Ram, 4*500Go HDD, 3*Eth 100 (upgraded from a faithfull Compaq Deskpro 400 Mhz "server")- web, mail, and bittorrent dowvnloader (torrentflux-bart) as well as "media server" connected to the xbox with XBMC)
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
Most DSL circuits, even sold by different vendors, go through the same facilities and sometimes the same equipment. For example, the local loop is usually the local telco's, no matter who your DSL vendor is. And many DSL vendors resell one of a few wholesale providers (e.g., Covad), so your data on both DSL lines could be going through the same wholesale provider's equipment/facilities. The same may be true of other technologies (e.g., fiber).
In trying to setup something similar, we finally settled on using cable for one circuit and fiber for the other. We know the cable company has its own local loop, and they assured us (FWIW) that they have their own facilities out to their upstream provider (e.g., AT&T, Sprint, etc.). Fiber would be Verizon. We would use DSL, but I'm concerned that it would end up in the same Verizon facilities.
Good luck. There are also routers that do fail-over, but I know that's not what you asked about.
Hotbrick makes a very good load-balancing soho router. They're a bit pricey but they seem to work quite well for exactly what you're describing. Take a look on ebay for their LB series.
I do have to second the suggestion of using Cable+DSL rather than DSL+DSL. Most places where there are multiple DSL providers, they're both operating from the same physical infrastructure with one reselling the service of the other. It's certainly better than one by itself, though.
Even people that believe in pre-destiny look both ways before crossing the street.
Honestly, I think that's not understanding how DSL works very well. In virtually all markets, there's one physical DSL provider, and a few dozen 'ISPs' which cost a little bit more to provide potentially 'unique' services on top. One monopoly for phone (and hence DSL), one monopoly for cable.
Er, the cheapest DSL is what, around $25, $30, for 256k? Double that, and you've got a price for very fast (8mbit or more) cable, including 256-512kbit upstream. Even if you have 2x256k, and the equipment to use it in a decently efficient manner, that's still some 512kbit, and two different IPs.
Only in a few situations can you use the bandwidth of both cooperatively for a single task, and the most common failure is based on when the physical link/line conditions deteriorate, in which case having two ports to the same network isn't going to make any difference at all.
Cable/DSL will provide the potential reliability you'd be looking for, I think. But, as a home user, some 98-99% (even if not 99.97%) uptime isn't good enough? For the additional cost, it's not worth the extra -average- hour per month of downtime you gain 'back'.
If your ISPs downtime is any more than that, you have every right to complain, twist their arm to fix whatever might be causing the problem.
"A Goddess rarely smiles for she is forced by others to be an island unto herself." - Zephiris
Seriously? Is your network infrastructure -that- unreliable that its actually worth *doubling* your costs for redundancy?
I have had maybe 10-15 hours of internet-only downtime in the last 8 years. Of that, maybe 4 hours affected me (ie I was awake and wanted to use the internet). I've had another 10-15 hours of power fail in the last 8 years, and even with backup power the internet was still down (routers, switches, etc in the upstream path weren't on backup power so keeping my 'modem' up isn't worth beans.
In any case, I can see a lot of situations where it would be worth another $2500 over that period to have had internet access for those couple hours.
If I were running servers (and I am), it might be worth it, but in practice its not worth the trouble. round-robin DNS just means every odd connection attempt fails if one of the links is down, and dynamic dns updates to take the downed link out of rotation would be great except most internet outages are over before dns updates are likely to propogate. So its just not effective.
If I wanted -faster- downloads, that might be worth 2 connections, but that's not what you claimed your objective was. And even then, it usually won't make a specific download faster, but will rather let you do 2 at once at full speed (in the case of a large http or download for example which only uses one connection) which may or may not be what you need. Torrents, using multiple connections, will of course benefit from the extra bandwidth capacity.
If you SERIOUSLY want redundancy, you might want to look at a router that can fail-over to dialup. That will actually stand of chance of being available during a power failure, and might not cost you extra in terms of service, since many ISPs give you some free dialup hours as part of your broadband. And the dialup infrastructure is often separate enough from the adsl/cable infrastructure that you'll be able to connect on dialup while adsl/cable is down.
You just get a Linux box with 2 NICs and start adding static routes :
route add 1.1.1.1 255.255.255.255 eth0
route add 1.1.1.2 255.255.255.255 eth1
route add 1.1.1.3 255.255.255.255 eth0
Etc, etc....
It might seem like a big job, but there's huge ranges of reserved addresses you can skip. Let us know how you get on.
What you want is a "dual wan" router. Which will give you two ways out, by default putting each connection between your local host and a remote host over a single WAN's route, but pool the two WANs so the less-full one gets the whole next connection.
Then you want to look into "bonding", or whatever the router vendor calls their version of it. It usually doesn't work, because the two different WANs usually take very different routes most of the way to the remote host, and the bonding has to accommodate all the hops between on each of the two WAN routes. But sometimes it does work, especially if the routers at both ends of the routes share the same bonding technique.
But you will indeed get immediate uptime benefits. Because if one WAN gives you, say, 99.9% uptime, that's 0.1% downtime, which is still over 31,000 seconds down a year, which is still almost 9 hours. But if you can get connections over either one WAN or the other (each at 99.9%), you can get 99.9999% uptime, which is only about 32 seconds a year, which is unattainable at reasonable prices for a home user.
--
make install -not war
Great, so you googled some shit. Maybe he wants to get some people's experiences with them? What is good or bad?
A witty saying proves you are wittier than the next guy.
I have quite a bit of experience with this, as I use two consumer ADSL circuits to provide very reliable 'net services at my office.
To an extent you either get to use two different services (for reliability) or combine them into one service for improved performance. Not both.
If you're going for reliability, you'll be using two different providers. That eliminates the use of multilink PPPoE to bond the two services into a single logical service with a single public IP address. It also eliminates ATM channel bonding, which is the other way to achieve the same end. This isn't such a great loss as you might think since I've *NEVER* found a provider (at least here in Australia) that knows what either is, let alone supports even one of them.
So, you're stuck with two ADSL circuits, each with separate PPPoE connections (or direct IP over ATM links; either way) and separate public IP addresses.
This sucks. You can't even load balance across them properly without the cooperation of a router/proxy on the other side of your ADSL links.
Load balancing your transmissions on a per-packet basis is obviously hopeless because any sane ISP has egress filtering based on source IP address, and even if they don't you'll still get replies back on the official source IP (so you won't gain much). SNAT won't help because if you SNAT some packets in a connection the recipient will have no idea they're part of the same connection as the unmodified packets leaving on the other connection. The only way that packet-level load balancing across multiple links with different IPs will work is if you're only talking to an endpoint (probably a VPN termination point) that is aware that you're using multiple connections and can combine them. You can use tricks like multilinked PPTP for this, or iptables trickery on each end. In any case, you're going to need access to a server with enough bandwidth to service both connections that's willing to route traffic for you. You probably don't have this.
So, packet-level load balancing is out. What's left? Connection-level, and per-protocol.
Connection level load balancing works well for some services. Outgoing SMTP, for instance, is well suited to being randomly allocated between multiple ADSL links (if you're unfortunate enough to have users who think that 100MB attachments are a good idea). Unfortunately most home user services like HTTP web browsing are not. You'll find that websites like to store session data with your IP address, so if you do connection load balancing with HTTP you'll find that websites keep on forgetting your login. To work around this you need to use "sticky" load balancing that remembers which connection was used to talk to a given host - but that, of course, reduces the benefits of the load balancing.
In the end, all you can really do is a bit of sticky connection-level load balancing when establishing new outgoing connections for some protocol types. If you want more than that, you need to do ugly things like say "all FTP connections go out ADSL1, and all SIP and other VoIP connections go out ADSL2" etc.
Personally, I don't bother even with that. I have both ADSL services listed as MXes for the company's DNS, so if one is down we still get mail. The A record points at a colocated server elsewhere on the Internet, so that's not a worry, but if it didn't I'd have to use some sort of ISP-level or colo load balancing to reroute traffic down whichever link was currently available.
Outgoing connections just all use the primary link when it's up, and fail back to the secondary link if/when the fast one is down. The secondary link is the primary MX, so when both links are up mail will tend to come in one link and everything else in the other.
If I wanted more than this, I'd probably have to route everything through another server colocated at an ISP or peering point. Unless I could get free traffic between it and both my ADSL circuits this would get expensive fast - and it'd also reduce the benefits of the redundant ADSL links
God, not another person saying this.
Slashdot articles aren't just posted for the question, but for the discussion. Yes, anyone can find an answer to anything they want with Google+Wikipedia+etc.
The point here is that maybe someone will take an interest in it that never thought of it before or cared enough to dig around Google.
Obviously from the author's point of view, multiple viewpoints by the readers would be helpful. However from the Slashdot mods (and community in general) it's an interesting enough topic to read on their own.
look for the Linux Advance Routing Howto
Somewhere in that site it talks about some of the problems of having 2 IP addresses, like confusing game servers and the like, but with a bit of tweaking you could get it functional. I don't think this solution explicitly provides failover functionality, but I suppose that could be scripted in somehow.
pfsense is a nice turnkey solution for this too, if you're not into spending a couple weeks solid trying to make your debian or lfs distro act like a router.
db
I am literally 3000 tokens away from the chaotic crossbow --Stephen
I produce a system that can do this. It's called Broadbond.
You can bond several ADSL lines, even from independent providers, and it will deliver the combined upstream and downstream bandwidth of the two. All traffic is load balanced across the two lines and can also be transparently compressed. The throughput of the lines is automatically measured to determine the optimal load balancing. Differences in latency on the two lines are compensated for.
The catch (there's always a catch!) is that you need to have a partnering system co-located with an ISP to handle the far end of the tunnel -- although I can also provide this if you would prefer.
The system is available as a software package that you can license to run on Linux or OpenBSD and also pre-installed and pre-configured on a couple of small embedded Linux boxes -- very low power (under 5W), no moving parts, good for up to 90Mbit/sec.
I bond two ADSL lines to my office, 4.4Mbit and 9.6Mbit, and I get around 13.5Mbit on file transfers.
If you're interested, contact me (details on the broadbond.org web page).