Affordably Aggregating ISP Connections?

An anonymous reader writes "Has anyone setup a system to aggregate multiple ISP connections to form a high bandwidth site-to-site link? Load Sharing SCTP looked interesting, but it doesn't look like it has been widely adopted. Multi-Link PPP appears to be more widely supported for clients, but I can't find any good guides for setting up both sides of the connection for a site-to-site link. The hardware solutions I've found are expensive for a small business. Does anyone have experience using hardware solutions from Mushroom Networks (Virtual Leased Line, p2 of this document), Ecessa (site-to-Site Channel Bonding), or others?"

Comments

Bonded VPNs

[Anonymous Coward]

I have bonded 2 IPSec VPNs running over 2 ISP's to create a bigger (and cheaper) site-to-site link on the cheap.

http://www.zeroshell.net/eng/faq/vpn/

Read Point 5 in the link

tomato

[angelbunny]

The cheapest way to do this is use the mlppp version of tomato on a wrt type router. You can check it out here: http://fixppp.org/

Re:tomato

[Anonymous Coward]

Only works if you've got DSL, and then again only if they use PPPoE. Then the remote DSLAM needs to support mlppp as well.

I would suggest OpenBSD + PF and just load balance the connections using PF. Takes all of 10 lines of code to get it up and going and is well documented. This won't aggregate your bandwidth, however if you have multiple streams open, it'll bounce those between two or more connections. I've personally done 4 lines like this (2x adsl2+ and 2x DOCSIS 2) and hit about 95% utilization across all lines.

Also with PF, both lines don't need to be the same speed, or even with the same provider, which gives you some additional fault tolerance.

printers/newpapers

[shareme]

Your local newspaper or medium sized printer will have such a setup. Buy their IT staff diner to get the information..

Multiple bonded connections

[davidwr]

In theory, you can bond multiple DSL, multiple cable, multiple T1, or even multiple dialup connections from the same vendor.

Even if you are in a small town where the best service you can get is 1Mbps DSL, if you've got enough wires running from the neighborhood box to your house you can ask for 2 or 3 or more separate DSL lines and see if the local telco will support bonding them.

Even 15 years ago some telcos offered on-demand, 0-24 circuit, bonded dialup. The idea was a business would use it as up to 24 voice circuits during times of the day they talked a lot and up to 24 modem/data circuits when they needed them, typically at night for batch data exchange. It was sold as an alternative to T1 or ISDN, the first of which was very expensive and not available in all areas, and the latter of which was expensive and roughly the equivalent of 2 phone-or-data lines.

DSL and later cable internet made this pretty much obsolete, at least in technically advanced areas.

Re:You can't do what you want to do

[Anonymous Coward]

Sure you can.

http://www.openbsd.org/faq/pf/pools.html

One simple example. Plenty of other options available with other software. As long as you load-balance per connection instead of per packet there aren't many issues with this, and those often don't apply outside of special use cases.

Re:Need More Infos

[mindstrm]

"TCP/IP doesn't allow for that, that I know of"

It sure does - it doesn't care what route the packets took - just that they got there. THe problem is if you split the stream over 3 links with varying latency - you won't see the performance gains you wan t- it'll more likely hurt.

If the goal is to end up with a virtual point-to-point link between two offices using multiple ISPs, you can certainly leverage multiple connections to do that. It also depends on the nature of the traffic.

You can set up multiple VPN tunnels and then run whatever protocol you want - you could do MLPPP - but that'll get ugly if the links don't have very similar characteristics.

The solution you mentioend in the end - Iv'e found that' susually the best - you can get most common *nix systems to do some kind of weighted load balancing of outgoing sessions... whether it's per-source, per-destination, per-protcol, or based on any other weird usage combination you had.

For an office situation Iw as once in - we had 1 2mbps and 1 x 4mbps lines (from separate providers) and a very high latency 1Mbps satellite connection.
I gave them a web page that had four buttons on it.
The first was "normal operation - 2MB + 4 MB". TCP sessions would be randomly routed over one orhte other, with double rpeference given to the 4 meg line.
The ohters were "ISP1, ISP2, and Satellite" respectively. At the push of a button the routes would flip, the state tables would flush, and everything would work. For practical puproess, it worked really well.

There is no magic way to simply aggregate bandwidth from separate providers over consumer connections with meaningful results... not like bonding multiple direct lines or anything like that.... 2 + 2 won't equal 4.... but depending on the use case, it can be just about as good.

Linux Iproute2 is all you need

[SectoidRandom]

A few years back I did this with a colleague, we actually investigated 3 solutions; 2 commercial and one linux script based, in the end the one that won easily was the Linux script.

Basically using iproute2 and some nice scripts gives you the ability to load balance your outbound packets and then using some relatively simple scripts to monitor each remote peer for automatic failover.

A quick google turns up this blogger who sounds (from a quick skim) like he's doing the same thing: http://blog.taragana.com/index.php/archive/how-to-load-balancing-failover-with-dual-multi-wan-adsl-cable-connections-on-linux/

Unfortunately I can't remember the commercial solutions we tested (this was 4-5 years ago!), but although they did exactly what you wanted perfectly, our problem was that we were doing this for a managed services company who ran 150+ IPSEC VPN's over those (at the time) 3 bonded ADSL connections, needless to say the commercial solutions had never imagined anyone trying to statefully balance that many VPNs! However with some tweaking (to be honest a LOT of tweaking) we got the Linux solution working a treat, even with nearly seamless failover.

Google is your friend on this one.

Re:What are you really trying to do?

[TheRaven64]

But that's the grandparent's point. What does he actually want from this setup? There are a lot of factors that will affect the best solution. Does he want:

• Individual connections to be faster?
• Total throughput to be more?
• Overall reliability to be better?
• Transparent fail-over if one connection goes down?

If he wants the last one, does he want:

• Existing connections to continue working?
• New incoming connections to keep working?
• New outgoing connections to keep working?

Some of these are trivial, some require a little bit of client-side configuration, some require additional support from the ISP. Without knowing what he actually wants to achieve, it's impossible to make a recommendation. You can do all of these things relatively easily with a stock OpenBSD install on your router, but exactly which ones you want depends a lot more on the requirements. For somethings, you want to run a VPN between the two sites with packets sent over some of the link with the most bandwidth. For others, you could get away with just a couple of routing rules. If you want more than just the two sites and you want existing connections to work then you need the ISP to support updating the routing tables when their link to you goes down.