Slashdot Mirror
Affordably Aggregating ISP Connections?
An anonymous reader writes "Has anyone setup a system to aggregate multiple ISP connections to form a high bandwidth site-to-site link? Load Sharing SCTP looked interesting, but it doesn't look like it has been widely adopted. Multi-Link PPP appears to be more widely supported for clients, but I can't find any good guides for setting up both sides of the connection for a site-to-site link. The hardware solutions I've found are expensive for a small business. Does anyone have experience using hardware solutions from Mushroom Networks (Virtual Leased Line, p2 of this document), Ecessa (site-to-Site Channel Bonding), or others?"
The hardware solutions I've found are expensive for a small business.
And leasing the line is too if you want multiple ISP's on it. One is easy, after that it gets hard.
If you want something fail-proof, just go for co-location in an established datacenter with many peers.
The more interesting question here is that if someone has done *peering* outside of major datacenters? It's of course costly, but it's something the submitter is looking for.
I have bonded 2 IPSec VPNs running over 2 ISP's to create a bigger (and cheaper) site-to-site link on the cheap.
http://www.zeroshell.net/eng/faq/vpn/
Read Point 5 in the link
What you have presented us with here is a "B C" problem. You want to achieve C, so you ask us how to do B. Unfortunately, you never specify what A is, so the best we can do is give you some pointers for B which are probably going to be irrelevant and useless to what you are really trying to achieve.
Most of the comments will probably be about trying to figure out what your A problem is. To that end, why don't you just get a faster line in the first place and forget about this line aggregation stuff you're asking about?
We've been using 2 Powerlinks from Ecessa (back when they were Astrocom). They work really well, and the price is tough to beat. We have one in our Dallas branch (with a T1 and business cable ISP) and one at our home office in Baton Rouge (a dual bonded T1 and business cable). They are channel bonded with each other, so the site-to-site VPN is more stable. They made my life a lot easier!
All of them?
Um, yeah: Whatever you say, kid.
We usually just use a Roadrunner account for the main office, just like all the other small business out there. It's faster and cheaper than a T1, and has better reliability than the PRI that handles our phones. (We also have a freebie account with a local WISP that we do some business with for manual fail-over, but we haven't had to use it in years.)
Kid-proof tablet..
When I first read it, I thought he was talking about connecting two offices together securely. Of course, I also hadn't considered that we here in Chicago tend to spend more money than people in other areas, either.
The cheapest way to do this is use the mlppp version of tomato on a wrt type router. You can check it out here: http://fixppp.org/
Unless you can get your ISP to bond several connections together about the best you can do is load balancing across multiple connections. I use pfsense (http://www.pfsense.com) as my router/firewall VPN solution that's free, you only supply the hardware to run it on. with it you can load balance and fail over to 2 or more connections automatically. Specif connections can even be setup to have certain traffic routed over them while all other traffic gets load balanced round robin style. there are of course other free *nix distros out there that will let you do the same type of stuff however I and many others have found pfSense to be far batter than most. AW
Have you looked at what Talari Networks (http://talari.com/) is doing? I'm pretty sure their products do EXACTLY what you're talking about. Might be pricy for you, but it should do the trick.
Your local newspaper or medium sized printer will have such a setup. Buy their IT staff diner to get the information..
Fred Grott(aka shareme) http://mobilebytes.wordpress.com
I had an idea for how to do this - has anyone tried using a HTTP proxy, and having it split up large downloads across multiple HTTP range requests, each going out of a separate WAN connection?
In other words, given N connections to the internet: Small requests are load balanced by simply doing round-robin. When the response starts coming in, if the object size is more than say 10MB, the proxy goes and issues N-1 additional range requests across each of the other WAN connections for equal sized chunks (or sized in proportion to the speed of each link, if they're different speeds).
And this could be done a lot better with some additions to the HTTP protocol. A "stride" parameter for example...
Of course it is not trivial but I think for static objects it is imminently feasible.
In theory, you can bond multiple DSL, multiple cable, multiple T1, or even multiple dialup connections from the same vendor.
Even if you are in a small town where the best service you can get is 1Mbps DSL, if you've got enough wires running from the neighborhood box to your house you can ask for 2 or 3 or more separate DSL lines and see if the local telco will support bonding them.
Even 15 years ago some telcos offered on-demand, 0-24 circuit, bonded dialup. The idea was a business would use it as up to 24 voice circuits during times of the day they talked a lot and up to 24 modem/data circuits when they needed them, typically at night for batch data exchange. It was sold as an alternative to T1 or ISDN, the first of which was very expensive and not available in all areas, and the latter of which was expensive and roughly the equivalent of 2 phone-or-data lines.
DSL and later cable internet made this pretty much obsolete, at least in technically advanced areas.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I read your comment as "of a tomato on a wrt typewriter."
Shades of 1909.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Sure you can.
http://www.openbsd.org/faq/pf/pools.html
One simple example. Plenty of other options available with other software. As long as you load-balance per connection instead of per packet there aren't many issues with this, and those often don't apply outside of special use cases.
The higher end dreytek business modems support at least two aggregate DSL links. The real question is, do you want a wider pipe, or a faster pipe. One is easy, the other not so easy. Bigger trucks in your tubes, or faster trucks in your tubes :)
(sorry couldnt resist that analogy)
a fucking book on how routing works
Now there's a fetish you'll only run across on Slashdot.
This side up.
It is possible as long as you have control of both endpoints. The routing book is probably still a good idea.
"And we have seen and do testify that the Father sent the Son to be the Savior of the World"
1 John 4:14
www.sharedband.com
Bonds both Up and Down stream.
Layer 3 so you don't have to bother your ISP.
I have seen people bond FiOSS with DSL and Cable modems.
Sold directly or through your ISP if they offer the service.
Reliable and very cost effective.
Keeps your ISP honest.
Cheers
Just get one of the commercial multi wan routers and jam a bunch of connections into them. It's not true link aggregation but it's as affordable as it can get. It won't become one giant pipe, instead the connections from machines behind the router will get load balanced out. In order to get true link aggregation, well, it's horribly expensive and I'm almost positive that it can't be done with multiple ISPs.
What people presume to know, sheesh! Bandwidth Aggregation: Combining Internet Connections to Incrementally Increase Bandwidth Capacity Bandwidth aggregation combines two or more Internet connections and gives Internet applications access to their total available bandwidth and increases reliability with link redundancy. PowerLink, ShieldLink and ClariLink bandwidth aggregation techniques (also known as multi-homing) support load balancing to route Internet sessions from congested links, to links with more available bandwidth. They also provide automatic failover of Internet sessions from failed links to functional connections to eliminate the Internet as a point of failure. http://www.ecessa.com/pages/solutions/solutions_technology_bandwidth.php
Wired has an article on Willie Nelson's setup in his tour bus running, http://www.wired.com/epicenter/2009/10/willie-nelson-broadban/ "Willie Nelson has tossed the satellite dish off the back of his corn-powered tour bus in favor of a little box that fuses wireless data cards from a variety of networks into a single connection."[Mushroom Networks PortaBella 141]
What exactly are you trying to achieve? .2) something repeatable for multiple remotes
Some scenarios:
A) remote to central with 2 ISP's at remote with "cheap regular" DSL type connection going to central where there is a "big fat pipe" (multihomed?)
A.1) a one of for a single remote
B) remote to central with 2 ISP's as A) and with (same?) 2 ISP's at central (also B.1 & B.2) as above).
Do we have any fixed public IP addresses anywhere in the equation (or is this out of budget too)?
In all cases in which direction is the data flowing mainly?
Also, what is the purpose mainly here? Getting higher speed? Higher redundancy? Less latency (hah!) ?
> The hardware solutions I've found are expensive for a small business
Can you define expensive, what type of price is out of it (both for hardware and for links)?
I would GUESS that the end result needed is to connect LAN-1 to LAN-2 , so it doesn't HAVE to "look" as a single channel for the routers involved, just that the paths
aggregate and are redundant... But a bit more information would be appreciated!
Sounds like you're trying to take a DSL, cable, and possibly a T1 or other technology and trunk them for combined throughput. That isn't possible because you'd have packets in the same stream taking different routes and TCP/IP doesn't allow for that, that I know of. I don't think any technology allows for that. For example an 8mbit DSL, 6mbit cable, and a T1 can't be combined to make a 15.5mbit connection. I suppose the same would be true if you were wirelessly connected to multiple networks.
You can, however, use all three gateways independently with a variety of load balancing software so that when a new request is made from any host it is routed through the gateway with both the quickest response time and the most bandwidth available. I'll let you look that up on your own - there are lots of free options. The problem is that the load balancer needs to be smart enough to not fuck up your active sessions. If you were communicating with a host via one route, went idle for a bit but didn't end the session, then sent more data via another route the host on the other end will most likely (if written correctly) not accept your new packets.
The way we handled it at "The Geek House" with three internet gateways was to just permanently assign gateways based on the role of the host, and made sure not too many were on the slower gateways. It's not perfect, and certainly could have been geekier, but it worked and we didn't have to worry about shit breaking in the middle of a frag fest. And if one gateway was down the hosts configured with that gateway just had to change their gateway.
No sig for you. YOU GET NO SIG!
It's got everything you'll need for Multi-WAN load balancing and failover, and supports many platforms.
pfSense Multi WAN / Load Balancing
Peplink multi-WAN router supports forming multiple site-2-site VPN connections over multiple WAN connection. Failover and load balancing VPN traffic is supported. Routing between sites is automatically configured. 256 bit AES encrypted. Supports static IP, DHCP and PPPoE WAN types.
you might want to have a read of that routing book as well, since it's possible to use 2 different isp's and still increase aggregate speed....
If you mod me down, I will become more powerful than you can imagine....
If you're going to recommend a pair of Cisco routers, then why not run GRE over IPSec? You can run EIGRP over the tunnel interfaces, and configure either equal-cost load sharing across the links, or use variance to proportionaltely allocate traffic over the links according to the expected bandwidth. If you want to keep some conversations pegged to a particular link, you can policy route those host/network pairs, while still maintaining failover if that link dies. You don't even need NAT/PAT in this scenario if the private address ranges don't overlap.
I'm not one to yell at noobs but I really can't imagine timothy did more than a Bing search because I see that pfSense comes up on the first page of results on Google when you query "multi wan".
PfSense is probably the go for this, but you are free to choose any other BSD or Linux based distro which gives you a nice pretty point and click web interface out of the box and good online documentation on how to use the features.
Hell, you don't even actually need physical hardware for this provided that you have two NICs available and a virtualization capable server.
Are you trying to bond all of your neighbor's WAPs together so you can aggregate their bandwidth? This could make bit torrenting an interesting thing.
Someone I know did this by setting up Linux routers with "Advanced Router" kernel features -- namely source-routing on established connections, so that established TCP connections could be consistently kept onto a single ISP connection. Without doing this then packets can be sent (or received) from an IP address not associated with the TCP connection, so they're dropped.
Obviously this won't work on UDP packets, since they're stateless; so if you have programs that need to stream data via UDP, that will be an issue.
Good luck with the project.
Admittedly, I have no idea if it works, nor do I have any idea how it decides to load balance between the connections.. But I ran across the feature the other day and it looked pretty cool.
In Mac OS X you can create a new "Aggregate" network device from any other devices and, in theory, do exactly what your describing. Again, I just ran across this the other day in Network Preferences and have no idea if/how it works, but it might be worth a shot (especially since it seems a lot easier to configure than a roll your own router with dd-wrt or tomato, though those likely offer more fine-tuned configuration).
appleguru.org
If you're trying to combine different types of access (leased-line, cable, DSL), I think you're out of luck with trying to aggregate everything into a single "super circuit". However, you can certainly utilize all of those individual circuits. Look up policy-based routing. Most every platform out there should support it through some method. Set it up so that email goes over the DSL, your database queries goes over the cable connection, and your VoIP goes over the leased-line. You'll probably need to tweak it a bit at first until you get a nice blend of traffic, and you'll want to make sure to set up some default routes to handle things if you have an outage on one of your circuits, but you'll see better performance on individual circuits and use all of them. If you've got the same type of access, but through different providers, you'll probably have to do the same. If you've got the same type of access through the same provider, then MLPPP or GRE should work.
> ...their cell phones strapped to the outside of their waists...
You strap yours to the inside of your waist? I'm trying not to visualize that...
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Linux & iproute2 should be more than enough for what you want.
"Routing for multiple uplinks/providers"
Once the load balancing has been stablished you can set up OpenVPN to encrypt the traffic between the two (I like using openvpn + brigde to do a Layer 2 vpn). You can even get more fancy and add traffic shaping to distribute bandwith, prioritize packets (for a lower latency in ssh or terminal server traffic for example).
- Human knowledge belongs to the world
I imagine the way sharedband works is that it's a VPN endpoint. If you use VPNs (essentially creating another IP layer on top of the existing one), you *can* aggregate multiple connections and even get faster single-session transfer speeds.. You just need an endpoint to connect to that has at least that much bandwidth. This appears to be part of what sharedband offers. The main issue I'd be concerned with, however, is latency.
For something natively supported in Linux, this really isn't something that I'd say is impossible...
Lets suppose you have networks A and B. Given N cheap broadband connections on each side, lets call them A1, A2 ... A(N) and B1, B2 ... B(N)
At host A, for each A(N), B(N) pair, you set:
* a route for B(N)_IP via A(N)_Gateway
* a VPN link with source address A(N)_IP and destination B(N)_IP
* a static route for private networks behind B via each A(N)->B(N) virtual interface
Repeat for host B and each B(N), A(N) pair.
Problem: if each link has very distinct latencies, you will end up with package streams arriving at the other side heavily out of order. Tune your TCP stack accordingly.
As many posters have pointed out, there are about a gajillion ways to do this (I'm a big fan of GRE, Quagga, and some judicious OSPF metrics :)
If you're talking about remote offices with workers who aren't IT-aware past "Oooooh, email" and you start adding layers of complexity to their Internet connection(s), you necessarily increase the risks of network downtime due to configuration errors, busted hardware, code bugs, etc... many times things you can't fix remotely. Some assessment of your target customer's tech-level for dealing with those issues should go in to the design decision. E.g. - implementing a Linux-based firewall on repurposed commodity hardware in an office without full time IT staff might make for a nightmare if the hard drive died; you likely would end up driving to that office to fix it, hiring a local "consultant" to assist if you can't drive there reasonably, or re-tasking someone's time in the office for your own nefarious IT purposes (instead of them being out there selling your employer's bread and butter).
If you're a centralized network manager at the company HQ, then the conversation that starts with "Powercycle the blue-and-white box and tell me what the LED's do" is a lot easier to deal with than "What does the screen say? Oh, well a kernel panic means something really bad happened..." You can mitigate those issues, but you'll inevitably end up on the phone someday with an office worker whose "Internet ain't workin". Sometimes it's easier to spend the money up front for a piece of dedicated hardware, rather than in the back-end on support costs (opportunity or actual).
Perhaps English isn't your first language...
The OP wrote "Has anyone setup a system to aggregate multiple ISP connections"
Had he written "Has anyone setup a system to aggregate multiple connections from an ISP" you'd just be a jerk, but as it stands, you're an ignorant jerk.
- real hackers don't have sigs -
Insert tab A into slot B...
Hot RJ45 action!
Wow. I'm not the AC but after that response I fully agree with him.
Your use of selective quoting is amazing, you got some big-ass internet cojones to ignore the rest of the very same sentence that you quoted.
When information is power, privacy is freedom.
Some people (Cisco, etc.) are working on developing the Locator/ID Separation Protocol as a core component of the Internet infrastructure.
If that ever takes off, you'll be able to buy a Provider Independent IP address block, advertise it through multiple ISPs (even Cable/DSL), and transparently load balance your upstream and downstream traffic across them, without bloating the core BGP tables.
The downside is, you'll have to use an MTU that's smaller than 1500, but I'd say it's a fair trade.
The Advanced Routing Howto on tldp.org - nuf sed.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
An Internet Service Provider (ISP) isn't involved in a point-point. It's just a service provider at that point. Multiple links from the telco for redundancy is silly as the vast majority of problems will take down both links (cut fiber, local CO issues, etc). If you're talking mixing Telco, Satellite and Cable for redundancy as someone else mentioned, then I'd guess you are are talking about an ISP and running VPN then? In that case there are options such as mlppp, etc.
(We also have a freebie account with a local WISP that we do some business with for manual fail-over, but we haven't had to use it in years.)
That sounds like code for "we haven't tested our backup plans in years"
Having a backup and not testing it is a rookie move.
[Fuck Beta]
o0t!
I don't work for Barracuda, but their link balancer (http://www.barracudanetworks.com/ns/products/link_overview.php) is ~$3700 US for the 3 connection device, with full support.
antipaucity
Relax, meme. Everything works fine. Not that it particularly has to, though: There isn't much that goes on in this particular small business which requires Teh Intarweb, anyway.
Assuming otherwise without further information is the mark of a real asshole. And just because it's a popular assumption over the past few days (Danger/Sidekick/T-Mobile/Microsoft) doesn't mean that it's universally true.
Now get back under your rock, #669689.
Kid-proof tablet..
Its long, at least read about Greenlight in N.C. and learn!
I am 100% positive you could do this with hardware that will run the DD-WRT, here is a list of DD-WRT supported devices, they have a search link, but I find that it does not work very well if you do not know the name of the router / firewall that you are looking for. So use the list and find a supported device.
You would need two of them and two different providers. You could even get a third one and do some special VLAN stuff to put some ports on all three on the same virtual network., many options.
These devices are very light weight, therefore shipping is next to nothing. The Linksys WRT54Gs' were great routers for the DD-WRT software. Costing over $75 when they first came out, dropping to $69 for years and finally hitting $15 or $30 when the stores were unloading them to bring in the new Linksys routers (none of which will support the DD-WRT software, except one that runs Linux). NOTE: there are BETTER routers than the WRT54G to run this software. The WRT54G will ONLY run the Micro version of the software. Do yourself a favor and get one that will run the Mega version of the software! (They cost less than $100 per and well worth the price.)
Linksys (Cisco) begin removing DD-WRT compatible firewall/routers from store shelves, replacing them with devices that are NOT compatible with the DD-WRT software in 2007/2008.
Get two DSL lines ($13 - $19 each), add in a NAT and a couple of these routers, probably need to do some secure tunneling to avoid the DNS of the Cable / DSL Companies and voila you are good to go. Your DSL speed will vary based on distance, but even far away you can get 1.5MB down and 384Kbps up. If closer you can get 3Mb down and 768Kbps up. (That is faster than 98% of Americans with Cable Modems because of throttling of service by Cable providers.)
Could you run the second DSL upstream over the first one? Thus saving the cost of a second telephone line, you would lose the redundancy that two telephones would provide, but save around $13 per month on a second phone line...probably better just to get the two lines, you total cost of ownership (TCO) will still be less than $60 per month and you will have redundancy. If one service gets stupid and starts throttling, drop them and get a different one. Politicians help us if they all throttle!
Solves allot of problems related to Cable companies throttling back service if you can create a secure VPN that their Deep Packet Inspection and/or Bandwidth shaping (throttling) service might have a harder time restricting (throttling). Granted they would still throttle you back by your IP address or MAC address of Cable Modem. Again, they do that now anyway.
A friend of mine was pissed that he was throttled back to less than 100K down and 0K up 85 - 95% of the time. He went on and paid his cable company the $10 burst / protection racket money / "give me a little more of what I am already paying for money" extra fee. Keep in mind that they were promising up to 8MP and delivering less from day one. He said he got a letter in the mail that they would be rolling out a new service in his area, the day after they started using that service, his bandwidth was throttled to next to nothing. (0 Kbps upstream, consistently less than 20Kbps). (There were 1 GB, 2GB and 3 GB ~ 1 second spikes ONLY, unless he was downloading a Linux distro, then he got 3GB - 4GB sustained with a 1 sec 6GB spike) He is convinced that they throttle him back because he uses Skype VoIP service (uses P2P packets) in a vain attempt to get him to switch to the Cable companies VoIP service. At less than $100 per year, Skype blows away any telco/Cable company offering.
Guess what his speed was after the switch over....Yep less than 100K (down) and 40K upstream 95% of the time. When he is throttled back to 0Kbps like I am, t
Is your Internet Throttled? Install DD-Wrt, OpenWRT or Tomato to learn the truth! Google: 1Gbps/1Gbps: 5 Communities
Link aggregation made easy (_easy_ as in ZFS making filesystems easy)
http://opensolaris.org/os/project/crossbow/
Any opensolaris distribution with the latest builds should have this (including Nexenta).
http://dilemma.gulecha.org - My philospohical short film.
Put the damn server in a cheap colo, already (or rent a dediserv and copy everything across).
Multiple SDSL's with bonding just to get a slightly better upload for a game server? You've gotta be mad.
A few years back I did this with a colleague, we actually investigated 3 solutions; 2 commercial and one linux script based, in the end the one that won easily was the Linux script.
Basically using iproute2 and some nice scripts gives you the ability to load balance your outbound packets and then using some relatively simple scripts to monitor each remote peer for automatic failover.
A quick google turns up this blogger who sounds (from a quick skim) like he's doing the same thing: http://blog.taragana.com/index.php/archive/how-to-load-balancing-failover-with-dual-multi-wan-adsl-cable-connections-on-linux/
Unfortunately I can't remember the commercial solutions we tested (this was 4-5 years ago!), but although they did exactly what you wanted perfectly, our problem was that we were doing this for a managed services company who ran 150+ IPSEC VPN's over those (at the time) 3 bonded ADSL connections, needless to say the commercial solutions had never imagined anyone trying to statefully balance that many VPNs! However with some tweaking (to be honest a LOT of tweaking) we got the Linux solution working a treat, even with nearly seamless failover.
Google is your friend on this one.
Two Mikrotik routers would also work very well (http://www.mikrotik.com). You can pick up the whole thing for about $250-$300 for two of 'em and set it all up inside of about 15 minutes a piece. They're extremely reliable and the 4.0 release includes all kinds of fancy things that you can do to monitor, automatically fail over links and more.
Just add {In Space!} to anything.
Get two DSL lines ($13 - $19 each), add in a NAT and a couple of these routers, probably need to do some secure tunneling to avoid the DNS of the Cable / DSL Companies and voila you are good to go. Your DSL speed will vary based on distance, but even far away you can get 1.5MB down and 384Kbps up. If closer you can get 3Mb down and 768Kbps up. (That is faster than 98% of Americans with Cable Modems because of throttling of service by Cable providers.)
Sorry to ask, but is the broadband that slow in the US? I had no idea. I was using 40mbps on adsl, switched to fiber/cable and I'm now at 100mpbs here in spain.
In my family's house in Brazil, I'm using 100mbps as well. Costs about 10 times the price you said, but still is 10x cheaper per mbps....
Yes, the author does rudely smack the original poster in the face, but as Captain Jack Sparrow said, he "may have deserved that."
The original poster didn't give us enough info. Aggregation from multiple ISPs is possible, but it's a lot dodgier performance-wise than aggregating multiple connections from the same ISP. On the other hand, your choices of possible solutions depend a lot on your problem - if you want to make a single fat TCP session go faster, for instance a big file transfer, that's a lot harder than load-balancing a bunch of smaller sessions. And of course most cheap consumer solutions are very asymmetric, so the upstream will be your limitation, and don't give you good Layer 2 feedback and probably aren't running TCP ECN either.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I have written a solution which does exactly this -- http://broadbond.org/
I imagine the way sharedband works is that it's a VPN endpoint. If you use VPNs (essentially creating another IP layer on top of the existing one), you *can* aggregate multiple connections and even get faster single-session transfer speeds.
Absolutely. See my post below:
http://ask.slashdot.org/comments.pl?sid=1406513&cid=29766837
Mod parent up!
I think therefore I am... a Linux geek.
In the general case, aggregating lines from multiple ISPs is not possible, but the original poster asked for aggregation for a single site-to-site link, which is possible. You'd put both of the sites in different private address ranges and then run a VPN between routers at each end. These two routers could send any encapsulated packet over whichever link had the most spare bandwidth and the receiving end would remove the encapsulation and deliver it. If you have lots of connections, however, it's probably easier just to configure multiple routes and load balance at the connection level.
I am TheRaven on Soylent News
What difference does that make? My original post applies - you need a device that supports port trunking. A "site-to-site link" can be created using one or more ISP's, so that criteria doesn't require any particular prerequisite. As written, the "multiple ISP connections" means "multiple ISP" unless you suggest the adjective doesn't modify the noun directly following it.
If so, the internet cojones apparently don't require intelligence.
- real hackers don't have sigs -
First you need a well connected end point. Here for example you can buy very cheap (30€/month) unlimited bandwidth colocated servers. You just start an OpenVPN connection between that host and your Linux router on the site through each of your DSL lines. Using iptables random match, you mark each packet with one of two values. Then you use policy based routing to direct packets to one connection or the other based on the mark.
Then you just configure nat on the colocated host and add the proper default route on the local router. Voila.
What difference does that make?
It makes all the difference in the world. All you need is the appropriate device at each site - not at the ISP. Set up a VPN tunnel across the multiple links that terminates at the other site and you can aggregate at the packet level just like any of the site-to-ISP aggregation methods. The only case where the ISP has to support link aggregation is where it is site-to-internet-at-large, not site-to-site.
If so, the internet cojones apparently don't require intelligence.
Considering that it now appears you've been proclamating without investigating, it is quite appropriate that you would say that.
When information is power, privacy is freedom.
So, you agree that site-to-ISP requires link aggregation...
There's a difference between a customer network and an ISP network.
An ISP provides a connection to the internet, by defintion. So, "site-to-internet-at-large" is what was the topic of discussion.
- real hackers don't have sigs -
At work we've just picked up a Draytek Vigor 2820 ADSL router, it's a bit fuller featured than your usual home modem, and it also allows aggregating bandwidth between the built in ADSL modem and either a USB connected 3G modem or any other IP connection connected over ethernet. We use the latter connected to a vanilla ADSL modem. As a whole it seems to work pretty well, the web interface is nonstandard, but after a while I got used to it, and you can set it up to either keep both connections active, or only fail over to WAN2 at a predetermined bandwidth usage or on failure of WAN1 (built in ADSL). Not sure what the OP is looking for, but the Vigor is just what we were looking for. (they do other versions with built in wireless and voip)
Although not with any of the solutions others have offered.
Two DSL links from different providers (would have worked with cable or satellite or GPRS/HSPA) -- the router uses iproute2 to set up multiple default gateways and use them in a round-robin fashion. Now I was going one step further; they wanted a single IP, so I extruded a multihomed IP from a colocated server and routed traffic for it over these multiple links (over L2TP tunnels, IIRC).
It worked really well, and you could aggregate more bandwidth simply by adding more connections. The colocated server had next to zero load. As links fell and came back, the ip-up scripts automatically adjusted and the overall bandwidth grew and shrank as they should.
If I were to do it again, I'd be a little smarter on the choosing of the link to use for a particular packet; i.e. have a management daemon that kept track of the average/burst traffic through each link and select the "best one" based on available bandwidth at that instant and expected return packet size.
I really don't see why all the head scratching here is about. A proper bit of research on Google would answer this. This is a fairly trivial task with most any linux distro on both ends (Tomato or similar flashed routers would likly be able to do it also). I guess the limit on this is how many connections can you plug in (your hardware or kernel handle), and how many will your isp sell you.
I have to use multiple ISP where I live because none are reliable for 100% up time, and none will sell much more than 4 mb-6mb per connection but they will allow me to stack dsl connections as much as I want. SSH is my choice for VPN solution, but I suspect any other VPN will do the trick with some tweaking of the iptables and such.
Living in Chile
An ISP provides a connection to the internet, by defintion. So, "site-to-internet-at-large" is what was the topic of discussion.
That's some funny ass shit dude.
The OP said site-to-site link and you think he meant not site-to-site link!
You crack me up. Are you stoned or just high on your ego defense mechanism?
Been fighting for peace too?
Fucking for virginity maybe?
When information is power, privacy is freedom.
The OP said "Has anyone setup a system to aggregate multiple ISP connections".
What's funny is how you keep ignoring the original premise and want to infer based on subsequent statements that don't support the challenge you're trying to make...since "site-to-site" links can be created over the internet, it's irrelevant to your argument yet you try to use it as a focal point.
On second thought, no, it's not funny... not "ha-ha" funny, at any rate. Read a book and get back with me.
- real hackers don't have sigs -
What's funny is how you keep ignoring the original premise and want to infer based on subsequent statements
Subsequent statements in the same sentence that serve to clarify his intent.
You just keep right on denying the obvious dude, safe and warm in your little house of meaningless semantics
When information is power, privacy is freedom.
Yes, I'm at work on a 640Kb down maybe 90 or 128 up connection. Yes thats bits. At home however, I can get sustained 2MB (yes bytes) down but still not much more than 150KB up.
Obviously direct aggregation isn't possible, as each line will have a different source IP. What works is load balancing, but load balancing sucks. If you do per-TCP-connection load balancing on multiple lines, lots of sites will give you problems, as multiple requests for the same session are coming from different IPs. Online banking doesn't like this, ads-supported sites often don't like this (as the ad was loaded from a different IP). So this leaves you with per source-host load-balancing, and this only makes sense if there are lots of people who are two share the lines.
Doing real aggregation (bonding) requires a remote endpoint obviously, located in a datacenter somewhere for example. Problem: There is no standard protocol that works for a combination of different lines, Multilink-PPP will only work for several identical lines from the same ISP (ideally using the same clock source at the DSLAM etc). Why is that? That's because if you use multiple lines, they will have different latencies / round trip times. And if you bundle those, this means that TCP packets will overtake each other in-flight. So in the end whoever is receiving the re-assembled stream will get it out of order. And TCP can not differ between reordered and lost packet - if an unexpected (too high sequence number) packet is received, it is dropped. And this can not be solved by buffering at the router/PPP-device, because this buffering would interference with TCP windowing. In the end most of your aggregated bandwidth will therefore be eaten by retransmissions.
So, people may tell you to try this and that, but in the end everyone who has ever REALLY tried it himself will tell you: Forget about it, the performance will always be really bad (unless you have multiple identical lines).
There is a small german startup I work for which has solved the problem by creating a new bundled VPN protocol running on the way between the router in your office and the one in the datacenter, basically running a man-in-the-middle attack on TCP to get rid of the packet reordering in-flight. See http://www.viprinet.com/ for the available products and background info on how it works. Pricing starts at ~1000 USD, but obviously you'll need two boxes - probably not what you'd call "affordable". And sadly we do not yet have distributors inside the USA.
That's it, right there! You're apparently not a tech!
"meaningless semantics" to you, "technical specifications" to me. I guess over 30 years in the industry will do that.
The specs speak for themselves:
Aggregate multiple ISP connections to provide high bandwidth site-to-site link, yet not expensive 'cause it's for a small business.
As stated, it's not technically feasable. You go ahead and keep imagining those are "meaningless semantics" or imagining they mean something else entirely. Have fun with that.
- real hackers don't have sigs -
Your meaningless semantics really are meaningless - they certainly aren't details that make a difference to solving the actual problem as stated.
As someone who has done precisely what the guy asked for, as previously described with a VPN, this 'not a tech' laughs at your continued denial of the obvious.
PS, this "not a tech" has 20+ years of tcp/ip stack and other misc internals experience, he knows exactly what he's talking about.
By your own demonstration in this thread, you don't.
When information is power, privacy is freedom.
We use BGP to combine multiple circuits into a single bound circuit. We have outbound and inbound traffic working over this.
We have the routers set to not cache routing. But you can't expect to turn two 1.5Mb/s T1's into the same thing as true 3Mb/s connection. It is close, but a single video stream is only going to traverse one of the two T1's. A P2P download will use all 3Mb/s.
If all you really want is to speed up outbound connections (not inbound to a webserver for example) you can use something like a ZyXel 100 or 50. They have load balancing built in;
http://www.zyxel.co.uk/web/product_family_detail.php?PC1indexflag=20040908175941&CategoryGroupNo=PDCA2008004
So, then you suck as a tech since you want to read things into the specs that aren't there. It's OK, I've seen plenty of crappy techs at IBM, AT&T, Boeing and McDonnell Dogulas before that. You'll still be able to make a living.
I've seen really good techs there, too, and they'd not misread the specs.
- real hackers don't have sigs -
So, then you suck as a tech since you want to read things into the specs that aren't there.
Dude, that's effing hilarious, "site to site link" is clearly in the spec. You want to ignore the spec and instead worry about trunking to the ISP, when the user doesn't care a whit about his packets being routable by the ISP.
3...2...1... initiate deliberate misunderstanding of the word routable to further protect miserable ego.
When information is power, privacy is freedom.
From the charter:
We switched from M0n0wall to PFsense (a fork of the BSD-based M0n0) because PFsense supported aggregation. Our experience has been very positive. We aggregate two biz class Comcast 50 Mb (down) x 10 Mb (up) (& eventually will include a Verizon T-1) on an COTS PC stuffed with server grade NICs. The PC is driveless and boots off a USB. Rock Solid, out of pocket expense ~$400 for the Lenovo PC but you could probably find a much cheaper hardware platform.
I ignored no spec - they need to go back to the customer for revision 'cause they're technically unfeasible. I guess I COULD talk myself into believing I know what the customer wants... ...but then I'd suck as a tech.
I haven't gotten where I am by doing that.
- real hackers don't have sigs -
I heard about a backpack-sized setup that you can get which takes a video input, compresses to HD quality, then splits the outgoing signal onto six separate cell phone data links (three are 3G, three are standard).
This was pretty expensive for 30 hours/month service but in theory would let you do high quality video without a satellite uplink or other special gear.
Presumably stitching the data streams back together is a pretty big hat trick especially with low latency.
--hongpong.com
That is one reason I put the subject as Pering and not Peering.
Yes we Americans are throttled back that severely, of course the Cable Company/Telco oligopoly (monopoly in many areas) will NOT call it throttling, they flat out deny that they restrict usage. And when they are caught red handed (Both Comcast and TWC have been in the past on numerous related issues, can you stay TCP/IP Stop Packets to interrupt an Internet users surfing of the Internet? Even Sprint one time blocked an entire range of IP addresses, I have forgotten and no longer care what their excuse was, as it was WRONG) they pull out the following BS excuses: pornography, child welfare, pedophilia, spammers, etc, etc, Ad nauseam. Like you are not smart enough to censor yourself and your family, please stop the FUD.
I, and many Americans, believe each family unit are the only ones who can independently determine if they want to self censor or not. It really is a freedom issue (net neutrality, bandwidth caps, bandwidth shaping, deep packet inspection and other bandwidth limiting strategies), a privacy issue, a simple Respect your customer issue.
As for bandwidth, it is a form of censorship, but much, much worse. It literally determines what content you and your family are able to get to, see and view. Hit the cap, you better pay up more or else you will be cut off.
As for bandwidth, less than 1% of the American population get the FCC definition of Broadband, which is 768K. Note my friend's speed test showed over 9000 Kbps down and over 900Kbps upstream. Yet he was throttled, restricted, prevented from getting bandwidths higher than 100Kbps downstream and 30 to 40Kbps upstream about 85% to 95% of the time. (We have talked and we both believe that we are throttled back about 98% of the time, however we wanted to be conservative in our criticism)
He is regularly throttled to 0Kbps upstream and it bounces from 0Kbps to 4Kbps or from 0Kpbs to 13Kbps, ocassionally up to 30Kbps. Not only can you not watch TV or video, but forget about using sites with allot of CSS or skins, for instance Digg and stumbleupon are not usable at that level of bandwidth throttling.
There are few if any off-the-shelf residential firewall/routers (DSL or Cable modem does not matter), typically costing less than $80 dollars per, that will show the home user actual logging information about what packets are and are not going through their firewall/router. Companies like Cisco/Linksys use the software as a product differentiating / limiting device to get users to spend more for more capability. However to get true TCP/IP and UDP packet information none of the Residential routers give you this information. Even the Linksys / Cisco firewall/routers costing in the $60 to $80 range vary greatly. I have a BEFSX41 (older non WiFi router) that would give me the outgoing and incoming IP addresses, but not packet information. When I purchased a newer (now old too) WRT54G WiFi (DD-WRT capable) firewall/router I expected even better logging. Yet the logging was even more limited. It did not make sense until you saw how they differentiated their products. Logical from a business pricing / marketing strategy, but still impractical to give home/residential users the opportunity to control and view their actual packet information. (Thus why I harp on the DD-WRT routers, so does my friend, once you have this capability and see what it does for you; the data/information that you now can use; you will not settle for anything less, I know I would not.)
In a "nut"shell many Americans take the speed test as gospel...what a crock, it is NOT. All it tells you is the devices between your PC and the ISP (DD-WRT enabled Firewall/Router and DSL/Cable Modem) will allow you to get that maximum speed at that specific moment. In the next moment all best are OFF. If you did a bell curve with my friends 9000 Kbps down / 900Kbps up. It would mean that he should expect to get the f
Is your Internet Throttled? Install DD-Wrt, OpenWRT or Tomato to learn the truth! Google: 1Gbps/1Gbps: 5 Communities
I ignored no spec - they need to go back to the customer for revision 'cause they're technically unfeasible.
Right, technically infeasible for someone with internet cojones that don't require intelligence.
I guess I COULD talk myself into believing I know what the customer wants... ...but then I'd suck as a tech.
You've already demonstrated that you suck as a tech. This entire thread is you showing the rest of the world your autistic compulsion to focus on one key word, "ISP," to the exclusion of everything else in the same sentence. You couldn't write your cojones rebuttal correctly (unless you intended to flame yourself) and it's not like it was a simple typo, you got the entire sense of it wrong. Just like you got this wrong starting with your first case of selective quoting. "I ignored no spec -- I just left out the half of the sentence that clarifies it..." lolz
When information is power, privacy is freedom.
The 0-24 line thing was available in fractions, before fractional T1 was common.
If you were a small business that had, say, 4 telephone lines at a branch office, you could use this device to give you 3 voice lines and a fax line during the day and 4 data lines at night. It was up to you to put one of these "bonding modems" at your branch office and your main office, the telephone company wasn't involved.
This was also back in the days before 53K dialup was common, you would use your existing modems and get probably 14K or 28K but possibly 33.3K per "data line."
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Carpentry?
No kilobytes per second? So how do you make an HTTP request at all? That's upstream bandwidth. If you're truly not getting any outbound traffic, this likely indicates that there is something electrically wrong, not that they are shaping traffic that hard. Either that or you have a clogged pipe from your ISP up to the outside world.
Check out my sci-fi/humor trilogy at PatriotsBooks.
No kilobytes per second? So how do you make an HTTP request at all? That's upstream bandwidth.
Tell me about it, it sucks! However let me clarify...
Since our service is throttled back most of the time to either 4Kbps or 0Kbps. And you are correct at that level of restriction hardly anything will load. I certainly can not use Digg, stumbleupon, hootsuite and a couple of other social media websites that load more CSS style stuff than other sites...those sites are unusable at upstream bandwidths less than 20Kbps in my personal experience. Whenever a website does not load or loads very, very slowly, I just switch to one of my other Linux windows where the bandwidth monitoring status is being graphed and I typically see bandwidths less than 20Kbps. Its shocking how often its only 4Kbps or 0Kbps. Its disgusting, especially considering its costing me more than $50 per month for cable modem access. That is too much for too little service. DSL at less than $30 per month, sustained bandwidths of 1.5Mbps down and 384Kbps or higher up, is looking very enticing! Thus this slashdot story, putting two DSL providers on your network starts to make allot of sense!
I think most of us would agree that only getting 20Kbps upstream is unacceptable for cable modem access to the Internet advertised as high speed broadband!
With the DD-WRT software you see a continuous graph for both your downstream and upstream bandwidth 24X7. (You do not have to let this bandwidth status graphing run if you do not want to.) With the upstream line bouncing between 0Kbps and 40Kbps. And a second downstream line bouncing between 4kbps and 100Kbps most of the time, however you will see higher spikes upstream...up to 300Kbps, 700Kbps, 1Mbps, occasionally 2Mbps and even rarer 3Mbps downstream.
The higher spikes upstream are fewer and farther in between, and rarely rise above 100Kbps. It becomes pretty obvious that the bandwidth shaping software is controlling users ability to watch any streams (IP TV, Video, Movies, etc...) by throttling (limiting) the upstream bandwidth. My guess is that if you had a sustained 300Kbps (even slower DSL services offer this) upstream you would be able to watch multiple video / TV streams without sputters and/or interruptions. You might need a bit more if those streams are high definition. Since most users are limited upstream to less than 40Kbps (dropping down constantly to 0Kbps) this is what prevents you from watching video via the web.
The FCC and our politicians could break this effectively by decreeing any bandwidth less than 786Kbps can not be considered high speed broadband. Make it FRAUD to declare otherwise. Quite frankly that is a very old standard, even though it was recently set. The 2000 broadband standard should have been 100MB/100MB. The 2006 standard should have been 1GB/1GB. If the telcos/cable companies had given us fiber for our tax dollars as they promised(1990s), that would be the standard today! Do you have respect for promise breakers?
Back to reality, ... With the DD-WRT software showing you your actual bandwidth, you see those lines drop to 0Kbps around 50% or more of the time. In other words, you spend much more time at between (0Kbps and 4Kbps) upstream than you do at 40Kbps or higher.
As far as equipment being damaged, that is the kind of FUD that even the Cable company will try to throw out at an unknowing and less knowledgeable public. Since the Speed tests show you that given that few seconds in time your equipment (firewall/router running DD-WRT + Ethernet network + optional hubs + more Ethernet + Cable Modem) works just fine, but ONLY during that moment in time. Thus the very speed test they tout as proving you have great service (even when you do not) also proves that your equipment is functioning just fine. (They artificially prevent it from working as well at other times.)
In fact if you have DSL or Cable
Is your Internet Throttled? Install DD-Wrt, OpenWRT or Tomato to learn the truth! Google: 1Gbps/1Gbps: 5 Communities
lol, that was a typo, I had to search for 0K to find it, comes from my first post, should have read 40Kbps as like me, he wants to be conservative in his estimate, the reality is worse than 40Kbps 85% - 95% of the time.
It should have read
A friend of mine was pissed that he was throttled back to less than 100Kbps down and 40Kbps up 85 - 95% of the time.
Since the other related quotes all have either 30K or 40K and state that the bandwidth is "shaped" to less than 0Kbps on a regular basis, I will assume that you are having some fun with me at my expense. Good catch and I have a great sense of humor. Thanks for allowing me to correct it.
Hopefully my point was well understood in spite of that typo...at least I hope so. The simple point is that both my friend and I are prevented from getting acceptable bandwidths to surf the Internet, with one Cable provider specifically, approximately 80% -95% of the time. In reality it is something higher than 95% of the time, but by stating either 80% to 95% or 85% to 95% of the time, we both are being conservative.
These quotes are from the two posts, related to the percentage of time (restricted, filtered, throttled, prevented, censored, , shaping, etc...) in addition to the one above:
From the first post, #29766339
Guess what his speed was after the switch over....Yep less than 100K (down) and 40K upstream 95% of the time. When he is throttled back to 0Kbps like I am, the videos sputter, gMail, twitter, Facebook and MySpace will not load because of the little extra bandwidth required for the skins and CSS markup language.
From the second post, #29766339
Yet he was throttled, restricted, prevented from getting bandwidths higher than 100Kbps downstream and 30 to 40Kbps upstream about 85% to 95% of the time. (We have talked and we both believe that we are throttled back about 98% of the time, however we wanted to be conservative in our criticism)
However he is getting less than 100K down and less than 40K up well over 85% of the time.
We suspect that every cable providing Internet access does this same thing and that most people do not have a firewall/router with software capable of showing them their bandwidth in real time; so in reality most people do NOT know.
Is your Internet Throttled? Install DD-Wrt, OpenWRT or Tomato to learn the truth! Google: 1Gbps/1Gbps: 5 Communities
However most UNIX-like OS support connection binding. You can do this with Linux or BSD for sure, the problem is that the binding needs to be in place on both ends, you can't have a fast transfer rate to an arbitrary non-participating site. I used to have an ISP who let me bind multiple dial-up connections to his DSLAM, which did give fast connection to the outside world, but it wasn't a telco ISP. I could do it in the hours when he always had many unused ports, like 1am-6am, etc. But it still took two cooperating end points, it was just that one was a DSLAM.
You seem to have gotten other useful information, and you can also do it through multiple vpn connections, although I know of no remotely automated way to do it, I used to make the two connections and then run a script, aggregating a DSL and cable connection.
My feeling is that it's enough trouble do be of limited value, but it can be done, which was your original question.