Slashdot Mirror
Is the Internet Shutting Out Independent Players?
"ISPs aren't advertizing routes for competing ISPs, and since IP blocks are heavily filtered upstream, this won't do much good anyway. The reasons for this are clear (Routing table growth was getting way out of hand), hence the introduction of CIDR ? , and the allocation of IPs to ISPs, with a resulting lockout on availability of routable IP space to individuals or smaller groups.
With the availabilty of IPv6, and the cost of RAM, I find it somewhat hard to believe that either IP address blocks are scarce, or that the size of routing tables are unmanageable any more. This might have been true with an 8MB Cisco 10 years ago, but surely it would be a negligible cost to put 1-2GB of RAM on even a reasonably budget router at todays prices.
Obviously, IPV6 isn't really here yet, but i would like to think that when (if) it arrives, we will see a more open routing system.
Is anybody working on returning some kind of equal standing to 'the little guys' when it comes to internet routing infrastructure, and how a more 'open' system could work in practice on tomorrow's (or today's) internet?"
Here - 217.53.98.174 - doesn't seem to be responding; use that one.
Having a multi-homed network is extremely stressful on the rest of the Internet, and you're going to have to pay for the privilege.
Yes, routers have gotten a lot more advanced, but if every Tom, Dick, and Harry wants to have their own APNIC-assigned IP block, it is going to cost a lot of money for the backbone providers and everybody else to accomodate the routing tables. Unless you're big enough to make a reasonably large dent in their bottom lines, they aren't going to care about making you happy because it's just too damn expensive. (And guess who would wind up paying for your pleasure? Every user of consumer-grade connections, that's who.)
You should be quite satisfied that you can even get high-speed connectivity (not to mention, connectivity from multiple providers at once) where you're at. Here in the USA, the most technologically advanced society in the world, it's difficult if not impossible to get *any* high speed service outside a major metropolitan area. Before my cable monopoly upgraded its network, I couldn't get any service at all that wasn't long distance dialup.
My advice to you: count your blessings, and find a different way to solve the problem.
Just my 2c.
~wally
Vintage computer games and RPG books available. Email me if you're interested.
You just called a Kiwi an Aussie... [gulp] its going to get mighty ugly ;)
These methods and models of doling out IP addresses leave some of our internet data centres hopelessly inadequate at providing something as trivial as fault-tolerant links thru two or more ISPs within the same country as each ISP would refuse to route blocks belonging to other ISPs.
However, I dont think that arguing the increased RAM capacities of routers being capable of storing the huge routing tables is the answer.
CIDR and its ilk was developed to partly address huge routing tables, but the key point it addresses is propogation of new route changes which need to be sent to more routers and thus generating more traffic instead of being confined to just the edge (in context) routers as used now.
If the propogation of new and changed routes could be addressed without generating additional traffic, and believe me when I say bandwidth isnt cheap in Asia, then I would agree with utilizing larger RAM in routers to store these tables.
Incidentally, I was a couple of minutes short of FP. :)
Even the in new Routers from Cisco you can't put 1 to 2 Gigabytes of RAM in them, most top out at 256 or 512MB. RAM for PC's might be cheap but most of the RAM for routers and such have not come down in price like the RAM for PC's.
/19 level. ARIN's minimum block size is /20 or for Multi-homed ISP's that qualify for a /21 also get a /20. But if you want you routes (and IP's) to be globaly distributed with no problems, then you need a /19 or bigger.
Here in the US there is similar requirments, BackBone providers often filter routes at a
An idea that I had been toying with was to buy 2 internet connections, say DSL and cable modem, then use NAT to use them both simultaniously. In a simple scenario, seems like it could be accomplished by picking up 2 of those cheap home gateways and setting up a non routeable network. Internally the machines would be set to use one of the gateways by default, if that connection went down you could switch to the other one. Externally multiple DNS records could be used to distribute the traffic among multiple ips, all of which point back at the non routable network.
Even though I concieved this idea for a low end home network, the basic idea should be applicable to a business that really wants a redundant connection. Just buy multiple connections from multiple sources, keep your machines in a non routeable network, then use some fancy equipment (a Cisco PIX for example) to make everything work. Bit of a kludge, but I think it's a viable solution.
Unfortunately, the very reasons you're eagerly awaiting IPv6 are probably the reasons that you won't ever see it, and you probably already know those reasons.
The Internet stopped being about information about five years ago (Or at least that wasn't the point anymore) and it's now all about eCommerce and BS like that. The very same companies that got on the Internet in the first place to deliver information are now delivering information only from their marketing departments, and not from engineers or researchers. Commerical interests have all but drowned out its original spirit, and are also partially the reason for the inception of Abilene (Internet2). Of course, it probably won't be long before that new promised land gets pillaged and raped. The Internet as we know it seems to be in an eternal state of loss of innocence, I'm afraid. I don't think the solution is to supplant or supercede the original 'net, but to just have a user-maintained network...kinda like what the network-area neighborhoods are designed to accomplish, except on a much grander scale. When the corporate interests don't exist, then the public can do with it as they see fit.
"Mod, mod, mod...and another troll bites the dust."
yes, but I believe the solution rests with a layer on top of the internet - namely something like peer to peer systems of today where nodes can shift more easily, appear and disapear without hurting the overall network.
the real problem is with NAT (network address translation). How to two peers behind such a NAT firewall anounce their presence to each other and then communicate without the assistance of a 3rd peer with a proper IP address and place on the internet. if anyone knows the answer to this quiestion, I'd love to hear it!
really, how do you announce a service behind a firewall? that seems to be the question of the day.
It's true, you can't get portable IPs of your own anymore. The advent of CIDR and the segregation of netblocks were in an effort to reduce global routing tables.
Putting in 1-2Gb of memory in a router is still incredibly prohibitive. It just can't be done in the mainstream (common) routers.
You can still be multi-homed with netblocks from one ISP to be received by another. This happens this way in the US, and I'm sure it happens with APNIC and RIPE-issued blocks. You get the same effect, without all of the hassles of truly having your own blocks. At least we don't have the /19 barrier for advertising that used to be prevalent in larger ISPs. There is some give and take. The give on that is that the larger ISPs have gone to regional aggregates.
For instance, I don't want to have to pay for my addresses in the US now thanks to ARIN. (Don't get me started.) My ISP takes care of that. The justification process of getting addresses isn't fun, but it's a lot better than the Inquisition your provider has to go through. I'm not saying that economy is bad, but it's a fact of life with IPv4.
It's possible that controls will be loosened in an IPv6 world, but I don't think so. We've been down that path before. With tiny fragmented blocks of IPv6, we're creating a nightmare of routing tables the likes of which we've only imagined with IPv4. Aggregation is here to stay, and I beleive the days of the portable netblock are long gone.
Of course, if you can justify your need for your own blocks, you can go directly to your registry. If not, isn't it enough to have your networks SWIPed to you?
The days for "vanity" addresses are long gone. Maybe you should think up a clever .com domain name instead while you still can.
Someone tried selling me on a box that did that, except it would take several high speed connections (like 4 or 8 ethernet ports on the box, you supply the other end) and then via NAT and then intelligently load balance the traffic across those connections. I think it had the ability to transparently redirect traffic based on protocol to these presumably cheap broadband connections.
The idea was that instead of buying another expensive T1 because everyone's reloading Slashdot all the time, you buy cheapie DSL connectivity as needed and run your "unimportant" traffic out this box and the business-critical gets more of the T1.
It's a neat idea.
I'm not sure why you want to go multihomed, with all the attendant problems that it brings. If this is a corporate connection, that's not got services (other than mail) being provided to the outside world, then I don't really see the point. I think you can provide the redundancy in other ways - here are some ideas, using 2 ISPs (and PA IP addresses allocated by each of them).
Put a mail server on each connection (or map an IP address from each connection through your firewall to the mail server). MX records will do your load balancing and redundancy for you.
Use NAT/PAT for users to connect to the Internet. If one conenction goes down, remove the internal routing to that connection - all your sessions will now go out of the other connection. I find that this is quicker than waiting for BGP to reroute connections via a backup/alternate path. It also gives you more flexibility in internal network numbering, and to move ISPs.
Host services with colocation providers - not internally. Colo service providers have already solved most of the service provision problems, and are well connected to the Internet - I don't think it's worth trying to do this in house.
How many computers do you have on this lan? Why do you think you need to 'own' the IP addresses? First off, you don't even need to own ANY ip addressed to do multihoming. You could NAT all of you LAN boxes up into the single /30 advertisement that your ISP(s) are going to give you for the serial interface on your router, and then have the ISP advertise that out to the 'net, and voila, you have multihoming. When one provider goes down, you can use your IGP to route across the other, OR, if you wanted to go a litte more high-class, you could buy a large router, and take full BGP tables from both providers, and differentiate intelligently based on the preferences sent on the routes. Now, if you don't want to do NAT, and there are a whole slew of good reasons you wouldn't, why are you hung up on ownership of these IP addresses? Why won't you let the IP-allocation process work like it's supposed to? If APNIC had to allocate IPs to every small business in the region it's responsible for, it would take 3 years to get IPs from them. Buy a block of IPs from your ISP(s), and if you transition to another ISP, re-number your network. Or, if you don't wanna go the cheap way, you CAN buy portable IP space from providers. Many of them buy whole Class As just for this purpose, it's just that you're going to have to pay more for these IPs than you would otherwise, as you should, since the ISP's netblocks can become non-contiguous if you leave. As far as your questions about IPv6 and router memory, the internet routing table is well up above 100k routes already, and there are many routers out there that are already having problems dealing with tables of this size. Many Cisco boxes will die in the near future if not upgraded, as their old routing engines run out of memory, and despite the fact that PC memory is cheap, router memory often is not. Especially when you have to install it on the tens of thousands of routers any decently sized ISP will have. IPv6 isn't really even a factor yet.. and when it is, many routers are going to need heavy upgrading (software, hardware, etc) to deal with it, which is why so many ISPs aren't rushing out to do it. So buy some portable IP space, get yourself multihomed, and go buy a good BGP book.
//Phizzy
"Most European technology just isn't worth our stealing," -- Former CIA chief James Woolsey, referring to Echelon
Oh so many answers, so little time.
First of all, one should note that IPv6, while supported in newer versions of Cisco IOS, has the slight problem that in BFRs, the hardware accelerated routing hardware has four times more work to do to look up a 128 bit IP address making performance somewhat of a problem. Add to the fact that a lot of the routers out there simply can not be upgraded past 128 MB of RAM and you run into a slight problem when you go to make your $150k router IPv6 capable.
Then there is the little problem of client operating systems and the "migration" to IPv6. As there are only a handful of people on this planet who use IPv6 exclusively, routers will have to support both until all the client software of the world moves over. Now, it is bad enough getting full IPv4 BGP updates, but getting them *AND* IPv6 updates?
Of course, next comes all the little hardware out there. From the terminal servers people dial up to, to the layer 4 load balancers, there is a lot of hardware that doesn't support IPv6.
So, as a large network service provider, one would have to justify the costs associated with IPv6 against the benefits. The benefits are pretty slim right now unfortunately. Ideas like a single roaming IP (pipe dream if you ask me), mandatory multicast/anycast support, fixed sized headers and IP level security are all fine and dandy, but when you are talking about replacing (or at least suplementing) millions of dollars in infrastructure to allow a handful of people to use IPv6 for years until the REST of the world follows, it starts becoming hard to justify.
Don't get me wrong, IPv6 has some lovely attributes, but until Cisco enables IPv6 by default on all the hardware they make, everyone upgrades their copies of Windows and MacOS to support it and all of a sudden the terminal servers of the world (remember dialup still exists) all start learning how to route IPv6 packets, it is an uphill battle.
So the question really becomes, how long will it all take? IPv6 really needs a killer application to the general public aware that they *need* it and ask their providers to provide it. Once enough demand is generated, ISPs will start asking their upstreams for it and the ball will start rolling.
The same problems have plagued multicast for some time and still, very few providers support it and even fewer have customers who use it.
Of course, that's just my opinion, I could be wrong.
The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
for those of you who are confused about the nature of multihoming :
multihoming involves connecting to 2 or more isps and BGP publishing your ip space through both of them. this (ideally) involves having your own ARIN assigned ip space & AS number.
the point of multihoming is to address redunancy for inbound as well as outbound connections. you can use 2 isps + nat + creative outbound routing to handle outbound traffic, but that does nothing for a potential web server you're trying give multiple inbound paths to.
read the multihoming faq :
http://www.netaxs.com/~freedman/multi.html
Not really, and no I'm not.
The Internet already is, always has been, and must be, run by large players. You cannot have an interconnecting network that spans the world and has that many users without someone very big to put the infrastructure (hardware and software) in place, and to maintain it afterwards. The only people capable of doing that are major corporations, and a few very large not-so-commercial bodies (the academic community, for example).
I'm sorry, but if keeping things efficient and practical for these essential big players means you can't play with precious IP address space, then that's the price you're going to have to pay. There just isn't space for everyone to play with their own blocks of IPs any more, and there isn't time for everyone further up the chain to account for them even if the space was there.
Yes, it's unfortunate that some of these big players have a monopoly, which is rarely a good thing. Yes, it's unfortunate that little fish get eaten by big fish. But unless you have a better suggestion, there are only two choices: (a) leave the big fish alone, accept that for now there will be issues, and have an Internet, or (b) get on your high horse about monopoly abuse, civil liberties, and any other subject of pontification you can find, and kill the Internet. Me, I think that's a pretty easy choice.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
There's a good article at onlamp that talks about where all the IP's went and why things have gotten so stingy. A sad story about misallocation in the early days of the net (do companies like GE or Xerox really need 16 million addresses?)
This was an extremely oversimplified view, more like "I think I need to have bar want to do foo, but I'm clueless what anything else".
/20 (Sprint and Verio are two notable cases). (Thus, if you have an IP range IP_A from ISP A and IP range IP_B from ISP B, and both ISPs advertise both ranges, you can still run into problems when one of htem goes down). Fortunately, lately, the wind started to change, and I think sprint already relaxed their requirement to /24.
There are many issues at work:
a) Assignment of PI (Provider-Independent) addresses:
Back in '94, as an end user, you were able to get a netblock directly from ARIN. Then, this block could be advertised (by BGP4) by your upstream[s], and thus you got connectivity. The problem here lays that these IP addresses were nonaggregatable and led to exponential growth in routing table size. (see http://www.telstra.net/ops/bgptable.html up to 1994). Thus, CIDR was born, and hierarchical assignment became the rule. Your upstream (call it foo) gets the IPs from their upstream (call it bar), and the whole internet sees needs only one routing table entry to reach all of bar's customers.
b) ingress filtering (filtering of traffic from customers to make sure only the source IP that are assigned to them are used). Yes, most ISPs do ingress filtering now, and it is now considered a BCP (best current practice) to do this (there's an RFC on that). Again, this is for a damn good reason: Without filtering, DoS attacks cannot be traced to their source, if one is spoofing the source addresses. With filtering, at least you know that the source IP address is likely to be the one attack is launched from (or one of 0wned machines attacking you).
Its well known that ingress filtering makes multihoming harder, as your upstream has to open up their ingress filter for the IPs that are assigned to you by entities OTHER than your upstream (say, your other upstream).
Since apparently you intend to advertise your network via BGP4, all ISPs who will talk BGP4 to you will have no problem relaxing their ingress filters. If all you have is a DSL line, you'll have fat chance of getting your upstream to talk BGP4 in the first place. See below for strategies to do this without BGP.
c) Even if you managed to get your upstreams to turn off ingress filtering and advertise your network via BGP4, you still may run into problems because many ISPs do not listen to network announcements less than
Bottom line is: if you want to have your "own" IP address range, you must advertise it via BGP4. If you can get your upstream to do that, you can get them to relax their ingress filters, thus your original complaint is silly.
Now, if all you have is two DSL lines and no cooperation with your upstream you can do the following (sometimes called DNS-based multihoming), _for inbound traffic_:
You set up two nameservers (A and B), one on each of the IP ranges that you have (range_a and range_b). Make all of the entries given out by nameservers have TTL of 5 minutes.
Make each nameserver have a DIFFERENT zone, containing only IP addresses on that range. (Ex, nameserver A will have an entry for www pointing to an IP from range_a, nameserver B will point to an IP from range_b.(both nameservers can actually run on same machine, bound to different interfaces).
Then, whenever someone tries to reach www.yourdomain.com, they'll hit one of the nameservers. If the one they hit first is down, they'll hit the other one, and get an IP address from the _working_ network. Voila, you are still reachable when one connection goes down.
Then, if you don't want your servers to actually have two IP addresses (one on each net), you can do some trickery with iptables/ipchains to redirect traffic to a single IP (probably on private network).
For the outbound traffic: All you have to do is to NAT your traffic to the correct interface/IP range (the one that's currently working). That is not very hard to do with a bit of shell scripting.
Actually, things are a bit more complicated because of this: Your machine (main firewall or whatever) that contains all these interfaces, normally has one routing table. Choosing of the correct interface is done by lookup of DESTINATION IP. Now, assume a packet comes over to IP_B. You _must_ make sure that it will go out BACK on interface B (if you send a return packet with an IP_B source address over ISP_A, it'll discard it because of ingress filtering). This is hard: again, remember, routing does not depend on your _source_ address, it depends only on destination address.
So, how do you solve it?
Luckily, Linux has policy routing, which allows you to have multiple routing tables and choose between them based on some criteria, in your case, it will be source IP. You'll set up two routing tables, one with default route pointing to ISP A, one to ISP B, and a rule saying "If a packet has a source on IP_A, use routing table A, if not, use routing table B"
(see iproute2 documentation for details)
Well, I think I should write a HOWTO on that...I glossed over quite a lot of details here.
I'm Tech Director for a Caribbean ISP, so I know the problems in getting bandwidth AND multihoming.
To be multihomed correctly you will generally need:
-a decent router that can do BGP.
-more than one connection to providers who will talk BGP with you.
-your own AS number and an allocated block of IP addresses
The expensive part is not really "paying the fees" of (ARIN, RIPE, APNIC), or complying with their conditions, but in fact having someone tech enough that also understands the POLITICS (yes POLITICS) involved in running BGP, and the ongoing cost of keeping your network in fact running in this type of situation.
You are just looking at the tip of the iceberg and saying "wow that's expensive JUST for a block of IP's", which on the surface might look correct, however:
-just about anyone can say "gimme a block please" (cheap).
-checking on who can actually utilise them or not is expensive.
Memory in routers is easily scalable (it isn't but lets pretend it is), but the problem is not lack of memory, but actually wading through all those blocks of IP addresses.
Most of the main tier 1 providers have serious filters in place to avoid filling their routing tables up with junk due to mistakes or due to people who just haven't made a transit deal with them, so even if you were "given" a block of addresses, it wouldn't always be that easy for you to get it routed.
My advice: as you are "small" (compared to a Tier 1 provider), my guess is that there are ISPs down there that will do a better job than you for getting redundancy. Spend a bit more money on linking up to one of these, and backup your link to them somehow, and trust THEM for your link instead of trying to do it yourself. It will probably cost you just about the same, but your uptime will probably be HIGHER, because when you do BGP yourself, you are adding in extra weak spots that you may at this moment not be thinking of (your internal routing policies and how they get propagated, the people you will need to make sure this runs, etc...).
Just my own opinion. Add salt.
Here's an example of the kind of ridiculousness that results from some institutions having lots of IP addresses. I'm a student at MIT, which has all of net 18. I've been the network administrator for my fraternity for a couple years, which uses all of 18.216.xxx.xxx. That's right, we've got some 64k IP addresses, of which maybe 60 are assigned, and 40 actually point to a running computer. That means %99.9 are being wasted.
Arrr, it be the infamous pirate, No Beard Pete!
As nice as it is to have Provider Independant IP Space, as you've found out it's virtually impossible to get without paying through the nose (you can just BS how many hosts you have, if you want to fork over the cash to pay US$2,500/year for a /20 block from ARIN here in the USA). Then there are less clueful orginizations that don't even know they have some, because the current IT staff didn't get along with their predecesor (for instance this block I found for my own local City).
/24 block from either traditional Class C space, or the 63/8 or 64/8 Class A blocks that were returned a bit ago. No one with a clue should be filtering a /24 from either location.
However, it's not required to multihome. Really what you require to multihome is an Autonomous System Number (ASN) and a
The biggest downside to using your upstream providers IP space is that it pins you to a single ISP as you must use their IP space, and leaving them requires renumbering (but can be done without downtime within a reasonable transition timeframe of a few days). What we did was pick the largest ISP out there (UUNET), and then one of the top 10 (Sprint) and use both IP space (although we could have chosen to only use UUNET's). We use both provider's IP space on any important box (email, mainly) so that if we were to disconnect from one ISP (not likely), we only have to remove their IPs from our DNS, and the other IPS's IPs are already there and live (plus it gets around odd local routing problems outside of our control, where one remote site can reach one ISP but not the other).
We announce both blocks out both ISPs (to announce UUNET's blocks out Sprint and have them come back the shortest route, we had to get UUNET to "punch a hole" in their larger block and announce the smaller block we had so that both UUNET and Sprint would be announcing equally specific blocks for us... same is true of Sprint announcing their own assignment to us more specifically so they'll route to Sprint or UUNET, as if we only announcing the smaller block out UUNET, then all traffic would go that way unless our UUNET connection was down).
Anyway, not to write a HOW-TO (see Halibi's Internet Routing Architectures ISBN: 157870233X), but that's how to do it.
You don't need a huge router to be multihomed. Even a 2501 would work (as you just take default routes announcements from both ISPs, with the point being to advertise out your own blocks). If you want to take full routes from two ISPs, a 2650 with 128mb of RAM will work fine. If you want to take defaults + ISP-direct-customers, a 2610 with 64mb of RAM will work (it handles ISP-direct-customers from Sprint and UUNET just fine for us).
Lastly, never forget that site redundancy is just as important as internet redundancy. If a backhoe takes out the fiber or copper pairs going to your neck of the woods, more than likely it'll be both ISPs.
Normally I'd never mention my certs, but here they're relevent:
I'm a CCNP (next step past CCNA) and CCDP (next step past CCDA). I've been working for an IT Consulting/Integrater firm for 4 years (help desk positions 3 years before), and we also have our own little ISP on the side. I've worked with all the top 10 ISPs (and plenty of the Tier2/Tier3 folks), and set up a couple hundred of multihomed sites, so I'm not just quoting what I read in a book somewhere.