Researchers Probe Dark and Murky Net

umm qasr writes: "Security Focus has an interesting article on blocks of internet space that are hidden from most users, it is based on a survey by Arbor Networks. The most common 'invisible sites' being .mil, which seems is unintentional. The survey suggests others, which seem more sinister...using unused netblock addresses to send spam. It's a bit short on the details but interesting none the less."

Comments

Dark address space?

[gabriel_aristos]

So.. Does this mean that if they find enough "dark address space", the Internet will eventually stop growing, and someday, billions of years from now collapse back in upon itself to start the cycle all over again?

-j

Re:Interesting

[ShaunC]

>Its kinda crazy thinking about all the stuff thats out
>there that no one will ever see. I always figured
>anything sensative for military use would be stored on
>a proprietary government network

Might already be that way and we just don't know it. Talk about "dark netspace," nobody holds more of it than the US military... A bunch of class A's - 6.*, 7.*, 11.*, 21.*, 22.* - not to mention the smaller, uglier blocks. I imagine they could be running some sort of TOP-SEC-NET (or maybe SEC-PORN-NET) on one of these, unbeknownst to the outside world.

Shaun

The Cause..

[fwc]

The article doesn't really do a good job of saying what this is really about, and the report several people have linked to does provide detailed information, but again you need to have some context to understand it.

What they are really saying is that there are large chunks of the internet which can't talk to each other. This isn't because of firewalling or "hiding" behind a NAT box or the like, but is instead a result of the peering "politics" (which better describes what goes on than policies) between carriers.

Let me explain. If I am ISP A and I connect via peering to ISP B, I can't talk to ISP C's customers through B even if ISP B and C are connected. That is, unless I have an arrangement with ISP B to provide transit to ISP C. ISP C also has to agree to accept my routes even if ISP B provides transit to me.

Generally the big "Tier 1" ISP's peer with each other and generally don't exchange or buy transit from each other (except in some limited cases). Smaller ISP's generally buy transit from one or more Tier 1 ISP's. Some of the smaller Tier 1's both peer and buy transit.

It is not altogether unexpected that with hundreds of ISP's out there that certain ISP pairs just plain do not have connectivity between them. It would be almost impossible both economically, politically, and technically to insure that each ISP could talk to every other ISP out there.

Add on to that that there are some ISP's who set arbitrary limits on how many addresses you have to announce together in one chunk (prefix) before they will even listen to them. If you have a small ISP with insufficiently sized address blocks you may find that your connectivity to the internet suffers.

The other piece which WAS said fairly well is that most people don't notice the problem as 99% of the people out there don't use more than the most popular 1% of the internet. And THOSE sites are almost 100% connected (and if you ran an ISP which wasn't connected to the big sites, you would quickly find yourself without a customer base).

Note that I've taken some liberties with this description so there is some minor technical/political breakage in the description above. Or probably better put, this isn't meant as a technical reference piece on peering policies....

Re:So spammers can grab anything they want?

[db279]

In answer to your question- it depends, but certainly in some cases- yes.

Route-filters help address this, but many people don't do aggressive route filtering. Route filters, at least in this context, allow you to describe which route announcements you will accept from who. You typically write route-filters to *only* listen to route announcements for the networks that the person you are peering with owns. If its a multihomed connection then this can be a pain. If its an ISP (especially a multihomed one with multihomed customers) it becomes even more of a pain and becomes a matter of trusting your peers to enforce the right policies at the edge of their network. Some people do things with BGP communities to make this easier, but many folks do not have the clue to do so.

As mentioned earlier in the article, aggressive route filtering can actually increase the discontinuties in the network, but failing to do the right filtering can create opportunities for antisocial/malicious behavior.

There were attempts, with some success to create truly useful route registries- the radb's. MCI and someone else (I'm pretty sure it was the route-arbiter project folks- in which Abha [from this report] played a significant role) maintained these. Some people used these to auto-create route filters, but I think that all got just to darn complicated. I could be totally wrong about this, but that's my recollection.

Not to rant (to late), but to my way of thinking this all is rooted in a basic issue with large multi-entity IP networks- a peer isn't just someone you exchange traffic with for free [or with settlements] it really is a *peer*. By exchanging routing information (especially if you do something like accept/honor MED's) you really do have to trust these people- that means you have to believe they are as competent or moreso than yourself- in other works, a peer- in the truest sense of the word. With extremely democratic large scale IP networks (like the Internet) the meaning and usefullness of the term peer becomes significantly diluted- and this means that the network as a whole is likely to not function at a fully optimized state (or even a merely completely working state) all/most of the time. That isn't a horrible thing, but it certainly does make you reevaluate certain assumptions many people make about IP networks.

Further, I believe that most if not almost all of the "scaling" problems in the Internet today are not as much technical capability problems as configuration/design/education problems. We now have a giant, dynamic network that usually works quite well- can it fail catastrophically? I believe it *can*, but the size, interconnectiveness and diversity tends to locally contain failure conditions- events that would have been extremely catastrophic just a couple of years ago.

I'll stop "lecturing" now, except to say that it is great to see folks like these, CAIDA, Packet Design, and assorted others starting to really try to formalize analysis methods for networks of this complexity- its a great step forward from the cult-of-the-few-geeks (The Internet Routing Cabal wasn't that long ago- not to say they weren't great people who made lots of personal sacrifices to keep things working)

As a footnote, Craig L. and Abha A. have done other related work (before they were with Arbor Networks). I know they presented some of their work on BGP reconvergence time at the Montreal NANOG. I suspect they've presented since then.

http://www.nanog.org/mtg-9910/converge.html