Slashdot Mirror
Researchers Probe Dark and Murky Net
umm qasr writes: "Security Focus has an interesting article on blocks of internet space that are hidden from most users, it is based on a survey by Arbor Networks. The most common 'invisible sites' being .mil, which seems is unintentional. The survey suggests others, which seem more sinister...using unused netblock addresses to send spam. It's a bit short on the details but interesting none the less."
So.. Does this mean that if they find enough "dark address space", the Internet will eventually stop growing, and someday, billions of years from now collapse back in upon itself to start the cycle all over again?
-j
Torg, come out of the spaceship. Nothing can stop Torg.
ICANN is changing the domain namespaces by adding new TLDs like .info, and accepting new conventions like non-ASCII characters.
The problem is that many software, libraries, and hand-made filters validate domain names based on simple rules like "only 0-9, a-z, dots, and it should end by two characters or com/net/org/edu" .
For instance, I guess that many web forms are currently refusing mail addresses like "john@johncompany.info".
These new, non backward-compatible domain names will probably belong to the "dark and murky net" too.
{{.sig}}
>Its kinda crazy thinking about all the stuff thats out
>there that no one will ever see. I always figured
>anything sensative for military use would be stored on
>a proprietary government network
Might already be that way and we just don't know it. Talk about "dark netspace," nobody holds more of it than the US military... A bunch of class A's - 6.*, 7.*, 11.*, 21.*, 22.* - not to mention the smaller, uglier blocks. I imagine they could be running some sort of TOP-SEC-NET (or maybe SEC-PORN-NET) on one of these, unbeknownst to the outside world.
Shaun
Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
Sorry 'bout the whoring..
Shaun
Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
From the article:
Because routers don't normally log such activity, murky address space could hide the full range of antisocial or illegal network behavior, says Labovitz.
Oh no, here we go again. Just because it's about the internet and contains a lot of words that are a little bit different to what "normal" people use daily - like "router", "hosts" and "routable address space" - it doesn't mean it's something dangerous. Not even new.
Can you imagine someone getting funds to look into the origins of "paper spam"? "Oh no, the spammers are using bogus return addresses!" "Bad guys can communicate pretty safe and unhindered by putting their messages in envelopes, stamping them and sendim them by mail!"
I can understand that the guys had to show something for 3 years worth of "research", but unless the securityfocus article is a very-very short, abridged version for the masses, they have no results.
What they are really saying is that there are large chunks of the internet which can't talk to each other. This isn't because of firewalling or "hiding" behind a NAT box or the like, but is instead a result of the peering "politics" (which better describes what goes on than policies) between carriers.
Let me explain. If I am ISP A and I connect via peering to ISP B, I can't talk to ISP C's customers through B even if ISP B and C are connected. That is, unless I have an arrangement with ISP B to provide transit to ISP C. ISP C also has to agree to accept my routes even if ISP B provides transit to me.
Generally the big "Tier 1" ISP's peer with each other and generally don't exchange or buy transit from each other (except in some limited cases). Smaller ISP's generally buy transit from one or more Tier 1 ISP's. Some of the smaller Tier 1's both peer and buy transit.
It is not altogether unexpected that with hundreds of ISP's out there that certain ISP pairs just plain do not have connectivity between them. It would be almost impossible both economically, politically, and technically to insure that each ISP could talk to every other ISP out there.
Add on to that that there are some ISP's who set arbitrary limits on how many addresses you have to announce together in one chunk (prefix) before they will even listen to them. If you have a small ISP with insufficiently sized address blocks you may find that your connectivity to the internet suffers.
The other piece which WAS said fairly well is that most people don't notice the problem as 99% of the people out there don't use more than the most popular 1% of the internet. And THOSE sites are almost 100% connected (and if you ran an ISP which wasn't connected to the big sites, you would quickly find yourself without a customer base).
Note that I've taken some liberties with this description so there is some minor technical/political breakage in the description above. Or probably better put, this isn't meant as a technical reference piece on peering policies....
While the proposed explanation is quite possible, there is a simpler explanation: The spammer's upstream ISP disconnected them. Cut them off, and their advertised BGP routes will automatically lapse -- resulting in the rest of the internet simply seeing a spam source followed by a withdrawn BGP route.
Tarsnap: Online backups for the truly paranoid
Dark address space refers to globally unique IPs (ie. not private IPs as defined by the RFCs) that should be accessible from anywhere on the internet but are not due to one of many reasons. The two reasons I am most familiar with are:
/24 or longer masks). This means that unless there is an aggregrate route for that block that will get the packets there eventually, the IP is dark for people using that provider.
.mil
Route filtering.
To reduce the size of the routing table in the memory of their core routers, some providers throw away announcements of small blocks (say
Some providers also filter blocks that are listed by the one of allocators (ARIN, RIPE, APNIC) as not being allocated or are reserved for special use. The article infers that this is what happens to lots of
Black holed routes.
Sometimes, either intentionally or accidently, providers announce routes to blocks that they actually can't reach directly. This is usually a misconfiguration or done on purpose to null route blocks containing a host performing a DOS or some other network misdemeanour. This is usually a transient state.
hth
Marty
"I can't buy want I want because it's free. Can't be what they want because I'm me." -Corduroy, Pearl Jam
In answer to your question- it depends, but certainly in some cases- yes.
Route-filters help address this, but many people don't do aggressive route filtering. Route filters, at least in this context, allow you to describe which route announcements you will accept from who. You typically write route-filters to *only* listen to route announcements for the networks that the person you are peering with owns. If its a multihomed connection then this can be a pain. If its an ISP (especially a multihomed one with multihomed customers) it becomes even more of a pain and becomes a matter of trusting your peers to enforce the right policies at the edge of their network. Some people do things with BGP communities to make this easier, but many folks do not have the clue to do so.
As mentioned earlier in the article, aggressive route filtering can actually increase the discontinuties in the network, but failing to do the right filtering can create opportunities for antisocial/malicious behavior.
There were attempts, with some success to create truly useful route registries- the radb's. MCI and someone else (I'm pretty sure it was the route-arbiter project folks- in which Abha [from this report] played a significant role) maintained these. Some people used these to auto-create route filters, but I think that all got just to darn complicated. I could be totally wrong about this, but that's my recollection.
Not to rant (to late), but to my way of thinking this all is rooted in a basic issue with large multi-entity IP networks- a peer isn't just someone you exchange traffic with for free [or with settlements] it really is a *peer*. By exchanging routing information (especially if you do something like accept/honor MED's) you really do have to trust these people- that means you have to believe they are as competent or moreso than yourself- in other works, a peer- in the truest sense of the word. With extremely democratic large scale IP networks (like the Internet) the meaning and usefullness of the term peer becomes significantly diluted- and this means that the network as a whole is likely to not function at a fully optimized state (or even a merely completely working state) all/most of the time. That isn't a horrible thing, but it certainly does make you reevaluate certain assumptions many people make about IP networks.
Further, I believe that most if not almost all of the "scaling" problems in the Internet today are not as much technical capability problems as configuration/design/education problems. We now have a giant, dynamic network that usually works quite well- can it fail catastrophically? I believe it *can*, but the size, interconnectiveness and diversity tends to locally contain failure conditions- events that would have been extremely catastrophic just a couple of years ago.
I'll stop "lecturing" now, except to say that it is great to see folks like these, CAIDA, Packet Design, and assorted others starting to really try to formalize analysis methods for networks of this complexity- its a great step forward from the cult-of-the-few-geeks (The Internet Routing Cabal wasn't that long ago- not to say they weren't great people who made lots of personal sacrifices to keep things working)
As a footnote, Craig L. and Abha A. have done other related work (before they were with Arbor Networks). I know they presented some of their work on BGP reconvergence time at the Montreal NANOG. I suspect they've presented since then.
http://www.nanog.org/mtg-9910/converge.html