Slashdot Mirror
Researchers Probe Dark and Murky Net
umm qasr writes: "Security Focus has an interesting article on blocks of internet space that are hidden from most users, it is based on a survey by Arbor Networks. The most common 'invisible sites' being .mil, which seems is unintentional. The survey suggests others, which seem more sinister...using unused netblock addresses to send spam. It's a bit short on the details but interesting none the less."
Kinda interesting what all is out there. Now, add on top of that all of those evil spam sending servers that are behind firewalls on 'reserved' ip blacks. Its kinda crazy thinking about all the stuff thats out there that no one will ever see. I always figured anything sensative for military use would be stored on a proprietary government network. But now that I think of it. If they put it on some obscure ip block and give it no hostname, who will ever find it? Wonder if they found my secret porn stash when they were probing all them blocks. =)
Can all fish swim?
"First Officer! Demurk!" ... Finished!"
"Yes Captain Spamford."
"Prepare spam... Bulk Email!"
"Bulk Emailing sir!
"Excellent, return to Murk space."
.
.
.
"Sir! it's an anti spammer!"
"What's he want?"
"He wants to shove our testicles up our noses and beat us to death with toner cartridges. He said something about sucking your eyes out with a penis enlarger as well."
"again?"
dave
So.. Does this mean that if they find enough "dark address space", the Internet will eventually stop growing, and someday, billions of years from now collapse back in upon itself to start the cycle all over again?
-j
Torg, come out of the spaceship. Nothing can stop Torg.
And all that time I thought it was just my ISP that sucked when the "dark side" was taking over the address space. "Oh, now I get it. errrrrrr I think?"
It seems like the article could have had more explanaton and real information on what dark address space is.. I'm still not fully clear after reading. Is "dark address space" just unconnected networks or more subtle. I guess you really need to be a network person to understand fully.
Reminds me of the raging debate over dark matter in Astronomy, and how it accounts for the mass of the universe etc... The debates always involve crazy theories that pretty much contradict eachother until they finally high-enough resolution data..
__ No registration required to read this message. They did it in the Matrix.
ICANN is changing the domain namespaces by adding new TLDs like .info, and accepting new conventions like non-ASCII characters.
The problem is that many software, libraries, and hand-made filters validate domain names based on simple rules like "only 0-9, a-z, dots, and it should end by two characters or com/net/org/edu" .
For instance, I guess that many web forms are currently refusing mail addresses like "john@johncompany.info".
These new, non backward-compatible domain names will probably belong to the "dark and murky net" too.
{{.sig}}
>Its kinda crazy thinking about all the stuff thats out
>there that no one will ever see. I always figured
>anything sensative for military use would be stored on
>a proprietary government network
Might already be that way and we just don't know it. Talk about "dark netspace," nobody holds more of it than the US military... A bunch of class A's - 6.*, 7.*, 11.*, 21.*, 22.* - not to mention the smaller, uglier blocks. I imagine they could be running some sort of TOP-SEC-NET (or maybe SEC-PORN-NET) on one of these, unbeknownst to the outside world.
Shaun
Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
Many discussion sites have marginal value because it is difficult to sort through the background noise to find intelligent, meaningful dialogue. Slashdot is interesting because it resists the typical Internet qualities of anonymity and egalitarianism. ...next thing ya know, they're gonna be using slashcode for missles
//radiotakeover.
IPv6 could lead to a lot of new problems. I think it's necessary but even with IPv6 we need better methods of allocation.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
Sorry 'bout the whoring..
Shaun
Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
From the article:
Because routers don't normally log such activity, murky address space could hide the full range of antisocial or illegal network behavior, says Labovitz.
Oh no, here we go again. Just because it's about the internet and contains a lot of words that are a little bit different to what "normal" people use daily - like "router", "hosts" and "routable address space" - it doesn't mean it's something dangerous. Not even new.
Can you imagine someone getting funds to look into the origins of "paper spam"? "Oh no, the spammers are using bogus return addresses!" "Bad guys can communicate pretty safe and unhindered by putting their messages in envelopes, stamping them and sendim them by mail!"
I can understand that the guys had to show something for 3 years worth of "research", but unless the securityfocus article is a very-very short, abridged version for the masses, they have no results.
I'm with an ISP in Vancouver, and I can tell you that 1 out of 5 sites I try will fail. If a site cannot be reached, a quick traceroute reveals that UUnet is the culprit. Always a 152.158.xxx.xxx address.
.... not so reliable. Has anyone else noticed a slow degadation in the performance of the 'Net in general? Or is it the crack again?
Over the last 6 months or so, it definitely seems like the 'Net is
What they are really saying is that there are large chunks of the internet which can't talk to each other. This isn't because of firewalling or "hiding" behind a NAT box or the like, but is instead a result of the peering "politics" (which better describes what goes on than policies) between carriers.
Let me explain. If I am ISP A and I connect via peering to ISP B, I can't talk to ISP C's customers through B even if ISP B and C are connected. That is, unless I have an arrangement with ISP B to provide transit to ISP C. ISP C also has to agree to accept my routes even if ISP B provides transit to me.
Generally the big "Tier 1" ISP's peer with each other and generally don't exchange or buy transit from each other (except in some limited cases). Smaller ISP's generally buy transit from one or more Tier 1 ISP's. Some of the smaller Tier 1's both peer and buy transit.
It is not altogether unexpected that with hundreds of ISP's out there that certain ISP pairs just plain do not have connectivity between them. It would be almost impossible both economically, politically, and technically to insure that each ISP could talk to every other ISP out there.
Add on to that that there are some ISP's who set arbitrary limits on how many addresses you have to announce together in one chunk (prefix) before they will even listen to them. If you have a small ISP with insufficiently sized address blocks you may find that your connectivity to the internet suffers.
The other piece which WAS said fairly well is that most people don't notice the problem as 99% of the people out there don't use more than the most popular 1% of the internet. And THOSE sites are almost 100% connected (and if you ran an ISP which wasn't connected to the big sites, you would quickly find yourself without a customer base).
Note that I've taken some liberties with this description so there is some minor technical/political breakage in the description above. Or probably better put, this isn't meant as a technical reference piece on peering policies....
If you could see one, it wouldn't be dark. And if you did see one, They would have to kill you.
I think this is just another .mil conspiracy - those sites and addresses aren't just parts of badly managed webspace - they are websites of black ops, dark projects, stealth planes and hidden agendas. An intranet for the Anti-Illuminati - the Shadows. :-)
Money for nothing, pix for free
While the proposed explanation is quite possible, there is a simpler explanation: The spammer's upstream ISP disconnected them. Cut them off, and their advertised BGP routes will automatically lapse -- resulting in the rest of the internet simply seeing a spam source followed by a withdrawn BGP route.
Tarsnap: Online backups for the truly paranoid
Dark address space refers to globally unique IPs (ie. not private IPs as defined by the RFCs) that should be accessible from anywhere on the internet but are not due to one of many reasons. The two reasons I am most familiar with are:
/24 or longer masks). This means that unless there is an aggregrate route for that block that will get the packets there eventually, the IP is dark for people using that provider.
.mil
Route filtering.
To reduce the size of the routing table in the memory of their core routers, some providers throw away announcements of small blocks (say
Some providers also filter blocks that are listed by the one of allocators (ARIN, RIPE, APNIC) as not being allocated or are reserved for special use. The article infers that this is what happens to lots of
Black holed routes.
Sometimes, either intentionally or accidently, providers announce routes to blocks that they actually can't reach directly. This is usually a misconfiguration or done on purpose to null route blocks containing a host performing a DOS or some other network misdemeanour. This is usually a transient state.
hth
Marty
"I can't buy want I want because it's free. Can't be what they want because I'm me." -Corduroy, Pearl Jam
The Internet was never a military network. This seems to confuse many people buts its quite simple. ARPAnet was created to allow the computer science community to share resources since all the new CS departments in the 1960's were calling for more and more government funds to pay for bigger and faster computer systems. It was though that networking them would allow collaboration and sharing of big iorn machines. Futile hope I know 8)
The confusion is based on the fact that Paul Baran at RAND had designed a network which would have used inexpensive links with multiple redundancies to ensure that communications would not be disrupted in a command and control structure for the Nuclear deterant. This idea was also being developed seperately in the UK and called Packet Switching by Donald Davis at the UK National Physics Lab on the first system to use this technology. It was later used as a basis for ARPAnet.
The important point is that when the ARPAnet was created the inventors had never heard of the RAND report and the Air Force had turned down RANDs plan to build a test syestem. It was civilian to the core. However when the military absorbed ARPA to form DARPA the created a nonclassified system called MilNet. This came later and is not the same as saying the Internet is built on a military system
Ok that was my 2c's worth. Any comments?
the phenomenon is generally not noticeable to average Internet users because most netizens only use a tiny portion of the Net. "Most people access five or ten web sites," Labovitz says.
Oh...(SHOCKED!) so does it mean out there are other sites besides slashdot...
Cool... do you need any special software to browse them ? I use K-Meleon. There's a green icon on my desktop - I double click it and it takes me to slashdot.org, where I read the coolest stuff and then I click the tiny X button ontop when I finished.
Heard about a proggie, though: Internet Exploder that would supposedly take you places where you wanted to go that thay - I always thought it's some travel/tourism/ticket booking application or stuff like that....
Gone researching how to get to the others 4 or 9 web sites...
__________
Don't belong. Never join. Think for yourself. Peace!
Consider the source they used for their data: Routing tables. Aside from announcing the main superblock that says 'Hey, I have these IPs', looking at a full routing table to find out where blocks really wind up isn't effective. I actually had this discussion with a colleague a few days ago. They may announce it, but that doesn't mean it's reachable.
.mil and broadband land as the largest 'offenders', for lack of a better term. Personally, I could care less if .mil hosts aren't world reachable. By and large, I know for a fact there's a lot that exist that you simply can't get to, or wouldn't want to anyway.
/24's up into /30's for interface numbering. Doing this produces a herd of four IP subnets. You immediately lose two IPs to Network Address and Broadcast, leaving you with two usable IPs, one for each end of the numbered interface, against 254 for a full Class C allocation. Do the math, and that's 64 point-to-point circuits.
The report cites
As far as broadband goes, as well as large NSPs, consider how much address space is simply lost to breaking
Companies like Cisco and Unisphere sport routers capable of numbering interfaces in the THOUSANDS. Even making efficient use of IPs when numbering ATM topologies (common for DSL implementations), you're still losing one IP per interface, in addition to whatever small block is allotted to the customer on the other end. In most cases, every hop you see in a traceroute is one IP of a four ip subnet (exceptions would be LAN topology based peers or transits). For the purposes of security, or simplicity, providers may simply choose to not announce routes to IP space allocated for interface use. Inside their own networks, interior protocols like IGP, ISIS and OSPF can handle local delivery, but the world doesn't really need to know how to throw packets at a router's interfaces.
Cable modems are less guilty of this than most, since they tend to allocate two or four class C superblocks to a neighborhood and mask them accordingly.
- billn
You're close to right, that IS possible. The problem is, that someone has had the block allocated to them. It's a simple lookup to the IRRdb or various other registry's to find the owner of the block and contact them. It *is*, however, a pretty damn sneaky move, which fully thwarts the most common tool used to identify a spam source: traceroute.
As far as the IPv6 issue, a lot will depend purely on accounting: How is address space issued? Do you get an IP with your driver's license?
Accountability will be everything, at that point. IPv4, as it's designed, is based on trust. America, as it's designed, is based on civil disobediance. Stop laughing, I'm serious.
- billn
In answer to your question- it depends, but certainly in some cases- yes.
Route-filters help address this, but many people don't do aggressive route filtering. Route filters, at least in this context, allow you to describe which route announcements you will accept from who. You typically write route-filters to *only* listen to route announcements for the networks that the person you are peering with owns. If its a multihomed connection then this can be a pain. If its an ISP (especially a multihomed one with multihomed customers) it becomes even more of a pain and becomes a matter of trusting your peers to enforce the right policies at the edge of their network. Some people do things with BGP communities to make this easier, but many folks do not have the clue to do so.
As mentioned earlier in the article, aggressive route filtering can actually increase the discontinuties in the network, but failing to do the right filtering can create opportunities for antisocial/malicious behavior.
There were attempts, with some success to create truly useful route registries- the radb's. MCI and someone else (I'm pretty sure it was the route-arbiter project folks- in which Abha [from this report] played a significant role) maintained these. Some people used these to auto-create route filters, but I think that all got just to darn complicated. I could be totally wrong about this, but that's my recollection.
Not to rant (to late), but to my way of thinking this all is rooted in a basic issue with large multi-entity IP networks- a peer isn't just someone you exchange traffic with for free [or with settlements] it really is a *peer*. By exchanging routing information (especially if you do something like accept/honor MED's) you really do have to trust these people- that means you have to believe they are as competent or moreso than yourself- in other works, a peer- in the truest sense of the word. With extremely democratic large scale IP networks (like the Internet) the meaning and usefullness of the term peer becomes significantly diluted- and this means that the network as a whole is likely to not function at a fully optimized state (or even a merely completely working state) all/most of the time. That isn't a horrible thing, but it certainly does make you reevaluate certain assumptions many people make about IP networks.
Further, I believe that most if not almost all of the "scaling" problems in the Internet today are not as much technical capability problems as configuration/design/education problems. We now have a giant, dynamic network that usually works quite well- can it fail catastrophically? I believe it *can*, but the size, interconnectiveness and diversity tends to locally contain failure conditions- events that would have been extremely catastrophic just a couple of years ago.
I'll stop "lecturing" now, except to say that it is great to see folks like these, CAIDA, Packet Design, and assorted others starting to really try to formalize analysis methods for networks of this complexity- its a great step forward from the cult-of-the-few-geeks (The Internet Routing Cabal wasn't that long ago- not to say they weren't great people who made lots of personal sacrifices to keep things working)
As a footnote, Craig L. and Abha A. have done other related work (before they were with Arbor Networks). I know they presented some of their work on BGP reconvergence time at the Montreal NANOG. I suspect they've presented since then.
http://www.nanog.org/mtg-9910/converge.html
One of the people conducting the study, Abha Ahuja, has passed away.
Not to mention stupid things like "ZIP" codes.
Guess what - other countries may have postal codes, but they don't always fall into a format of five contiguous numbers...
Just today, Yahoo told me that I had an impossible 'zip' code, so I did what I usually do in that case - enter "02134", which as many of you know. is pronounced "Oh!, two-one, three-FOUR!", especially if it follows "Box 3-5-0, Boston Mass", which I fill in whenever some braindead php monkey has never heard of my particular prefecture...
-- My Weblog.
I always figured anything sensative for military use would be stored on a proprietary government network
It's called SIPRNET, and is well protected.
When I worked for a company that made routers and other networking equipment (Gandalf, now part of MIke and TErry's Lawnmowers), we had a very large address block. I forget how big it was, it might have been a class B or even an A. But I know we had assigned to our lab three class Cs, one that we used for computers we put on the internal net, and two that we used for computers we put on test networks. Usually the two class Cs on test networks were only connected to each other through a router or bridge that we were testing, not to the internet at large.
Actually, this was a pretty interesting project to many slashdot readers. Using an extremely early version of Linux (SLS 1.02 with kernel 0.99pl14e, I seem to recall), we had a laboratory full of 486s and 386s with two ethernet cards. One was a standard card that was connected to the company lan, and the other was a special programmable card that could be commanded to do stuff that ethernet cards aren't supposed to do, like short packets and bad ethernet headers and the like. This card was connected to one of the lans on one side or the other of the unit under test. There was an automated program running on each box under control of the master box, which ran a script in a custom scripting language that could tell one box to emit a packet, and another box on the other side to check if it got it, and more sophisticated stuff.
It was very cool, and a very early use of Linux in a commercial environment.
The next Cmdr Taco duplicate will be ready soon, but subscribers can beat the rush and see it early!
ZIP code 12345 is a special ZIP code belonging to GE in Schenectady, NY.
Please don't mod down though, I am sure others here probably have the same question!
/24's, /30's - and definitions of classes (A, B, C, etc), as well as what it means when you see like an IP followed by a /nn (like, oh, 27.141.102.18/24 or similar).
Can you explain (or better, point me to a source explaining) what is meant in networking terminology when you say
This is something I have been curious about for a long time, and would like to learn more (whether it would be useful to me or not).
Thank you for any help you or others can provide...
Reason is the Path to God - Anon
By definition, any classified machine CAN NOT be connected to the Internet. Try it, and you could be looking at a lifetime vacation in Leavenworth.
When I worked for a defense contractor, we were exceptionally paranoid about this sort of thing.
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
Is the "dark address space" made up of strange websites? Or perhaps charmed ones?
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
Is that what they told you?
This next song is very sad. Please clap along. -- Robin Zander
In many cases, it is the military;s fault - not the ISP. Take for example, www.gordon.army.mil. You can't hit that site because the administrators are so paranoid they have closed it off to outside access.
I am a cable-modem user (don't believe the damn commercials) and recieve a broadcast address of /32, thereby using up less IP addresses than would normally be necessary. Not to mention that the DHCP server is not using a valid IP address. They obviously are doing a good job in changing their ways.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
See my post in this earlier