Researchers Probe Dark and Murky Net

umm qasr writes: "Security Focus has an interesting article on blocks of internet space that are hidden from most users, it is based on a survey by Arbor Networks. The most common 'invisible sites' being .mil, which seems is unintentional. The survey suggests others, which seem more sinister...using unused netblock addresses to send spam. It's a bit short on the details but interesting none the less."

Comments

.info and other new TLDs in the dark, too?

[chrysalis]

ICANN is changing the domain namespaces by adding new TLDs like .info, and accepting new conventions like non-ASCII characters.
The problem is that many software, libraries, and hand-made filters validate domain names based on simple rules like "only 0-9, a-z, dots, and it should end by two characters or com/net/org/edu" .
For instance, I guess that many web forms are currently refusing mail addresses like "john@johncompany.info".
These new, non backward-compatible domain names will probably belong to the "dark and murky net" too.

Arbor analyzed ISP mail logs?

[ShaunC]

From the article,

Arbor Networks' researchers went to the mail logs of a local ISP and compared several thousand unique mail sources with "murky" addresses spotted in their monitoring.

Am I reading this right? If so, am I alone in feeling uneasy about it? It would be interesting to know what ISP allowed "some research company" to look through their mail logs. I suspect Arbor was only interested in source IP addresses, but it still smells.

Shaun

The Cause..

[fwc]

The article doesn't really do a good job of saying what this is really about, and the report several people have linked to does provide detailed information, but again you need to have some context to understand it.

What they are really saying is that there are large chunks of the internet which can't talk to each other. This isn't because of firewalling or "hiding" behind a NAT box or the like, but is instead a result of the peering "politics" (which better describes what goes on than policies) between carriers.

Let me explain. If I am ISP A and I connect via peering to ISP B, I can't talk to ISP C's customers through B even if ISP B and C are connected. That is, unless I have an arrangement with ISP B to provide transit to ISP C. ISP C also has to agree to accept my routes even if ISP B provides transit to me.

Generally the big "Tier 1" ISP's peer with each other and generally don't exchange or buy transit from each other (except in some limited cases). Smaller ISP's generally buy transit from one or more Tier 1 ISP's. Some of the smaller Tier 1's both peer and buy transit.

It is not altogether unexpected that with hundreds of ISP's out there that certain ISP pairs just plain do not have connectivity between them. It would be almost impossible both economically, politically, and technically to insure that each ISP could talk to every other ISP out there.

Add on to that that there are some ISP's who set arbitrary limits on how many addresses you have to announce together in one chunk (prefix) before they will even listen to them. If you have a small ISP with insufficiently sized address blocks you may find that your connectivity to the internet suffers.

The other piece which WAS said fairly well is that most people don't notice the problem as 99% of the people out there don't use more than the most popular 1% of the internet. And THOSE sites are almost 100% connected (and if you ran an ISP which wasn't connected to the big sites, you would quickly find yourself without a customer base).

Note that I've taken some liberties with this description so there is some minor technical/political breakage in the description above. Or probably better put, this isn't meant as a technical reference piece on peering policies....