Map the Internet... In One Day?

rjbrown99 writes "There have been numerous stories over the past few years on Bill Cheswick's Internet Mapping Project. The Lumeta folks even created a company out of it. Well, now there is a competitor. A single guy with a single computer is working to accomplish the same feat - within ONE DAY and using open-source tools to do it. The new project is called Opte and can be found at www.opte.org." He's made some progress and is looking for volunteers.

Comments

Creepy

[Seanasy]

When I first saw the image on the right it looked like a human brain. It would be creepy if the Internet had a sort of fractal self-similarity to our physiology.

Re:Here ya' go...

[BadCable]

Why don't they make maps like that of say the telephone network?

That'd be very interesting to see with very similar benifits.

rsync

[bigjnsa500]

Why can't somebody just rsync the Google search cluster? Wouldn't it have the same results this guy is looking for?

I have considered something similar

[fv]

As the author of the free Nmap ("Network Mapper") tool, I have also considered creating a map of the entire Internet. I would have focused on end hosts (where they are, what operating systems and services they run, trending, etc.) instead of routing. Rather than try this from a single high-bandwidth machine (as with Opte), I was going to take a distributed approach. I would release a P2P-like application that users could run and each scan small sections of network space to be contributed to the global database. The app would be called Nmapster :). I also liked to think about it as a "caching service", so that you don't have to spend the time rescanning the Microsoft network if someone else has done so in the last N hours.

Then I came to my senses and decided to work on more practical and less controversial projects such as Nmap Version Detection. But the subversive in me still hasn't given up entirely on Nmapster :).

-Fyodor

bad for business

[glassesmonkey]

Maybe people already take this into consideration, but won't this impact webhosting? Won't people try to get their webpage/company closer to the main trunk / center of map? When you look for a hosting service (basically an IP address) right now most people don't consider where in the map the host is.

I mean with this tool, I would look up where my new IP would land me and try to find a host closer to the main backbones. Is this already done now by most people?

(on another subject the maps remind me of the species origin stuff)

Re:Been there, done that

[jd]

I believe this new project uses the CAIDA tools. The maps look like the output from their Java-based network mapping package.

However, it looks like it's one map a week, not one a day, and that's only with more power. Based on the charts on the site, it's going to take between 3-4 months to map a decent portion of the Internet, and he's only going to Class C resolution.

Further, he's mapping as a spanning-tree. This means that tunnels, load-balancing and multipath connections cannot be shown at all.

Also, it only shows IPv4 unicast nodes, so you don't see any IPv6 or multicast paths.

Networks that are gatewayed, NATted or otherwise complicated, will also not show up. It would be good if gateway nodes where the other side of the gateway cannot be seen were marked, so that we can see where obscured parts of the net are.

Re:Hierarchy

[daves]

Has anyone noticed that nearly all of the maps have a more or less tree-shaped structure?

No matter where you are on the net, your view is going to look like a tree with you at the center. Traceroute-type mapping will not capture the redundancies.

limited by the speed of light?

[Elminst]

So if we assume the electrical signals of his packets travel at the speed of light (186 000 miles per second) across the internet (which they don't really, but we'll ignore that for this argument), then logic tells us that the internet must have less than 16,070,400,000 miles of cable in order for this to work. Because his data cannot travel any faster along the pipes.

And that's only one way... Assuming query and response, his packets have to effectively travel double the existing cable lengths.

So do all the (public) networks in all the world total less than 16 trillion miles of cable?

Internet Topology

[Tacoguy]

I have followed various projects related to mapping cyberspace through the years and have always found An Atlas of Cycerspaces to be fascinating.

Mapping by Lumeta is one such methodology and I even have a poster of theirs printed by Peacock Maps (server down just now) in my office.

I have noticed that these mappings take a long time to complete and being able to map in a short time frame could be beneficial in much the same way that Internet Traffic Report can be to visualize traffic patterns or disruptions.

Taco

Re:Pfft! Kids today

[hburch]

We still have that picture.

Since I linked to his site, I should mention that Martin Dodge has gathered a nice collection of maps of the Internet on his CyberGeography site, including many historical maps. CyberGeography also includes many other interesting types of maps.

AS mapping would be more useful

[cpghost]

Why not map Autonomous Systems instead? Routes to AS are being advertised by BGP, and a set of well placed looking glasses would be all it takes to get a big picture. I never saw anything like an AS mapping, with the ASes as nodes and the (BGP announced) routes between them as links.

Of course, some AS span multiple geographical areas, but this is also true of class C networks.

The big advantage of mapping ASes is, that there are not so many of them, compared to class C nets, thus resulting in much simpler graphs. Moreover, the graphs would nicely show the boundaries between institutions/organizations, rather than artificial boundaries based on numerical addresses.