Slashdot Mirror
Latest Maps of the Internet
mnmonte writes "Yesterday morning Opte.org announced that they have successfully mapped the entire internet. They are currently compiling a LGL map for all to see. Currently they have a LGL map that has 'over 5 million edges and has an estimated 50 million hop count'. Also only took them 252.68 hours to complete."
Maybe I'll finally be able to find out where I took that wrong turn...
"Also only took them 252.68 hours to complete."
If they can do all that, then they likely won't suffer too much from the slashdot effect. That is unless, enough of us get our grubbies on their 2.8meg PNG map from Nov 23...
"Mapping engine status: Stalled (Damn Slashdot Bastards!)"
I know it's a LGL map, but wouldn't it be cooler to position connections on a mock surface of our planet? That might actually be something to behold. These maps just appear to be link/traffic pointers or something to that effect.
So when are one of us nerds going to invent a better way to tell what geographical location is associated with what IP/URL? Servers could have a kind of location grid address. That'd be neat. That way you could tell how far your data was going, and where. You could avoid posting in certain countries, or try to post in others. The flipside would be that it would cut back on privacy and the anonymity that makes the web special. Wouldn't it be kinda scary if your IP told people where to find you? I can think of a few angry gamers that might want to do me in, I don't know about you!!!
I'll print it and give it to my wife. If she ever needs directions again on the web, I'll give her the map, slap my forhead and mumble 'oh yeah, that's right : women can't read maps'
:-)
ooh i'm so bad
When will I end this grieving ? When will my future begin ?
Quick! They've done something really cool and clever. Let's Slashdot them back to the stone age. That'll teach them!
Reality is defined by the maddest person in the room
"Hey, I can see my house from up here!"
Trolling is a art,
Their site seems to be crawling. I thought for a minute we'd have to take them off the map.
________________________________________________
suwain_2
... I'll be starting to produce maps of IP addresses to latitude/longitude by IP address soon... Been sourcing the data.(See the sig.)
Simon
Physicists get Hadrons!
...and it only took 6 minutes to /. their server.
"...all the labours of the ages, all the devotion, all the inspiration, all the noonday brightness..." yada yada
So now I finally know why the internet is so damned slow... people are purposely wasting bandwidth trying to 'map out' the internet, and all this time I thought it was a DDOS attack... or at the very least some new worm...silly me.
I wonder if there are people driving around during rush hour trying to 'map out' the city...
---
Programming is like sex... Make one mistake and support it the rest of your life.
I think this is neat. They are apparently releasing the map data itself, not just the pretty pictures. This means that the map is usable for research. There are lots of things you could look at, from average network distance between nodes to routing redundancy and who knows what else. Since it's open source, maybe others can come along and improve it, perhaps associating the nodes with geographical location.
Wouldn't it be kinda scary if your IP told people where to find you?
Umm... this has been possible for quite a while: See Geobutton: http://www.geobutton.com/IpLocator.htm
The IP map I'm (slowly) making will locate you to a city, eventually. It's only been going for about 15 days atm, but we're already up at ~15-20% successful at locating cities, according to visitors :-)
/24 for the time being, although with DSL companies giving static /29's I'll probably adapt to that soon enough...
I doubt that locating to city has any privacy implications, and I'm only doing it to
Simon
Physicists get Hadrons!
How do they check for outages. IF I'm going from A->B for the first time and suddenly the route drops or changes surely that changes the map. So any idea of how many times they check a route?
Rus
Cheap UK and US VPS
Torrent of their images, data, VRML, etc.
"[A] high IQ is like a Jeep; you will still get stuck, just farther from help!" --Just d' FAQs, c.g.a
The 'You are Here' arrow?
Sure makes for a nice project I just threw up a /26 for some hosts they're not included in the map, so aside from novelty what real purpose does this serve? I'm not trolling I'm just trying to look at this from a different perspective outside of 'yay look what I did'.
Side note to clarify those scared clueless crybabies who made a statement about the "magic" perl script I posted, please read on cluebie. You should check wtf your talking about the script does nothing more than what it just did scare luzers and makes for a nice honeypot. FYI the script is from Deception Tool Kit, if you dug around you would know this. Only line I added was at the bottom, which is nothing more than print
MoFscker
[grin] It sort of guesses wrong on purpose if it doesn't know - my theory is that people are more likely to correct it if it's wildly wrong than slightly wrong :-)
:-)
:-))
I'm working on the lat/long stuff this weekend, then there's a bunch of networks that can be automatically located. With this map of the net, I can start intelligently looking at IP's as well, rather than probing random ones that might not exist
Tx for the correction
Simon
Physicists get Hadrons!
What point??? While it may not pinpoint *your* location, it *does* show what provider you are connected to, and the exact location of that provider... If someone wants to find you badly enough, knowing where the provider is located is a good start... From there, the provider's records can be subpoenaed (or an employee can be bribed, etc) to determine where *you* connected to the provider.
If it is broadband, the provider knows exactly where you are. If dialup, the phone company knows where to find you...
"Also only took them 252.68 hours to complete" is also the amount of time before their slashdotted servers will be back up to make the file available for download!
(1st sig) If this were a snappy sig, you'd be reading it right now. (2nd sig) I'm a karma whore. >Insert FUD here
As a woman I am betting that most of you men would still refuse to stop and ask for directions, no matter how big the map is.
In any case, I think that's missing the original poster's point, who said it would be scary if your IP told people where to find you, not who your ISP is.
You are not alone. This is not normal. None of this is normal.
A "You Are Here" sign... Reminds me the HHGTTG and the cookie machine which drove the wife crazy.
4Z5TX
So when are one of us nerds going to invent a better way to tell what geographical location is associated with what IP/URL?"
Geo::IP [cpan.org] is per-country, or per-city if you pay for it and the city's in america.
Google did something using zip-codes it found on websites to identify a country. That's useful, because the location of your webserver has sod-all relevance to the location of anybody using it, whereas zip codes are the company address. Actually it wasn't google, but someone winning a google competition.
If someone wanted to use the WHOIS data, they all have zipcodes too, although you'd have to ask nanog'rs where to get the full list of domain-registrar addresses. (or ask spammers...)
There should be a UK post-code database somewhere, which has geographic regions. It's rather expensive though, especially given that the public paid for it to be compiled.
Even if all you can find is a city name on someone's website, you can compare that against a public-domain database of all cities in the world [xplanet.sourceforge.net], to get a location. If you can tell the difference between namesake towns, that is. I'm in London but not in Canada.
Someone had an extension to Geo::IP, where they were asking website visitors to volunteer their approximate locations, then using that to map end-users.
And of course, many US servers have their lat/long in a database somewhere for precisely this sort of mapping.
I'm working on a mirror of the pretty pictures. It's available at http://leela.lasthome.net/maps/.
Only on slashdot can a posting be rated "Score -1, Insightful".
I guess that black hole on the map is where OPTE used to be.
I'm having troubles understanding conceptually how this is possible?
Did they spider every registered domain? Is that list available?
Or did they just spider, like google? In that case, how do they know they didn't miss some?
Assuming they did, I'll be the party pooper.
*registers theonetheymissed.com*
HAHA! They do NOT have the entire internet indexed.
Clif
Blogzine.net
clifgriffin > blog
The large map is available here.
Douglas P. Price
Just an idea, maybe you could use Wikipedia's lists of locations, such as its list of Flemish municipalities instead of letting users choose them by themselves.
Also, how do you handle ISP's with dynamic IP's? Especially with dial-up, the same IP can be in totally different locations on different times.
This sig under construction. Please check back later.
The internet is 1-dimensional? They must be describing it metaphorically. Because it sure as hell has a geometry, and an N-dimensional geometry at that (where N is larger than 1, even if I don't know the exact value).
For example, I've experimenting with networks that have a regular geometry, where every router might have 6 links to other routers, arranged in a 3d grid type of geometry. In the logical sense, a router is certainly "to the right" of another, or "above" another. In such a network, it's easy to see that it has 3 dimensions. With the internet, the geometry is very irregular, even 'organic'.
All that said, should I Subject this post with "Fr0st t3chn1cal p0ts" ? Even an hour later, everything seems to be lame kiddy banter....
I don't really mind what people use, so long as it makes sense.
:-) which ought to cope with whatever people type in. If it's not recognised, the system will then try a soundex match on the name, and present a list, or ask for a new name. Only the recognised names will then get through (which will help with the dupes as well :-) This isn't ready yet, but it will be soon...
/29's over (eg) ADSL ...
I now have some 15000 or so place-names in Belgium (not in the public DB
As for dynamic IP's, well obviously I can't. I can flag up when people use an IP range for more than one city, and mark it as dynamic, but over in the UK, the infrastructure is ATM, and geography plays no part in even static
For the time being I just report back whatever was last input. I'll come up with something better once other problems are sorted out (mapping, peers, etc.)
Simon
Physicists get Hadrons!
You are HERE ----->(X)
If you don't understand anything I post, please accept that I ate paste as a small boy...
They use animated gifs to represent WiFi users.
heh, i have a comcast IP. i'm in NH and it says I am in Michigan.
/20 and i would do it ONCE for a large mushroom pizza.
As an aside, updating that physical location information is really easy. For instance, in north america, all our IPs are dished out by the American Registry for Internet Numbers (ARIN). ARIN wants any contiguous block of IPs larger than 7 to have the information filled in. if lying doesnt work (your admin figures out that you're just too far out of the way, like 2 states over) you could just bribe them. or flatter them, like "heh, wouldnt it be funny if you could update that info... hahaa too bad you cant. yeeep. cant do it. be a genius if you could though." i'm an admin for a
slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
The 1st time I encountered the internet was in the early '70s in a graduate level CS course at the University of Illinois. I remember the prof saying he had just come back from an early conference of net sites and everyone was excited because you could no longer draw the whole ARPAnet on a 3x5 index card, you now had to use an 8.5x11 piece of paper. Of course, even then the official map didn't have every site. There was a big grey box in one corner of the Center for Advanced Computation machine room that connected to the internet through U of I's router and reportedly went to some hush-hush military installation somewhere, but the map didn't show this connection at all. It was a real bulletproof router, though - made to military standards and looked like you could pound on it with a sledge as long as you wanted without causing it to drop a packet.
As opposed to mapping interconnected hosts, I wonder how it would look if we took internal networks (192.168./16, 10./8 172./8, etc..). Instead of mapping the Internet, it would be interesting how many computers are actually connected to the internet, even if by means of NAT, (transparent) proxy, interior routing, etc.. Although it's an impossible task to do without contributors, but I think it would be a very interesting visualization when combined with a map like this.
One of the earlier works appeared in Slashdot, for instance here in 1999. But neither that column nor this hits for me on a search for military despite the military implications.
Specifically, there was a paper about this work in the 2000 USENIX Annual Conference. It mentioned detecting a loss of network connectivity during the NATO bombing of Yugoslavia during the period of their study, something the military could use to monitor the efficiency of their campaign.
Mencken had it right. So glad that's old news.
I said with so much certainity mainly because earlier research into the topology of the Web, by Barabasi et al had already suggested that it is a so-called scale-free network (my post in the earlier story on this had more references), which, arguably, are fractals.
More than mere navel gazing.
Fragmentation is a problem, however, I am not seeing it as fargmented as I thought it could be. I would say about 99% of the ip's in /24's route to the same last two hops. Some isp's break it up a lot, but that's on their small part of their networks. When that happens it is relfected in the image with "less resolution" of the edges.
The scanning is done using random /16's that are checked out from the valid address space, they are scanned by 3 different nodes on different backones in different parts of the world. Soon they will be done from maybe about 6 different backbones.
Also, using the old term, "Ceci n'est pas une pipe" (this is not a pipe), the "map" is a working image of data, nothing more. It's art, math, whatever you want to call it, but I never say "this is the internet."
The best image I have now is stated as: "This graph is by far our most complex. It is using over 5 million edges and has an estimated 50 million hop count."
If you want to see the data for yourself and play with it, you should download Alex's LGL view .jar file and load the data. You can zoom in on every single route on any point and then turn on the ID (or IP of that point) run some traceroutes of your own and you will see that your traceroutes match the data in the images.
Anyway, back to my point: No image can be "the Internet", it is just an image. :) Depending on what philosophy you subscibe to.
This was a small project that grew, we are prducing neat maps and we are solving issues with them as the days go on. This is about 4 weeks of work and it's neat, but not anywhere near done.
I'll be looking forward to help for anyone. :)