Slashdot Mirror
Google and OpenDNS Work On Global Internet Speedup
Many users have written in with news of Google and OpenDNS working together on The Global Internet Speedup Initiative. They've reworked their DNS servers so that they forward the first three octets of your IP address to the target web service. The service then uses your geolocation data to make sure that the resource you’ve requested is delivered by a local cache. From the article: "In the case of Google and other big CDNs, there can be dozens of these local caches all around the world, and using a local cache can improve latency and throughput by a huge margin. If you have a 10 or 20Mbps connection, and yet a download is crawling along at just a few hundred kilobytes, this is generally because you are downloading from an international source (downloading software or drivers from a Taiwanese site is a good example). Using a local cache reduces the strain on international connections, but it also makes better use of national networks which are both lower-latency and higher-capacity."
Why does the GNAA website have a .eu domain name?
I never get my downloads that slow when downloading from Taiwan...
Oh ya, that's right, I am in Taiwan...
Akamai has been doing this the proper way for years now.
I prefer this is implemented by the content provider, not by the ISP
How is this different from P2P ?
-- It is the mark of an educated mind to be able to entertain a thought without accepting it. -- Aristotle
Looks like someone forgot to check the little box marked "Post Anonymously."
> The service then uses your geolocation data to make sure that the resource you’ve requested is delivered by a local cache
This will make censorship much easier. No more corrupt foreign data in [your favorite oppressive country].
lucm, indeed.
If you had read it, you would understand that his post is the second step in applying for GNAA membership.
Doesn't say anything about posting it under a registered username.
Isn't this little more than an expensive band-aid for the underlying bandwidth problem? Delivering content from strategically located caches is an OLD concept, and it's always been trouble-prone, with some sites not receiving updated content in a timely manner and others getting corrupted.
Quite frankly, I wish some of the big players with vested commercial interests in a good-performing internet (like Google, Amazon, or Microsoft) would pitch in on some investment funding to upgrade the infrastructure itself. I know Google has experimented with it on a small scale, running fiber to the door in a few U.S. cities. But I'm talking about thinking MUCH bigger. Fund a non-profit initiative that installs trans-Atlantic cables and maintains them, perhaps? If a nation wants to censor/control things, perhaps they'd reject such a thing coming to their country, but that's ok.... their loss. Done properly, I can see it guaranteeing a more open and accessible internet for all the participants (since presumably, use of such circuits, funded by a non-profit, would include stipulations that the connections would NOT get shut off or tampered with by government).
A network where only big players can afford fast delivery is not neutral. CDNs starve the actual network in favor of local caches. Money that would have to go to bandwidth improvements now goes to the CDNs, which in turn are only used by global players. This leaves us with an anemic network that can deliver Youtube clips quickly but chokes on broadband communication between individuals on different continents. And if that weren't bad enough, they abuse DNS to do it.
If I type google.com into my browser, I actually want to get an english web page from a computer in the USA, not a redirect to a page in the language of my country, and not a page served by a computer in my country either.
Sorry, I may just have woke up on the wrong side of the bed this morning.. but wouldn't doing as TFA suggests just open the door to another MITM attack method?
Are YOU using the TOOL, or is the TOOL using YOU? Think about it!
"downloading [drivers] from an international source"
So that's what you kids are calling it these days. Heh.
What I like about this entire thing is that we can save on bandwidth, which should also lead to some power savings overall, not a lot, but just another drop, those drops could add up and become something useful in the future.
if you see me, smile and say hello.
I realize that IPv4 is going to be with us for quite some time, but is this going to be worth the effort? It requires a bit of jiggery-pokery to repoint your DNS, the kind of thing that appeals to the Slashdot crowd but which your grandma will never, ever pull off. ISPs could help, but will they do so before IPv6 makes it irrelevant?
Description seems a little simplified. Sounds like an Akami presentation from over a decade ago.
So is this a commercial competitor to Akami, or a non-commercial competitor, or a freeware / public competitor, or is it something somewhat different, like a squid proxy set up for transparent caching from 2002 or so?
Speaking of squid, its 2011, is squid ever gonna support ipv6? There's not much software out there that doesn't support v6, and squid is probably the most famous.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
When I open an HTTP connection to some web service, they should already have 'my' IP address
By the time you make an HTTP connection, you've already chosen which mirror of the web service to use. According to the article, this spec would allow DNS servers, such as an ISP's DNS server resolving on behalf of the ISP's customers, to use the prefix of each user to determine which mirror to recommend. It's like a network-topology-aware version of round-robin DNS.
If a nation wants to censor/control things, perhaps they'd reject such a thing coming to their country, but that's ok.... their loss.
You mean like, EVERY FUCKING NATION ON THE PLANET?
Quite frankly, I wish some of the big players with vested commercial interests in a good-performing internet (like Google, Amazon, or Microsoft) would pitch in on some investment funding to upgrade the infrastructure itself.
You'd have several hundred lawsuits from several dozen companies that have a vested interest in keeping control over the existing infrastructure. You'd have antitrust investigations being called for, contract lawsuits against the cities that promised them monopoly access, and several billion dollars poured into lobbying to make sure that it cannot and will not happen.
And keep in mind, on the low tiers the vast majority of laid fiber is dark, just waiting for someone to actually plug into it and use it. And I'd be shocked if that didn't include the transatlantic lines as well. On the higher tiers, many (most) cities don't have proper conduit installed, which means installing fiber to the front door is going to be an expensive, messy, time consuming process even if they got through the legal and bureaucratic nightmare.
Getting more bandwidth to more places is great.
Using the bandwidth you have efficiently is not a bad thing.
Not sure how efficient use of resources is a bad thing.
Why is it so hard to only have politicians for a few years, then have them go away?
I'm all for this if and only if all protocols are fully complied with.
HTTP gives plenty of leeway and in fact was designed with caching in mind. So long as the involved parties do not violate the protocol, I'm ok with it. Cache control directives must be honored, for example. No silent injection of random crap.
The DNS protocol must also be honored to the extent that deviations from same have not been expressly authorized. OpenDNS offers typo correction and filtering services on an opt-in basis. NXDOMAIN hijacking and whatnot is foul play.
Just don't fuck with the protocols, and you can do whatever you want.
How about we get rid of all the linkages to crap sites like facebook, digg, mysapce, etc. These are the links that are almost always responsible for slow page loads for me. I rarely have any issue at all with downloads or videos, local or international. Interesting too when using noscript and some webpages literally twitch when something like facebook is blocked as they try over and over to reload and reconnect. And as others have said, these local caches are just another security risk waiting to be exploited.
Google could just not provide a service that inserts themselves into the DNS path. The problem isn't "the internet" or DNS, it's that Google's DNS servers have no relationship to the client systems. If people were using DNS servers that had some relationship to their network -- such as the one provided by their IPS -- then this wouldn't be an issue.
Plus not using Google's DNS gives you a little more privacy. Privacy of course being defined as not having every activity you do on the internet being logged by one of Google's many methods of invading your space (DNS, analytics, search, advertising, blogger, etc.)
And I'd be shocked if that didn't include the transatlantic lines as well.
They're all lit all the time. Maybe just by second string "if we lose a strand, your strand becomes their protection ckt" but they'll be lit. I used to be in the telecom business.
Where you find dark fiber is on shorter local hops where there simply isn't the current demand, at any reasonable cost.
You'll never completely satisfy demand between stateside US-HI. You can supersaturate demand to the point of dark fiber between little-city and cow-village.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
While this is a commendable effort, the biggest offender when it comes to horrible load times is the embedded advertising content. Most of these ad providers -- I'm looking at you Google -- deliver their content at a snails pace, delaying the delivery of the actual content.
Case in point, I used chrome's network developers tool to analyze load time for the various elements on http://www.slashdot.org
The top 5 longest durations go to *.doubleclick.net and range from 242 to 439ms, with the total page load time at a little under 4 seconds.
Keep in mind that Google, Amazon, Akamai, etc. had already created geographically distributed networks to reduce latency and bandwidth. Improving the accuracy of geolocated DNS responses through a protocol extension is basically free and makes these techniques even more effective.
Also, Google cares a lot about latency. A major component of that is backbone transit latency, and once you have enough bandwidth to avoid excessive queueing delay or packet loss, I can imagine only four ways to significantly it: invent faster-than-light communications, find a material with a lower refractive index than the optical fibers in use today, wait for fewer round trips, or reduce the distance travelled per trip. This helps with the last. Building more fiber wouldn't help with any of those and would also be a lot more expensive.
Full disclosure: I work for Google (but not on this).
Happy to answer questions about this work.
# Hack the planet, it's important.
Messing with DNS is doing it the Wrong Way. All of these CDN services are based on HTTP. When you're using them, that's an HTTP server you're talking to. It's perfectly capable of geolocating you by IP, and it can either hand you back links to a local CDN, or redirect you to another server.
Why the hell must we mess with DNS to do this? This is a solution which only works if you use Google DNS, OpenDNS, or sometimes if you use your local ISP's DNS. What if you're just running bind for you local net vs the root servers? Bzzt. Doesn't work.
The most insane thing about this is it's Google we're talking about here. They damn well know how to implement this entirely in their servers without resorting to DNS hacks. Why are they promoting this net-neutrality breaking, layering violating botch? We need less people to use this, not more.
Maybe it's about time the Windows operating system was given the ability to cache DNS queries locally BY DEFAULT. It would speed up searches for most used sites by the user, and take the load of the internet from running requests for sites the user keep on visiting.
Take Nobody's Word For It.
If you went out of your way to Disable Client-Side DNS Caching, that's hardly something you can blame Windows for...
Just to reuse the title from above, for more effect. I am guessing Akami owns the easy and obvious ideas in pretty tight patents. That is one of the reasons they have limited competitors.
So...the summary states "they've reworked their DNS servers so that they forward the first three octets of your IP address to the target web service". Uh, doesn't my browser send my WHOLE ip address to the web service when I make a HTTP request anyway? How is this different/better?
/24 requests www.itunesdownload.akamai.net, then that specific IP should be cached for all requests from the same /24 for the TTL specified by the site operator, but for all other sites that will probably not have been hit recently, and thus not cached, I only see this adding more latency.
If what they meant to say is that the resolver sends the first three octets of my ip address to the destination's name servers when doing a recursive lookup, then how is this any better than using any old DNS? In other words, the big advantage to Google DNS for example is that it's free and fast. If their DNS server now has to ignore any cached records and do a recursive lookup for EVERY request, doesn't that negate the speed advantage? Obviously once someone in my
They should just do it in the web stack instead of trying to do it down in layer 4. User navigates to the download page, then the webserver has your full IP, geolocates you and redirects you to a download from a local server.
If you have a 10 or 20Mbps connection, and yet a download is crawling along at just a few hundred kilobytes
I would love to have a connection that "crawls along at just a few hundred kilobytes (a second)", most of the times, when it crawls, it does so at a few tens of kilobytes a second (sometimes even less than that).
>(downloading software or drivers from a Taiwanese site is a good example)
not if you live in Taiwan
For Akamai.
Having to work for a living is the root of all evil.
Evil and OpenDNS Work On Global Internet Speedup.
I think from now on simply replacing the word Google with Evil should be an auto-correct feature.
Even if your first connection is to a server in Djibouti, you may be redirected
Which costs a TCP setup and teardown to Djibouti.
to a server in Canada, and then that one may again redirect you to a server in Sweden
Which costs a TCP setup and teardown to Canada.
There are two factors that affect the performance of web (HTTP) lookups: latency and bandwidth. Latency depends on the distance between client and server. You won't be able to send data faster than the speed of light. Bringing the data closer to the client helps to reduce latency, especially for small lookups. Bandwidth becomes the limiting factor when you transfer (large amounts of) data over under-dimensioned pipes. In general, I'd be a much more happy person if people would use HTTP caching headers (Expires and such) more often, as then a Squid proxy can bring substantial performance gains.
No, they're trying to turn it into cable TV. One way delivery, only "approved" content providers get distributed, everything that's not on the official local CNS gets throttled way down. Easier censorship is just a side benefit
An Internet speedup would involve adding the ability to carry more bytes per second, analogous to changing your delivery vehicles from donkey carts to vans. This is just improving the logistics of the donkey cart-based delivery service.
"When information is power, privacy is freedom" - Jah-Wren Ryel
Comment removed based on user account deletion
What we need is less junk in web pages. The amount of dreck in web pages has gotten completely out of control. The home page of Slashdot has 3424 lines of HTML, not counting included files. This is not unusual. I'm seeing pages over 4000 lines long for newspaper stories. Pages with three to five different tracking systems. Hidden content for popups that's sent even when the popup isn't being loaded. Loading of ten files of CSS and Javascript for a routine page.
CSS was supposed to make web pages smaller, but instead, it's made them much, much bigger. Being able to include CSS from common files was suppose to improve caching, but instead, many content management systems create unique files of CSS for each page.
And you get to pay for downloading all this junk to your smartphone. It doesn't matter what route it takes through the backbone; eventually all that junk crosses the pay-per-bit bandwidth-throttled air link to the phone.
Where bandwidth really matters is video. There, the video player already negotiates the stream setup. That's the place to handle choosing which source to use. Not DNS.
I was preparing to hate this (I don't trust Google that much & OpenDNS do some questionable things), but I can't find anything wrong with this from a privacy or openness perspective. I think there has already been huge DNS pools that return a random set of records, so the idea of DNS being universal and reproducible is gone, if that idea ever existed (and I see no reason why that would be a problem).
Isn't this what anycast is supposed to solve? For example when using 6to4, one can specify a single, globally known address, but it takes you to a local 6to4 gateway (if it happens to work at a given ISP..). Would anycast support commercial entities setting up a CDNs with a few anycast addresses, that route to the nearest server farm? I don't know if I prefer this, since it requires putting some "intelligence" in the routers. Going by the adoption rate of IPv6, this wouldn't be supported by ISP before 2030 anyway...
Sorry for researching this, but if anycast is better supported in IPv6, then that could be why the explanation page lacks any mention of IPv6. Sites would have to supply a few normal addresses in addition to the anycast address in DNS for redundancy, but browsers could try the anycast one first for speed. I think this would be a good solution, but I don't know if it's better than Google and OpenDNS's suggestion.
So which one is it? Does google have a separate cache for each /16? I'm interested to know how they deal with this. I guess it's not a dealbreaker, but all options involve some trade-offs.
I don't think that's true at all, many transoceanic links have dark pairs. I know the Google-cable was going to be only half lit at installation. The existence of dark fiber on transoceanic links is driven by many of the same economics as dark fiber on land, only magnified since the cables are so much more expensive, the installation makes trenching look cheap, and the lead times are measured in quarters instead of weeks.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
w/ large HOSTS files (Windows LOCAL DNS clientside cache service).
YES - Windows LOCAL DNS CLIENTSIDE CACHE SERVICE "chokes" with larger HOSTS files & lags you. I noted this to MS' Senior mgt. in their "Windows Client Performance Division" in fact (Foredecker/Mr. Richard Russell who used to post here quite a lot in fact) here on this very website years ago:
http://slashdot.org/comments.pl?sid=1467692&cid=30384918
Along with the fact that a SMALLER & FASTER/MORE EFFICIENT BLOCKING "IP ADDRESS" CAN STILL BE USED IN Windows 2000/XP/Server 2003!
(Where it could be used in VISTA (& onwards in Win7/Srv2k8) up until MS "Patch Tuesday" on 12/09/2008)
That FASTER/BETTER one, is 0 - Used as the blocking address (vs. the largest/slowest in the 127.0.0.1 loopback adapter address, OR the slightly better & just as compatible 0.0.0.0 blocking address (no loopback hit is incurred there either mind you)).
He never got back to me, though he said he would here, AND in email... I didn't like THAT very much!
Still It's nice to see that GOOGLE & OPENDNS are doing this though! (both of whom provide DNS services, OpenDNS also filters vs. phishing/spamming & possibly malware (Norton DNS does for sure on the latter though, so I use it alongside OpenDNS + ScrubIT DNS in my routers &/or IP stack settings)).
APK
P.S.=> HOWEVER/"Bottom-Line" here, is this: Nothing GOOGLE &/or OpenDNS can do can resolve IPAddress-to-HOSTS/DOMAIN names, as fast as locally hardcoded favorites in the HOSTS file either... that's not all I use it for, but mostly for blocking out 1,580,313++ KNOWN bad sites/servers/hosts-domains, botnet C&C servers, sites KNOWN to serve up malicious script content or malware-in-general (for security) AND, adbanners (For more bandwidth/speed that I pay for) which also lends not only to speed online, but also security (since adbanners HAVE been found to house maliciously scripted content as well)...
... apk
Isn't this little more than an expensive band-aid for the underlying bandwidth problem?
Not really. There is always a finite quantity of bandwidth. It only becomes a "problem" when you have applications which assume infinite bandwidth or are forced to assume this for legal or political rather than technical reasons.
Like, oh let's just say for example, streaming video.
Streaming is the anti-caching. It's a terrible technical non-solution to a legal problem. It clogs the tubes and wastes bandwith by design just to retain control over the obsolete idea of "broadcasting" so that copyright control and advertisements can be retrofitted into the stream.
But doing video right would require re-engineering our entire economy -- which will have to happen sooner or later when the IP crash comes -- so we'd rather just break the Internet by design and then attempt to retrofit some kind of weird fixup after the fact to make some preferred partners work sort-kinda okay.
You are not a brain: http://books.google.com/books?id=2oV61CeDx-YC
Why would they use this rather involved mechanism rather than anycast IP addressing? TFAs don't go into why they've gone to the effort of reimplementing something which essentially already exists.
Sorry, but you have no clue what this is about. This has very little to do with bandwidth and much more to do with latency.
Say you live in Germany.
From California to Germany you're talking about 150ms minimum. If you're working with an interactive site like Gmail or Google maps the experience is going to suck. Say it takes the server 100ms to do the work to respond to your request. That's already faster than most website servers are designed to respond to. This means that 250ms is the time it takes to get, process, and respond to a single request. This means you can only really do things at a rate of 4 updates per second. This is very slow for highly interactive web apps.
If a site can redirect you to a server in europe you can cut 125ms off that request. That's half of the time. It's also below the 190ms it takes your brain to respond to visual changes.
http://en.wikipedia.org/wiki/Mental_chronometry
I'm not the idiot who designed the local DNS clientcache service in Windows that "flakes out" when it encounters a largish HOSTS file (this problem does NOT occur in Linux, mind you, either).
---
Now, as to your off-topic adhominem attack, cowardly little ac troll? Well, time for an application of "ReVeRsE-PsyChoLoGy":
".toidi gnikcuf a er'uoy esuaceb s'tahT" - by Anonymous Coward ANOTHER "ne'er-do-well" /. COWARLY OFF-TOPIC ADHOMINEM ATTACK UTILIZING TROLL on Tuesday August 30, @07:31PM (#37259622)
"???"
Uhm... Could we get a translation of that off-topic "troll-speak/trolllanguage" of yours, please?
---
* And, you're an off-topic troll - no questions asked...SEE MY SUBJECT LINE ABOVE!
APK
P.S.=> Yes, it must have just have been another off-topic done nothing of significance with his life troll spewing his off-topic b.s. again & not contributing to the ongoing conversations. Oh well - No biggie!
(My "ReVeRsE-PsYcHoLoGy" template, below, for trolls - Courtesy of this code template of mine, by "yours truly", generating YOUR response in less than 1 second flat):
---
#TrollTalkComReversePsychologyKiller.py (Ver #2 by APK)
def reverse(s):
try:
trollstring = ""
for apksays in s:
trollstring = apksays + trollstring
except:
print("error/abend in reverse function")
return trollstring
s = ""
print reverse(s)
try:
s = "Insert whatever 'trollspeak/trolllanguage' gibberish occurs here..."
s = reverse(s)
print(s)
except Exception as e:
print(e)
---
... apk
You forget the reason why these services popped up to begin with. In general often your ISP can't be trusted. There are ISPs out there who run DNS servers which break some of the fundamental functionality of DNS used by various applications, such as the ability to honestly say "no server found" rather than directing you to a search page.
Then there's ISPs who are too incompetent to run a DNS server providing an undersized box which can't cope with basic traffic. It's sad that in some cases Google's DNS servers actually respond faster than ones located in your own city.
I personally use a 'Local Cashe' of all of the music, and videos, that I may wish to enjoy. I have found that this does indeed resolve any bandwith issues that are associated with repeatedly streaming content.
I would personally like to see how 'Their Plan to Speed up the Internet' avoids upsetting the RIAA...
How are the first three bytes of 2001:470:a:6bb:21f:29ff:fe87:88c0 going to tell them anything about my location?
-- I have a private email server in my basement.
OR, did he not state that a larger file (in this case, a HOSTS file) would take longer to read?
APK
P.S.=> Do I have to quote he saying that? No, don't think so - not yet @ least...
HOWEVER, just for laughs: Well - Let's let a little TRULY anonymous COWARD see what he has to say in rebuttal to that point I noted above, & then, I will quote he in that much... making YOU out, just "too, Too, TOO EASILY" as per your trollish cowardly usual, to be the FOOL once again (as always on YOUR part)...
... apk
I remember going to the global crossing data centre in chile and seeing that google had (apparently) purchased 1/4 of the deep sea fibre around south america. But i don't hink they are thinkimg altruistically when they purchased it, and that was 4 years ago
This seems pointless.
1. A smart ISP can use a caching proxy like squid. This speeds up content for anyone.
2. The webserver knows where you connect from anyway, there is no need to tell it.
(It can get your ip address from the socket, it knows for how else would it be
able to send the page back to you?)
So it can redirect you, if it knows that a closer server has the same content.
No need for this newfangled protocol trickery.
Full disclosure: I work for CDNetworks and have been part of this initiative.
As rightly pointed out, content delivery networks already have a widely distributed platform and use intelligent algorithms to route user traffic to the "closest" CDN server. These algorithms use geo-location whenever possible. But in some cases, it is hard to figure out the location of the end user. This initiative aims to address that.
To get an understanding of the inner workings of the technology, you can read my blog here: http://www.cdnetworks.com/blog/global-internet-speedup-initiative-a-look-under-the-hood/
Also, this is doesn't cost anyone, anything. It is just a more transparent communication between the CDN provider like CDNetworks and DNS provider like Google which enhances the user experience.
Arijit
I actually tried to read it out of curiosity. The links are dead.
It's official: according to Netcraft, GNAA is dead. Or at least 404ing in their cut and pasted diatribes. Even the new ones.
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
Foredecker had to admit apk's right that a smaller faster hosts file results using 0 over 0.0.0.0, bigger/slower, or 127.0.0.1 worst of all is slowest/worst and apk's right that it ought to be put back into vista and windows server 7/2k8 too because no reason is offered for it not to be.