Slashdot Mirror
Akamai -- The Other Huge Distributed System
Frisky070802 writes "Technology Review, the MIT alumni magazine, has an article by Simson Garfinkel that compares the huge distributed systems run by Google and Akamai and speculates that Google might even consider buying Akamai. It also discusses the flame-out of Akamai after its tremendous IPO."
I think there's plenty of room for both groups to be successful. One thing Google and Akamai have in common is their desire to hire extremely skilled people instead of making it up with large numbers of code monkeys.
;-)
I assume this is true, at least, because at some point each of these companies have hired a friend of mine.
"TV is great! Every New Year's I make a resolution to watch more TV." - Ann Coulter
Wow, those are strong words. Real hard news here. News for Trolls maybe.
Mike Bouma
MCSE, MCSDT, Microsoft Office Expert, Well Respected VB Scripting Genius
Mike Bouma
MCSE, MCSDT, Microsoft Office Guru, Well Respected VB Scripting Genius
According to this Google already outsources their DNS load balancing to Akamai.
Read Epic the first RPG novel.
Lets hope we don't Slashdot google. Anyone have a mirror?
Vote for new mod!!! Score:-2,Imbecile
Google & Akamai are similar in that they both use clusters of computers to do extremely high performance tasks. While there could be some great possibilities by combining the two, this is definitely not a "no brainer". Their models are different enough to make it difficult.
Akamai's business is distributing servers around the Internet, to maximize the efficiency of the web connections to them. They distribute the workload, and minimize the network distance needed for each person to connect. So, they need a large number of sites, each with a small number of servers (small relative to Google).
Google has a small number of sites, with a huge number of servers. Those servers are heavily dependent on one another. As mentioned in the article, they use Google's file system technology to aggregate to huge database. If that same structure was divided up into smaller chunks that were highly distributed, the back-end cluster performance would suffer because of the WAN links interconnecting them.
I'm sure Google will continue to grow, and create more data centers. But, they will need a different structure than Akamai uses.
You know, in the past few months, I've heard more about one company buying another company than I'd care to hear:
IBM will buy SCO
Apple will buy Real
Microsoft will buy everyone
And now this. Don't people realize there is more to 'buying' a company than ordering fries and a coke? Also, sometimes its advantages not to buy a company, but rather, to create a partnership, or even to just buy or license IPO.
The *other* way companies of similar persuasion exist at the same time, other than just eating each other, is to COMPETE.
That is the point of our economy. Rather than having large fish eat the small fish, and then be left with nothing but big fish and us (fish food), the big fish and the small fish should compete for our business by making their offerings more attractive.
They are in a minor supplier/customer relationship. Akamai does DNS load balancing for Google. There's something Akamai does for cheaper than Google can do themselves...
.
Akamai doesn't deal with end users ever.
Google has lots of smart people thinking about end user applications for distributed systems.
Akamai has lots of smart people thinking about business applications for distributed systems.
Akamai has a more widely distributed network
Google has a more centralized network.
They're probably of a comparable size.
Merging the networks would be brick-stupid.
Applying good ideas from each network to the other could be very advantageous.
Giving both groups of smart people a slightly different distributed system to work with might be very productive.
It'd be a good way for Google to grow it's headcount.
(Please, contradict me if I'm talking stupid. Happens all the time.)
There are no trails. There are no trees out here.
One of Akamai's hidden talents basically safely oversubscribe their systems because there's no way all of their customers can be at their peak resource usage at the same. Web usage is in part a zero-sum game... if thousands of people are running to their computer after being invited the same URL by a Super Bowl commercial, it's safe to assume that those thousands of people are not hitting CNN.com. Sure, some people not interested in the game might be at CNN's site, but they're not going to be part of the throng headed to the advertised site.
They don't really need to have enough systems so that every site can have its peak usage all at once. They just need to be able to absorb their market share of the entire World Wide Web activity at any given moment. They don't particularlly care which site you hit... they know that any spike at one is most likely going to come at the expense of other sites, and that they run a good chunk of those sites that are going to have the corrisponding decreases in traffic. They're basically assured that almost nobody downloads an iTunes song and watches a TechTV video clip at the same time.
I understand that we are not allowed to imagine beowulf clusters of these.
pity
...he repeated that figure of 1,000 queries per second--but he said that the measure was made at 2:00 a.m. on December 25, 2001. His point, obvious to everybody in the room, is that even by November 2002, Google was doing a lot more than 1,000 queries per second--just how many more, though, was anybody's guess.
What's obvious to me is that the metrics were taken at 2am on Christmas morning... not that they were taken a year earlier.
What cod piece?
...at least from a customer's prospective.
We're one of Akamai's larger customers.
We use them because the traffic patterns on our websites include 10x (and up) spikes in traffic during news and weather events.
These events are specifically times when we CANNOT be unavailable. We live and die by those events.
But, those events are not very often - perhaps a few per month.
Akamai allows us to serve this massive traffic spikes without requring us to maintain a massive overhead in servers and bandwidth that goes unused most of the time.
Each site in our network has a geographically localized audience, but across the network as a whole, we have users everwhere.
Edge Serving allows us to provide extremely low latency service to all of those users - and providing a much greater resistance to core internet connection issues.
Further, Akamai provides us with massive redundancy. A single (or group of few) datacenter, not matter how large and well designed, is still not as redundant as the Akamai network.
Finally, if our origins become unavailable for whatever reason, our sites live on, completely available on the edge (albeit, growing stale as time goes on) while we restore origin connectivity.
Then we have EdgeJava, Akamai Network Storage, the video serving, etc.
Our latest web project (which will become quite popular in mid-late August) will be served entirely from the Edge using Akamai.
No, I think you've missed the point of Akamai. Akamai is in the delivery business, but bandwidth is only part of that. Akamai delivers content, and is capable of delivering CPU cycles as well. You have also forgotten about the other last mile: the content provider's Internet connection. Most people don't go buying themselves a direct-backbone connection to put up their websites- that's left to their ISP or their ISP's ISP.
Take, for example, a website linked to in a Slashdot front-page article. The HDD cannon today seems to have been hosed pretty badly by the Slashdot Effect. First problem was that the provider's bandwidth was not nearly enough to serve what was apparently a graphics-heavy page (I don't know- I never even got to see it!). The second problem was that even if it had been a simple page, it still takes a fair amount of power to serve a large number of simultaneous requests.
Had that web site operator used Akamai's services, the Slashdot Effect might not have been able to make the content unavailable. Instead of one last mile to the provider being clogged, the traffic is distributed among all of Akamai's "last miles". At the same time, no one server has to cope with answering all those requests in a timely manner.
Google can get away with a few datacenters full of servers. The bandwidth to any one Google datacenter can probably be planned for and new pipes provisioned pretty readily as they grow and expand services. Akamai is there for other uses- for example, hosting video streams of immensely popular but short-shelf-lifed sporting events. If the sanctioning body for a sport invested in enough infrastructure to provide it themselves, it would be underutilized out of season. If Akamai does it, they can host video streams of the baseball World Series for MLB, then the Superbowl for the NFL, then March Madness for the NCAA, and those organizations don't have servers sitting around twiddling virtual thumbs in the off season.
Little Debian: America's #1 Snack Distro!
Assuming those numbers are correct, and assuming they use several year old algorithms:
Google can break an RSA-512 key. 12 times a day.
It would take them 8 months to break an RSA-1024 key.
Of course this glosses over some of the technical difficulties (such as memory bandwidth, RAM, etc) but the interesting thing is that if they directed their gaze towards a problem of for even an hour, they could solve some truly monumental problems.
But, according to Slashdot, Google is good today, not evil, so we can expect them not to use their power for bad.
-Adam
Thanks to archive.org, you too can join in on the caching fun! If you want to post a web page's URL to Slashdot without having it, um, Slashdotted, you could use Freecache. If you run a major ISP or university IT department, Freecache could use you.