Slashdot Mirror

one of the passengers on one of the planes that came down on 9/11 (it was the one that crashed in the field, IIRC) was a founder of Akamai Networks, one of the load sharing/distribution companies that allow bandwidth to scale according to demand. As his plane came down, his company was entering one of the most demanding days in its history, as more people were targeting news sites at once than ever before.

It's organisations like that which will assist in the next big news item.

-- james

In case anyone in the Washington area missed the story, the author spent several minutes this morning discussing the highlights of his article -- omitting any semblance of fairness -- on a local cable television news channel ("News Channel 8"). Here's the RealVideo file.

Segal: The world of pirated online music is still alive--
Channel 8 Talking Head: Like huge!
Segal: Yeah, huge.

The Washington Post has an extensive content deal with NBC (General Electric) and Microsoft, but is owned primarily by the Graham family and Warren Buffet.

You do, of course, realize what Akami is... they do not, themselves, serve up any of that content to you. They are a hosting outsourcing and web caching service. Check it out.

Would you refuse to go into a grocery store, just because it was physically located right next to a telemarketing office?

Distributed network of proxy servers specifically designed to withstand (D)DoS. FBI own servers use it, and now it looks like Akamai will be getting even more business. Good job, kids.

"The NOCC's proactive stance and unique view ensures a seamless response to network conditions - enabling Akamai to deliver rich Web content, regardless of network traffic volume ."

you probably are being transpaently proxied. a few years ago apple made a deal with Akamai to proxy all there content, thats why an apple download will max my DSL, my ISP has there own set of akamai servers :D

What happens is that the air is broken down TO PLASMA between the wires so that it conducts electricity, just like lightning 8-D. The spark then convects upwards due to the very hot air. After it's shot off, air is broken down at the bottom again, and another spark is started.

The best photos are probably HERE, but they're yellow sparks (i think that's to do with the gas) which isn't in my opinion as cool as brilliant blue ones :). TechTV also has a page on it and a cool-ish video if you can view asx files. Their JL is pretty weak though, because it stops before the spark "falls off" the end - meaning the wires are too far apart for the voltage to be that small to be able to turn the air in between into plasma.

MS: 'Everybody raise their hands if you trust your data on this man's server'

Imagine being able to pull up 2001: A Space Odyssey at 2:38 a.m...

...so, to wrap it up, my review of 2001 the year is the same as my review of 2001: A Space Odyssey; it went on too long, it was hard to follow, and you could only enjoy it if you were really, really, really really stoned.

The whole point of broadband is _not_ to download websites faster.

This is true, but the whole economics of broadband relies on it being used to download websites faster. Websites, E-Mail and NNTP.

The $52/month you're forking is chump change compaired to what the ISPs are paying for that bandwidth.

It's time some of you had a little bit of insight on how they can give you that bandwidth and still make money.

NNTP and E-Mail are easy, you're not actually using the ISP's bandwidth to download these. You're only using the link to your ISP. We all know local network speed is cheap. You're using their local network for these protocols (As long as you are using their news/mail servers). The web part of your bandwidth usage is a little bit harder to handle, but not much. One of the biggest helpers, and a company that has made high speed access for what we are paying possible, is Akamai. They've given some nice 1U rack mount content servers to almost every ISP out there already. Even small local ISPs will have some 1U servers in their server room. Symantec, (ping liveupdate.symantec.com , you'll see it's probably one of your ISP's IP addresses) Best Buy, Washington Post, Trend Micro and Barnes and Noble are just a few examples of their customers, and sites that will be using mostly your ISP's local network. (Taken from Akamai's site.

The next step to save even more of your traffic from hitting the ISP's big fat expensive pipe is caching servers. Inktomi and Compaq teamed up to give a nice setup. It's expensive (Somewhere in the neighbourhood of $75,000) but you save that in bandwidth pretty quickly. It's going to cache any semi to frequently viewed pages, and alot of the streamed media you watch. Five hits to ESPN.com just becomes one, and four local requests. The sites you hit and the files you download that are not cached or served from the content servers are a small matter, because a good setup and alot of "normal" internet users will actually be hitting the local servers about 80% of the time.

But you cannot cache P2P traffic, you cannot cache internet gaming traffic, and you cannot cache incoming traffic (Hence why that $52 is not enough for you to be serving up content.) The things you cannot cache are the things that will run an ISP out of business. Everyone here has heard the price of T1s in previous broadband articles or has priced them out themselves. Most realize the economics don't work. This is the only way to make it work. Traffic on alot of protocols just isn't cost effective to be given at high speeds.

I hope this clears up alot of peoples views on broadband access and how it can all work. I can't say if it's right or not, it's just the way things right now have to work. The only thing I can say is if you want a fast connection that you can use for Internet Access and not a fast connection to your ISP, you are going have to pay for it, and it's going to be alot more then $52/month.

The whole point of broadband is _not_ to download websites faster.

This is true, but the whole economics of broadband relies on it being used to download websites faster. Websites, E-Mail and NNTP.

The $52/month you're forking is chump change compaired to what the ISPs are paying for that bandwidth.

It's time some of you had a little bit of insight on how they can give you that bandwidth and still make money.

NNTP and E-Mail are easy, you're not actually using the ISP's bandwidth to download these. You're only using the link to your ISP. We all know local network speed is cheap. You're using their local network for these protocols (As long as you are using their news/mail servers). The web part of your bandwidth usage is a little bit harder to handle, but not much. One of the biggest helpers, and a company that has made high speed access for what we are paying possible, is Akamai. They've given some nice 1U rack mount content servers to almost every ISP out there already. Even small local ISPs will have some 1U servers in their server room. Symantec, (ping liveupdate.symantec.com , you'll see it's probably one of your ISP's IP addresses) Best Buy, Washington Post, Trend Micro and Barnes and Noble are just a few examples of their customers, and sites that will be using mostly your ISP's local network. (Taken from Akamai's site.

The next step to save even more of your traffic from hitting the ISP's big fat expensive pipe is caching servers. Inktomi and Compaq teamed up to give a nice setup. It's expensive (Somewhere in the neighbourhood of $75,000) but you save that in bandwidth pretty quickly. It's going to cache any semi to frequently viewed pages, and alot of the streamed media you watch. Five hits to ESPN.com just becomes one, and four local requests. The sites you hit and the files you download that are not cached or served from the content servers are a small matter, because a good setup and alot of "normal" internet users will actually be hitting the local servers about 80% of the time.

But you cannot cache P2P traffic, you cannot cache internet gaming traffic, and you cannot cache incoming traffic (Hence why that $52 is not enough for you to be serving up content.) The things you cannot cache are the things that will run an ISP out of business. Everyone here has heard the price of T1s in previous broadband articles or has priced them out themselves. Most realize the economics don't work. This is the only way to make it work. Traffic on alot of protocols just isn't cost effective to be given at high speeds.

I hope this clears up alot of peoples views on broadband access and how it can all work. I can't say if it's right or not, it's just the way things right now have to work. The only thing I can say is if you want a fast connection that you can use for Internet Access and not a fast connection to your ISP, you are going have to pay for it, and it's going to be alot more then $52/month.

Actually, Ximian RedCarpet does update your whole system too and shitload of more, for example, provides you with all demogames from Loki (and yes, it also does provide all those redhat updates as rhn (allthou rhn might provide content earlier than pushing the stuff to public ftp's))

And yes, Ximian does develop "truly useful and unique linux app". Its called Evolution. Ofcourse its matter of taste but to me, there's no real comparison in *gui* emailclients field. There's only evolution.

Ive now read some of the most moderated posts and they all claim this is a good thing because ximian pays for bandwidth. While this most likely true, i want to question this a bit. I use redcarpet allmost daily, but as "educated" user, i use mirrors. No, i urge you to go into and look those channel urls.

Lets start with Debian Potato channel..

http://http.us.debian.org/debian/dists/potato/ma in /binary-i386 (thats hardcoded path in channels.xml)

Now tell me how ximian pays bandwidth for that mirror ?

Most of the other channels are then again located in strict list of mirrors that provide ximian's one filestructure. Lists contains sites like sunsite.dk, rpmfind sites, few lug sites and universities. It kinda strikes me odd that these sites that serve free (and linux) software for free are new being used for commercial purposes.

We dont want to forget either that atleast in the past, ximian has used akamai network (atleast when i first installed ximian gnome i was so amazed how frigin fast it installed straight out of network and then i did traceroute to the install server which was akamai host and hell, *only 2 hops* from my laptop. Data was coming from machines that where located in my work's datacenter.)

So lets summarize this scheme a bit. Ximian is providing a tool called redcarpet and ximian generated xml metadata for redcarpet and providing content from basicly, publicly and freely acessessible ftp sites. And they want me to pay 10 us dollars for that ? Or are they now starting to use akamai for also redcarpet channels which then again, really shouldnt cost that much because transfers are onetime only (im not familiar with akamai's prices)

In the end, hell no, im going to pay 10 us dollars for eyecandy ftp browser even thou i use it quite a lot. It will be a lot easier for me to remove redcarpat and install autoupdate (which has no gui but its really great tool for keeping anything rpm based up to date, hell, i have even setup my own ftpsite for my company's servers and they poll stuff nightly from one place, thus i can verify all upgrades then then put them on one place and everythin is updated next morning)

My conclution is that ximian will not succeed with this mind of thing. Even thou i really really really do want them to do just that. I use ximian as my desktop and i wouldnt do any real work if there where no Evolution. I hope you guys best and really do hope you manage but personally, im a bit pessimistic.

You mean like what Akamai does? They even do it seamlessly. Of course, they're not free, and they don't do everything you want, but I'd say it's a good step, if I understand you correctly. (No, I have no relationship with them)

• Every single traditional news outlet basically failed yesterday, and if it weren't for mirrors of such sites' data, we wouldn't understand the events of 9-11 as clearly.

• Every single traditional news outlet basically failed yesterday, and if it weren't for mirrors of such sites' data, we wouldn't understand the events of 9-11 as clearly.

Daniel Levin, the CTO and a founder of Akamai, was on board the American Airlines Boston to LA flight that crashed into the WTC towers today. I have only met him once, but I was impressed with his inteligence, poise and personality. He will be sorely missed.

He was 31.

More details can be found here

Here is the press release from Akamai itself.

Caryn Brownell
Media Relations
Akamai Technologies
617-444-4661
cbrownell@akamai.com

AKAMAI TECHNOLOGIES MOURNS THE LOSS OF CO-FOUNDER AND CTO DANIEL LEWIN

Cambridge, MA, September 11, 2001 - With great sadness, Akamai Technologies, Inc. today announced the passing of Daniel C. Lewin, co-founder, chief technology officer and board member of the Company. American Airlines confirmed that he was on board the Boston to Los Angeles flight that crashed in New York City today. Danny was 31 years old and is survived by his wife and two sons.

George H. Conrades, chairman and chief executive officer of Akamai said, "Danny was a wonderful human being. He will be deeply missed by his many friends at Akamai. Our thoughts and prayers are with Danny's family, friends and colleagues during this time of national tragedy and personal loss."

The rumor about Akamai's co-founder appears to have been true - here is a link to their press release on the subject. Akamai Press release

I can't connect to that site, but the news is on Akamai's main page.

Real Audio Clip http://mfile.akamai.com/920/rm/thepost.download.ak amai.com/920/msnbc/ms091101-2v.rpm Not very good quality, no sound, but its something and its up.

That one didn't seem to work -- slashdotted, maybe? -- but a simplified one does:

http://twimedia.download.akamai.com/2611/2001/06/1 8/0000522955.rm, which seems to go straight to the Akamaized content, rather than through the Akamai caching system.

--

Some spelunking through the mess of javascript turned up this:
http://mfile.akamai.com/2611/rm/twimedia.download. akamai.com/2611/2001/06/18/0000522955.rm
It plays at ~100kbit for me.

In some cases, it is not their fault. One of our customers (a bookstore with several stores) ordered a T1 line to their "corporate headquarters". When our installer arrived, it was to a warehouse. The job of "network administrator" was a secondary job for someone; his primary job was driving the forklift. Would a company ask the forklift driver to do their accounting? Why should they ask the same person to manage what they are now considering an important part of their business infrastructure? As the Internet (and networking in general) "matures" and becomes a more important of business, companies will have to realize that they can't get away with just picking the employee that seems to recognize a computer when you drop it on his foot and calling him the "network guy."

Part of the reason so many ISPs either keep raising prices or go out of business is that people expect their ISP to do their network support. When we tell a customer that our resposibility ends at the ethernet port of the router (because they want that T1 line cheap), they get irate. Our choices are to try to help them, even though we don't know a thing about their network (and because of that we may screw things up more) or to tell them it isn't our problem (which makes them mad and may cause them to move their service). The small "mom and pop" type ISPs can afford to do more of this kind of help in the short run, but they can't maintain it in the long run (been there, done that). The same thing holds true for residential (dialup/DSL) customers. People (including me) love to complain about the quality of most tech support groups, but try asking an ISP how much of their revenue goes to support. You get what you pay for.

I handle our abuse email, and we get all kinds of reports. We've had people complain that our DNS server is attacking them on port 53 (the DNS port), that our Akamai content distribution servers attack them every time they go to CNN, and so on.

We had an Army network admin call us a couple of weeks ago because he was getting flooded with reports of an attack originating from our Unix shell account server. It turns out that someone from his network had connected to our server via SSH. When you make an outgoing TCP connection, a random port is chosen for your end of the connection. The port his computer had chosen happened to be one on which some old Cisco switches had a security hole. Every single packet this guy received (he was connected for 4 hours) cause an alert on the Army firewall. The network admin didn't understand what was happening, and instead of going to the computer within his network that was the "target", he jumped on us.

I could go on :-), but I better stop now.

• $6.23 -- Retail Markup
• $0.85 -- Co-op advertising and discounts to retailers

The labels can do away with these by selling directly to the consumer. Mail-order music subscription services such as Bertelsmann's BMG and Columbia House already do something similar.

• $3.34 -- Company Overhead, Distribution, and Shipping

If they ever decided to get a Clue and sell their albums online as 192 Kbps Ogg Vorbis secure downloads, they could reduce this to the cost of Akamaized bandwidth.

The picture is from This page. Which describes their network tech. here is the orginal picture.

Enjoy

Freenet is anonymous, it's archetecture automaticly does proxying and multiple layers at that. While the first node you connect to can figure out who you are they can't easilly determine if you initiated the request and any nodes down the further down the chain don't have a clue about who you are. Unlike Gnutella when a Freenet request is fufilled the requesting node doesn't directly connect to the node that has the data.

Secondly by creating many copies of popular data you are decreasing the amount of bandwidth used in exchange for increasing the amount of space used. Say you have a popular site in Europe that's getting lots of hits from the US. Without Freenet every time someone in the US wants the data their request has to go over congested international links, very wastefull that. With Freenet a copy of the data, probably many copies in fact, would soon get stored in the US reducing the load on those slow international links. Even though the data has to go through many nodes in between, and has multiple copies of it made, you still get a significant savings in bandwidth usage. And any copies made while en-route are likely to be used again by other people because those copies are by-nature "close" to the keyspace that node serves.

Ever heard of Akamia? They do essentially the same thing with their network to reduce the load on sites around the net.

Now currently Freenet doesn't do a very good job of this because it doesn't take network speeds into account when choosing nodes while routing messages. But it will be quite simple to add support for that and will be done after a few more releases.

Except that Microsoft's DNS is now being provided by Akamai on (apparently) Linux 2.1 servers. See this story in The Register.

InterVu (bought by Akamai) comes to mind. They were one of the first to directly broadcast live events and are now one of the best.

You can then have more time to focus on your web site and managing the content, not the service.

2) Automatically executing code from a remote, untrusted source is bad, kids. I haven't seen a web bug that actually executes remote code on the local client machine unless you consider JavaScript code to be unsafe. Sure JavaScript can be unsafe if your browser's intepreter has an implementation bug or you consider certain information like screen resolution, local timezone and other browser options to be private, but we are not talking virus risk here.

The Web Bug FAQ for more information. In particular note that it does list some non-evil uses for web bugs:

Another use of Web bugs is to provide an independent accounting of how many people have visited a particular Web site.

Web bugs are also used to gather statistics about Web browser usage at different places on the Internet.

E.g., If you want your site to run at the fastest posible speed, you might host static HTML with a globaly traffic managed web caching or hosting company like Akamai or Speedera But you still would like to get logs directly for anaylzing traffic to your site and comparing with the web hosting company's bills. So you place a web bug on your pages directly back to your origin site (or third party like LiveStat). The user experence is still fast if done right, because the slow logging to your server occurs after the page is rendered.

Think how much faster and more efficient the Internet could be if instead of always connecting you to a central server every time you click on to a website, your computer would find the source that housed that information nearest to you [...]

Sounds like SF's next project will be to travel back in time a few years, say to 1995, and then start up Akamai.

-- Give him Head? Be a Beacon?

What we need is smarter protocols that distribute the content closer to the edge of the network. The commercial Akamai service does this by placing content caching servers at several thousand ISPs. These caching servers hold frequently-requested images, video clips, and other large files from major content providers. When Joe ISP user downloads a video clip it comes from his ISP's Akamai server instead of going out onto the Internet, crossing peering points, etc.

Worthy of note is that Helix Code is an Akamai customer. So, when you install Helix GNOME, it's not coming from some arbitrary mirror site, but automagically from the closest upstream Akamai server.

Akamai is awesome, but it would be nice to have an open-source, open-network way to implement this at the ISP level. Does anyone know of any such effort underway already?

An example of a typical personal firewall log entry that indicates normal web traffic might look like:

The firewall has blocked Internet access to your computer (TCP Port 1189) from a512.g.akamai.net (216.200.119.18) (HTTP).

Generally a TCP server listens on a well-known port &lt 1023 (for example port 80 for HTTP), and a TCP client connects from a port &gt 1023 assigned by the operating system. So a connection from port 80 of the Akamai server to port 1189,1192,1188,etc... on the client machine, is a normal HTTP transaction.

If you are seeing this alert when you start up your dialup connection, it is likely that some program on your computer is initiating an HTTP transaction with an Akamaized web site when dialup networking starts up.

We've notified Zone Labs about these alerts and look forward to working with them to resolve this issue of alerts for normal web traffic.

If there are any further questions, please feel free to send them to ccare@akamai.com

- Kevin
Akamai Technologies Customer Care

--

Like, if I request a URL from www.flibbertygibbit.com, can't the browser be smart enough not to request further resources from, say, ad.doubleclick.net (but be smart enough to request resources from pix.flibbertygibbit.com)?

Yes; the trouble is that many sites have offsite images load from a perfectly normal and harmless third-party server. Akamai is the best example; companies from Altavista to Apple to Andover store their graphics on Akamai's distributed servers for faster load times. If you prohibit all third-party graphics, you prevent these graphics from loading, thus breaking many pages.

Wasn't this capability in Mozilla until recently? How hard is it to put back in?

Yes, it was; see this older slashdot story for details. The good news is that Mozilla retains the capability to block off-site cookies, which doesn't totally eliminate the web bug problem but does take a huge bite out of it (along with the whole DoubleClick-privacy problem in general).

Personally I suspect that the offsite image problem could be 99% solved with a little special-casing and some creative DNS work. But I don't know that for certain.

The bottom line is that, because of this one incredibly simple feature, Mozilla is currently the most privacy-friendly off-the-shelf browser that I know of. Of course, if you are really concerned about privacy, you could try add-ons like Junkbusters or IDcide.

Jamie McCarthy

That's actually the beauty of the techniques used by companies like Akamai and Sandpiper. They take the burden off the end user for finding a local mirror. When you try to access a pages on their servers, it automatically selects the closest one to you.

On the contrary, this would presuppose that users chose their own routers, which is simply not the case. Perhaps there should be greater caching at ISPs, but this is difficult in a world of personalization, streaming content, and pages dynamically generated from real time data feeds.

Technologies such as those offered by Akamai claim to be able to address this. And, IIRC, it runs on Linux.

Think about it: Internet 2, the move towards optical switching for the net's backbone, multiple levels of DSL service: all these are going to be have v. have-not services. Even now, the web pages can afford it get acceleration through services like Akamai, while the rest of the internet groans under the weight of thousands of new users every day.

The rising tide of network speed will not lift all boats, and while that's unfortunate, you can't be surprised. As averse as we all are to consider it, this is the kind of situation where you look at a tax-flush federal government to make sure everyone can keep up.

-jpowers

but then I started thinking about me as a developer and sites that I've built to make heavy use of services like akamai. now _none_ of my images will show up in your browser because all of the images are served out of akamaitech.net (akamai's distributed network). now it doesn't sound so great...

unless I could setup akamaitech.net as an exception, and therefore acceptable to fetch images from (but how many average joe users of netscape would actually understand why my page looked bad in the first place?). but then, if I allow akamai images, the ad banner folks will just start akamaizing their delivery (not a bad idea to begin with), and I'm back to square one...

just my (probably incoherent) ramblings...

- mark

Stripes starts his reply:
Assuming you can figure out where they all are form the IP addresses in the root.cache file, and traceroute, or other similar tools, and maybe a bit of social engenering, it shouldn't be any harder then any other 12 randomly selected machines.

Define "explosions"

Stripes, The poster to which you responded did not specify what type of explosions were available to them. If they're nuclear explosions, they'd probably need only 8-10 strategically placed explosions to wipe out all of the current neameservers (with or without social engineering). If they're lucky, they might take out the "shadow root servers" as well. Given the location of some of the root servers, they'd probably cripple alot more than just DNS. They'd effectively take out a good deal of infrastructure as well as the Internet engineers necessary to repair it, not to mention start a worldwide panic.

The Internet would still recover though, much as you described in your post. Anyone can setup a redundant server cluster within a matter of minutes given a set of pre-staged root and first level zone data.

The more interesting problems are due to corrupted data rather than doing denial of service attacks on nameservers. Some bad data in Network Solution's database can make various interesting parts of the Internet suck really bad. When one root server has data corruption, the whole net feels it. Imagine if some NSOL staffer garbled the nameserver data for "Yahoo.COM." or "IN-ADDR.ARPA." to point to 255.255.255.255 instead of the real servers?

For anyone else interested in DNS DoS...

An easier method

One of the easiest way to kill DNS is to try a coordinated DoS attack against all of the nameservers. Each of the world hundreds of thousands of resolvers is configured to use any of 13 root nameservers. Just like a 15-year-old kid did with HTTP requests, one could probably start a distributed DoS attack against DNS. The "heftiest" root nameserver is rumored somewhere in this discussion to be able to handle 6000-8000 hits a second. With 13 published nameservers, one needs only about 100000 hits per second to saturate the current capacity of all of the servers. Let's say that I was a bright hacker (which I'm not) that I could find my way into 1000 machines around the world that each had a T1 connection or better. Can we agree that this is a difficult but not unreasonably impossible thing to do? If one were not smart enough to do it themselves, one could perhaps go to a hacker convention or local user group and bribe a script kiddie seeking infamy and fortune to go forth an find 1000 machines to hack. Another way is to unleash a time-dated virus onto the net that will do your bidding at a specific time. Each machine would gather a list of 100 addresses, perhaps starting with the history file of a user's browser to get a list of second-level domains. It could also look for addresses using a popular portal directory or search engine and interpret results to get domain names. With 100 domain names, it would query 100 names per second (less than one megabit) from each of the few registered root nmeservers. While the traffic isn't overwhelming, it will overload the root servers fo rthe number of transactions per second, and nothing short of hunting and killing half of the query servers would reduce the effectiveness of the attack. To make the attack harder to stop, one could double or quadruple the number of query servers or use methods of masquerading your attack (I won't go into detail here) to keep network administrators from being able to shut down query servers. Another way to scale the attack is to use they heavier TCP protocol for most of the queries instead of the lightweight UDP.

fin.

The technology needed to exponentially increase the ability of the root servers to perform is not out of reach. With the proper motivation (a DoS like I described), one million dollars of capital (compare $1m to the current valuation of NSOL), and perhaps 30 man-weeks of time, one can make a farm of servers able to handle two orders of magnitude more requests than the current set of servers.

The IBM server announcement by Network Solutions disappoints me. It's sad.

Any of the following are good candidates that I know about for scalably solving root DNS infrastructure problems...

• UltraDNS - DNS service provider with an interesting spin on distributed scalability
• Nominum - the knowledge and knowhow to make fast scalable DNS servers and software
• Akamai/Sandpiper - a distributed operations infrastructure onto which one can install root clusters.

One can also implement interesting filters on such a proxy server to reduce the effect of stupider resolvers or lame DoS attacks.

--
Eric Ziegast

PS: Slashdot probably isn't the best forum for this, but if you know a better forum, feel free to point them toward this post.

There's no sense pushing around a lot of bits if you don't have to, and while you can make the rest of everything pretty fast by periodically dumping dynamic content to static pages and serving them up with Zeus or khttpd, reserving PHP3 or mod_perl use for the very few things that really must be completely dynamic each and every time, and you can use Layer 4 switches to load-balance between servers at a site, and distribute your servers around the world and distribute the load to the closest unloaded server by using customized nameservers, the graphics is at least one thing that can be solved by using the Akamai network.
--
Brad Knowles

I think one of the first developers of squid is the CTO of Akamai.

The CTO of Akamai is Daniel Lewin; his bio page at Akamai says nothing about Squid.

You may, perhaps, be thinking of Peter Danzig, who is the VP of Technology at Akamai; his bio page at Akamai says:

His background in Internet information systems also includes work on the federally-funded Harvest Information Discovery System, or 'Harvest Project.' His collaboration on this project at the University of Southern California resulted in one of the earliest designs for caching Internet backbone traffic. Danzig led the Harvest Web cache and helped design the Harvest indexer projects from 1992-1995.

I think the Squid project was originally derived from the Harvest cache; the NetApp NetCache software was also originally Harvest-derived, although much, perhaps most, of it was done at Internet Middleware (a company founded by Peter and bought by NetApp) and NetApp. (I suspect much of Squid might also be non-Harvest code.)

I think one of the first developers of squid is the CTO of Akamai.

The CTO of Akamai is Daniel Lewin; his bio page at Akamai says nothing about Squid.

You may, perhaps, be thinking of Peter Danzig, who is the VP of Technology at Akamai; his bio page at Akamai says:

His background in Internet information systems also includes work on the federally-funded Harvest Information Discovery System, or 'Harvest Project.' His collaboration on this project at the University of Southern California resulted in one of the earliest designs for caching Internet backbone traffic. Danzig led the Harvest Web cache and helped design the Harvest indexer projects from 1992-1995.

I think the Squid project was originally derived from the Harvest cache; the NetApp NetCache software was also originally Harvest-derived, although much, perhaps most, of it was done at Internet Middleware (a company founded by Peter and bought by NetApp) and NetApp. (I suspect much of Squid might also be non-Harvest code.)

The backbone doesn't need to handle the bandwidth, if people like Akamai and InfoLibria have their way. They put servers at the ISP headend, and charge content providers for cacheing their material at the "edge of the web".

Consider that much news footage is shot in DV25 these days. This is a 25 Mbps format. I think VDSL + edge servers looks really interesting for full-quality video delivery.

I'm not forgetting the quality-of-implementation and distance-from-CO issues; I just figure they'll get worked out eventually.

The company is called akamai and according to an article in wired, they serve content for ESPN, Paramount, and Apple amongst others. Sorry about not including the link before, but I could have sworn it was a slashdot post instead of being in Wired.

Not a bad idea! ;-) A new internet startup called Akamai does something very similar. The company has ~900 servers around the world, caching their clients static web content. For example, jcrew.com generates their HTML with image URLs pointing to (say) http://a1240.g.akamaitech.net/7/1240/969/ffe0a8c13 22031/jcrew.com/images/ sep99/e2home/clearance.jpg. This distributes the load off of jcrew.com's server to the Akamai servers nearest the user.

I wonder if someone could write a freebie script that does the same thing using free web space like GeoCities or Xoom. The script could automatically create accounts on those sites and shed the load for serving static content to these free servers. >:-) I bet GeoCities would soon figure out a way to block this behaviour, though.. but it still might be fun.

I'd wager to say that despite all of the dynamic content on heavy load sites (e.g. Yahoo, CNN), static content is still a major issue and bandwidth drain. Specifically, images are a real big performance drag.

While your server farm might be able to handle the bandwidth, the pipe inbetween you and the eventual downloader may not be so fat. For example, no matter what a site in the UK is running, it's going to seem slow here in the US. This is why many major web sites have contracted with Akamai to move static data (e.g. images) to sites around the world closer to users. Next time you visit Yahoo, you'll probably download images from an Akamai server closer to home, and it'll be really fast.

Point being, most big sites send out more data in static images than in dynamic HTML (Slashdot comment forums excluded).