Slashdot Mirror
The Linux Kernel Archives
Jeremy Andrews writes "KernelTrap offers an interesting look at the history behind the Linux Kernel Archives, home of the Linux kernel. They start from the beginning in 1997, when kernel.org ran on a generic "white box PC" using a shared T1, to the present where it runs on multiple quad Opterons each with 24 gigabytes of RAM, 10 terabytes of disk space, and a gigabit link to the internet. Much of the article is based on an interview with Peter Anvin, also including quotes from Linus Torvalds, Paul Vixie of Internet Systems Consortium, Inc who donates the bandwidth, and Matt Taggart of Hewlett-Packard who donated the hardware."
The normal bandwidth used by kernel.org is between 150 to 200 megabits per second, at times when "nothing major is happening," Peter said. "Quite honestly, the test releases aren't even a blip on our radar," he added, referring to the -pre and -rc kernels, explaining that they don't noticeably increase the amount of bandwidth that is consumed. Only when an official stable release is announced does kernel.org see a spike in traffic. For example, with the upcoming 2.6.12 release Peter predicted, "I expect it go to the high 200's, for about a day." He noted that even with a direct link from a busy website such as Slashdot, that was about as much bandwidth consumption as they see from a kernel release. "What really drives up the load average is when one of the distributions that we mirror makes a release," he explained, "such as one of the Fedora cores. The kernel is only a few tens of megabytes, whereas a fedora core is a couple of gigabytes." With the upcoming release of Fedora Core 4, Peter predicts that both gigabit links will probably be saturated for 3 or 4 days. "This is largely speculation, because never before have we had the capability of serving that much traffic."
Hulk SMASH Celiac Disease
Boy what I could do with that and BitTorrent... *rubs palms together*
Very interesting...will have to check this out.
2006: generic "white box PC" using a shared T1 -- AGAIN
!!!HA!!!
It actually runs on W2k3 w/IIS as the backend. That's why the uptime is so great.
Beep beep.
Dear Patriots:
Remember, what I told you back in November 2004:
Faith, Family, and Values won't defend the world's most dangerous and inarticulate "leader" from high crimes and felonies
Sincerely,
Kilgore Trout
Iraq: When Was The Die Cast?
John Prados
May 03, 2005
John Prados is a senior fellow with the National Security Archive in Washington, DC. He is author of Hoodwinked: The Documents That Reveal How Bush Sold Us a War (The New Press).
Coming just days after the release of the original secret legal advice given to the British government on the lack of foundation in international law for invading Iraq, a fresh leak out of London now reveals with stunning clarity that the goal of overthrowing Saddam Hussein was set at least a year in advance.
Emerging in the final days before the UK's parliamentary election, a memo leaked to the London Sunday Times reveals that Bush decided to go to war by April of 2002, and that by July of that same year it was clear that the United States would fabricate the intelligence necessary to justify the war.
The Bush administration's pious rhetoric about strengthening the United Nations was strictly for public consumption. Its talk about alleged Iraqi weapons of mass destruction--as Lord Goldsmith's legal opinion demonstrates--was crucial because the only avenue offering a fig leaf of legal justification for war was to claim to be enforcing U.N. disarmament resolutions. And President Bush's repeated assertions that no decision had been made about attacking Iraq were plainly false.
Decision Made: November 2001-April 2002
Military planning for Iraq actually began in November 2001, while the campaign in Afghanistan absorbed the public's attention. In his memoirs, American field commander General Tommy Franks tells us that on December 4, in his very first briefing of the existing U.S. contingency plan for Iraq, Franks told defense secretary Donald Rumsfeld that, "I am assuming the principle objective will be to remove the regime of Saddam Hussein." Rumsfeld replied that the president would make the ultimate decision but that, "That is my assumption too." After several weeks of fleshing out the preliminary concept, General Franks presented it to George W. Bush at the president's ranch in Crawford, Texas, on December 28. At that meeting Franks told the group that regime change and WMD removal were the working assumptions behind his concept, with "a murmur of assent" being the reaction of those at the table or watching the teleconference. At the end of the presentation, Bush expressed confidence that diplomacy and international pressure would make military action unnecessary.
Neither in his various statements to the media nor in interviews--including those with Bob Woodward--has Bush ever recounted his evolving thinking or detailed his actions. However, reports show that at the same time of Frank's planning--around the end of 2001--the president signed a directive authorizing the CIA to act against Saddam. Bush subsequently targeted Iraq as a member of his invented "Axis of Evil" in the State of the Union address in late January 2002. When asked on February 6, 2002, about the administration's desire for regime change in Iraq, Secretary of State Colin Powell replied, "We are looking at a variety of options that would bring that about." This was the day before General Franks presented a more detailed war plan to Bush and the National Security Council at the White House. Bush specifically told the press on February 12, regarding his options on Iraq, "I'll keep them close to my vest."
The following month Vice President Richard Cheney made an extensive tour of European and Middle Eastern nations which failed to enlist much support for action against Iraq. This made the attitude of the British a vital question for Bush. Prime Minister Tony Blair visited the United Stat
...especially having dealt with something like this (on a much smaller scale) recently.
We were having bandwidth limitations on RubyForge; it was getting up to 80 GB per month at the end of 2004. Mirroring out releases helped get usage back down to 15 GB per month. Many thanks to our mirror providers!
The Army reading list
Good Grief that's a lot of pipe! Saturating a PAIR of gig links? Certainly tends to make one stop and consider how many people are actually USING linux nowadays. Good to see!
Thinking outside my Head
Way to slow.
Mod +5 funny -5 irreverant
WTF?!!!
This was a great article! I can attest the there is quite a difference with the new hardware, I got a 500KBps download last night while downloading rc3-mm2.
Can we please have the same kind of article about slashdot hardware?
"think of it as evolution in action"
The 'kernel.org' domain name was picked because by that time in 1997 the more logical seeming Linux dot names were already taken. The Transmeta domain was intentionally not used to avoid creating the false perception that Transmeta owned Linux.
I wonder what would have happened with Transmeta and Linux if they had used the Transmeta domain to host the kernel archives. Would IBM have gotten involved with Linux? Would SCO have sued Transmeta instead of IBM? Would Linus have left Transmeta?
Why mention ISC or HP at all in this article except as 'free advertising' for those firms.
Want to learn how to read past the PR BS here?
When Linus Torvalds purchased his first computer on which he began writing the Linux kernel, the state-of-the art PC with 4 megabytes of RAM and running at 33 megahertz was too expensive for him to buy outright.
Oh my god, it's a diesel!
-1, Disagree is not a valid option. Troll, Flamebait and Offtopic are not a substitute.
Lite i 5 straining
At least this is a better story and more technical (aka nerdy) than oh say....what Google's homepage looked like in 1997.
...it runs on multiple quad Opterons each with 24 gigabytes of RAM, 10 terabytes of disk space, and a gigabit link to the internet...
Do I smell a challenge?
Where are all the:
/., you are slipping.
"I'd like a beowolf cluster of these..."
posts.
Don't forget to disable JS; don't say I didn't warn you.
I'm in awe of that box. It just pushes so much data, all the time. And 1000Mb/s of bandwidth?! That's more bandwidth than Google!*
* I strongly suspect this not to be true.
Get your own free personal location tracker
Referring to 32-bit systems, Peter noted, "we learned that the Linux load average rolls over at 1024. And we actually found this out empirically."
Can you even get the server to TELL you what the load is when it's that high?? That's INSANE!
teeker
It's already nearly slashdotted. mirror
this post contain no useful information, no need to mod it down
As we have some figures for the numbers of machines in the early days and surely we have the traffic figures for then as well...
We should be able to make a reasonable guess at the number of machines out there with Linux on them...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
I'm really happy to see HP giving so much support. I'll definitely remember this the next time someone asks my opinion about what server hardware to buy.
It doesn't surprise me that being linked from slashdot is just a minor effect. A kernel package is tens of megabytes, while a single visit will likely consume less than 100KB.
see a Text Widget
...is get themselves mentioned on Slashdot on the same day that there's a simultaneous release of a major distribution and a Linux kernel.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
DONT USE APACHE.
It's a horrible resource hog and expecially for static files there are FAR better solutions out there. Tux, lighttpd etc.
http://www.lighttpd.net/benchmark/
and no, apache 2 isnt much better.
so good i had to fondle myself while reading it.
It would make a nice warez server.. Put that bw into good use!
where it runs on multiple quad Opterons each with 24 gigabytes of RAM, 10 terabytes of disk space, and a gigabit link to the internet.
will it run linux?
My problem? I was perfectly gruntled, until some numbnuts came by and dissed me.
So, you are going to recommend HP for the simple fact that they are supporting kernel.org? Shouldn't you recommend them based on their actual product? How is this any different than Bush putting his cronies in high-ranking jobs? It is easier to make an informed decision when you don't involve personal politics.
However, in this post from hpa, it looks like the tools are not ready.
Love salty crackers? catchy electronica? Try !
I wonder what operating system they are running. :)
wayner@pobox.com -- Wayne A Arthurton -- www.pobox.com/~wayner
I guess H.Peter Anvin petrified Natalie Portman with his Beowulf cluster! Watch for Soviet Russia to pour hot grits down his pants!
C|N>K
I worked at Globix when we offered free bandwidth to kernel.org. In the beginning, when things were going well and we had hundreds of millions of dollars to spend, we used this to leverage our poision in the open source community. Of course, when the bubbled bursted Globix tried to get rid of all the free riders first. It was done very selectively, though. While some were cut loose as fast as possible (like kernel.org), others were kept because they had better connections to some of the executives. I don't want to name names, just that much: it is one of the better known nudie magazines. It was quite a qide to work at Globix. They are still around, barely.
If it's an announcement of a new kernel, it is likely that at least some percent of the /. crowd will download the new kernel.
It takes a man to suffer ignorance and smile
Be yourself no matter what they say
"where it runs on multiple quad Opterons each with 24 gigabytes of RAM, 10 terabytes of disk space, and a gigabit link to the internet"
Can you image a cluster of these...no wait...
I don't have occasion to use rsync, and I'm not too familiar with its design, but I think it synchs directories by checksumming the files in them to see if they differ. So Peter is saying above that the server's bottleneck is checksumming. I would think that on a server like this, checksums could be cached - why checksum a stable file more than once? Once you have a checksum for linux-2.6.0.tar.bz2, why calculate it again?
This would require a bit of bookkeeping when files change, but wouldn't it be worth it on such a busy system? (Or am I confused?)
...I guess that:
a) kernel.org doesn't think the 'enterprise readiness' of RH Enterprise Linux is that great, (vs what Fedora offers) even in what should be considered one of the most mission-critical sites in the Linux ecosystem (or that the difference with Fedora is worth paying for)
b) No one at RH is bright enough to be embarassed by this and offer kernel.org some free licenses...
Yes, the torrent would sure be a releaf.
I'm really happy to see HP giving so much support. I'll definitely remember this the next time someone asks my opinion about what server hardware to buy.
I believe that was the reaction HP's marketing department also expected. Admittedly, providing the hardware was a very nice gesture, but in reality, it's a brilliant marketing move.
Furthermore, I hope you will take other factors and datapoints into consideration when someone asks you for your advice, though. The servers donated were relatively high-end -- they might or might not be reflective of all HP hardware.
I'm Trappped at Berkeley.
Where is the swap size = phys. size x2 rule?
Linux doesn't support the swapping of 48 GiB, max. is 2 GiB, :P
I read the email Linus sent out introducing his creation where it says it won't support anything other that AT Hard Disks. Cracks me up. The history of the Linux kernel would make an interesting read, moreso than other OS's because of recent events, and the fact that it wasn't created by IBM, but a lone Grad Student. I think it would make a fascinating read.
I'm not a troll, but I play one on Slashdot.
thats where we need somekind of link between cvs and a peer to peer system.
Did the interviewer forget to ask what the longest uptime is for kernel.org?? That would have been interesting to know.
the torrent would sure be a releaf.
... resist ... urge ... to ... ask ... why ... bit ... torrent ... would ... put ... leaves ... back ... on trees.
Must
"I don't know, therefore Aliens" Wafflebox1
At least on Linux, the load average includes
processes waiting for in uninterruptable sleep.
That would be disk IO, mostly.
What's different about rsync is that it does not ordinarily use a single file checksum (and therefore copy whole files if changed). Instead, to save bandwidth, it uses a more sophisticated system to ensure that only changed parts of a file are transmitted - and it detects changed parts by comparing (many) checksums, I believe. The report sums it up like this:
(Disclaimer, I have only skimmed the rsync report and that was some time ago, but I am a longtime and happy rsync user.)
you had me at #!