Slashdot Mirror
Kernel.org Needs Some Help, Perl Foundation Got Some
Dante wrote in to say "I just read this on the Linux Kernel mailing list, it's from Peter Anvin, one of the ftp.kernel.org maintainers...
H. Peter Anvin writes: "The recent troubles we've had at kernel.org pretty much highlight the issues with having an offsite system with no easy physical access. This begs the question if we could establish another primary kernel.org site; this would not only reduce the load on any one site but deal with any one failure in a much more graceful way.
Anyone have any ideas of some organization who would be willing to host a second kernel.org server? Such an organization should expect around 25 Mbit/s sustained traffic, and up to 40-100 Mbit/s peak traffic (this one can be adjusted to fit the available resources.) If so, please contact me."
In related news, mbadolato wrote in to tell us that "there's a press release over at dyndns.org announcing that they've donated $20,000 to the Perl Foundation!
'Thanks primarily to Perl and other Open Source technologies, we are able to provide DNS services to over 180,000 members of the Internet community,' said Tim Wilde, founder and chief executive officer of DynDNS.org. 'This is our way of giving back to some of the people whose tireless devotion to writing quality software has enabled us to provide our services to the Internet community over the past three years.'
The donation page for the Perl Foundation can be found here
But I would imagine that everytime a new kernal is released that world+dog go to view the site. I serious doubt that everyone who goes to the site downloads, most just read - lots of people reading still requires a fair old chunk of bandwidth.
/. effect seemed to bring that site to its knees, but as regular news sites see linux as more and more relevant to their audience they too will link people in, adding to the problem.
/. effects which stop people going to your site and could potentially discourage them from going there in the future.
As we saw earlier in the week the
Realistically they are victims of their own success - people want information about the new kernal and doubtlessly they want to download stuff too.
As these once limited interest sites become more mainstream, then it's clear that they need to maintain quality of service, and that means no
Just my 2c
i agree with putting limiting direct access to kernel.org. save it for the mirrors, and for key developers.
as for the rest of us, how about having words with major shareware sites.
Or possibly some sort of pay-per-mb-download scheme from official mirrors? that would certainly improve the popularity of patching.
just ideas. flame as you see fit.
IMHO we have to move away from the idea a central resource allocation for projects is good. The currecnt debate about SourceForge and VA is the best example.
It is just dangerous to rely on one or two main sites run by corporates. Why not try to find many corporations that can share the load and also minimise the risk for the project of being affected by companies woes?
One main server that is the central source for many, many mirrors and without direct access for the end-user might be the way ahead.
Regards,
REB
Sounds like a bad idea to me. Centralised control is a good thing on occasion - at the moment we have kernel.org and a list of mirrors. I think this is a sweet spot between being too centralised (bandwidth problems) and too distributed (control problems)
So if we have a decentralised system (I assume you are inferring P2P style? If not, then surely the mirrors system is adequate?) how are we to stop people abusing the system? I could host a trojan version - where do people go to verify the MD5? How many people actually check MD5s? My point is that we need a trusted pool of servers; the truly paranoid can still check their MD5s against those at kernel.org, the rest of us can be assured that this is one of the trusted mirrors, and not a server owned by j03 133+ h4X0R.
Hey - my acct works again!
25 Mb/s = 3.125 MB/s = 187.5 MB/min = 11.25 GB/hr = 270 GB/day = 8.1 TB/month
Looking for any old 8-bit Heathkit/Zenith software/hardware - http://heathkit.garlanger.com
I know system admins (if they can be called such) that don't know how to patch. Granted, it's not an intuitive process.
Also. some are not updating from the last kernel, wich requires more than 1 patch.
I slso believe that such tool, that downloaded as many patches as needed, should be explained and incentivated in the kerne's site motd. If they don't show it on the front page, and say it's an advantage to the user, then few people are going to get it.
-
Roses are #FF0000, Violets are #0000FF, find / -name '*base*' |xargs chown -R us && mv zig greatjustice
Didn't google say recently how they save so much money with Open Source, etc etc etc?
:/
They probably have that kind of bw...
How 'bout if people who use P2P like Edonkey, who downloaded the kernel-source, just put it in their shared directory? That would distribute the load a little bit
Could some form of broadcast or streaming help?
What about Net-News it is an existing system that could distribute the patch to many of the people within a day.
The new kernel could be released,
mirrors and approved developers could have access to kernel.org for the first 3 days. Then only be patch downloads from kernel.org for the next 4 days.
BUT through net-news and most people would have it in a day.
Asking for a big chunk of bandwidth and centralized management is the problem. It's expensive. Instead:
- Use the existing file sharing networks
- Netnews (I can get the file faster from my ISP's news server than anywhere else), and software like pan makes getting all the pieces trivial.
- Are there any open file sharing projects that we could use? Something that limited to a single download per user wouldn't be onerous. There are lots of cable/DSL linux users.
Can You Say Linux? I Knew That You Could.
1) Only allow access by mirrors and those ACTUALLY working on the kernel (ie, the kernel maintainers).
2) Get more mirrors. We're talking like several thousand here. As an ISP, I know I would not mind hosting a mirror, but I cannot afford $25,000/month in bandwidth. Splitting up the load using a large number of mirrors would make it MUCH cheaper to mirror the kernel files.
3) Use a highly-efficient load-sharing/balancing mechanism to direct people to mirror sites. Make it so the user can browse/select the files from the main kernel.org site, but the downloads are redirected from there to the mirrors.
4) Use a better patch process to reduce the size of the average download: 1) The x.x.0 release is the only full download, 2) use a patch system that downloads all the necessary updates, applies them to the x.x.0 version (or whatever the version the user already has) to get the latest version, and 3) MD5 checksums EVERY file to verify that it was patched correctly.
-SS "Teach the ignorant, care for the dumb, and punish the stupid."
Just do what a large number of larger sites like Yahoo do, and ask the mirror list (currently over 100+ sites) to act as full mirrors, and round-robin the dns.
Further, make kernel.org alpha.kernel.org, and have alpha be the site everyone mirrors from, and restrict access to it to only core kernel developers.
Overnight, you'd have taken care of the problem.
GPL'd web-based tradewars themed space game
Burris
That's an idea! Linus should ask Bill G to front the green for the Linux kernel site. I know Billy-boy would do it. He's all for helping the community... ;-)