Calling for Smaller Kernel Sources?

FrozedSolid asks: "I can understand that the kernel contains many drivers and support for a lot of platforms, but the fact that the full kernel download can amount to 32mb doesn't make it any easier to download with a 56k modem. Kernel patches are nice, but obviously only apply when you have access to an entire kernel tree beforehand. Is there a way you can download a leaner linux kernel source? Is there a place that carries sources for x86 only or possibly sources without some of the less popular drivers?"

Comments

Actually,

[GreyWolf3000]

32MB is not too much to ask. On 56k, that amounts to like a few hours of download. Let's assume 5K/s (I used to get 2 in Windows, but 7 is Linux for some reason). That's 300K/min. That's 1 MB every 3.5 minutes. That's 32MB in ~120 minutes, or 2 hours.

If you haven't gotten cable, and you're using Linux, the distros themselves are at a magnitude greater in size; I doubt that kernel sources are the real problem.

Re:Actually,

[King of the World]

You do realise that you're turning your back on the potentially brilliant kernel developers who are physically unable to leave a download going overnight and start "hacking" the next day, don't you?!?

What sane person would ask such a thing.

Re:Actually,

[kerasineAddict]

Sometimes, every byte counts.
I'm in a University residence, where we're allowed 500MB / week...This alone can get eaten up pretty quickly when you're downloading updates to software...Or, for example a DSL provider around here charges over 5GB/month. Now in these examples 10-20 megs won't make or break you, but it does point out that these things can matter.

Part already answered

[gmhowell]

Part of this has already been answered. If you just pick x86 (or PPC or Alpha etc.) the size does not change that much. The vast majority of the kernel is not architecture specific. That's a good thing!

I don't know of any sites, but let me say a few things. First, your distro probably has a binary package with almost everything either compiled in or a module. Barring that, when I used to be stuck on dialup, I'd get the most recent kernel and then download the patch each time. It was a pain in the butt, but not as bad as downloading the full sources each time.

RSYNC!!!

[Jeremiah Cornelius]

You need rsync.

It was devised to combat just the problem you cite.

rsync://rsync.kernel.org/pub/[wherever you want to go]

Thank you, TRIDGE!

Online make menuconfig

[cybermace5]

Seems like a person could set up a few webservers, let people select kernel configuration options, and send the much smaller bzImage (and compiled modules) through email. Sure, the size would vary wildly based on how many modules were selected, the architechture, etc., but on average I'd say it would be much smaller.

The benefit is less bandwidth wasted for people downloading 35 megs of source to recompile a 900K kernel image. The disadvantage is processor time required, well, how many Athlons do you have to buy to serve the same number of kernels per day, and how does it compare to bandwidth costs?

Yes, *I* would like to have the sources to myself, I have a few source files I need to tweak to get my machine working properly. But many people just have the burned CD from the friend of a friend, and would appreciate a recent kernel without a mammoth download.

Maybe someone's already doing this, I don't know.

Re:Online make menuconfig

[FueledByRamen]

Hey, this does sound interesting. I have a few spare computers around and a little knowledge of PERL... I think that I'll look into this tomorrow. That would be _really_ neat. If I do manage to get something working, I (of course) will insure that it won't remain that way for long by posting it back up here...

It really shouldn't be too hard. I've been staring at your post thinking for a bit, and the best way to do it that I can think of is to read in the config template file from the kernel source tree (have several selectable versions) and generate a huge page full of radio-select buttons. Once that is submitted, an MD5 hash is applied to the generated config file. If it matches an existing package (unlikely at best), simply serve that up. Otherwise, make a new build tree named "builds/$VERSION/$MDSUM" and copy the config file into it. Build the kernel, tar.gz the resulting modules and kernel image, and email the links to the person.

This would require quite a bit of CPU horsepower, but it would make for a nice, small kernel download and a sort of set-and-forget build. Set the options and press the button on your lunch break, and have the link sitting in your inbox when you get home (unless it's slashdotted, in which case I'll come home to a hole melted in my floor where a server and accompanying cable modem used to be).

Seek bandwidth elsewhere.

[toybuilder]

Never under-estimate the bandwidth of a plane full of CD's... ...or the CD-RW drive at your nearest Internet Cafe.

This is nothing

[0x0d0a]

Incredibly, Mozilla 1.2 is going to have built-in, enabled-by-default prefetching. The amount of bandwidth this will waste blows the mind. Imagine every single Joe on the Net suddenly using up 20 times as much bandwidth downloading stuff that he will *never see*. The intermitent activity on lines turning into constant load -- and ISPs rely on being able to oversell their lines.

Back in the day when the now-unpopular "web accelerators" were getting big, I always brushed them off as used by network abusers who didn't know what they were doing. Now this abuse has been legitimized.

The people who are going to be the real losers in all this are the techies, the ones who tend to have several browser windows loading at once, or a n ssh connection, or a server running. Up until now, they've been somewhat subsidized by the fact that ISPs can charge cheap prices because the other 98% of users only use their line 10% of the time. Now that everyone's lines are going to be under continuous load...goodbye Quake.

The entire idea of single window browsing is simply awful. It places extremely tight constraints on bandwidth and latency. When the user clicks a link, they want the new page there, now, and damn anything that has to be done to get it there. If you work with several windows downloading at once, so that you're reading one while another is coming in, you never run into this problem, since even a modem is easily enough to comfortably handle web browsing of nearly any site...as long as you're not waiting around staring at a progress bar while the image loads. Prefetching simply feeds this flawed single-window user-behavior model.

For once, a Microsoft program (IE) is actually less of a network abuser than its competitors. Awful.

Re:This is nothing

[jensend]

This is false. The Mozilla prefetching is only for pages which explicitly request to be prefetched by a or type construction- a slideshow, for instance, might use Moz's link prefetching (since the probability that someone will proceed to the next slide is rather high), but most sites won't.

Of course, they ideally ought to implement blacklist blocking for prefetching so people could exclude sites which use it in ways which affect network traffic adversely enough to be a worry, but my guess is that people won't start abusing it until IE does it as well.

I had the same feeling of shock when I first heard about it a week ago- until I read the FAQ. Remember- any large project like this is unlikely to make highly visible stupid decisions. You linked to the FAQ; please read it.

Re:The catch with rsync

[robin]

I thought that recent gzip compressed files were "rsyncable", in that the blocks they consist of are designed to remain as invariant as possible given the slight differences in content of the files. See, for example, this patch.