Slashdot Mirror
Calling for Smaller Kernel Sources?
FrozedSolid asks: "I can understand that the kernel contains many drivers and support for a lot of platforms, but the fact that the full kernel download can amount to 32mb doesn't make it any easier to download with a 56k modem. Kernel patches are nice, but obviously only apply when you have access to an entire kernel tree beforehand. Is there a way you can download a leaner linux kernel source? Is there a place that carries sources for x86 only or possibly sources without some of the less popular drivers?"
If you haven't gotten cable, and you're using Linux, the distros themselves are at a magnitude greater in size; I doubt that kernel sources are the real problem.
Slashdot: Where people pretend to be twice as smart as they really are by behaving like children.
Part of this has already been answered. If you just pick x86 (or PPC or Alpha etc.) the size does not change that much. The vast majority of the kernel is not architecture specific. That's a good thing!
I don't know of any sites, but let me say a few things. First, your distro probably has a binary package with almost everything either compiled in or a module. Barring that, when I used to be stuck on dialup, I'd get the most recent kernel and then download the patch each time. It was a pain in the butt, but not as bad as downloading the full sources each time.
Jesus was all right but his disciples were thick and ordinary. -John Lennon
It was devised to combat just the problem you cite.
rsync://rsync.kernel.org/pub/[wherever you want to go]
Thank you, TRIDGE!
"Flyin' in just a sweet place,
Never been known to fail..."
i have dialup as well and i got Download Accelerator Plus (DAP) it allows you to start a download overnight, and it will disconnect for you, whenever the download completes, this, along with a pause\resume feature make it invaluable for slow access over the net.
Currently gentoo has a kernel-source package (actually, several) but they don't really get much out of portage/ebuild. If Gentoo had USE support within the kernel, and seperated the sources into PPC/x86, driver, etc. parts, it would be almost like a normal gentoo package and almost like what you're looking for.
Seems like a person could set up a few webservers, let people select kernel configuration options, and send the much smaller bzImage (and compiled modules) through email. Sure, the size would vary wildly based on how many modules were selected, the architechture, etc., but on average I'd say it would be much smaller.
The benefit is less bandwidth wasted for people downloading 35 megs of source to recompile a 900K kernel image. The disadvantage is processor time required, well, how many Athlons do you have to buy to serve the same number of kernels per day, and how does it compare to bandwidth costs?
Yes, *I* would like to have the sources to myself, I have a few source files I need to tweak to get my machine working properly. But many people just have the burned CD from the friend of a friend, and would appreciate a recent kernel without a mammoth download.
Maybe someone's already doing this, I don't know.
...
Never under-estimate the bandwidth of a plane full of CD's... ...or the CD-RW drive at your nearest Internet Cafe.
The last time I was poking around the kernel source SRPM for a Mandrake install, the stock linux-2.4.? tarball was in there (along with all the other patches they apply to it).. Your distribution may have the same thing. Unpack the SRPM to get a linux tarball, unpack that to get the source tree, then download and install the patches to bring it up to the version you want to run.
Granted, this doesn't answer your question, but it may ease the download times a bit..
I don't think making a smaller kernel tree just for those people who unecessarily wants to be bleeding edge, don't like CD distributions, don't like downloading patches, won't or can't have broadband at home, and doesn't have access to broadband at a friends place, library, workplace, school or anywhere else is worth the trouble. But if you feel otherwise, just go ahead and make it.
Incredibly, Mozilla 1.2 is going to have built-in, enabled-by-default prefetching. The amount of bandwidth this will waste blows the mind. Imagine every single Joe on the Net suddenly using up 20 times as much bandwidth downloading stuff that he will *never see*. The intermitent activity on lines turning into constant load -- and ISPs rely on being able to oversell their lines.
Back in the day when the now-unpopular "web accelerators" were getting big, I always brushed them off as used by network abusers who didn't know what they were doing. Now this abuse has been legitimized.
The people who are going to be the real losers in all this are the techies, the ones who tend to have several browser windows loading at once, or a n ssh connection, or a server running. Up until now, they've been somewhat subsidized by the fact that ISPs can charge cheap prices because the other 98% of users only use their line 10% of the time. Now that everyone's lines are going to be under continuous load...goodbye Quake.
The entire idea of single window browsing is simply awful. It places extremely tight constraints on bandwidth and latency. When the user clicks a link, they want the new page there, now, and damn anything that has to be done to get it there. If you work with several windows downloading at once, so that you're reading one while another is coming in, you never run into this problem, since even a modem is easily enough to comfortably handle web browsing of nearly any site...as long as you're not waiting around staring at a progress bar while the image loads. Prefetching simply feeds this flawed single-window user-behavior model.
For once, a Microsoft program (IE) is actually less of a network abuser than its competitors. Awful.
May we never see th
rsync is great when what you are downloading is a newer version of something old and is uncompressed (ISOs tend to work quite well). Unfortunately, things like a compressed kernel tarball (or an RPM) seem tend to compress differently each time leaving relatively little in common causing the speed up from rsync to be very small (if there at all).
Just an FYI for people getting into kernel stuff with RedHat-ish systems:
Getting Linux via bitkeeper.
Also don't forget.
-
- make install doesn't work with grub, so you have to do your thing manually now
- recommended compiler is gcc-2.9.5.3 [for 2.4 and 2.5 now], I always have extra compilers ready to go just in case. Make sure all the tools are the proper version, and that you have a recent ksymoops (if you need to do any messing around looking for problems ), modutils - etc.
If the build fails, find the offending code and remove from selection, or try to hack it if you need it.
I would like to also mention cvsup and FreeBSD. I like cvsup quite a bit and its free and open. I only wish the linux kernel was using the same method FreeBSD does. I like FreeBSD for its coherency speed and ease of maintenance, and that the kernel is released with a system for a very smooth ride. If you havent tried FreeBSD, please try it.
rsync is also very good. use it. I would also like to promote the purchasing of very cheap CDroms to get your started, and FreeBSD CD is great because you can use CVSUP to diff the whole thing with minimal bandwidth abuse.
Legalize the constitution. Think for yourself question authority.
download once
use the patches for incremental upgrades
problem solved
don't forget to install the Trojan code, too.
It's a good idea, but how do I *trust* that webserver that just compiled a new kernel for me? I trust kernels from the disto, kernels I build, and maybe a few other places, but that's about it.
The living have better things to do than to continue hating the dead.
I've been thinking about this same problem for a while. Especially before I had broadband.
Checking the posts, I see someone mentioned a solution similar to what I am thinking about, but for binary kernel downloads. Whilst nice, I think few people actually trust a binary they didn't build themselves.
I've been wondering why kernel.org didn't create a download configurator somewhat like the one that existed for djgpp in the delorie days.
The poster is right in a sense, there are a multitude of drivers in the kernel that a large percentage of people will never use, but which are still invaluable for the people who do. So why not allow a custom download somewhat like make menuconfig. Select the packages you *could* end up building, download them, and then build your kernel on the local machine.
Given that I have absolutely zero knowledge about the kernel source, it's highly likely that I am overlooking something rather basic... is there something I am missing?
Video meliora proboque deteriora sequor - Ovidius
Has anyone considered the possibility of modular kernel sources? Break the sources into several modules. Things like kernel-base, kernel-scsi, kernel-reiser. Download the ones you need. extract all to the same location. things like `make menuconfig` would scan for your "modules" and give you options based on which modules you have extracted. Is this a possible method to making smaller downloads?
What about similar to what the MSIE and Netscape installers do - choose what parts you want to download...? Have the equivalent of a 'make menuconfig' that you run on your local machine, which will then just download the required/selected source, or even better calls some kind of CGI that custom packs your source tree into a tgz file for you?
I think 'make depend' may kill it tho... unless that information can somehow be downloaded ahead of time also.
- RR
I should put something clever here. Maybe someday.
i dont think it should be left up to third party people compiling sources... why not go to kernel.org either upload your config or enter everything you need and download just the source you need... however all this could be avoided if people would read the changelog and realize they dont need to upgrade to the latest greatest kernel coming out, especialy when none of their items have had any changes at all
-- botsex is {grep;touch;strip;unzip;head;mount}
You are correct, but you made one bad assumption -- that people designing web pages are interested in responsible use of the network.
You've got a web designer who can "improve his user experience" by marking everything as prefetchable -- what do you think is going to happen?
And the overwhelming majority of web designers these days use GUI tools. After word gets out that websites designed with tool "foo" are snappier (because it uses prefetch by default), and it becomes a selling point...
After all, what web designer wants to believe that people won't delve deeper into his site, and hit those pages *anyway*?
May we never see th
You took the words right out of my mouth. I *always* use wget, even on Windows. Cygwin and wget are godsends. No need for spyware on my boxen.