Slashdot Mirror


The Linux Kernel Archives Gets Major Update

hpa writes "The Linux Kernel Archives, kernel.org, has gotten a major facelift! After suffering with insufficient bandwidth for far too long, we are now operating with a dedicated 100 Mbit/s connection from Globix. VA Linux gave us a really nice new box to run it on, too; so it is really a wonderful setup we have gotten. Not only should this resolve the bandwidth shortage, but hopefully we'll be adding new services soon. We have already added anonymous rsync services for the benefit of unofficial mirror sites. I'd appreciate hearing requests for new services on kernel.org, just email me. "

36 comments

  1. Re: a REALLY generous donation from Globix by Mr.+Gone · · Score: 1

    Keep in mind that Globoix donated a rack and the 100Mb/s connection. This is a non trivial expense and it signifies a willingness on their part to play nice with the Linux community.

  2. Wow, they weren't kidding by Poisoned+Coyote · · Score: 1

    Even though its undoubtedly being slashdotted as we speak, it loaded before I could blink.

  3. 100 Mb/s? by Eccles · · Score: 2

    Man, you're going to need a quad-processor host with 4 ethernet cards running NT to keep up with that bandwidth... :-)

    Oww! Stop hitting me with that mackerel!

    --
    Ooh, a sarcasm detector. Oh, that's a real useful invention.
  4. Re:OT! Transmeta. by chris_se · · Score: 1

    from the transmeta.com html source:

    "
    There are no secret messages in the source code to this web page.
    There are no tyops in this web page.
    "

  5. Rsync on gzip by Anonymous Coward · · Score: 1

    It is true that rsync does not work effectively on gziped data? I heard something about some guys at our lug wanting to get a change into the debian version of zlib, so that rsyncs from debian would be faster and use less bandwidth. Apparently the change would be compatible with the existing gunzip algorithm, but the gzip algorithm would be changed so that if it generates the gziped file in chunks. The way it is now, if you change one byte in the data that is to be compressed the whole gzip file is changed and so does not work well with rsync. The new method would result in slightly larger files however. Can someone confirm or deny this? Add some more details? Seems like a worth while change if this is correct.

    1. Re:Rsync on gzip by Anonymous Coward · · Score: 1

      Unfortunately it's true, gzip kills rsync very badly. I've often found that decompressing the file on both ends actually speeds up the transfer!

      The compressed chunk idea sounds interesting. I'm not certain if it would work, but it would be nice. A similar alternative would be a block compressor back-end to rsync that ungzips the source, block-compresses it, then sends/compares that over the wire. On the other end, the same thing could happen, gunzip block-compress, compare. A severe drawback to this is it's extremely CPU intensive so I think chunk-gzip would be better so long as the "slight" size increase is indeed small. (I don't think your typical user would appreciate a significant packing size increase to help out rsyncers.)

    2. Re:Rsync on gzip by hpa · · Score: 1

      That's because rsync is trying to do the compressing itself, obviously redundantly. You should have a "dont compress =" directive in rsyncd.conf for the appropriate files, to avoid this problem.

      In other words, it's not a problem for a properly configured rsync site.

  6. Re:Pre-built kernels? by pwhysall · · Score: 1

    When I first hit reply, I was going to say something along the lines of "Don't be silly" then I stopped and thought a bit and there's no reason for this not to work, because of kernel modules.

    You'd have to be able to pick a kernel from a list, depending on your architecture, but after that, compiling half-a-dozen kernel modules isn't going to kill your elderly 486, is it?

    In my perfect world, the ONLY time you'd ever recompile the whole thing (or download a new one) was when the version changed enough for you to want/need it.

    Can any kernel gurus tell me how feasible it would be to restrict the kernel compilation stage to new modules?
    --

    --
    Peter
  7. Thank you to VA by suprax · · Score: 1

    I think once again we owe a big thank you to VA Linux for providing yet another non-commercial site with the hardware they need to successfully provide their service for the number of hits they receive. I can't name how many times we have seen this, and it's just great that it's going on. A true sense of the linux community spirit. :)

    --
    Scott Miga

  8. Check out Transmeta's site... by Jerenk · · Score: 2

    I noticed on the new Kernel.org site is a link to Transmeta. It seems that they updated the site. Cool.

    No Secret Messages Here - Uh-huh.

    Later,
    Justin

    --
    Mu. P.S. The address you see is real. =)
  9. Re:VA is so cool by Bowie+J.+Poag · · Score: 1

    Agreed. VA has a long track record of being very, very generous when it comes to helping out the community. I can attest to that, personally..both of the projects i've managed have been sponsored by VA in one form or another.

    All helpfulness aside, it actually makes sense for them to be so generous. By doing so, they strengthen their foothold in the market, increase their name recognition in the industry, and get to enjoy a return on their investment once these various projects bear fruit. By sponsoring all these community projects, theyre ensuring their own survival in what may turn out to be a horribly competitive server market in a few years. Its damn good business sense, if you ask me.

    Keep in mind, tho, VA isn't a charity organization. Theyre a business, like any other..and pretty soon a board of directors will be calling the shots. Lets hope they see the same benefits. :)

    Bowie J. Poag

    --
    Bowie J. Poag

  10. Interviews ? by Sylvain · · Score: 1

    I would love to find interviews of kernel hackers each 2 weeks or each month. They would explain us the state of their work, why they did such or such a choice in their code, what they plan to do and how we can help.

    Maybe it would help to find new hackers if the actual guys speak about their work.

  11. Re:OT! Transmeta. by Anonymous Coward · · Score: 0

    submitted this the other day....
    http://www.tbtf.com/archive/current-issue.html#s01
    basicially says transmeta is gonna try to get around intels patents on the x86 arch. also says that the win2k dev team has advance knowledge of transmetas product(s).


    geoff

  12. Re:Nifty page. by Utter · · Score: 1

    Yes, download the patch.

    And instead of make menuconfig or make config you should do a make oldconfig, which only asks for new options.

  13. How much does it cost? by Tet · · Score: 2

    Just curious... how much does 100Mb/s bandwidth to the Internet cost? It's not gonna be cheap (at least, if the prices of 2Mb/s leased lines in the UK are anything to go by).

    --
    "The invisible and the non-existent look very much alike." -- Delos B. McKown
  14. no more all nighters... kinda by invictus · · Score: 1

    Neat, now I don't have to worry about the DL time of those kernal sources.

    make dep; make clean; make bzImage

    p.s. I find it amusing the little, Operated by Transmeta, all the way at the bottom.

    --
    --Ks9
  15. I really didn't notice by lubricated · · Score: 1

    I really didn't notice that is untill I got a cablemodem. Then it makes sence. Downloading a new kernel used to take forever. The world is much difrent when you think 10k/s is 30 times to slow.

    --
    It has been statistically shown that helmets increase the risk of head injury.
    1. Re:I really didn't notice by dalraun · · Score: 2
      Regular users like you and I should be using a mirror, like ftp.us.kernel.org.
      Click here for more details on the kernel archive mirror system.

      As a side note, I still haven't heard a reasonable explanation for how and why there is a kernel mirror in Antarctica.

    2. Re:I really didn't notice by hpa · · Score: 1

      If you'd looked closer you'd notice there isn't actually a mirror *in* Antarctica... just a mirror who has volunteered to handle Antarctica traffic. I doubt it adds significant to their bandwidth consumption :)

    3. Re:I really didn't notice by jfunk · · Score: 1

      Funny, it's located in Canada.

      Wrong pole, guys :-)*

      Actually, the FTP login message explains most, if not all.

      (hey, the +1 bonus is back! I guess I'll celebrate by hitting the "No Score +1 Bonus" button below)

  16. This explains something... by Unknwn · · Score: 1

    I had wondered why my logins to kernel.org recently had been suceeding. In general, I try to use mirrors, but when I know that a pre-patch has been released very recently (aka in the space of the past hour or three), the chances of mirrors having it are pretty small :(

    Anybody know of any good mirrors that update on a very regular basis, or even better, are push-updated?

    --
    Jeremy Katz

  17. They should link to more sites... by tilly · · Score: 3

    They can start with my favorite kernel site, Kernel traffic. If you want to have a reasonable sense of what is going on with the kernel but don't want to follow the mailing list - then visit this site every week.

    By and large the sorts of services that people need are already available. They should recognize that, list a few, and then move on.

    I would say that some advice on kernel programming would be good. Sprinkle said advice with links to a few of Torvalds' rants on sending patches. :-)

    Cheers,
    Ben

    --
    My usual seat in the cluetrain is at A HREF="http://pub4.ezboard.com/biwethey.ht
  18. Nifty page. by pb · · Score: 3
    But remember, guys, when slashdot announces the next super-duper must-have kernel update early, and the ftp site is swamped, relax, take a deep breath, and...

    Check the mirrors!

    Download the patches!

    ...because more bandwidth never means sufficient bandwidth.
    ---
    pb Reply rather than vaguely moderate me.

    --
    pb Reply or e-mail; don't vaguely moderate.
    1. Re:Nifty page. by sporty · · Score: 1

      One day linux mirrors should implement cvsup servers. Good compression, you don't download everything, only changes. Most people would be doing so after an initial version.

      Only people who want to download the entire thing will (by deleting the kernel source dir).

      Its also great for going back in history for kernels and downloading those too.

      Yes, I know most people will think *bsd. But linux can borrow this as well. Heck, they borrowed the entire concept of unix by cloning it. heh

      --

      -
      ping -f 255.255.255.255 # if only

  19. this will be good for when 2.4 comes out by didjit · · Score: 1

    its great that kernel.org has more bandwidth so that it can handle more traffic. now, when the new kernel comes out, more people will be able to get it quicker. although it may not be as revolutionary as 2.2, 2.4 should bring a lot more people onboard and the linux community will continue to grow. personally i can't wait to log onto kernel.org and get 2.4. good job, all of those who run the site.

  20. OT! Transmeta. by j+a+w+a+d · · Score: 1

    I clicked on the "operated by transmeta" link, and it took me to the updated transmeta page. how come this wasn't a slashdot story!?!

    instead of saying "This web page is not here yet!" it says "This web page is not here yet! ...but it is Y2K compliant. "

    (Hey! You moderator! I said it was offtopic in the subject!)

    --
    i dont display scores, and my threshhold is -1. post accordingly.
    Discuss /. policies
  21. Guess this means... by paleck · · Score: 1

    That I will be able to retire my cron script that would wait until 4am to download the latest kernel.

    Now I'll be able to get a new kernel anytime I want!


  22. that site deserved more since a while by Anonymous Coward · · Score: 0

    good. after all the kernel is actually all that linux os is. it definately desrved some serious attention for a while now.

  23. VA is so cool by mplex · · Score: 1


    It is so cool how VA helps out as much as they do. Is it just me or do they donate to everything linux be it boxes or bandwidth. I bought my last box from them and while a little on the expensive side, they used the absolute best parts and when my power supply died there was another one waiting for me in the morning without a hassle. Think about it though, half the linux sites seem to have some connection to VA. Any other thoughts on them. I mean maybe it is a little PR but I dont even care.

  24. Re:Pre-built kernels? by uncleFester · · Score: 1

    Is the 486 the only machine you have? I've pondered building 3/486 kernels on my celeron box using the cpu-specific flags in config (ok, menuconfig.. sue me). I'd think theoretically it should work.. but I'd at least make damn sure the compilers match (or were at least the same vintage) between boxes should I build add-on modules later in the future.

    --
    -'fester
  25. Antartica by Forge · · Score: 1

    You do realize that Penguins live in Antarctica right ?

    --
    --= Isn't it surprising how badly I spell ?
  26. Re:Pre-built kernels? by StarFace · · Score: 1

    Actually, distributions sometimes offer just what you are looking for. I know, off the top of my head, that RedHat and Debian both have "packages" that have generic images of the latest kernels. You'll have to be proficient with handling modules however because the typical strategy is to include as much as possible modular to satisfy the masses. I know that the debian kernel-image packages are very slick, they install all you need to safely boot up with the new kernel without being too risky about it and allowing you to return to your old kernel if things go bad.

    --
    V
  27. VA is so cool -- I agree by Ken+Broadfoot · · Score: 1

    The really do know how to make friends in the Linux Community. I bet there will be far less "static" on their inevitable IPO than Redhat had on theirs.

    --
    Bitcoin pyramid: Join here: http://www.bitcoinpyramid.com/r/1427 it's FREE!
  28. bzip2 and rsync? by hab136 · · Score: 1
    "bzip2 compresses files in blocks, usually 900kbytes long."
    http://www.bzip2.org/bzip2/docs/ manual_2.html#SEC8

    This would seem to suggest that it would work better than gzip (without the above-mentioned change to the gzip process). Rsync would only have to send the updated ~900k block, instead of the whole file (and part of that block may be identical). Still not ideal, but the best thing for now. bzip2 compresses better anyways..

  29. Pre-built kernels? by Rayban · · Score: 2

    What about offering some sort of tarball of pre-built kernels for some of the newer kernel versions? My poor 486 can barely handle the strain of compiling 2.0.3x. I'm really wary about moving up to 2.2.x, though I'm not really sure that would be a good idea anyways. I mean, if it's not broke, right? ;)

    I wonder how much of the kernel could be precompiled anyways for those of us who like to be on the cutting edge, but hate taking those 5 minutes out to recompile the latest unstable kernel. Oh well.. I can always use that time to get a cup of coffee or something. :)

    --
    æeee!
    1. Re:Pre-built kernels? by TRyanC · · Score: 1

      I"ve been doing this for a while. I've got a 386 (25MHz, 6 MB RAM) and a 486 (33 MHz, 16 MB RAM) both running 2.2.5. I compiled their kernels on my PII 450 in an NFS exported directory, doing the make bzImage and make modules on the PII, then did the make modules_install and copied over the zImage (or bzImage, I forget which I used) on the 386/486.

      They've been up for months since then, with no problems.

      Out of curioisity, does anyone know how long it would take to compile a 2.2.5 kernel, and about a dozen modules, on a 25 MHz 386 with 6 MB of RAM?

      --Ryan Cleary