Slashdot Mirror


How Google Uses Linux

postfail writes 'lwn.net coverage of the 2009 Linux Kernel Summit includes a recap of a presentation by Google engineers on how they use Linux. According to the article, a team of 30 Google engineers is rebasing to the mainline kernel every 17 months, presently carrying 1208 patches to 2.6.26 and inserting almost 300,000 lines of code; roughly 25% of those patches are backports of newer features.'

11 of 155 comments (clear)

  1. Release the patches already by Dice · · Score: 5, Interesting

    They monitor all disk and network traffic, record it, and use it for analyzing their operations later on. Hooks have been added to let them associate all disk I/O back to applications - including asynchronous writeback I/O.

    I. Want. This.

    1. Re:Release the patches already by Anonymous Coward · · Score: 5, Informative
    2. Re:Release the patches already by Anonymous Coward · · Score: 5, Funny

      Can we donate some money and buy these people a site that _doesn't_ look like a goatse link?

  2. Re:Is it worth it? by Rockoon · · Score: 5, Insightful

    This company had about a million servers last time I cared to find out. I dont think 'more cheap hardware' means the same thing to you as it does to Google.

    --
    "His name was James Damore."
  3. Low memory conditions by jones_supa · · Score: 5, Interesting

    Google makes a lot of use of the out-of-memory (OOM) killer to pare back overloaded systems. That can create trouble, though, when processes holding mutexes encounter the OOM killer. Mike wonders why the kernel tries so hard, rather than just failing allocation requests when memory gets too tight.

    This is something I have been wondering too. Doesn't it just lead to applications crashing more often than them normally reporting they cannot allocate more memory?

  4. Re:Is it worth it? by dingen · · Score: 5, Interesting

    Ooooh... efficiency.. I'm curious what the net savings is.. compared to buying more cheap hardware.

    We're talking about Google here. They have dozens of datacenters all over the globe, filled with hundreds of thousands of servers. Some estimate even a million servers or more.

    So lets assume they have indeed a million servers and they need 5% more efficiency out of their server farms. Following your logic, it would be better to add 50,000 (!) cheap servers which consume space, power and require cooling and maintenance, but I'll bet you paying a handful of engineers to tweak your software is *a lot* cheaper. Especially since Google isn't "a project" or something. They're here for the long run. They're here to stay and in order to make that happen, they need to get the most from their platform as possible.

    --
    Pretty good is actually pretty bad.
  5. Re:Open source is the coat tails that Google rides by IamTheRealMike · · Score: 5, Insightful

    Hmm, you realize that Android alone is over 10 million lines of code right? That's a pretty big open source contribution right there. But then there's also over a million lines of code across 100+ smaller projects too. So I am not sure what your definition of "table scraps" is but it's significantly more lines of code than most companies do.

  6. Re:Is it worth it? by Rockoon · · Score: 5, Insightful

    Also consider the fact that Google has been basically deploying new servers non-stop for many many years. They are already purchasing cheap hardware at a very high rate. Even a tiny 1% improvement in efficiency for the existing and future servers is a huge huge win for them.

    That could amount to hundreds of millions of dollars saved over the next decade, and it doesnt take a genius to realize that a couple dozen programmer salaries will be a hell of a lot less than that.

    --
    "His name was James Damore."
  7. Re:Does Google give coade back by MBCook · · Score: 5, Informative

    Yes, they do. Since they use older kernels and have... unique... needs, they aren't a huge contributor like RedHat, but they do a lot.

    During 2.6.31, they were responsible for 6% of the changes to the kernel.

    --
    Comment forecast: Bits of genius surrounded by a sea of mediocrity.
  8. Re:Togh by pathological+liar · · Score: 5, Insightful

    Yeah great work Linus.

    The distros STILL stick with older versions and backport fixes, because who in their right mind is going to bump a kernel version in the middle of a support cycle? It's even MORE broken because the kernel devs rarely identify security fixes as such, and often don't understand the security implications of a fix, so they don't always get backported as they should.

    The Linux dev model is NOT something to be proud of.

  9. Re:Togh by grcumb · · Score: 5, Funny

    The Linux dev model is NOT something to be proud of.

    Indeed:

    "The Linux dev model is the worst form of development, except for all those other forms that have been tried from time to time." - Winston Churchill

    ... Oh wait, no. That was me, actually.

    --
    Crumb's Corollary: Never bring a knife to a bun fight.