Paid Developers Power the Linux Kernel

Hugh Pickens writes "Believe it or not, there is still this illusion that Linux and open-source software is written by counter-culture, C++ programming cultists living in their parents' basements or huddled together in Cambridge, Mass. group-houses. Now CNet reports that the Linux Foundation has found that 'over 70% of all [Linux] kernel development is demonstrably done by developers who are being paid for their work.' That Linux is primarily developed by paid developers should come as no surprise considering that Linux enables many companies — hardware, software, and online services — to be more competitive in their markets and to find new ways to generate revenue. 'What's important about how Linux and open-source software is created isn't the side issues of politics or how its developers are perceived; it's that its fundamental methodology produces better software,' writes Stephen Vaughan-Nichols."

Comments

C++ programming cultists?

[razvan784]

It's written in C, not C++.

Re:C++ programming cultists?

[SigmundFloyd]

C++ programming cultists often write in C.

Those are C++ programming occultists.

Re:C++ programming cultists?

[TheRaven64]

The Linux kernel currently compiles with GCC, Clang/LLVM, PCC, Path64 (development branch only - some of the inline asm isn't handled correctly by the released version), ICC and (I think) XLC. An earlier version also compiled with TCC. It's not really that dependent on GNUisms. Or, rather, the GNUisms that it depends on are pretty well supported by other compilers.

I'm not surprised

[jimicus]

My employer uses F/OSS extensively - and as the sysadmin, I've started to notice a pattern.

F/OSS products which scratch an individual or a small group of peoples' itch generally get developed to a certain point and then stagnate. If you're lucky, that point is acceptable to you.

The products that do really well - the "best of F/OSS", if you like - are almost invariably the sort which scratches a very common itch. They're usually bankrolled by a number of companies (the Linux kernel falls under this category) or become self-funding when the project leader sets up a company to sell a commercial version with support and possibly extra features.

Re:I'm not surprised

[Bert64]

Well the same is true anywhere, the more potential users a given piece of software has the more interest there is in developing it...

Commercial software works the same way, something with mass market appeal is highly likely to be developed and either result in multiple competitors or serious effort to stifle competition...
But something small with a very limited market either won't exist at all, or will be extremely expensive if that niche market has the money to pay for it. Niche products also tend to be rather buggy.

OSS serves certain niches very well, ie those niches occupied by technically minded people who are capable of writing what they need for themselves... Other niches are served somewhat less well because those who would be capable of producing software have no need for it, and those who need it aren't capable of writing it.

You also get a lot of businesses and individuals using software which is extremely poorly suited to their needs because they aren't able to customise it for themselves, and the only people who would be capable of doing so aren't willing to.

I'm sure there are many things that could be improved by being computerized, but where the people capable of producing such software either don't realise or don't care.

Re:I'm not surprised

[jc42]

One real benefit is that if you are a company developing kernel code and contribute it back you will get goodwill and you will enhance the competence of your employees.

Actually, I've long thought it strange that the business/industrial world has an objection to supporting things like an OS kernel, runtime libraries, etc. The obvious parallel comes to mind: Lost of companies farm out part of their operations to subcontractors. They routinely subcontract for cleaning, delivery, electrical services, for example, not to mention their phone, water and sewage systems. They don't seem to be taken aback by the fact that the companies that supply these services also subcontract to "the competition".

The idea of paying a separate company for software development and supports services is also hardly new. That is how IBM has made much of its money, after all. Paying a company like Red Hat doesn't strike me as very different from any of the above. It doesn't take much management genius to understand that paying a contracting firm for software support at the "system" level is a fairly good idea. That way, you can share the cost with all the other companies that hire the same software firm, and everyone can get the benefits from having the software organized by people who (hopefully ;-) know what they're doing.

So why is this even a story? You'd think there would have been enough sensible businessmen all along for lots of Red Hats to prosper.

A related question is all the propaganda against "open" software. Systems such as water, sewage, electrical, etc. all have "open" designs, with everything published and the detailed specs easily available to anyone. Companies don't often buy fleet vehicles without shop manuals, which give the detailed specs for the innards of the vehicles. Why would people classify open software as "hippie" or "communist", when they don't say the same about shop manuals or electrical diagrams? You'd think that sensible managers who approve of open standards for these other things would also want software that follows published standards (e.g., POSIX), and whose specs (i.e., the source code) is easily available to everyone.

But for some unexplained reason, business people keep buying software systems with hidden, "proprietary" innards. They wouldn't do this with delivery vans or electrical wiring; why would they accept it with software? Exactly the same reasoning says that software should be open, standardized, and accessible to anyone with the technical training. And the same reasoning that supports specialized firms to do common tasks should also support specialized firms for software needs.

It may be yet another example of a theory that keeps popping up: Whenever a computer is introduced anywhere, all precedent is forgotten, and people have to relearn from scratch all the things that they knew from before there were computers. I wonder what it is about computers that causes this social amnesia and inability to see parallel situations?

Old News

[PiAndWhippedCream]

Do even the editors read anymore?

Not a total non-story

[Sycraft-fu]

This perception of OSS still remains rather strong. You will notice that a common advocacy for OSS is the "many eyes" thing. The idea seems to be that there are just tons and tons of developers out there with amazing amounts of time that will jump on a project and help, if only it was opened up. The advocates then point to things like Linux or Firefox or MYSQL and how great they are. What they miss or don't understand is that these high profile, top-flight OSS projects have heavy financial backing. They have developers who's full time jobs it is to work on the software. That's wonderful and all, but don't then try to claim that you'll get that kind of development on a project just because you open up the code.

Re:Not a total non-story

[Attila Dimedici]

Actually, most of the people who miss that are not listening closely enough. The point of the "many eyes" thing with Open Source software is that if an OSS project is of interest to a company (in particular a company that hires programmers anyway) that company can have its people look at the software from the perspective of the way that company uses the software and they can make changes (either bug fixes or otherwise) to make the software better for their company. If they were using proprietary software, they would be at the mercy of the company that owned the rights to that software for any fixes or improvements.

Re:Not a total non-story

[cduffy]

Many companies have learned the hard way the true cost of custom software, which in many ways can be worse than proprietary software.

If you're making custom software out of your OSS software, you're doing it wrong.

If you're doing it right, you're submitting your changes back upstream -- so the software doesn't "stand dead-still", as you put it, even on those times when you aren't shoveling man-hours into improvements. If you happen to be curious for some examples, google around for patches under my name submitted to open source projects over the last decade. Just about all of those were paid for by my employers -- from the OS X VNC plugin bugfix to the feature enhancements to libvirt to improved cover page generation for HylaFAX.

For the work I did at Dell, we worked together with Red Hat to get as many of the libvirt and qemu improvements we wanted as possible into the RHEL6 release schedule, enabling some of Dell's internal QA tools to work out-of-the-box with RHEL6 (whereas those same tools required heavy tweaking on RHEL5). Sure, we could have gotten the same thing done as a professional services engagement rather than a friendly collaboration between engineering groups... but this way was far easier, cheaper and lower-paperwork (and by building the patches in-house, we made sure that we got exactly what we wanted).

No surprises in this article for me.

Not sure about the conclusion

[Registered Coward v2]

No, what’s important about how Linux and open-source software is created isn’t the side issues of politics or how its developers are perceived; it’s that its fundamental methodology produces better software. That’s why businesses invest in Linux’s development. Linux works. If it didn’t, big business wouldn’t bother with it.

I am not sure the methodology produces better software; it does however produce software companies can use and modify to their needs without paying ongoing license fees. That's why they use it - it allows them to develop other, higher-value, products and maintain control over the source and not be beholden to some third party company. The methodology does work to produce high quality software since many people are looking for bugs, with that part I agree - but companies don't invest in software because others are QC'ing it; they invest because it enables them to make money. While what the author says is, IMHO, true about the Linux kernal, I don't think it is true for many other FOSS projects; unless they are used as part of a larger product, such as a server. There simply isn't the same incentive to spend time and money on an application, such as an office suite, that you can't use to sell something more profitable.

I'm one of those paid kernel developers

[Izaak]

I've been working as an embedded Linux developer for almost a decade now, and yes, most of us kernel hackers are paid for our work. For example, right now I'm working for a major microchip manufacturer that wants to make sure their products are fully supported by Linux. Consequently, they fund teams of open source developers (often hired through big name consulting firms) to port the kernel to their latest CPU's, develop drivers for integrated peripherals, etc. Just look at the email addresses in the submit logs for major open source projects. You will see ibm.com, intel.com, ti.com, redhat.com, windriver.com ..., and many, many more big commercial technology companies. Its been this way for a while, which is why I would always laugh whenever some MS fanboy would try to denigrate Linux programmers as a bunch of basement dwellers. I make a better than average living from Linux coding, with multiple job offers right now, even in this horrid economy.

This is also why I have no worries about Microsoft ever killing off Linux. There are far too many companies making far too much money from Linux based products in market niches that MS has no traction in. The embedded and mobile markets are pretty much owned by Linux, and those are pretty much the only tech sectors seeing strong growth right now. If you haven't yet added Linux skills to your resume, do it.

If anyone wants to ask me about the Linux / embedded / open source consulting world, go ahead and post your questions. I'll check back and answer if I can.