Has Linux Development Become Too Political?

1010011010 asks: "Has Linux development become too political, bottlenecked and ego-driven? Witness the recent exchange between Hans Reiser, of ReiserFS fame, and Alexander Viro (VFS maintainer) on Linux-Kernel; Hans, and others, were griping about Viro refusing patches and ideas on principle, and Viro keeps telling people to shut up and read the code. It's obvious that the kernel needs better documentation and fewer penis-size contests, but it also needs better 'roadmaps' and plans -- i.e., documentation in advance of changes. Does anyone have a current "org chart" for Linux kernel development? Is Linux Kernel Development sustaintable as a coalition of little feifdoms?" Such things are concerns, but sometimes it can be that very ego that drives a project. Behind technology, there are the politics of that technology. Being human, is there any way we can avoid this? Should we?

Comments

If you want to read up on the situation yourself

[Anonymous Coward]

Start here.

(Posted anonymously to avoid karma whore allegations)

Concurrent systems, docs, and MMU guarantees

[Morgaine]

It's going to take more than just good documentation to keep Linux from developing out of control and spiralling down towards instability. Most non-trivial software heads in the general direction of loss of control as it grows, but operating systems are specially vulnerable to this syndrome because they are highly concurrent systems, and concurrency offers opportunity for problems to emerge with the greatest of ease.

For Linux to remain stable, one of two things are going to have to happen. Either extremely heavy-handed personal control by people with a supreme grasp of everything that's going on in the kernel and who won't be deflected from their straight and narrow path --- that's where we are now, so be careful not to throw the baby out with the bathwater when knocking it. Or, fight the problem with technology and abandon the single monolithic unprotected space approach and partition the kernel into a large number of interacting but separate domains using the MMU plus well-specified and relatively stable internal interfaces. Needless to say, the second of these approaches would result in an utterly different kernel altogether, but at least it would have a longer life expectancy.

The current kernel is as brilliant as it is because of the brilliance of the people that are keeping it from falling apart under force of change. But there is a limit to human intellect, at least in the current pre-nanotech timeframe. The current developers should accept that, and work towards a kernel design that reduces dependency on their brilliance by providing an effective assortment of hardware-assisted guarantees. Learn from the user-space design of Unix. It's a good model.

It never happened

[Rik van Riel]

At least, while there were some overheated email going in both directions, there never were political arguments involved there's no reason to fear that Linux development will fork or halt as a result of infighting.

Truth is that in the heat of the email thread both sides wrote up some things that just weren't there. And after yesterday's invention of the "VFS flamefest" here on Slashdot, it seems logical that some of the people who weren't on linux-kernel but were on Slashdot yesterday get a bit worried.

Well, I guess this is how rumours come into existance :) Lets forget this whole thing and get back to coding like we always do...

Read the exchange for yourself...

[artdodge]

I encourage Slashdot readers to look up the recent exchanges between Hans, Al, Alan Cox (all-over-the-kernel guy), Rik van Riel (VM guy), and others, and form your own opinions about the involved parties.

Calling the kernel a "coalition of little feifdoms" is wildly inaccurate, because as the core systems of the kernel become more integrated with each other (think f.e. of the tangled web of interaction between VM/VFS/buffer cache/dcache/icache/swap cache/etc...), the "heads" of each work more and more cooperatively. Sure they still flame each other (time was, being called "pinhead" in linux-kernel was a mark of high esteem), but the amount of respect and degree of deference shown to each other's expertice is truly admirable.

Of course, politics show up every now and then... there are still people who think DevFS integration is the seventh sign of the coming of the antichrist. But even those who vehemently opposed it are now helping to make it really good by adapting the dcache and VFS to work better with it.

IMVHO, Hans made something of a spectacle of himself on the list. He ran around accusing everyone he could find of being corrupted by corporate influences (RedHat was his favorite target, since Alan and SCT of ext3 fame both work for them), while simultaneously admitting he had a strong financial incentive to get ReiserFS integrated with the kernel.

Of course, there are two sides to every story; Hans has been working on/with this thing for a long time, and has sunk a lot of his own blood, sweat, and tears into it, so I don't blame him for getting a little fanatical about it now and again. :-)

It's our right to make noise

[SurfsUp]

Making noise is one of the things that keeps the open source system healthy. I think that in general the Linux kernel has had good stewardship, and though I personally have not had direct interactions with Al Viro, I think that extends to him as well.

On the other hand, I think the system could be improved a lot. There are some annoyances. For example, there is little, if any, documentation available on the one of the most critical aspects of the kernel, namely the buffer cache component of the VFS. It seems that you are expected to either learn about it yourself by reading every post on Linux Kernel, and every line of code in the filesystem tree, plus all patched versions. If you get stuck, you can play 20 questions on the kernel list, trying not to appear too clueless and at the same time trying not to appear so clueful that you will get flamed on grounds that you should have figured it out yourself. If your post to the kernel is phrased correctly then one of the VFS gurus will answer it - usually clearly and accurately, but not necessarily completely. And the game goes on.

The good news is that you *can* get the information you need. The bad news is that you are expected to jump through a lot of hoops to get it, and the information seems to be dolled out as a kind of payment for playing the game correctly and politely.

I think this should change. I think that the design documentation we need should be readily available - it has to be posted somewhere where everybody can get at it, and contribute to it. There's already a place for it: kernel.org. Why isn't there more there than just kernels and patches?

(The Linux Doc project is a good source of documentation, but not for in-progress kernel work.)

There's no shortage of people who would be willing to do this kind of documentation work if they were able to. Right now there seem to be some roadblocks in the way.
--

When you're the maintainer, it's your call.

[Bedemus]

It seems to me that the issues above shouldn't even be referred to as political -- it can't really be compared to politics because a maintainer is granted absolute control over what's officially in a piece of software, he or she has to be in order to effectively do their job without a huge bureaucracy getting in the way of decisions being made in a timely manner. Since that involves making judgement calls based on one's own opinion (though hopefully taking other viewpoints into account!), there are bound to be cries of "unfair!" from the development community at times.

I'm the author of an open source project entitled NeoMail. As an aside -- yes, my real nick is Neo, not Bedemus, everyone started calling me that after I spent way too much on a Neo outfit for my work's halloween dress-up day and started wearing an ankle-length trenchcoat during the colder months -- I know, I'm a loser. :) When I set out to design NeoMail I had a certain audience in mind, essentially the same one as all open source authors when they start out -- myself.

Now, once I had something that worked pretty well for me (I had poured at least a month or so of effort into it at that point -- was taking on the project just to see if I could, teaching myself Perl in the process), I posted the code on the web and announced its release on Freshmeat. Immediately, the mails started pouring in -- some bug reports, which are always appreciated -- but more and more suggestions as to what people would like to see. Most of these suggestions were from non-programmers, and therefore included no patches or anything, just opinions as to where NeoMail should head.

I quickly realized that I'd better start narrowing the scope of NeoMail beforehand, otherwise I'd NEVER reach 1.0! A few things were decided up front:

• NeoMail would require no extra software be installed, aside from the necessary Perl 5 distribution and included modules, a web server, and an MTA.
• NeoMail would not be designed to do POP3, IMAP, etc. There were plenty of solutions doing this already, and it would have been easier to use Net::POP3 than to code local mbox spool handling myself, but it just wasn't the direction I wanted to head in. NeoMail served a certain niche quite well, and a chief benefit was that it didn't require users to even have access to a real system account for it to work.
• Plenty more -- you get the idea. I'd list everything here but how do I know people have even read this comment this far? :)

Next, the arguments started.
"Why don't you use thus-and-such module to add thus-and-such feature?"
"Because NeoMail doesn't need that feature, and it'll make for a hairier install for admins having to hunt down additional modules to install."
"Yes it does need that feature."
"No it doesn't, and if it does, code it yourself -- that's why I made it open source!"
"But I don't code, why don't you add the feature and just make it an option so it doesn't increase system requirements?"
"Because I only have so much spare time, and after a while when you're dealing with an interpreted (yes, compiled before execution, but you know what I mean :)) language, extra features that aren't getting used just become bloat unless they're modularized, and I just don't want NeoMail to go there -- it adds complexity for the user."
"Fine, then I'll be forced to go elsewhere for my webmail solution." (This one always floored me! Made it sound like I had a commercial interest in them using my software!)
"That's your right, thanks for trying NeoMail anyway though."
"Wait, I was just kidding -- PLEASE won't you add this feature?"
And so on, ad nauseum.

Eventually, come arguments actually convinced me to add something, but only if I thought it would benefit the majority of users. I was writing a webmail solution for normal people, not geeks like me. :) When people started saying they wanted NeoMail to import and use their pine or mutt settings, I tried to explain that the vast majority of people that actually are using NeoMail aren't the admins installing NeoMail, but their users, who likely don't even know that pine's a mail client, not a tree. That's the problem when you release your work to a community of people like you -- eventually if you don't fight the urge to do it, your tool will become so big and complex that only people like you will be able to use it. That isn't always a good thing.

Now, I understand that filesystems are much more complicated than my humble webmail solution, but I think that what people need to realize is that just because someone is the maintainer of a piece of code that allows user contributions doesn't mean that the individual is morally obligated to include everything that comes across their desktops, no matter how well thought out or convincingly argued the matter is. The individual is, after all, the maintainer, and if you don't like it you're always free to add the feature for your own use -- that is, unless you're seeking to satisfy your own ego by getting your code included and a credit in the CHANGES file. The door swings both ways, you know.

I'd post more -- have a lot of thoughts on this -- but if I don't click "Submit" soon, I can almost guarantee nobody's going to read to the end of this little rant!

--
NeoMail - Webmail that doesn't suck... as much.

Re: Has Linux Development Become Too Political?

[Al Viro]

[apologies for over-the-head reply, folks]

Has Linux development become too political, bottlenecked and ego-driven? Witness the recent exchange between Hans Reiser, of ReiserFS fame, and Alexander Viro (VFS maintainer) on Linux-Kernel; Hans, and others, were griping about Viro refusing patches and ideas on principle, and Viro keeps telling people to shut up and read the code.

Sigh... Folks, what about getting the story straight? 1. Hans (or anyone in Reiserfs team, for that matter) did not submit any patches _to_ VFS. The only piece that modifies VFS itself is, as all sides agree, ugly but fixable and did not cause any objections. 2. Patch contained a large chunk of code that obviously was not reviewed for a couple of years, which is a bad thing, especially for interface code. Again, all sides seem to agree on that. 3. "Others" in that context probably means Richard with devfs stuff? Well, the problem with devfs was that it did _not_ change VFS. Instead it tried to do everything in a filesystem and that gave a huge bunch of races and major ugliness. Richard refused (search l-k archives) to move the relevant parts of changes into VFS. Many times. After being asked to do that, since that would make things much cleaner. 4. Ideas are good when the author knows what he is talking about. In particular, if the idea starts with "let's make this to do that" it would better be supported by understanding how "this" currently works or willingness to redo it completely and make sure that nothing breaks. If it is there - fine, things get discussed (tons of examples in archives). If not - too bad. 5. Reading the code is definitely useful thing - especially if you want to change that code. No? 6. Rumors about the need to submit stuff through me are greatly exaggregated (read: provably are pure BS). Discussing the stuff on maillists _is_ needed. _Submitting_ it to me (or anybody other than Linus) is not. 7. During 2.3 there had been several large changes of VFS. If you are claiming that Andrea, Ingo, Stephen and I are the same person... <shudder>
As for the "external pressure" - get a grip. Really. As far as I'm concerned this stuff is done on technical merits. You can't build a pressure sufficient to make me act against that. And I mean it - I consider self-respect more valuable than having a paid job. YMMV. You can't force anyone of us to do what you like. You can learn the kernel and start submitting patches that would be acceptable to Linus on their technical merits. That's how it works. I would be glad to see anyone joining the VFS/fs work. Learn it and go ahead. Or keep wanking - nobody can deprive you of that right.