Behind the Scenes in Kernel Development
An anonymous reader writes "Some interesting changes took place in the way the Linux kernel is developed and tested. In many ways, the methods used to develop the Linux kernel are much the same today as they were 3 years ago. However, several key changes have improved overall stability as well as quality. This article takes a look behind the scenes at the tools, tests, and techniques -- from revision control and regression testing to bugtracking and list keeping -- that helped make 2.6 a better kernel than any that have come before it." We might as well mention here (again) that a couple of new kernels are out: leif.singer writes "2.6.3 and 2.4.25 are out, fixing another vulnerability in do_mremap()."
I wish I could wrap my head around even the smallest part of the kernel. There is so much code in there and aside from main(), it is hard to find a good place to start studying.
Would these tests be a good starting place?
I have been pwned because my
I can't say how much I appreciate the automatic tests. This is applying computers to a thankless task that they're suited for.
Now if they only had a web dashboard portal showing the latest results in an easily-assimilated color coded HTML table....
"Provided by the management for your protection."
I don't know what they've done in terms of changing things, and I didn't RTFA. What I do know is that I've been using the 2.6 kernel on my new ThinkPad T40, and the machine is FAST, and stable. Hats off to all reponsible.
libertarianswag.com
However, several key changes have improved overall stability as well as quality.
...
I have a suggestion : how about not calling development kernels with an even version number?
- 2.6.0-beta-something kernels were bad (okay fair enough, it was beta, and Linus admitted having called a 2.5.x kernel 2.6 in order to lure early adopters and get them to test it).
- 2.6.0, 2.6.1 and 2.6.2 were unstable for me, with doozies such as oopses while rmmoding and random crashes using ide-scsi (yes I know it's deprecated, but some of us need it).
I now run 2.6.3-rc3 and it's the first time it seems stable enough to be called a 2.6 kernel. There are some problems left, but overall it's getting decent. But then why are the others "2.6" kernel called 2.6 at all? they were really 2.5 kernels imho.
This has happened before, with the beginning of the 2.4 serie. I only felt it was getting good enough at version 2.4.6 and above (I'm not counting the failed 2.4.11 release). When 2.4.0 went out, I thought it meant it was ready for prime time, like 2.2.0 was, or at least was more, but no it was crap. I was slightly annoyed with Linus then, but I thought he had been pressured by commercial Linux shops and that he wouldn't do it again. But no, he did it again with 2.6.
It's really quite annoying, because those who follow Linux know the first "stable" kernels aren't stable at all, therefore avoid it, therefore defeat the point of testing it for Linus, but beginners think "cool, a new stable kernel", try it and are disappointed, giving a bad name to an otherwise great kernel. Too bad
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
The announcement for 2.6.3 and 2.4.25 was yesterday, and the vulnerability to which the link in the text above refers was with mremap, not munmap; there's also another vulnerability with mremap mentioned yesterday as an *update* to the kernel announcement.
Emacs: for people who just never know when to
It'll be Debian so you needed worry, your colleagues wern't going to be using it anyway.
Without RTFA (of course), I tried to find any reference to "sco".
:)
The only match was "a misconfigured system".
Was here in yesterday's thread about 2.4.25 and 2.6.3 releases.
Software is not supposed to be about how to work around a useability issue. - Ken Barber
Now if they only had a web dashboard portal showing the latest results in an easily-assimilated color coded HTML table....
Then you'd become a PHB.
I want to delete my account but Slashdot doesn't allow it.
Oddly enough most of that explains a lot about Linux. Not in a good way mind you.
Then you do not understand the Tao. Go meditate until you are enlightened.
fixing another vulnerability in do_mremap() ah, good old Mr. Emap.
2.2.0 had a bug where the system would instantly reboot when any user ran "ldd". I wouldn't call that "ready for prime time" :)
.0 release" is pretty sage advice, and of course we know Microsoft's track record. You're not going to be able to catch all of the bugs before something gets truly widespread testing, no matter what you call it or how long you work on it.
(I remember this because we were waiting for 2.2.x to come out, having just gotten a dual P-II 350 server [2.0.x didn't have SMP support]. Fortunately, we managed to hold off for the first few revisions.)
It's not as if this problem is unique to the Linux kernel. "Never use a Red Hat
WMBC freeform/independent online radio.
I think they forgot to test the framebuffer in 2.6.x kernels. If I can't see Tux, then I ain't booting it! (radeon)
[the author] wrote:
I grant that sometimes you have to simplify history to avoid digressing in an article, but this is a bit too inaccurate to let stand.Bitkeeper wasn't suggested by anyone; it didn't have to be. It was developed from the ground up to Linus' requirements. Larry McVoy had a discussion about source control with Linus years ago, in which Linus said "none of the products are good enough" and Larry said, "ok, I'm going to write one that is". Apparently he had this on his mind anyway, and so he started Bitmover Co. As bitkeeper became a usable product Larry continued to take Linus feedback and improve it until it was good enough for Linus to use... at which point Linus started using it.
This is still a simplification of course, but it's closer... and as you can see, there were no third party suggestions involved.
- Design
- Documentation
- Ease of use
- Friendlyness
In other words, all the stuff users actually care about. The "TAO" of Linux also means being exclusive and derisive of new ideas. It means being a bit of an asshole and blanking anybody who doesn't agree with your opinion because you "understand" the TAO.Last but not least it apparently involves making "cute" Confuscious style stories about the "TAO" in an attempt to explain away these difficiencies instead of actually doing something about them.
It's still amazing to me that a project as large as Linux was able to be so successfull BEFORE the changes that were made to the development process. It lacked a centralized CVS, coherent bug tracking, automated testing... These are all things I use in the smallest of professional projets. Many eyes goes a long way towards compensating for having many hands in a big project, but some structure seems like it's helped.
i thought it was just:
/usr/src/sco /usr/src/linux
cp -R
ducks
My problem? I was perfectly gruntled, until some numbnuts came by and dissed me.
That's what bothers me. How long will the distros wait until they use the 2.6 kernel? I hear the scheduler is improved amongst many other things. So what's the hold up? Is it just that there's no one willing to be the guinea pig?
http://tinyurl.com/3t236
Just wondering if java has got its ugly head in somewhere in the kernel development release, maybe someone uses jedit perhaps, or even a JSP or servlet web site.
Maybe real kernel hackers do not use something that is half the speed of C but maybe the odd exception likes java.lang.Exceptions.
Offtopic but I just like to know where a 5 year java programmer might fit in besides maybe jakarta from apache.
Be Free: Free Software Tuition
I would like to see Linux written in C++. I've had to dig into the code a couple of times, and it's really not easy to understand what's happening under the hood,for example all those function pointers to that could be replaced by virtual functions. C++ being slower than C is a myth, it may have been true few years ago.. but not anymore.
The benefit of writing something like Linux in C++ is that the more complex things can be abstracted so they can be understood by more programmers and computer science students (or according to Open Source philosophy; more eyes, fewer bugs). Object orientation, instead of modular C programming, would also mean that it would be easier to refactor parts of the systems, which could ease things up for students that want to test their theories in action, and then donate them to Linux if they are better
C can be thought of as a subset of C++, so there is no reason to stop using neat C tricks (not everything should be an Object - C++ is not Java!). This isn't about stop low level programming. It should be about making the code more understandable. C and C++ can live happily together, but some Linux coders are just stubborn and won't stop using new as variable names and etc.
I don't like to start new threads but I didn't see this: A general "Thank you for your time, effort, and a job well done" to all of the kernel hackers out there. They're fixing kernel level bugs that are almost at the hardware level while M$ is still patching their web browser. I don't think there's any doubt which system is ultimately more secure.
Can anyone take a guess how many low-level memory exploits are in Windows XP, 2k, or others? Perhaps it's irrelevant. Who needs to crack the low mem when there are so many ways into the system at the document level?
+++ATHZ 99:5:80
Is that true that Linux is developed using the proprietary BitKeeper? Why not Subversion or CVS or RCS? I just tried to download BitKeeper and there's a strange question in the download form: "Do you anticipate implementing a source management system in the next: [Please Select]" What's the deal? Don't just tell me that if I want to contribute to Linux, than I cannot contribute to Subversion... If that is indeed the case, then what are they nuts? No wonder that Subversion is maturing so slowly, when all of the Linux contributors are legally obligated to not help it. But is that really fair? I'm sure Linus Torvalds would never choose software with such an EULA, so who has made that decision? Was BitKeeper always proprietary and anticompetitive?
I highly suggest picking up a copy of the Linux Core Kernel Commentary. The first part of the book contains the code of the majority of the "core" kernel components, and the second part explains the code. It's slightly out of date, but still a good read.
JUnit does a nice job of presenting "visually" pleasing results if you want it to. I usually use it in combo with Ant (if the JUnit test fails, don't deploy/check in/etc)
:-)
Haven't looked at the c++ port CppUnit, but I'm sure it's results can be presented just as pretty.
You need to get a life and learn of Geoffrey James.
Linus would put out a new release every day or so,
sometimes even twice in one day. We all ran the
latest code on our desktops and even servers, so
it got plenty of testing. When the kernel wasn't
solid, e2fsck got tested too!
Important bugs get reported again and again.
Spurious bug reports don't get re-reported, so
they get forgotten. This eliminates the problem
of year-old unreproducable "bugs" sitting in a
bug-tracker.
CVS sucks. You can't even rename a file. You have
to run a server, instead of having peer-to-peer
operations between equals. Linus himself, using
just microemacs and the patch command, makes for
better revision control.
Hey, if a million linux users [read: geeks] were trying to do ME, I would be feeling vulnerable and exploited too! No wonder he's rebelling.
Berto
I've encountered many problems with external hard drives using USB 1 and 2 interfaces. Locking up the entire system on large file copies was the main issue. (Copying small numbers of files was never an issue. Lockups occured on different drives, different external chipsets, different 2.4.x kernels though supposedly fixed in the latest 2.4.x releases.)
I've finally gotten the nerve to run a few days of tests on 2.6.1 to see if this has been really resolved, and I'm happy to report that this now works like a charm.
If you've encountered similar problems with 2.4.x, give 2.6.x a try.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
I can't think of a x.y.0 release of any software project that's been properly stable.
.0 releases, and have never been burned. YMMV, of course, and they are an exception, not the rule, I suspect.
OpenBSD comes to mind. I faithfully rely on their
The wise programmer is told about Tao and follows it. The average programmer is told about Tao and searches for it. The foolish programmer is told about Tao and laughs at it.
If it were not for laughter, there would be no Tao.
The highest sounds are hardest to hear.
Going forward is a way to retreat.
Great talent shows itself late in life.
Even a perfect program still has bugs.
Listen, it's very simple: Subversion wasn't (and still isn't) up scratch. CVS or RCS are so fundamentally broken as to be irrelevant to this discussion.
Wrt. the "must-not-develop-a-competitor" angle: This does not prevent people contributing to Linux. It simply means that they cannot use BK to do it! (If they plan on being able to work on any RCS). You may argue the fairness of such a stipulation in the BK license, but the fact remains: Linus chose BK.
Except the Geoffrey James stuff is The TAO of programming which is actually pretty insightful and clever. Not the TAO of Linux, which is actually pretty insightful for the wrong reasons and not very clever as it's just ripping off Geoffrey James.
The Slashbot hears of the TAO of Linux and goes fwapfwapfwapfwap!!!
TAO of Programming != TAO of Linux
TAO of Programming > Turd on your shoe > TAO of Linux
2. I don't think that "stable" in this context means what you think it means.
Lasers Controlled Games!
How about JBoss, or Apache Geronimo? User Mode Linux is also important for Linux, like in Java hosting companies.
What I'm wondering is other OSS going through a similiar "growing up" process, and what does that mean for OSS development in general?
Because there are several posts that ask how to get into Linux kernel development, I would also ask here (a bit OT): How to get into GCC development, especially backends?
GCC is also an amazing feat of software and creating custom backends would make many experiments possible (porting to old architectures, porting to virtual machines, c++ on virtual machines) etc.
There are per-CPU caches for memory allocation.
The correct way to spell
"friendlyness" is "friendliness."