Slashdot Mirror
Kernel Traffic #64 And The 2.4 Kernel TODO
sohp writes: "Alan Cox's summary writeup of the things remaining before 2.4 highlights Kernel Traffic #64. It's quite a long list -- I'm not holding my breath for 2.4 " Kernel Traffic is a pretty cool overview thing if you can't handle the burden of actually subscribing to the list itself ...
>But you can exclude nearly everything related to development, most network daemons, databases, most
>libraries, most scripting languages, and most applications.
Win2K has a complete scripting language, Windows Script Host, so it would be fair to include a Perl or Python in the mix. And, since WinNT/Win2K have a rudimentary POSIX system, it seems only sane to give Linux a rudimentary Win32 system and include Wine.
We should also include a modern browser, probably either Communicator or Mozilla, either of which will raise our bug count severely.
Win2K also includes some basic network functionality that NT was missing (under the strange moniker "Services for Unix"), like a telnetd and the like, so removing network daemons is not actually fair. There's Dynamic DNS, so we'll include BIND, which actually is less capable. Also encrypted filesystems, Kerberos (hey, it's not strictly correct, but it's in there), IPsec, telephony, OpenGL & DirectX...
And, IIS, XML parsing, COM system, PPTP, Distributed File System, ActiveDirectory -- you'd need Apache, PHP, probably Mozilla for XML, some ORBish system, any of the horribly buggy PPTP implementations for Linux, Coda? AFS?, OpenLDAP....
The list goes on and on. To say that Win2K is analagous to a stripped-down Linux distro is really not correct -- there's a LOT in there, way more than in the average Linux distro.
None of which is a defense of Win2K; I just think it's important not to use the 63K alleged bug count as FUD ammo -- I could file 63,000 bugs against ANY Linux distro if our definition of 'bug' is the same one used for the Win2K count -- anything that displeases the tester, even as small as tiny aesthetic visual issues with a program.
>Of course, we also don't have Microsoft's legions of full-time, paid developers and testers working
>on the core components of a Linux distribution, either.
That's kind of an excuse, and gets really pretty close to saying that expensive commercial software should have fewer bugs than Open Source since they have more money to throw at it.
--
In all fairness, comparing 95 known bugs in the Linux kernel with 63K bugs in all of Windows 2000's kernel, GUI, services, userland programs, games, icons/graphics/media files, POSIX subsystem, scripting language, and so forth is comparing apples to oranges.
It would be a more fair comparison to get hold of an entire distribution's count of todo's and bugfixes, including X, Wine, sendmail, python, GNOME, and on and on. I bet you could get right up there near 65K pretty easily if you aggregate all the TODO and KNOWN BUGS lists in all the SRPMS in Red Hat 6.2, for instance.
--
Thanks... now to work out how to put some of the other nifty features of the 2.3.99'ish kernel to use...
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Thanks for pointing all this out - I'm familiar with fstab, just not familiar with shm use in Linux... reading the Documentation/Changes file now, which also appears to answer my other questions related to how to find out more about the new features in the new kernel.
:) And I'm happy to say, it's been totally worth it all along.
For the longest time (been a Linux *user* since 1994, the days of Yggdrasil, pre-RedHat), I've never even bothered to really read the kernel docs, I'm ashamed to admit. Most of the time, for me anyway, it's been a matter of build the kernel, install the kernel, run the kernel (for a year or so), then upgrade a year later...
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
It would be a more fair comparison to get hold of an entire distribution's count of todo's and bugfixes, including X, Wine, sendmail, python, GNOME, and on and on. I bet you could get right up there near 65K pretty easily if you aggregate all the TODO and KNOWN BUGS lists in all the SRPMS in Red Hat 6.2, for instance.
Obviously comparing the kernel to Windows 2000 is outrageous, but comparing Windows 2000 to a full Linux distribution is also unfair. The average distro ships with some 3,000 or so packages, the vast majority of which are applications that you won't find an equivalent for in Windows 2000.
Windows 2000 is more comparable to the kernel, X, glibc, GNOME, samba, Apache (maybe-- does IIS come with the base OS now?), some common libraries and small utility applications. Probably a few other applications, too. But you can exclude nearly everything related to development, most network daemons, databases, most libraries, most scripting languages, and most applications.
And then, you have to factor in the fact that most Linux distributions run on several hardware architectures, many of the libraries and applications in that distribution run on various operating systems, too.
But still, you do have a point: we've got our share of bugs, too. Of course, we also don't have Microsoft's legions of full-time, paid developers and testers working on the core components of a Linux distribution, either. Imagine what you could do if you had a billion and half dollar development budget (which is about what they spent on Windows 98 and IE4/5).
The result of the massive changes is likely to be a diminishment of the amount of time Al Viro has available for other things, such as sleeping...
If you're not part of the solution, you're part of the precipitate.
There is a problem with the "when it's ready" philosophy, which is that the release schedule of a piece of cooperatively developed software is bistable. In one mode, releases are frequent, so the "penalty" of your favourite feature being held over to the next release is small, and so the pressure to get features into the current release is reduced, and it is easier to make frequent releases. In the other mode, the opposite happens. Everyone wants their feature in the current release, because they don't know when the next one will be, so it gets harder and harder to get releases stable and out.
Steve
First of all- to preempt the question that I KNOW is going to come time and again here: KERNEL 2.4 WILL COME OUT WHEN IT IS READY AND NOT A MINUTE BEFORE. Incase anyone is wondering, Mozilla will also be released WHEN IT IS READY. As will KDE2. As will any other project your waiting for.
:)
For those of you new to Open Source, the reason you can run linux for months without a crash is because the developers take great pride in it. We don't release our software as a final version until it is ready and fairly bug-free.
If you truly want to know when it will be coming out, subscribe to the kernel-dev list and read the status. Or, if that's too much mail, at least check out kernel traffic. It makes for good reading.
This particular list has been on kernel traffic for atleast a month. Alan has been updating it and revising it for a while now, with no huge changes. Why it's slashworthy now--I have no idea. Oh well, a minor rant.
Erik
I don't think you understand the philosophy of linux kernel development.
Every part of the kernel is to have a maintainer. When something changes, like the VFS layer, the maintainers of the components depending on that layer, like UMSDOS, must update their components to work with the changes.
If a particular thing isn't working, because of a change, it's probably because 1) there is no maintainer for that part or 2) the maintainer is lazy/busy/no longer interested.
As for other kinds of problems... that's the price of innovation. Linux kernel development, in in the development trees, trys to improve. You don't innovate,and manage to keep *Everything* working *perfectly* the whole time.
Keep that in mind. Maintainers do their own code. When something big changes, it gets announced on the mailing list. Someone who changes X isn't expected to go fix Y, Z, Q, A, and B that depend on it. The kernel is simply too big for that kind of process to be expected.
pid=`ps -ax | grep $1 | head -1 | awk '{print $1} '`
kill -9 $pid
echo "Killed $1, pid: $pid"
now you can run ./killg netscape
What I find strange about that is that Windows 95/98/ect runs in a similar fashion. Apps are launched in a user-mode virtual machine that is either 8, 16, or 32 bit. This is part of the reason M$ software runs a little tighter than third party stuff, they have full access to the vm control components where no-kernel-access-granted people only had published documentation.
I'm a loner Dottie, a Rebel.
Interesting, but I think there is a slight flaw:
Take your favorite linux distro. Remove inetd completly, install X with kde since it is releatively similar look and feel wise to linux. Install kde progs to match standard Win2k utilities. Add smb and a stable kde gui frontend. Remove all console utilities that have no equlivant in win2k. Now install a natile portman desktop theme on both computers. Compare the amount of known bugs
I believe that X is a Client/Server applications and thus relies on inetd to function. We at least need a loopback configured.
Besides - my opinion is that KDE is full of bugs and because it's still not fully operable you should halve the version numbers.
There are a significant number of changes to the kernel internals, especially w/ regard to the VFS layer (and internal layer that lets the kernel treat each filesystem uniformly) Making all these changes causes many things to break, until they are all updated to conform to the new internal interface layer. The VFS is expected to evolve even more for 2.5/2.6.
Why make all these changes, instead of minor incremental changes to the internals? So that the kernel can handle such things as advanced new filesystems (XFS, ReiserFS, etc.) w/ larger file sizes, advanced features, etc. Also, Linux has incorporated a logical volume manager (LVM) allowing people to link filesystems, grow filesystems, etc. Without updating the VFS abstraction layer, XFS and others would have to resort to kludges, that would be difficult to maintain in the future.
Basically, Linux's VFS layer is a VERY powerful concept, allowing one to plug new file systems into the kernel, and have the rest of the kernel automatically use it without tons of rewriting. However, as people get more ambitious with their filesystem dreams, the VFS must evolve to provide the proper abstraction of features.
Also, one more neat thing (unrelated to VFS). I'm running 2.3-pre6 (released today), and the Soundblaster Live! OSS drivers have been included (emu10k1). That is great news! Considering I never expected my SBLive to work under Linux when I bought, it is great to see that it is now in the mainstream releases.
"It's overkill, of course. But you can never have too much overkill." - Anonymous Slashdot Coward
>Obviously, you have never developed software on such a huge scale as the linux kernel.
Actually, I have. I was in the OS groups at two of the early UNIX-SMP companies, and I've worked on a half-dozen other kernel-oriented projects where the complexity was comparable. The point is that it's precisely _because_ of the large scale of something like a kernel that a more disciplined approach is needed. Fast and loose works fine for small stuff. What we're seeing is precisely the sort of breakdown that always occurs when you apply the same approach to big stuff. It's avoidable, but only if the developers exhibit some maturity. As you have amply demonstrated, though, that is a quality often lacking in this community.
>Before you start insulting the (majority) of people that give their FREE TIME to develop YOUR KERNEL, realize there is a lot more to programming than your simplistic view.
Typical slashdot. Someone says something critical of Linux, and respondents assume that the poster is a non-programmer. Au contraire. The whole reason I object to seeing this kind of breakage is because I know this stuff can be done better. I've done it better myself. I've worked alongside hundreds of others who know about basic software hygiene, who developed many of the techniques and algorithms (and sometimes the code) that has since been recycled in Linux. Linux can be improved by a willingness to learn not just technical stuff but also organizational stuff from the experts, instead of having to learn every lesson and reinvent every wheel the hard way.
An example: on my current project, I have often made changes that have caused breakage in msync in the Connectathon tests. This is exactly correspondent to the msync breakage in Linux right now. The difference? I go out of my way to _find_ that breakage by running appropriate tests, and then I _fix_ it before anyone else - even my own team members - is affected by it. There's no reason at all why an open-source developer can't do exactly the same thing. The reason many don't has nothing to do with philosophy or organization structures. It has only to do with individual laziness and lack of self-discipline.
Slashdot - News for Herds. Stuff that Splatters.
>Basically, Linux's VFS layer is a VERY powerful concept,
...which has been around longer than Linux itself. As a filesystem developer, I was stunned to find that Linux didn't have a VFS layer to speak of years ago. The value of such an abstraction, and the methods for implementing it, were old news even ten years ago when I started working on UNIX.
Slashdot - News for Herds. Stuff that Splatters.
>The same is true of O_SYNC: it is not vital to many people
Are you kidding? True, it's not vital to that many people, but to those who do need it there is no substitute. A filesystem that doesn't properly support O_SYNC (and yes, I'm aware that ext2fs never did, even before the current breakage) is simply not suitable for mission-critical apps. It's that simple.
Slashdot - News for Herds. Stuff that Splatters.
>The whole reason I object to this objection is because you don't seem to understand the idea of "development release".
I understand the concept of "development release" just fine. I've been involved in more of them than you. I'll put this simply so even a moron like you can understand it.
Even for a development release, there are certain kinds of breakage and certain levels of carelessness that are unacceptable.
It's not a difficult concept, and what we're seeing in 2.3.x is that we're on the wrong side of that line. Some people just need to clean up their acts, and that's all there is to it.
Slashdot - News for Herds. Stuff that Splatters.
>That's simply not true. If you have win98 running
>in Linux, then VMWare is a Linux process that
>emulates a virtual machine for win98.
So, if you do a ps, you will see a process that has win98 running inside it? Cute.
In that case, that is fairly similar in effect to the user-mode port. The basic design is entirely different, though.
Jeff
I use Gnome. Really.
With regard to the foot menu, it's not an essential part, and it's easy to remove (right click on the foot->Remove from panel).
For that matter, the entire panel could be removed, or easily configured in almost any way that suits you. Or you could have several of them, each configured differently.
In any case, WM does have a root menu, and how this is different from a foot menu is difficult to fathom. Except that there's an icon on the panel for it, but like I said, there doesn't have to be.
--
"I have a good idea why it's hard to verify programs. They're usually wrong." --Manuel Blum, FOCS 94
What hype? I really don't expect 2.4 to be very different as Linus said it would not be ( One of his emails posted on Slashdot a while ago said they'd pack less into each kernel so it could be released more often )
Is there anything in 2.4 you are looking forward to? A device driver you lack support for?
2.4 has some new stuff I'm looking forward to, mostly usb support and I really hoped reiserfs would be included, but heck kernel patches are fine
Oh, and btw - 2.1 went way into the hundreds of alpha/beta version before it was released, 2.4 is young compared to that. 2.2 was just released last year, settle down.
Since devfs is a file system, you can add it to your backup rotation. You might choose to dump it to tape nightly or rsync it to a remote host hourly.
If you want higher reliability, add a journal to the filesystem. If you want high availability mirror changes to the other servers.
Linus made it so you can add as much catastrophic failure tolerance as you feel is necessary.
I understand his complaint about numbering sysctl branches and nodes, but it's not as big of a deal as he makes it out to be- things don't get drastically changed with this practically ever.
The fact is, being able to get your data with one or two syscalls is fast. A lot faster than having to drag the VFS code into things. Case in point? Here's a look at top on a sourceforge box:
PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
7014 bugg 18 5 1156 1156 844 R N 0 6.8 0.2 0:02 top
6.8% of the processor. (this box has 238 processes)
What kind of processor is this?
model name : Pentium III (Katmai)
stepping : 3
cpu MHz : 598.506870
Unacceptable. Procfs is great for users who want to read things, but programs need to have a lower level interface for the sake of speed, and code simplification. It's a lot easier to use sysctl than it is to open up a file and read the data from it, hate to break it to you.
-bugg
That would be a neat poll, but you'd have to and an extra one that you forgot.
9)Hemos
comparing a bugcount of the two kernels is not fair because the windows kernel is not really seperate from the rest of the OS, while the linux kernel could be placed on a system without any of the traditional unix tools and a custom init and work
--- Justin Dearing http://www.justaprogrammer.net/ We're just programmers.
Can you provide more details about how to set up /etc/fstab, or point me (us) in the direction of a web page that has a reference for how to use these new features in the Kernel?
I'd be really happy to get into using 2.3.99, but I don't really have a lot of time to wade through the sources looking for details on how to use the new features. I'm sorry if this is a stupid question, but maybe there's a page out there that contains a guideline for the new features and more importantly - how they might be used?
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Yes, this is a classic "Me too!" comment. Kernel Traffic is wonderful, a godsend to someone like myself who really is interested in what's going on in kernel-land, but can't possibly read 100 messages a day (or whatever) on the mailing list. Anyone else in the same boat (and I'm sure heaps of Slashdotters are) would be mad not to check it out every week.
And, if you're not aware, Kernel Cousins is a collection of "cousins" to Kernel Traffic, for other mailing lists. Currently the Gimp, Wine, Samba and Debian HURD mailing lists are summarised weekly or thereabouts. So if you're interested in the bleeding edge of any of those projects, there's something for you too.
Massive kudos to Zack Brown and the other traffickers for these summaries!!
Incidentally, the portion of the Kernel Traffic discussion where Linus discusses devfs , mentioning thus:
This is one of those things that shows that Linus most definitely does have a clue. Further devfs changes will likely have an impact on VFS code, and thus be "injurious" to Alexander Viro. And it looks like there may be some side-effects whereby /proc gets nearly "reimplemented." And I can see the glimmerings of the VFS changes providing the kernel support needed to make managing ACLs and kernel capabilities a whole lot better.
It may take some time, and may not be complete until 2.5, but there is definitely some ongoing Good Stuff getting implemented in the Linux kernel.
If you're not part of the solution, you're part of the precipitate.
I am talking, of course, Larry McVoy's thoughts on scalability and SMP clusters. Here is a link on the problems with SMP, and here are the slides without explanation.
:-) The other piece of the puzzle is making a cluster work like one machine, and Ron Minnich has been doing some work there.
:-) :-) :-)
The theory goes like this. In an SMP system all of the CPUs have to be made to pay attention when any of the CPUs wants to do something where races would be bad. To do that you need good latency, which means that you need to fine-tune what is locked where and for how long. This introduces a lot of overhead.
Instead what Larry wants is to have a machine with a lot of CPUs turn itself internally into a cluster of Linux machines that just happen to network Really Fast. There are good theoretical reasons why this should scale Really Well.
One of the key items in this vision is the ability to run virtual machines within Linux. Guess what User Mode Linux is?
In 2 years, care for a 1000 CPU multi-threaded database server? With failover?
Cheers,
Ben
My usual seat in the cluetrain is at A HREF="http://pub4.ezboard.com/biwethey.ht
Reading the first couple of pages where all of Linus' quotes are in red text, I had some Deja Vu. Reminds me a bit of the New Testament.
>But this software is BETA, which is why it's in the UNSTABLE kernel numbering scheme.
Unlike some of the other BS people have posted in response to my original comments, this may be a genuine philosophical difference. You see, I've been working for ten years in a world where "beta" means that the people responsible for the product have done everything they could to ensure that it's free of major defects, but they want to get some real-world experience before they "close the book" and call it done. A beta isn't expected to be perfect, but it shouldn't have any _known_ defects of a certain severity level and adequate testing should already have been done to verify that nothing's in there that will later seem "obvious". Linux 2.3.x clearly does not meet this standard. Maybe this standard is too stringent and inappropriate for Linux 2.3.x, what with the cooperative development model and all. In fact, I believe that is probably the case. However, I think the current state of Linux 2.3.x fails to meet _any_ reasonable quality standard, even one more appropriate to the situation.
I still believe that there's a certain level of diligence that should be expected before beta, before alpha, before hand-off to QA, even before other team members get their hands on a new piece of code. Obvious standard tests should be run to check for regressions, for one thing, and the code should not leave that developer's hands while such obvious regressions exist. For example, I run Connectathon tests before I check in even to a development branch, and if I see a new failure I don't check in. That doesn't mean I'm special, either; it's basic stuff that should be required of every developer. What I'm seeing, and what I'm complaining about, is that even these very basic rules clearly are not being followed in Linux development.
Nobody in their right mind expects any software to be perfect, least of all something clearly labelled "unstable", but I think it's perfectly reasonable to expect that things won't be broken _more than they have to be_. And that's not just a philosophical thing, either. Detecting and fixing bugs sooner rather than later is also more efficient. What if fixing O_SYNC requires more than superficial changes, requiring everyone else to change their code yet again after they'd already changed it once to go along with the new VFS stuff? Wouldn't it have been better if the person who broke O_SYNC had been required to fix it _before_ the whole suite of related VFS changes was sent out to affect everyone?
Slashdot - News for Herds. Stuff that Splatters.
> It seems to me to be like a VMWare that only
> does Linux (to put it simplistically).
That's sort of the effect from the user's point of view, but if I understand vmware, it slides underneath the two OS's and makes them run side-by-side with one appearing in a window in the other. With the user-mode port, it is really Linux inside Linux. If you run it and do a ps, you will see a whole bunch of "linux" processes, plus what they really are inside the virtual machine.
> trying out new
> kernels/distributions/configurations without
> needing to mess with your current setup.
This part is fun. The kernel boots out of a file in your normal filesystem. I've got Red Hat, Debian, Slackware, and SuSE filesystems. This makes it a lot easier to play with new distros.
> One also wonders then if Linux could be ported
> to other call interfaces
There's been talk of a windows port. According to one of the guys on my mailing lists, 95 is out, but 98 and NT look possible. The really important thing is the ability to intercept and annull system calls. If that's there, everything else can probably be made to work.
Jeff
The two most exciting things here for me are being able to look at the kernel in user-space while running, in ways that wouldn't be possible on a traditionally running Linux, and trying out new kernels/distributions/configurations without needing to mess with your current setup. For kernel developers in particular this could be very valuable.
One also wonders then if Linux could be ported to other call interfaces - Linux under *BSD/*dows/etc for dual booters who need to do something quickly in Linux while still in their other OS.
The web page is here.
tangent - art and creation are a higher purpose
postmoderncore - art and creation are a higher purpose
I'm not holding my breath for 2.4
;-) If you've been running 2.3.99 you'll know there are still a few "issues" remaining, though these kinds of things hardly stand comparison with the bug list of a certain monopol^H^H^H^H^Hsoftware provider I could mention.
:-)
That's very wise
On the whole though, when it is ready, and it's a lot closer than you'd think from the jobs list, it's going to be a real killer. Just a couple of things: Built-in pcmcia and USB; extensive support for video, including USB Webcams; vastly improved SMP support; a new virtual device filesystem that makes major/minor device numbers go away; bags more drivers for all kinds of things; many, many other goodies I didn't mention.
I'll weigh in with a guesstimate of 3 months to 2.4.0 then another 2 months before you see 2.4.x start appearing in distributions. Very, very definiately worth waiting for, or just download it now and use it if you can't wait
Don't forget to put an entry into fstab for shm (shared memory) - this has now become part of vfs, and your distribution won't know about that.
--
Life's a bitch but somebody's gotta do it.
1) Linux 2.4
2) Debian 2.2 (potato)
3) Mozilla 1.0
4) XFree86 4.0 (The real version)
5) The multi-headed XFS/Reiserfs/ext3 beast
6) KDE 2.0
7) Evolution (The upcoming GNOME email app)
8) (insert your favorite software-under-development here)
Anyone willing to make a few bets as to which one we're going to see first? Hmmm, maybe this would make for a descent poll... (c:
--Cycon
Your Brain + EEG + LEGO Robots = Brainstorms
Excluding the "Done" and "Probably Post 2.4" sublists, they have exactly 95 issues to address. And even that number includes quite a few "Fixed but not yet merged" items.
If it was Microsoft, they would have shipped it 62,905 fixes ago.
C'mon, guys. If you raise customer expectations you're gonna wreck the industry. It's waaaay too expensive to ship shit that actually works.
--
Sheesh, evil *and* a jerk. -- Jade