Slashdot Mirror

Ok, I'm tired of getting modded 0 and 1 this post will be 100% serious for the TFA solely because my jokes weren't funny enough. Here it goes: Useful Info~ If you're interested in getting involved with kernel development please visit: http://kernelnewbies.org/ which is run by Kroah-Hartman. An interview with Guido van Rossum, the creator of Python: http://www.newsforge.com/blob.pl?id=1a2eb8656310db 65fa03461a41a91866 and finally Jeff Waugh, the creator of Ubuntu and the newest edition which will be talked about in this interview: http://www.newsforge.com/blob.pl?id=7e977d3c0eafee a86808086c275e87cf

Some stuff I found interesting on the human-friendly changelog.

Block queue IO tracing support (blktrace). This allows users to see any traffic happening on a block device queue. In other words, you can get very detailed stadistics of what your disks are doing. User space support tools available in: git://brick.kernel.dk/data/git/blktrace.git

New /proc file /proc/self/mountstats, where mounted file systems can export information (configuration options, performance counters, and so on)

Introduce the splice(), tee() and vmsplice() system calls, a new I/O method.
The idea behind splice is the availability of a in-kernel buffer that the user has control over, where "splice()" moves data to/from the buffer from/to an arbitrary file descriptor, while "tee()" copies the data in one buffer to another, ie: it "duplicates" it. The in-buffer however is implemented as a set of reference-counted pointers which the kernel copies around without actually copying the data. So while tee() "duplicates" the in-kernel buffer, in practice it doesn't copy the data but increments the reference pointers, avoiding extra copies of the data. In the same way, splice() can move data from one end to another, but instead of bringing the data from the source to the process' memory and sending back to the destination it just moves it avoiding the extra copy. This new scheme can be used anywhere where a process needs to send something from one end to another, but it doesn't need to touch or even look at the data, just forward it: Avoiding extra copies of data means you don't waste time copying data around (huge performance improvement). For example, you could forward data that comes from a MPEG-4 hardware encoder, and tee() it to duplicate the stream, and write one of the streams to disk, and the other one to a socket for a real-time network broadcast. Again, all without actually physically copying it around in memory.

A wiki can be a great tool for sharing information in a company or another community.

However, the information does need to be organized, otherwise you can only really put info into it and nobody will ever find it. Luckily Sinorca4moin provides a wiki editable navigation menu, that allows you to put some minimal organization on top of your wiki.

This has allowed me to migrate the Kernelnewbies site to a wiki. Now it gets regular updates again...

For those who don't know, the Kernel Janitor
project is dedicated specifically to fixing the bugs that are largely considered unrewarding
or uninteresting to fix.

It's a great way to learn about kernel hacking. It's educational to just lurk on the mailing
list before you're willing to actually get your hands dirty.

If anyone's interested in patching the kernel, a good place to start learning how is here. They have HowTo's on getting started and have a list of bugs that need to be fixed and another list for bugs that should be fixed.

The "comprehensible changelog" is slashdotted. Why is the high-level feature list of the release such a low priority, though so demanded? I know programmers prefer writing C to writing English (or Finnish, or Hindi, or German). But what good is code people don't install because they don't know what it does for us? There are so many people hanging around OSS projects who can't or don't contribute to the code. Surely some of those people can help by at least distilling the changes into a brief description. Release notes might not be the most important product of a release cycle, but they often control everything that product consumers do after the release is published.

Well, there are plenty of examples of people working on unglamourous things for reasons other than money; in Free software there's the the Linux Kernel Janitor Project and in Real Life there are folks like public sector healthcare workers and teachers. Section I.4.13 of the Anarchist FAQ also addresses this issue.

From the way I understand the question it seems nobody understood what the poster meant.

It seems to me like the guys is looking at some kind of guide to writing drivers for the different *nix flavours. Telling him to write open drivers with open documentation and specs isn't helping him.

As far as Linux is concerned, a good place to start is here. *BSDs probably have a similar way of working: almost all the communication between the kernel/driver developers is done by email on mailing lists. IRC channels are also used.

Many of the free unix flavours (linux/bsd/etc.) share open source drivers; ethernet card drivers is a good example of that. The interface with the kernel may be slightly different across platform but the low-level hardware access remains fairly similar.

My best advice would be to look at existing drivers. They are all open source so you can look at the source code all you want.

I will assume that you are serious.

While testing is one way, you might try doing janitorial work or some of the simple stuff.

If you have good coding skills, try newbies, otherwise, you may find testing is up your ally.

I couldn't agree more. A month ago I started serious kernel programming and my code wouldn't have been what it is if I hadn't found out about the free, online version of the second edition.

Some other useful sites :
the Linux kernel API reference

Linux cross reference, especially the `identifier search'.

the driver porting series over at LWN.net (which Rufus211 alreay pointed out).

the module init tools FAQ

this document, aptly titled `the linux kernel'

apart from these there are many more interesting links, but mostly those have to do with specialty domains, such as networking or memory management. I pasted my bookmarks.html to my personal website.

And lastly, if you want to do cross-version /dev (mknod and devfs), mmap, module or networking development, please take a look at our project's cvs server through our website (webcvs) at ffpf.sourceforge.net. The directory ffpf/srv/v1.1/generic contains some files I created that work on both 2.4.24 and 2.6.1 and which I'll test on a 2.3.99 system shortly.

Please someone tell me why I have been moderated as troll(parent post). I am just curious. Has this problem been solve already? Am I worrying for nothing?

I don't understant it seems like a legitimate concern to me. I search in google and found the following things concerning this issue:

this

this

this

this

so it has been discused plenty but it is far from being resolved. Meanwhile the companies try to stretch more and more the meaning of the GPL.

visit kernelnewbies.org

So.. I just have to ask: Where's Linux headed next?

Well, right now they're readying for the release of the 2.6.0 kernel, which they were saying in July would be ready in "less than" 7 months and will start showing up in boxed products shortly after that (probably whenever 2.6.2 comes). The new kernel brings, among other things, an O(1) scheduler, improved responsiveness for user actions, and vastly improved support for linux in embedded devices. One can also expect ReiserFS to begin to gain wider acceptance in this period. After that it looks like the schedule seems mostly to be to improve support for enterprise-class ("big iron") environments.

On the side of things more directly related to the user, GNOME is readying for their 2.4 release, which they expect by spring. 2.4 promises an unprecidented degree of polish, and may well prove to be the release that finally reaches the point "normal people" can deal with it-- if for no other reason than that it will FINALLY offer a clear, sane, graphical way for X users to *change their fricking screen resolution*. 2.4 will probably also be the version that Sun uses once they start shipping GNOME as the default desktop environment for Solaris.

On the other side of the user-interface fence, the competing KDE project will be releasing version 3.2 in december. Among other things will include inproved font support and a bundled groupware suite. After that it appears that among their plans is work on abstracting their theme display stuff by moving toward SVG-based graphics, which among other things will improve accessability by improving the support for those who need high-contrast or large-type displays.

Sounds pretty busy to me. That answer your question?

Does someone know if packet writing is in?

In kernelnewbies status list it is listed as pre-2.6.0 stuff, and the patch has been around for ages. I very much hope we will finally be able to use CD-RW's instead of the antique floppy drive. It is frustrating and somewhat embarrasing Linux still does not support this feature. I assume DVD-RAM/-RW/+RW etc. also depend on this?

Pretty, pretty, pretty, Please!

KernelNewbies?

If you have, you may want to consider starting out as a kernel janitor (basically doing assloads of grunt work to build a name for yourself).

I know this isn't quite what you're looking for, but you can find major feature milestones here: kernel newbies list

Sorry, this is an incomplete list for the 2.6 series, while the kernel released is 2.4.22. For a almost complete list of features going into 2.6 you can see Linux Kernel 2.6 Status

I don't know why your questioning who was first, but hyperthreading has been in 2.4.x since at least Linux 2.4.17-pre5 on 2001-12-06. Release kernels later than this clearly had it. (This obviously includes 2.4!) Though work may have begun in early pre-2.4.14 or "SMT P4" is what Linus called it. So this means when he forked 2.5, it already had it, and this supports why I couldn't find any mention of hyperthreading as a major addition by someone who has kept a record of 2.5.x release additions.

Heres some help

Once you have downloaded the kernel extract it somewhere (/usr/src/linux is the usual place). Then, run make menuconfig (or make xconfig for a graphical interface) to get to the kernel confiugation program. This allows you to customize your kerenel to your hardware.

Then go through all the options. Modules can be tricky to setup so i recommend you compile stuff directly into the kernel. There are a lot of complex options in there, which is overwhelming even for experts. But most options have documentation about what they do. If they suggest that you should say Y or N then you should as that is the safest option if you are not sure.

Once you have finnished save the config by going to exit on the main scree and say yes to it.

Then run make. wait until it says its ready. Then copy the kernel which is located in arch/i386/boot/bzImage into /boot. You should rename it to bzImage.2.6

Then conofigure your boot loader to recognize the new kernel. for grub you edit /boot/grub/menu.lst or on lilo /boot/lilo.conf.

Then reboot. You should see an extra option on the boot menu, that is the new kernel. Select it and you will boot. If there is a kernel panic then you must of forgotten to add support for your filesystem or you forgot to add the root option on the bootloader.

The 2.6 kernel is VERY hard to configure so I reccomend you compile a 2.4 kernel first to get the hang of things, as the chance of messing up is VERY high on 2.6. Other sources of help include your distribution's documentation, and sites like kernel newbies. But remember, you are compiling development kernels at your own risk.

I personally can't wait to skim the change logs.

http://www.kernelnewbies.org/status/latest.html

Detailed status and a general overview of incoming kernel's features.

Here's a nice overview by Guillaume Boissiere:

http://www.kernelnewbies.org/status/latest.html

And a document by Dave Jones:

http://www.codemonkey.org.uk/post-halloween-2.5.tx t

I've been a fan of Reiserfs ever since I started using it, and I've been very impressed by its speed. I'm really anticipating Reiser4, and I hope it'll get merged into 2.6. However the last I heard about this is that Linus hasn't read your docs yet, and the status page on kernelnewbies shows Reiser4 to be "pending". Can you give us any insight on what's going on?

Hi!

The following links should tell you what you want to know:

Dave's Post-halloween document and The Kernel Status Page

Cheers
Stor

My article Why We Should All Test the New Linux Kernel gives some tips on how to get started building new kernels (although it emphasizes testing the development patches).

The #kernelnewbies IRC channel has a website at www.kernelnewbies.org that you will find helpful.

And finally there is of course The Linux Kernel HOWTO.

Why isn't Sistina's LVM making it into the kernel? SUSE has been including it as standard in their distribution for some time.

2.5 kernel has initial support for USB2, bluetooth is no longer experimental., dunno 'bout FireWire2. This is all here

There's some kind of totally rewritten kernel managment coming, should make it easier. Linus loves his monolithic kernel. HURD is still in it's infancy. I'm not switching until you do.

To see whats already in 2.5, check kernel status

Well, damn as nearly. Linux is only in catchup when the manufacturor will not release spec on how to use thier hardware.

When it comes to NUMA machines, Linux is up there. It may not excel at everything (yet - I'm sure that it will get there if it's not already). I'm mostly talking about the 2.5 kernel series.

From the status list

New scheduler for improved scalability (Ingo Molnar)
Support for Next Generation POSIX Threading (NGPT team)
Syscall interface for CPU task affinity (Robert Love)
Hotplug CPU support (Rusty Russell)
NUMA topology support (Matt Dobson)
Per-cpu hot & cold page lists (Andrew Morton, Martin Bligh)
NUMA aware scheduler extensions (Erich Focht, Michael Hohnbaum)

The biggest performance changes in 2.5 seem to be in the many thread and many CPU region, including NUMA.

I'd trust it. (Yes, I do do scientific supercomputing).

That list is just the list of features that are not yet merged and thus need an imminent decision before the feature freeze next Thursday. It's also not especially long or impressive, since these are minor features and a much greater number of patches of that kind are already in. Of the stuff on that list, probably only IPSEC and one of the LVM replacements (needed since LVM1 has been removed) will impact most users, though the crash dumps would also be nice.

The significant changes in 2.6 will be the new block layer and attendant performance/scalability improvements, the new NPTL thread support, ALSA, and the XFS and JFS merges. See Guillaume Boissiere's list for more.

Serial ATA support is in Linux 2.5.35 and up, as noted here.

An executive summary:

This is a very difficult list to read.

This is much easier, and yet hopefully more condensed [executive summary anyone?]:

* Support for USB 2.0
* Preemptive kernel option
* ALSA (Advanced Linux Sound Architecture)
* Next Generation POSIX Threading
* New VM with reverse mapping
* JFS (Journaling FileSystem from IBM)
* XFS (Journaling filesystem from SGI)
* New IO scheduler
* NUMA topology support
* etc

The kexec patch should do the main part of the trick. And its status is Ready.

ALSA is defintely in, as is things like USB2, Access Control Lists, new NTFS support (it doesn't completely trash partitions now!), hot pluggable CPUs, software suspend (hit a key combo and save the whole system state to the harddrive), support for drives >2TB, and a whole lot more.

BTW, I have to love a community where this sort of thing is discussed on a site called KernelNewbies:)

o in 2.5.31 Support insane number of processes (Linus Torvalds)

This page contains a complete list of every new feature that has gone into 2.5, and other features waiting to be integrated and their status:

http://kernelnewbies.org/status/latest.html

Here is a good place to find out about the state of various features...

badhack

Of course, we moved to OFTC, which is run very well by a number of ex-OPN staffers. As an added bonus, their ircd has some nice protection against flooders and spammers, so the move to OFTC has technical advantages too...

• Major rewrite of the disk IO layer meaning better harddisk performance for Joe-user and high-end database servers as well
• New and faster scheduler
• Pre-empt scheduling for better interactive performance

• ALSA sound infrastructure
• Video for Linux redesign
• ACPI interface and other power-control patches. Especially a new software suspend-to-disk feature that does not involve Windows specific BIOS magic.
• Lots of high-end features (High memory, 64 bit processor support, per-CPU infrastructure, hot-swap CPU etc.etc.)
• JFS - Journaling filesystem from IBM (where's XFS?)
• Bluetooth
• USB-2.0

• Access Control Lists (ACL) which gives fine-grained security
• Per-process namespaces (some Al Viro hackery. Someone please tell that man to slow down a bit)
• Plugable quota system

• General Info
• The Linux Kernel API
• /proc in general
• just for network tuning

It's made for people just like you. *nod*

For those who don't have time to follow every kernel release, there's a page someone put together to follow all the big changes going into the 2.5 developement series here

I guess you missed the slashdot story a while back which claimed someone was able to compile the Linux kernel on some ungodly 32-way numa machine in under 8 seconds! That's right, 32-way! Linux is making _huge_ progress in the 2.5 developement series, such as the new O(1) scheduler, NUMA support, Block I/O, you can check out the 2.5 status page to see it all. Perhaps 2.4 can't scale like solaris can, but 2.5 will kick ass once it's ready. In the mean time, alot of the important patches are being backported to 2.4 (such as the new scheduler), I recommend the JAM patchset.

Also, this confirms that the 2.5 roadmap is fairly accurate. In yesterday's update, ALSA was listed as "ready [for inclusion]".

Other goodies are listed as "ready", are:

• IBM's JFS
• CML2
• lm-sensors
• User-mode Linux

In addition, things like SGI's XFS, LVM v2, and a new VM from Rik van Riel (with rmap) is in beta and likely to enter "ready" soon.

Check it out for yourself, it's very interesting stuff. :-)

--
Tore Anderson

• A sourceforge project with no files uploaded, no description, and a few messages asking, "What happened?"
• A glossary which describes SnapFS as "A defunct experimental filesystem". There's also a broken link to linuxcare.com.au/projects/snapfs .
• A diary entry from somebody named Martin Pool. On 18 August 2000 he writes,

  I wrote up some documentation for my snapfs filesystem, which has been shelved for some months now. I get mail every so often asking what happened to it, so it seemed good to answer it once and for all. Perhaps I'll get back to it eventually.

  
  
• And of course, Mountain View Data (or the equivalent snapfs.org), which now owns the project and seems to have renamed it "MVD Snap". But there is very little information here, and nothing to download.

Thanks to everyone for all the help!

Here's a good place to start, the web page and the irc channel on irc.openprojects.org #kernelnewbies.

Happy Hacking!

the page and irc channel are great resources.

I especially liked the 0.01 kernel walkthrough