Slashdot Mirror
Breaking Into The World Of Kernel Hacking?
crow_t_robot asks: "In
the past couple of months I have become increasingly interested in
kernel programming and have finally decided to take the leap and
'get my hands dirty.' I have searched around the web and read a few
docs and FAQs on getting started with the kernel but I was wondering
what kind of personal experiences those in the Slashdot crowd have
had that are so bold as to start goofing with the kernel code. For
those that have become competent kernel programmers, how did you
'break in' and what advice would you give beginners?"
...www.kernelnewbies.org is supposed to have a lot for the aspiring kernel hacker.
I think the kernel docs would be a good starting point.
/usr/src
2 .4.17.tar.bz2
cd
wget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-
tar -jxvf linux-2.4.17.tar.bz2
cd linux/Documentation
ls |more
and start reading all the documentation in there. It would probably make a good starting point.
Plus you should read Kernel Traffic and get on the Linux Kernel Mailling List.
Does the name Pavlov ring a bell?
One of the best sources of information is the O'Reilly book on Linux Device Drivers. It contains a lot of good information to get a kernel hacker up and running.
Index of Documentation for People Interested in Writing and/or Understanding the Linux Kernel >>c kers-docs.html
http://jungla.dit.upm.es/~jmseyas/linux/kernel/ha
The info was compiled by Juan-Mariano de Goyeneche after folks on the kernel list asked the same questions time & time again.
If you celebrate Xmas, befriend me (538
Linux Magazine's Gearheads Only is a great column to read for this, especially the mouse driver and Alan Cox's articles.
Their web site should have archives.
Sumner
rage, rage against the dying of the light
It also means that when you screw things up (if you don't, I'd be surprised; I bet even Alan Cox screws up now and again), you won't lose anything. And don't give me anything about ext3; if you screw up enough in the wrong place, your filesystem is hosed.
Have a look at the Kernel Janitors Project and perhaps KernelNewbies.org .
Well I think you just dont go into the kernel to be wandering and looking for inspiration. Think about something specific you want to do and get in to it, you'll allways find you have to mess around with several modules or at least get to know 'Where the hell iz thiz thing going??' ;-).
There are some books at amazon which can helps, I like 'Linux Kernel Internals' by Michael Beck
"May the force be with you" Jorge Ivan Suarez Murillo Medellin, Colombia
Nothing taught me more about kernel modding than spending a few dollars less on the hardware I used on a linux box - and then try to get it to work.
You become very familiar with code that might be close, get to pour over specs that may or may not help, and find a small comunity of others who saved a buck or two.
The best part - when it breaks, you get to keep both peices. When if finally works, ahhhh....
+++ UGUCAUCGUAUUUCU
First stop for a total newbie is download an early kernel (try 0.01!): _much_ less intimidating than the current monster. This should help start you in the right direction. KernelNewbies channel and site are both useful
.. is recommended. It's a medium-low level view of the entire kernel, following the source code but making it more readable. If you've taken an Operating Systems or Unix class you should be fine.
Linux Journal reviewed it.
- j
Try to do something simple, like finally adding module parameters to the framebuffer (sorry, personal irritation, but still haven't had the time to do it myself). Then keep enhancing and bug fixing small things, and move on from there to bigger changes... Post your patches on the LKML, and listen to their comments (and 'flame' them if you disagree)...
:))
That's how I rolled into the business..
(And if you get me those module parameters, I'll big you the best hug you've ever had
xer.xes -- 4181
Give the FreeBSD or OpenBSD kernel a try. I jumped right in with these. They have tons of documentation, are friendly and relatively forgiving (if you can call a kernel 'forgiving'). My first attempts involved a simple "undelete" feature. This is a nice starting point as it requires not only knowledge of the kernel, but of the file system as well. A great learning tool. But BEWARE: you first attempts shouldn't be on a machine with your PhD thesis or 36 GB mp3 collection. Set up a new, raw machine and start hacking away. Have fun!
This might not be the sort of answer you're looking for because it's expensive and will take a few months (at least) but if you haven't already, look at taking a formal course in operating systems. At my university it's a 300 level course and has some heavy pre-reqs but I've seen it offered at other universities with only the requirement of prior programming experience.
The course I took in opiples that make an operating system "go" -- scheduling, physical and virtual memory management, we touched on sockets and pipes (it was a *nix-centric course but we discussed other operating systems including Windows, as well).
If you dont want to spend the time or money on a formal course, I'd at least reccomend the dinosaur book. While lacking in code examples (it doesn't try to show you how to write a *nix-like OS in C, it tries to explain concepts that can be applied to any platform, any language, etc.), it is extremely thorough and makes hard concepts easier to learn.
While I haven't actually read it (yet), I hear Tanenbaum's Modern Operating Systems is also excellent.
If you like to see how things work, the best way is usually to write some code that tries it out. With a little extra work, you can create a small test case. Then submit the test program to the Linux Test Project.
Linux desperately needs more serious testers.
LTP - http://ltp.sf.net/
I was glad to see Alan Cox say so, and here's a little personal experience:
:-) You need only drop into root to lilo/reboot
For the past two and a half years, I've been putting some logging code into various parts of the filesystem (ext2, vfs and block device driver.
This has involved creating new syscalls and user level code to transmit logged data to a remote machine via UDP so we won't adversly affect the FS by logging _and_ writing log data to the same FS. Really trivial stuff in retrospect: the hard part was figuring out where to put what and how to do it without crashing (I worked via ssh and if I hosed something, I was out of luck until I could physically return to campus - which could be as long as a week).
My favourite book for this has got to be "Linux Kernel Internals": consice, precise with decent examples (IMHO).
Funny, I worked on module device drivers for a VHDL class _after_ I worked with the kernel directly, so I won't say to you: play with device drivers first to get your feet wet (although, load/unload kernel sure beats the hell out of rebooting). Try doing something simple like changing the messages (printk) within the kernel as a way to gain a small understandinig of what's happening under the hood. And another thing... the source tree I played with was owned by me (so I could cvs, nfs, coda it to my laptop without fear while playing with code (on an OSX PB) on the road
Cheers.
because of two reasons:
- module guide or howto. You have to be armed with good C knowledge, and some idea of how a computer _really_ works (understand interrupts? memory addresses?). If you want to write for a current kernel (say, 2.4.17), you _will_ have to look at the source code, and actually try to understand what is happening (told you you'd need to know C). You will also need to know, or at least have some idea of how the peripheral you're writing code for (say, if you're writing a driver module) works.
a)The elitist attitude of some, who will tell you to go RTFM. And unless you are willing to write code for a really old kernel, you won't find much documentation. And even so, the little free documentation you will find, is not that helpful for newbie kernel hackers (and I don't mean to piss on the efforts of those who have written docs, but they're complex to digest for real newbies).
b)The recipe-minded newbie. Let's face it, you won't find a connect-a-b-and-c-shake-bake-and-wham!-You-have-a
My experience: I was an absolute newbie to kernel programming, so I started reading the code for the tty driver on kernel 2.4.0, which seems to me is one of the simpler parts. After a lot of reading, I wrote my own module (it was just for fun, so don't think it was anything useful...) as a simple device you could write data to, then read it back. Try it on a machine you don't mind screwing up, though, because you will have tons of oopses and kernel panics.
No, seriously, I just come here for the articles.
If you don't have a spare PC laying around, VMWare is a pretty spectacular way to get your hands dirty without having to worry about completely hosing your development machine on accident. The only thing you really can't do with VMWare is hardware level drivers, for obvious reasons.
/proc interface ... and it was pretty easy to get into. I started with a basic understanding of how things work (inodes, etc.), and it came pretty easily. If you're going to be working on file systems, make periodic copies (backups) of your virtual drive image (if you're using VMWare/Bochs) ... it's a life saver for the first few weeks. :)
...
If you really want to get gritty on the emulation level, I highly recommend Bochs. It's a fully functional x86 emulator, and since it's open source, it comes with some nifty options, and the source code is on hand if you really need to get tricky.
I think the key to working efficiently with an x86 emulator/virtualizer is having a significant amount of RAM, and a RAM disk to mount your virtual drives on. Since RAM is so cheap these days, there's no excuse not to have at least a gig of it in a development box. It makes rebooting the virtual machine (which happens a lot) a bearable process.
If you need to work with hardware, you can build yourself a test platform for around $300 now, including a cheap monitor - unless you need exotic hardware.
My experience kernel hacking is in on the VFS level, inserting and modifying a few core routines to report through a
The kernel is complex, but so is any major piece of software. Thankfully, lots of bits in the kernel are well engineered and compartmentalized, so chances are you won't have to worry about screwing up the TCP stack if you're in the midst of inode.c
have fun!
Let me say that, at least for me, this was not like debugging any of the userspace programs that I had done before. If you're like me, when your program crashes, you first up gdb, load the core, and backtrace/step from there. First of all, there's no core dump. In this case I didn't even have the luxury of an oops readout; as I would find out later this particular bug was locking the computer even before the kernel could flush its output buffers and print to the screen.
So I had to start meticulously reconstructing the function call stack using printk(). It took me awhile before I figured out why none of these were getting printed (for the reason I just mentioned.) So that didn't work either.
I searched high and low but never did find a way to debug the kernel that was as easy as using gdb to debug a userspace program, and that's not saying much. No stepping, no backtraces, nada. The "bug" in my particular driver consisted of a single offending line which wrote an 8-bit register and was not to spec. I would have never ferreted it out if I hadn't "stumbled" across the NDA'd specs myself.
Anyways, moral of the story: kernel debugging sucks for really hard bugs. If anyone knows of better tools to use kindly inform me of them.
I think there is a world market for maybe five personal web logs.
You can find a half baked kernelhacking-HOWTO at http://www.kernelhacking.org
Start reading the MBR, then follow the chain of execution until you are deep into the kernel...
That's how I started on the FreeBSD kernel and the Linux kernel.
Sure at the first there's some dependencies - and you are sent on a journey through about 20 H files just to find one #define ition, and to know A you need to know B, which requires C, but it worked for me.
--jq
I actually wrote something up that was recently posted online at the linux journal site: http://www.linuxjournal.com/article.php?sid=4715 Since I wrote it over a year ago, some of the links are a bit out of date, but the general principles are still good.