Slashdot Mirror
Breaking Into The World Of Kernel Hacking?
crow_t_robot asks: "In
the past couple of months I have become increasingly interested in
kernel programming and have finally decided to take the leap and
'get my hands dirty.' I have searched around the web and read a few
docs and FAQs on getting started with the kernel but I was wondering
what kind of personal experiences those in the Slashdot crowd have
had that are so bold as to start goofing with the kernel code. For
those that have become competent kernel programmers, how did you
'break in' and what advice would you give beginners?"
...www.kernelnewbies.org is supposed to have a lot for the aspiring kernel hacker.
...RTFM! ;-)
Seriously, though. Try and find FRIENDLY help. Once you have that, you should be good to go. A lot of kernel hackers are very elitest, and don't take too kindly to newbies, so find yourself a good support group and go from there.
Good quote, too many chars. Seriously, the slashdot 120 char limit sucks!
(Now taking bets on whether this first hits -1 Troll or +5 Funny).
I think the kernel docs would be a good starting point.
/usr/src
2 .4.17.tar.bz2
cd
wget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-
tar -jxvf linux-2.4.17.tar.bz2
cd linux/Documentation
ls |more
and start reading all the documentation in there. It would probably make a good starting point.
Plus you should read Kernel Traffic and get on the Linux Kernel Mailling List.
Does the name Pavlov ring a bell?
One of the best sources of information is the O'Reilly book on Linux Device Drivers. It contains a lot of good information to get a kernel hacker up and running.
Index of Documentation for People Interested in Writing and/or Understanding the Linux Kernel >>c kers-docs.html
http://jungla.dit.upm.es/~jmseyas/linux/kernel/ha
The info was compiled by Juan-Mariano de Goyeneche after folks on the kernel list asked the same questions time & time again.
If you celebrate Xmas, befriend me (538
From Alan Cox's interview posted to slashdot a couple days ago:
"Ignore everyone who tells you kernel hacking is hard, special or different. It's a large program, and bug fixing or driver tweaking can be a best starting point. It is however not magic, nor written in a secret language that only deep initiates with beards can read.
Play with it, try things, break it horribly and enjoy yourself. I started on the networking code because it didn't work very well. Everything I knew about TCP/IP I had downloaded the same day I started hacking the net code. My first attempts were not pretty but it was *fun*."
25% Funny, 25% Insightful, 25% Informative, 25% Troll
Linux Magazine's Gearheads Only is a great column to read for this, especially the mouse driver and Alan Cox's articles.
Their web site should have archives.
Sumner
rage, rage against the dying of the light
1. Learn to code.
2. Learn to code in C.
3. Figure out what YOU want to add to the source.
4. Read the kernel source.
5. Understand what you read.
6. Make changes/additions to the source, per step #3.
7. Test out the changes/additions on your own system.
8. Make it work for you.
9. Send in your contribution.
10. Have it accepted/rejected.
It also means that when you screw things up (if you don't, I'd be surprised; I bet even Alan Cox screws up now and again), you won't lose anything. And don't give me anything about ext3; if you screw up enough in the wrong place, your filesystem is hosed.
Have a look at the Kernel Janitors Project and perhaps KernelNewbies.org .
Buy the O'Reilly book on device drivers mentioned above, and pick a driver you use. Try to understand it, and then tweak it a bit. Since they can be loaded and unloaded, device drivers are a little easier/quicker to play with. And there is a good book on them. ;)
/. a bit ago - Have Fun!
Working on it during part of an 8 hour work day, in about 1 month I was able to hack tab support into the s390 vm console driver with nothing more than reading code, searching the net, and using that book. And that was probably a little on the slow side. (see here http://www.eagle7.org/ibmlinux.html
And like Alan Cox said in the interview posted to
_sig_ is away
This Linux Journal article gives a really quick introduction into writing kernel modules.
It doesn't go into all that much detail, but I found it a good starting point for messing around on my linux machine at home.
Hth =)
If you don't have that much of experience, I would start making a device driver for a relatively simple device.
dude, don't take the bait! he's just trying to get someone to write a driver for his mp3 player.
Blaze a trail to the New World
Nothing taught me more about kernel modding than spending a few dollars less on the hardware I used on a linux box - and then try to get it to work.
You become very familiar with code that might be close, get to pour over specs that may or may not help, and find a small comunity of others who saved a buck or two.
The best part - when it breaks, you get to keep both peices. When if finally works, ahhhh....
+++ UGUCAUCGUAUUUCU
The assignment was to add a system call that would return the number of processes created and killed up to that point. The only difficult thing was to grok the system call table. Please look at this option as a good introduction to the kernel.
"Drug related crime" is a misnomer, "prohibition related crime" is the more accurate and correct phrase.
Its not like youre going to break anything, real, or permanent, this is all hocus pocus digital alchemy anyhow. If you kernel flies or not, it information, either on what you did right or did wrong. Have fun break some shit. Its no fun it works right all the time anyhow.
....uncomfortable to say the least.
....what are you going to hack, or just patch and compile, Look through the tree for stuff you know, see how they do it , see if you think you can do better and try it, start small....... Me, ive never personally done anything that need make it into the tree on a permanent basis. but hey its been 8 years of all kinds of fun.....
I also highly suggest the IRC channel
#kernelnewbies
I went there today for the first time looking for Rik van Riel I had some qustions about the rmap 11b patch and Guess what he was there and told me the 11c is coming today , REAL time info from people that really know their stuff.....
Kernel hacking can be fun if youre not in a rush IMHO, doing it against a dealine can be
Im assuming you know huw to extract the source tree and apply a patch , if not start there. Im also assuming you know C if you dont
One last point DO IT OFTEN !, Dont let yourself get rusty, Set a goal, a kernel hack a month, or at least a patch and compile a week if youre not cutting code....
Sig went tro...aahemmm.....fishing........
.. is recommended. It's a medium-low level view of the entire kernel, following the source code but making it more readable. If you've taken an Operating Systems or Unix class you should be fine.
Linux Journal reviewed it.
- j
Probably the best place to start is to find some definite itch you want to scratch, and so badly that you won't stop until it's done. For me, that was getting use[ful|less] features into the kernel. As many as I could cram in, without the hard disk exploding. And then fixing all the incompatibilities, as best I could. (Phew!! There are generally a lot!! That's why many of the FOLK patches I produce are unstable.)
What you do next depends on the "itch":
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
i bet that guy can hack a kernel but can't figure out how to change his oil or replace an alternator or something like that
That is true, but in the spirit of open source, we just borrow somebody else's car.
Do you have Linux and a DotPal? Click here now!
XINU = XINU is not unix.
it is a very simple OS with multithreading and a bunch of other stuff.
the full source is available and only like 8000 lines or something. it steps you through it in the book, and it is EXTREMELY easy to read.
this was what was used to teach my OS classes in college. you can actually get in and hack the thing away and know what you are changing right from the start.
MARIJUANA, SHROOMS, X: ONLINE?! - E
Rather than just hacking in general, you should identify a particular area where kernal development has lagged. That way, you can make incremental improvements in long-neglected code rather than trying to one-up the preeminent kernel hackers.
For instance, I've noticed that there is a sad lack of resources devoted to incorporating practical jokes into the kernel. Everything is so "write to disk, read from disk, move bytes around, manage processes" boring.
I've got some ideas you might want to consider for your first project. Implement these babies, and I'm sure you'll garner a great deal of attention.
- Fake "blue screen" crashes: When "root" is logged on locally, intermittently go to a blue screen with memory dump info for a few seconds, then switch back to console mode as if nothing happened.
- "Ha! Just Kidding!" memory manager: when an app requests a memory allocation, periodically claim that it has failed for no reason at all. That'll keep 'em laughing forever!
- Unionized thread scheduling: implement the concepts of lunch breas, smoke breaks, and overtime into thread scheduling. Union threads should refuse to work with non-union threads. Periodic strikes for better working conditions, and so on.
Do a good job with this stuff, and I'd be shocked if it wasn't included in the main tree!
Cheers
-b
If I wanted a sig I would have filled in that stupid box.
A good starter project is a device driver for something simple. Even easier is if you find a device Net/Free/OpenBSD supports and Linux doesn't, port the driver, that way there is somewhat less code to write.
That could be harder for you then me since Linux has so many drivers now (I started with 386BSD 0.0, so there were maybe 15 supported devices...my friend and I ported the MACH SoundBlaster driver). If all else fails, write a driver for something that already has a driver.
After that you can wonder off into the kernel proper and do some "real" stuff. I did IP traffic shaping, but someone seems to have done that to Linux already...
The publisher has a sample chapter online (though their HTML looks weird to me; I hope it looks better in your browser). Also, you can read a little more about the book, find links to online reviews, get errata listings, and so on, at its support site.
Oh, and I, er, happen to know the author. :-)
``Life results from the non-random survival of randomly varying replicators.'' -- Richard Dawkins
We were given a problem to solve: build a kernel patch to any version of the linux kernel that allowed a third party who is writing is driver to have the driver interact with a software emulator instead of real hardware. The end goal was to enable people to write drivers based only on the hardware specs and an emulator. The emulator would instruct the kernel to route certain hardware calls to it. After the driver was written the real hardware could be dropped in place and the driver would not have to be re-written, it could be used out of the box. Essentially it was hardware abstraction without knowing what hardware to abstract before hand.
Personally, if you are just looking at getting into it just find yourself a problem and solve it. It doesn't matter what problem it is, any problem, useful to the world or not. Remember that the end result of this step is to learn, not solve 2.4 VM problems, or to build a better SMP design. (That's the next step!)
If you like to see how things work, the best way is usually to write some code that tries it out. With a little extra work, you can create a small test case. Then submit the test program to the Linux Test Project.
Linux desperately needs more serious testers.
LTP - http://ltp.sf.net/
I recall a certain Finnish nerd, who sat around in his bathrobe in his mom's apartment all the way thru college. Apparently with those credentials he managed to write a pretty decent OS...
Get down and dirty with a kernel project. I chose Reiserfs on alpha.
This taught me a lot about lkml politics, which is probably the first skill (and some larrikins would say, the only skill you need) you must master to be a successful long term kernel hacker. First lkml hint: don't slag off anyone. Don't piss off a few people in the know until you get to know them, and then...
Then, don't talk - do. Respect is directly based upon your skills with patches, and their acceptance rate.
Patch submission. Follow the standard guidelines (found elsewhere), but know now that Linus sucks at code control. The mainline kernel development process is slow, prone to serious lossage, allows regression, and is irreparably harmed by Linus' refusal to adopt modern code control practices. So when you submit a patch, don't worry if it's not accepted. Every time the kernel is revved, re-do your patch and re-submit. It'll eventually be accepted, particularly if it helps the kernel boot. For example, it took nearly a month of my submitting a two line patch to allow the alpha to boot before it was accepted into mainline 2.4.0pre development. That's why I ditched Linux for a while - dickheads in charge. All the *BSDs have better kernel development practices, and their bleeding edge kernels are far more stable than any stable Linux kernel. However, for various reasons, I get attracted back to Linux on a regular basis, like a fly to a pus-filled boil.
Anyway, the things that need desperate attention are:
the kernel janitor project (clean out the cruft!)
the linux kernel testing project
These are far more important than any single feature you might want to add, and in particular the kernel janitor project will help you get familiarized with the kernel the quickest.
http://sourceforge.net/projects/ltp/
http://sourceforge.net/projects/kernel-janitor/
Andrew van der Stock
Let me say that, at least for me, this was not like debugging any of the userspace programs that I had done before. If you're like me, when your program crashes, you first up gdb, load the core, and backtrace/step from there. First of all, there's no core dump. In this case I didn't even have the luxury of an oops readout; as I would find out later this particular bug was locking the computer even before the kernel could flush its output buffers and print to the screen.
So I had to start meticulously reconstructing the function call stack using printk(). It took me awhile before I figured out why none of these were getting printed (for the reason I just mentioned.) So that didn't work either.
I searched high and low but never did find a way to debug the kernel that was as easy as using gdb to debug a userspace program, and that's not saying much. No stepping, no backtraces, nada. The "bug" in my particular driver consisted of a single offending line which wrote an 8-bit register and was not to spec. I would have never ferreted it out if I hadn't "stumbled" across the NDA'd specs myself.
Anyways, moral of the story: kernel debugging sucks for really hard bugs. If anyone knows of better tools to use kindly inform me of them.
I think there is a world market for maybe five personal web logs.
You can find a half baked kernelhacking-HOWTO at http://www.kernelhacking.org
Ok, now you can go back to read all these good advices that other
When I had time, I went and fixed the old keyboard, and rearranged the keys as a Devorak keyboard while I was at it. Unfortunately, the USB-PS/2 dongle that came with my IBM keyboard doesn't work with my Dell PS2 keyboard. However, I currently have the IBM keyboard hooked up to my USB port and my old Dell keyboard (with rearranged keys) as my PS/2 keyboard.
Luckily, keybdev.c (both the one in drivers/input and the one in drivers/usb) is astonishingly short. There's all of about 5 functions in there and most of the complexity is hidden in other modules.
The USB keyboard driver interfaces the PS/2 subsystem IIRC (don't know where I read this, maybe the hid.c documentation on linux-usb) so you can't have seperate keyboard mappings, unless you munge the keycodes inside the USBdriver. As long as you don't lock up your USB keyboard driver or have a buffer overun, you should always be able to restart.
I have LILO (GRUB, actually) setup to boot me into either a 2.2.20 kernel or a 2.4.17 kernel. That way, I can ensure that my hacked module won't be loaded by hotplug if I screw it up.
The Steps I took /usr/src/linux (after untarring the 2.4.17 source) and changed the line near the to from "EXTRAVERSION = " to "EXTRAVERSION = -maxmodular"
this changes the name of the kernel from vmlinuz-2.4.17 to vmlinuz-2.4.17-maxmodular and
makesa seperate directory in /lib/modules for your hacked modules. (I called it maxmodular because I even made the "misc binary support", "a.out binary support" and "floppy drive support" as modules.) Potential Gottcha: IDE support is no enabled by default, and comes after the IDE-parport questions in the config menu, so I must have recompiled 10 times trying to figure out why it couldn't mount the root partition.
Downloads I downloaded the 2.4.17 source code and used apt-get to install kernel-package (makes Debian kernel packages, so your nightly apt-get upgrades won't destroy your work).
Renaming I went into
Snooping arroundI poked around the drivers/usb/ kernel source and documentation some to try and figure out what needed to be hacked. I incorreclty identified drivers/usb/keybdev.c and after none of my printk's worked, I tried drivers/input/keybdev.c (this does not affect the PS/2 keyboard).
Printk() is your friend. Add printk()s (printk() works just like printf(). Make sure to do your kernel module insertion from a non-X virtual terminal to minimize the time it takes prints to get to you in the case of an impending crash.) to the beginning of every function in the driver, letting you know that it's being called (and optionally the arguments). Then recompile and load your module. This gives you a feel for how the driver works and confirms that you have the right driver.
Trim prink()s to only print out the information you need. In my case, I got rid of the printks from everything but the keyboard event handling function and the emulate_raw function kalled by the event handler. I printed out every keycode that was pressed (but not it's release) so that I could write down the keycode for each key I needed to remap. (And some of the neat colorful buttons IBM added above the funtion keys. I later used these to turn on and off the devorak remapping and turn on and off the printks in my module.
After that, what you do is pretty specific to your module. For keyboards, they send a 16-number, called a scancode that is one of the arguments to the keyboard event hadler function. All of the normal keys have scancodes less than 256, so I just made an unsigned char[256] usb_key_remap_table and did something like
if (code < 256 && code >= 0) {code = usb_key_remap_table[code];}
The more astute amoung you will remeber that I'm only remapping the USB keyboard, which is my good keyboard and therefore unmodified. Oh well, I figure having the incorrect letters written on the keys discourages me from looking down.
Anyway, the USB subsystem seems very readable for the higher level drvers and USB keeps getting more and more devices, so I'd recomed starting with a USB subsystem... too bad RadioShack is out of USB cue:cats :-D
Copyright Violation:"theft, piracy"::Anti-Trust Violation:"thermonuclear price terrorism"<-Overly dramatic language.