Slashdot Mirror
Custom Kernels Used In Comp. Sci Programs?
pdowner asks: "I've just finished and handed in my latest Operating Systems Inmternals assignment for my degree in Computer Studies. It involved writing/modifying a timer module for a Microkernel implemented in modula 2, which is written by one of my lectures and is available from here. I am interested in finding out what microkernels other universities use to teach their students the insides of Operating Systems."
Plus it's Mach-based, and thus gets the "trendy" sticker on it for using a microkernel! Woohoo!
I took the second (out of three) Operating System course here at state.
The first one (CSC202) is basically threads and stuff. Maybe you'll get to modify a fake scheduler; woo hoo.
The one I just took (CSC451) is pretty cool. Everything we did was usermode, but still... The course is taught in C. We wrote a typical utility (sortuniq -- roughly equivalent to 'sort | uniq -c', but as one command; I just used a binary tree), a simple file system ("MiniMinix"; it's the Minix filesystem with stuff taken out), a shell (implements pipes and redirection, but no other real metacharacters or built-ins) and a tftp client-server (that should work with real tftp; the server is threaded).
The next class, CSC452, is being discontinued in favor of CSC492, I think. This basically gets students working on projects with industry. However, I don't know what kinds of OS projects we're going to find; it should be interesting...
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
We used it at Taylor as well. As a sad note (or maybe not...) I learned more C than I did about OSes. Also I learned about how group projects are supposed to work, as a group! All in all it was a great experience.
-I just work here... how am I supposed to know?
Here at Brown, we have a class where you write your own operating system kernel, entirely from scratch. Some userspace code (libc and a linker-loader) is given, but that's it. (it also runs in a clever `virtual machine' using mprotect tricks)
:)
The kernel is actually fairly similar to Linux (or any UNIX for that matter). It used to be a C++ object-oriented microkernel, but obviously that was changed. (I think it was because of student interest, actually)
Admittedly, about all it does is boot and run programs on a System V filesystem with simple virtual memory (no swap even, unless you do extra credit), but it's still a pretty cool project
Daniel
Hurry up and jump on the individualist bandwagon!
Custom kernels are very useful in University. I doubt I would be able to make it through midterms without my Orville Reddenbacher.
At doc.ic.ac.uk, the second year students get a lab exercise to modify Minix (which is, of course, a microkernel design). Last year it was adding a new kernel task for debugging and a user process which sends messages to this task to step through programs and set breakpoints. The number of layers a system call goes through is rather scary. Still, it's no worse than on Linux or any other complex OS, just more explicit.
-- Ed Avis ed@membled.com
> Guess which one University of California at Berkeley uses. Some nachos thing.. they should use BSD.. mb
... hey get them a copy of NT quick!
Why? Just because they created it doesn't mean they're stuck with it. If it wasn't for Berkeley, we wouldn't really have Unix either, it'd still be a buggy stripped down bastardized commercial OS
--
I've finally had it: until slashdot gets article moderation, I am not coming back.
>What commercial grade microkernels are
>written in Modula-2?
Technically I don't know of any microkernels written in M-2, but the AS400 operating system is written Modula-2. It might not be anymore, but it was originally. I know because I interviewed with IBM Rochester in 1988 and that's what they were doing. The AS/400 was new back then and they were selling like hotcakes. I had Modula-2 experience so that's why they considered me.
If tits were wings it'd be flying around.
Oh man, that brings back memories. When I took my first Operating Systems class at Illinois, I was thrust into C++. It was the same kind of deal. "Hello class. All of your MP's for this class will be written in C++. If you know C++, great. If you don't, we recommend that you learn it." At least we didn't have to use templates and exceptions. Never did have any classes in Modula-2 though.
You gave the answer right there: Module-2 is a great teaching language. That is a good enough reason to teach it for beginning CS students. But at the same time it shouldn't be used for upper division course work.
When I went to university, the first CS class was Pascal. That was a primo decision on their part. I learned how to program instead of how to fiddle with syntax. Today, Pascal would still be a good choice for CS101, but I would also add Java and Python. After that, the next class was assember (PDP-8).
Then, to our horror of horrors, we were thrust into "real" programming. "This is Unix, this is C, you know nothing of them, but the first assignment is due this friday". It was rough those first five days, but since we were firmly grounded in programming as opposed to syntax, we made the transition with flying colors.
A Government Is a Body of People, Usually Notably Ungoverned
I hope Satya still runs it at CMU. He is truely the best of the best, a brilliant OS expert and among the best professors I've ever worked with, he can take difficult concepts and make them into very easy to understand ones and do it completely and in a nearly poetic way. If you can take his course, take it. Anyhow, when I did yalnix they gave you a virtual machine and told you how to compile code for it and that was it. I believe we did 5 projects, 1) was the standard freeby, had to debug a multithreaded queue application. 2) was a serial terminal driver, only difficult if you didn't know how it was supposed to work and had to grapple with understand that as well as writing the code. 3) Was where it hit the fan, we implemented the kernel here, 5 or 6 weeks, we implemented the syscalls, the scheduler, and the memory managment. 4) Kind of a lob, you made yalnix run arbitrary ELF-sparc code, I don't remember if this was intended to be a project of if they did some resheduling and gave us another freeby because of the difficulty of the first. This project essentially meant adding a few syscalls and pluging in some ELF code that they gave us. 5) we implemented the yalnixfs, this was a (bitch**bitch) yalnix wasn't multithreaded so we needed to invent a technique for dealing with it (lot's of goto's and a state tracker) Part 3 was the scariest and part 5 was the hardest. At the end Juan Leon, the TA that year, was talking about figuring out ways to reduce the pain/knowledge gained ratio, he thought it was way too hard for what we learned. It was one of the few projects at CMU I was really proud of, I always learned stuff in classes but that class kicked my ass at times and I was very happy to do as well as I did and I felt like I learned a lot.
I think Nachos and Minix are potentially harder projects, the class I did Nachos with had it pretty easy. Minix works on real hardware and the development cycle and tool chain can be a problem as well as just understanding your hardware, I know sparc and alpha well but I'm not an x86 guru and it's not the most logical architecture, there are lot's of special rules that you only learn from experience. It may be bias but I think yalnix was the best one of the three, CMU made it very hard but it was doable and you really learned the concepts by implementing them. Further, the OS class at CMU is the best I have seen. Again, my hat is off to Satya and Leon, I think the class was a good balance between real world implementation and concept; Satya pretty much taught the state of the art concepts in lecture even though the Yalnix OS is very primitive in a lot of ways. I think the OS class in general is a difficult one, they could give you Linux and ask you to screw around with various components, that could be incredibly easy or incredibly difficult. They could give you most of a kernel and ask you to implement a few syscalls and change the MM stuff. They could give you disk space and an editor and make you do the whole thing. Dialing in a good pain/signal ratio is hard. I would expect CMU to possibly make it easier, the general theme when I was there was that the school promoted a geek and hacker culture and not enough of an entrepreneural one and that there were 3 girls in SCS and 200+ guys and that wasn't right. (there were more female professors in SCS than students) The compiler class was similar, both OS and compiler pretty much took a semester of my undivided attention to get A's.
This is my signature. There are many signatures like it but this one is mine..
We' used XINU for the 4th year OS course here at UBC. Its not really a unix-alike system, and its extremely simple... but it allows students to play around with stuff like implimenting semaphors, ipc, etc.
The size and simplicity makes it possible for a student to understand it in a single term course, so I can understand why they went with a non-real life os like xinu vs BSD or Linux.
Check out some of the funky pictures of penguins. ;)
http://www.cs.washington.edu/451
(\(\
(^.^)
(")")
*beware the cute-bunny virus
Let's try this question: Would you be able to kick the ass of your present self if you went thru a full CS curriculum?
I just finished the last assignment for OS I at Columbia. We used Red Hat running on vmware running on Red Hat.
Karma: Good (despite my invention of the Karma: sig)
At Brown the students write their own kernel on top of a simulator after being given some minimal framework (a buffer cache, a loader and lots of header files). Then those that take the optional lab component of the course implement processes and kernel threads (including a simple scheduler), a VFS subsystem (without permissions among other things), Virtual Memory (without paging to backing store), and a simplified System V filesystem. Once it all works they get to run the shell they wrote as the first assignment on their own kernel. Its a ton of work to do in one semester (and the drop out rate is usually very high) but those that finish come away with a uniquely deep understanding of UNIX flavored OSes for an undergraduate course. (sorry for the blatant plug but I owe my current job to this course).
NACHOS provides basically everything you need to do a meaningful OS academic implementation. And its writen in C++ (with a hair of assembly), whose features make it easier to write the OS in than plain old C. NACHOS runs on top of the SPIM simulator (which simulates the MIPS instruction set), so you can run NACHOS on linux or unix w/o having to recompile your programs which run under NACHOS.
Georgia Tech recently added a new course to the CS lineup, an optional 'hands-on' OS course to follow up on the OS Theory course that's required for both Computer Science and Computer Engineering majors. I was lucky enough to be in this class the first semester that it was taught (I love this kinda stuff!).
Of course, the school didn't want us messing up the kernels on computers in the school labs for our tinkering... but Compaq Research Labs was kind enough to loan out some single-board computers that they've been developing (in exchange for free 'field testing').
The operating system being used for these computers? Linux, in Debian flavor. I'm no Linux expert, I've yet to have a Linux machine of my own (though when I've got money enough for a New System, it will be Linux), but I was glad to learn the ins and outs of Linux in this class.
---
Good judgment comes from experience.
Experience comes from bad judgment.
If instead you modified an OS and ran it in a virtual machine, the OS dying in the virtual machine wouldn't need to be fully-rebooted - you just restore the image, patch in the new kernel, and try again. You'd still have to boot, but you'd gain the ability to debug the kernel in the virtual machine with better control than attempting to debug the kernel while running it on the actual machine (which mostly consisted of printk statements to the kernel log). The first assignment was a killer because messing up the scheduler basically meant a reboot followed by a fsck.
Although it was definately cool to learn about the differences between Linux and other OSes and why the Linux scheduler sucks and the Windows one does too but in different ways. (Linux could use preemptive multitasking, where a process that should have the CPU gets it as soon as possible, while Windows needs some work to prevent it from starving processes.) It also gave me a new appreciation for kernel modules and just how cool that technology was. (So, is Linux really a monolithic kernel? The kernel modules put it into this nice limbo between monolithic and micro.)
You are in a maze of twisty little relative jumps, all alike.
I hate how CS programs force pinko, academic tools like Modula-2 onto students. This is the reason that I took a quickie little 2-year college program instead of a full CS degree. Even then, I ended up with 3 or 4 classes that were based on Modula-2 (on VMS, just for more fun).
I realize that Modula-2 is a great teaching languge, but c'mon -- how many commercial-grade microkernels out there are written in Modula-2?
Slashdot: come for the pedantry, stay for the condescension.
We start with some basic tools (a loader, C (or C++) compiler, some printf code and a frame-buffer driver. Then we write a message-passing real-time kernel. The kernel does very little - actually only starts a couple of initial processes, passes messages between processes and provides an interface to interrupts. Everything else is performed in a separate process. During the course, this kernel is developed and then each group (of 1 or 2 memebers) also has to write a system to control a train set or a robot. The real-time program is more work than the kernel creation, but every group ends up with a working nanokernel in about 5 or 6 weeks.
Each group is given alot of freedom to do what they would like, with only some basic guidelines to follow. The kernel must be message-passing and it must support real-time operation. By real-time, this means the scheduler is predicatable. The system setup loaded the kernel and programs into RAM directly so disk I/O was avoided. There is a serial terminal attached to a PC and the PC had a frame-buffer. Part of the requirements of the final project were to use the frame-buffer to display some graphics relating to the real-time actions being controlled (position of trains on the track showning movement for example).
"You need a license to buy a gun, but they'll sell anyone a stamp." - Red Green
The first university to be taken to the Supreme Court for aiding and abetting a corporation.
- I don't care if they globalize against free speech. All my best free thoughts are done in my head.
Isn't this a contradiction on your part?
"I realize that Modula-2 is a great teaching language, but... I hate how CS programs force pinko, academic tools like Modula-2 onto students."
Regardless of your contradiction, a full CS degree should be teaching language *independent* stuff, for the most part, such that it is irrelevant that "pinko, academic" tools are forced onto you.
With a decent grounding in predicate calculus, semaphores, locks, synchronization, threads, etc... you can forget Modula2 the minute class is over, and still be doing cool things.
To rephrase, the class should be teaching you things at a level abstract from reality. You should be able to grab, say, C++ from a *one* term class at a local college, or if you're really masochistic, out of one of those 21 day books ^^
Geek dating!
GPL Deconstructed
Back in 1993-1994 at the University of South Alabama, we used uCOS. You can see more about it here. It's a very small RTOS kernel -- small enough and well enough documented that you can actually master the whole thing in a short while. It (or at least the version available back then) isn't a world-class OS, but it's excellent for a hands-on introduction to the internals of IPC, scheduling, etc.
Linux is an excellent learning tool as well, and some of us at USA used it in projects in later courses, but its very complete feature set, size and complexity, and not-always-clear documentation makes it a bit challenging for novices. Sometimes smaller is better...
Yep. We did the same project at Rice Uiversity. Working with Yalnix. Given CMU's lead in kernel design, I'd suspect Rice modelled their course after CMU's. Great course, really. Between this and the compiler course (we built a funtional compiler), I have a much firmer understanding of what goes on under the covers. It was a bitch at the time, but I'm damned glad I have it under my belt now.
--Be human.
This guy has no clue. Carnegie Mellon is 10x the undergraduate university that MIT will ever be, at least in terms of the education it provides (there's something to be said about living in Boston instead of Pittsburg).
Look, I knew a couple of guys (one CS, one Mech E) that transferred from my school (Rice) to MIT. They both said their workloads dropped considerably, while their grades rose. They were very aware of the fact that they weren't getting the same education that they were getting at Rice. But, MIT has the name.
And one thing I've realized since I've graduated. There are only two other schools I've come across where the grads are as competent as my peers at Rice (truth be told, most of us Rice guys are complete goons socially, though). One is CMU. The other is Harvey Mudd College.
Just my $.02
--Be human.
I'm not sure they still do this as it was pretty ambitious, but in my OS class at Rice University (I think in 1991 or so) we had to build the whole kernel. The basic features we had to have were multitasking, memory protection ( I believe it had to handle page swapping ), and task scheduling (possibly some kind of I/O system?). There probably were other features I forget to mention.
We also had to write a few sample apps to work in it, and I think we were given some apps as well that had to run within our system. It may also have had to integrate with the multiuser file system we had to build earlier in the semester, but I can't say for sure if that is true.
Though a lot of work, that was one of my favorite clases ever. Later on for a grad class we were to use the Mach kernel for some research, but that fell through and I never did get to do the project.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
My school, WPI, recently received a grant from the NSF to build a Linux lab, which includes about 30 PIII-600mhz machines run SuSE 6.4, each of which is assigned to a pair of people to do with as they wish for the term. Last term it was used by the OS I class, followed by Distributed Systems (OS II) this term. The experience of coding low-level operating system software and in the process modifying the kernel yourself was in invaluable experience.
Interested in open source engine management for your Subaru?
At UWaterloo we use Nachos for our OS course.
Nice things: it is microkernel based (unlike Linux), and it is written in an OO language.
Bad things: the OO language is C++.
The OS course at CMU is a 400 level class, though students can take it as early as second-semister sophomore year. (Typically people take it in their junior year). It's a project class that is genearlly done in pairs, but can be done individually if you are clinically insane ;)
;)
The course involves 4 projects: You write a shell that runs on Solaris, then you write a terminal driver, a kernel, and a filesystem for the kernel, all of which runs on a virtual machine that you link as a library to the executable. I'm assuming this is what the original poster meant by "codebase", because you get nothing beyond the
hardware and prototypes for functions to interface
with the hardware.
The course is a *lot* of work, but it is definately worth it. I doubt I'd understand filesystems and kernels as well if I hadn't had to write them from the ground up
WRCT Pittsburgh, 88.3FM
Can you say "Holy HTML error Batman!"?
Buying a Dell computer is equivalent to dropping the soap in a prison shower.
http://www.cse.ucsc.edu/~elm
Anyway, the obvious plus was that the instructor could answer any questions you needed answered about it. The minus was that it was still a bit buggy, and a bug in the custom gcc compiler about stove in our heads on one of the assignments. We implemented monitors, traps for input and output to a user program, a keyboard buffer, a shell, 2-level page tables, virtual memory, and a custom filesystem. The preexisting code wasn't always as self-explanatory or well-commented as it could have been, but it was nice to be actually writing this stuff and not just plugging the holes like we would have been in NACHOS. And it was in C, whiche we were plenty comfortable with.
It involved writing/modifying a timer module for a Microkernel implemented in modula 2, which is written by one of my lectures and is available from here.
;-D
So what software do you use (Powerpoint?) to create your lectures that gives them the capability and AI features to create kernels and microkernels? Please reply with the link
______________________________
Eric Krout
If you celebrate Xmas, befriend me (538
(Two weeks after assignment 4, in which some deadlocks and synchronization issues are resolved in the scheduler...)
Microsoft releases SP2 for Windows98 SE as part of IE5.01 release
Geek dating!
GPL Deconstructed
I can flesh that out a little bit.
:).
The codebase is only really
- Standard Solaris functions for manipulating process contexts. Solaris is a great environment for all this, since everything is provided for you
- Special memory tricks. Rather than deal with the real SPARC pagetable structure, their support library uses a memory mapped file (mapped based on a simple 1-level page table provided by the kernel we write). This makes writing the paging code much easier.
- Signal handlers for things like SIGSEGV and all, as well as a timer that goes off. These are translated by the support library into kernel traps (which we write). So when a user program gets a SEGV for a memory access somewhere in the memmapped file, we change the page table some, the support library remaps things approrpriately, and we continue running. The timer is much slower than a real OS's timer, so a process's timeslice is slow enough a human can watch it go by (maybe 1/2 second or so).
So yeah. For all you people who thought you were doing everything, there really was a support library behind the curtain pulling the wool over your eyes. Norm and I cought the little man in there a couple of times doing funky things a couple of times (ever notice how you never have to worry about page table addresses when you switch from realmode to virtual memory? in theory you should have to be very careful to keep your page table from, say, falling across a physical page boundry when you allocate it with VM enabled. the support library ignores such details
Basically this ends up giving us a real kernel that's very very close to the Sun hardware. I actually asked Greg (professor this semester) why they didn't use Nachos. The answer had to do with the oversimplifications Nachos makes. Yalnix is a much more complete environment.
Actually, Greg says he's planning on redoing project 3 for next semester. He wants to take DR-DOS (or possible his own little minimalist DOS-like OS. This project would be on Intel intead of Sun hardware) and replace interrupt handlers one by one until you have UNIX. Since the DOS disk stuff would still be in place, he could then add demand paging to the project. He said this was the Clemson graduate OS project. For CMU undergrads, he'd write the asm utility functions, and also we'd have the advantage of VMWare or Plex86 (so when we crash ourselves all is not lost; this is a major advantage of Yalnix -- when you crash the OS, you get a core dump). The cool thing would be, when you were done, you could boot your OS for real on the hardware.
A note to people who aren't familiar with the CMU OS course: it's considered the hardest undergrad CS course (18 units, that's 6 credits). The last two projects are a month long each, easily 30-50 hours a week in the cluster. But when you're done and can say to recruiters, "Yeah I've taken OS and I got an A" they offer to hire you on the spot.
Ok, enough wasted time. I need to go back to my filesystem (project 4, due Friday at midnight. eep!)
"Today class, our good friend Mr. Gates has given us some of the Windows 98 SE kernel to use as an example. We're to modify this fine example of coding and add some new features to it."
(A horrifying silence ensues)
(After a long two minutes, one of the students, Craig, raises his hand.)
"Yes, Craig?"
(Craig clears his throat, and tries to speak as slowly and clearly as possible)
"Sir, um, did we do something wrong to deserve this?"
At CMU, our operating systems class writes a kernel called Yalnix. It's a user-mode Solaris implementation, and we're provided a codebase to work from -- we're basically responsible for implementing the kernel call API, the scheduler, and the memory manager. Details available here.