Slashdot Mirror
Behind Menuet, an OS Written Entirely In Assembly
angry tapir writes "MenuetOS is an operating system written entirely in assembly language. As a result it's extremely quick and compact (it can even fit on a floppy disk, despite having a GUI). It can run Quake. Two of the developers behind MenuetOS took time out to talk about what inspired them to undertake the daunting task of writing the operating system, the current state of Menuet and future plans for it."
Point one: assembly languages are compiled (from a human readable text file to a machine-readable object file). Did you think they wrote it in hex?
Point two: it's therefore possible to decompile it.
That would be disassembly, which they already mentioned separately as being prohibited. Putting "Point one" and "Point two" in front of clearly incorrect statements doesn't improve anything.
If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
I agree, the major advantages of assembly for that other 95 percent would be negated by a good C or higher compiler that will probably write more efficient and faster binaries than most programmers could using assembly.
for anyone wanting to tinker - it had been forked before they closed it
http://wiki.kolibrios.org/
http://www.kolibrios.org/?p=SVN&kind=dir&loc=/kernel/trunk
Hm. No. You're assuming a particular reading of their licence: that by saying "disassembly or decompilation" they are implying that there is assembler and higher level language, rather than "disassembly or decompilation" meaning "reversing it, whichever term you prefer".
Furthermore, disassembly is a basic stage of decompilation, and they may also be saying "you aren't allowed to produce C from it", which is usually possible even if it wasn't written in that language. C code can be created by code generation tools from the product of disassemblers.
The "entire" OS isn't written in Assembly (Quake and I think the web browser were not. As a matter of fact, there is a C runtime lib at http://menuetlibc.sourceforge.net/). But certainly the core bits are. The choice for Assembly was to keep the design uncomplicated, not for performance.
"When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
Since all links in the article submission seem to be slashdotted, an offshoot of Menuet OS named KolibriOS is located at http://www.kolibrios.org/?&lang=en.
Quake is a highly parallel program with dozens or hundreds of threads running at once just to support its physics, AI, and player control operations. The original DOS version brought its own threading library (compiled in); as Quake was written on more modern OSes and ported to DOS, the original implementation used OS calls while the DOS version used an application-level thread manager.
Support my political activism on Patreon.
it can even fit on a floppy disk, despite having a GUI
Those of us who still remember floppy disks should also remember that having a GUI with an OS that fits on a floppy is nothing new. For the most recent example, see the QNX floppy demo. (Was QNX written in assembly?)
I agree. Every time I've ever profiled code, I've been amazed at how much time is spent in such tiny portions of the code.
It's an interesting exercise though. I've only done a little assembler, but my thoughts were that if I did more of this I could get pretty good at it. The trick in any kind of programming is to learn to express *ideas*. I learned several different programming paradigms over the course of my career, and while I'm doing OO like everyone else these days, having tried functional programming makes me a better programmer.
I could imagine doing non-trivial systems in assembler, but mainly if the problem domain and its solutions are very well understood. When you see that 1% of your code is taking 99% of your execution time, you *could* tighten that code and get an immediate payback, or you could try understand the problem better in order to find a more efficient algorithm. If you can improve your algorithms, that's almost always going to be a bigger win.
We've been making essentially modern operating systems (virtual memory, process scheduling etc.) for a long time. If you just wanted to implement the textbook approaches for everything, and didn't care about architecture portability, it seems pretty feasible for a couple of guys to make a reasonably credible OS in assembler, provided they knew their OS stuff and were very good assembly programmers. Obviously the C with assembly for tight loops approach is a quicker path to a usable system, but the fact that they're assembly enthusiasts probably means they'll get more benefit out assembler where it is most useful and less benefit out of C where assembler is least useful than a sane programmer would. And it's always worthwhile to demonstrate the limits of conventional wisdom.
SInce you work in embedded systems, I don't have to tell you that upgrading your processor in six months isn't always an option.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Given that most assemblers are macro assemblers, I'd imagine that disassembly doesn't give the original source code back. You get an equivalent source code, but it might be considerably harder to read (depending on macro usage, obviously).
HAND.
..and thats why the fastest encryption libraries are written in high level languages, right?
..and to top it all off, if you arent using ICC then your compiler is shitty even compared to other compilers. The easiest optimization a VC or GCC programmer can make is to switch to ICC.
oh, erm.. encryption algorithms need the exact things you claim compilers are great at, yet the fastest encryption libraries are in hand-written assembler.
The fact is that this "compilers are better than humans" meme is complete bullshit and has always been bullshit. This fact can clearly be demonstrated by looking at the change logs of those compilers. For well over a decade this meme has been out there and for well over a decade there has been continual performance improvements to those compilers..
What it all boils down to is that HLL's use an abstract machine that has nothing at all to do with the real world details of computing on a specific architecture, and that my friend is why hand written essembler will always win the performance war.
"His name was James Damore."
Go look at the assembler that some of these compilers produce. It's frightening to see the amount of overhead they cost on even simple assignment operations.
I doubt this kind of code is being generated in *RELEASE* builds. I often check the code being generated in inner loops and most of the time it's the Right Thing (tm). I'm pretty amused to see that the compiler can aggregate calls to sin/cos with the same argument into a single fsincos call, or vectorize some loops over arrays. That's like having the best of two worlds: human readable code that maps directly to the problem at hand AND very well optimized generated code. And given a new CPU and a compiler that understands its architecture and can take advantage of it, my higher level code will profit from it with minimal change. PS: higher level code -> C++
What's academic about writing an OS in assembly?
Actually writing an OS in assembler on a 680x0 was part of my engineering degree. Also had to write a real time multithread scheduler.
Tesla was a genius. Edison however was a overrated hack who liked to torture puppies.
Actually, that would be Mr. Spock (to Edith Keeler), from TOS episode, "The City on the Edge of Forever":
Please turn in your Geek card :-)
It must have been something you assimilated. . . .
Of course, if they had to integrate Quake with the OS parts, then wouldn't the OS also fall under the GPL like the quake source?
It depends on how one defines "integrate". If "integrate" just means porting to its API, then porting a GPL app to MenuetOS doesn't bring the operating system under GPL any more than porting it Windows. Both GPL versions 2 and 3 have an exception for linking to an operating system's System Libraries.
You lose a lot on a disassembly though.
Comments for one, and non-public symbols for another (including stuff like variable names, structure names and composition, etc), plus any oddball globals referenced through a base pointer. You lose enough that the disassembly is hard to look at, and this is stuff you lose with a basic 2-pass assembler.
More advanced assemblers include functionality like macros, call handling (write stuff like "function do_something(param1, param2, param3)", "call do_something(r0, local_variable, constant)", and "return blah" and the assembler sets up the callstack, figures out a place for the the locals, etc), even conditionals (write "if (variable > something)" than a series of compare/test and branch instructions) and structure support. Often called "high-level assembly" because it ends up appearing a mix of assembly and C. Far more advanced than the basic 2-pass, but less advanced than a full-fledged C compiler. It's a great way to generate entry points that are architecture ABI compliant, while for the fast call stuff you can use your normal branch instructions.
Interestingly, their homepage does not tout execution speed as a motivation: "Menuet has no roots within UNIX or the POSIX standards, nor is it based on any operating system. The design goal has been to remove the extra layers between different parts of an OS, which normally complicate programming and create bugs."
Layers complicate programming and create bugs? Really? As in, "the whole modular programming thing is just a fad"? If these guys can actually honestly claim that they program better in the big ball of mud paradigm, they must be super-geniuses, and trying to collaborate would be impossible for the normal geek anyway.
Writing in a compiled language is easier, faster, and usually has a better set of pre-written functionality, but never, ever claim that it's going to be more optimized. Even with pipelining updates from the compilers that help the look-ahead caches on the CPU, there's very few times that hand-coded assembler isn't going to be faster.
As CPU's become more complex, this is less and less true. For example, even on the PS2, they created a tool called VCL that preprocessed assembler for you for maximum usage of registers, unrolling, and pipelining. It generated better optimized code than 99% of of coders out there could do on their own and at a fraction of the effort to seriously fully optimize pipelined assembler -- not to mention the asm code was much more maintainable since with handrolled manually pipelined code you may have to re-optimize everything if you change a single register or just a few instructions. Also, things like the auto-unroll and auto-register-assignment a were huge time saver (it could unroll a loop NX and allocate NX registers to interleave SIMD operations to hide latencies).
That's not to say that a really great ASM programmer wouldn't do better than VCL, but he would have to work much much harder. You'd have to track all the registers yourself, remember the pipelining rules and latencies for every single instruction, be willing to experiment with unrolling every loop and counting cycles between unrolled (with prologs/epilogs) and normal versions, know all the possible instruction swaps, synchronize the integer and vector pipelines manually, try nearly all possible instruction orders, etc. Certainly an average assembler programmer wouldn't do as well.
No, that's not academic, that's community college.
Damn Small Linux and Puppy Linux work well on older machines. Give 'em a try.
http://www.damnsmalllinux.org/
www.puppylinux.org/
"This post is an artistic work of fiction and falsehood. Only a fool would take anything posted here as fact."
Apple stole their GUI (and mouse) from Xerox.
Score: 4, Insightful
Sarcastic moderation on Slashdot? Fuck yes.
I write bullshit
It's copyright, not readright. The copyright limits what you can -copy-, not read.
In this case, under the terms of the license you are free to use (i.e. "read") the program, free of charge, but as soon as you disassemble it your right to use the software is revoked. What's tricky with software, is that in the simple act of "reading" involves copying the program several times (moving it from one drive to another, loading it into ram, loading the machine code into the cpu cache, etc).
Now, reverse-engineering is legal, so this is not much more than a CYA on their part, and an attempt to discourage the disassembly of their product. However, technically speaking, if you run it through a disassembler your right to use the software is revoked. If it went to court, though, it would probably be considered a fair use given the case-law behind reverse-engineering.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
We would write the code in assembly, then run a couple of command line programs over it to compile it into something that would actually run on the chip, then burn it to the firmware.
That's an assembler, not a compiler. Assembly language is a unified set of mnemonics that translate into machine code. For assembly language to be assembled into a binary that the machine can run, you need an assembler that assembles assembly language into machine code.
Compiled languages (like C, C++, C#, Fortran, VB, etc) run through a compiler, which compiles the high-level language into what is the equivalent of an assembly language, before assembling it into the binary machine-readable code.
It's called a compiler because it takes the commands you entered in the source code and compiles all the instructions necessary to execute those commands into machine readable code. In assembly, you are already writing all the instructions in machine code format, you are simply using mnemonics to make it easier. The assembler replaces the mnemonics with the correct machine instructions and away you go.
There are also interpreted languages, like Java or Python and others, which are not compiled - they are assembled on the fly by an interpreter.
Programming in assembly is basically removing a step in the process, and it is very hardcore as far as programming goes. The only thing more hardcore would be programming directly into machine language, which needs no assembling or compiling. Of course, if you do that your code will only work on one specific type of CPU - like the intel 80386 or the 80486 or pentium pro, etc. The machine language for each of those is slightly different, programming in assembly avoids that pitfall. Programming in a compiled language makes your program portable across hardware architectures, and interpreted languages are potentially portable across all architectures and OS's - so long as a compatible interpreter exists for a given setup.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
Yes, Synthesis is very cool. Apparently file buffers were handled with on-the-fly code with the buffer pointer embedded within it, and without explicit checks, instead relying on an unmapped page just past the end and a page fault handler to refill the buffer (at least that's how I remember the author explaining it to me). I believe the author is also the one who original wrote the Superoptimizer.
I found a better description of how it essentially does just-in-time compilation, treating even the pointer to the file, buffer etc. as constants, thus resulting in different compiled code for each file opened.
(argh, link was borked in the previous post)
Whoosh. The point of doing number crunching in FORTRAN is that FORTRAN does not reorder expressions. This is critical to maintain accuracy in certain computations.
Who cares how fast it is if you come up with the wrong answer?