A Look At Bootstrapping

markmcb writes "OmniNerd posted an interesting article on the often-overlooked process of bootstrapping. The author does a nice job of showing how to take an x86 system from BIOS to OS once it's powered on. A complete set of commented code is provided and explained in the article."

Comments

Tomorrow: How To Use A Mouse

[QuantumG]

Here I was hoping it was going to be a story about the commercialization of space.

x86 is a fossil

[kyouteki]

All this article says to me is how old and convoluted the x86 architecture is. I mean, all these memory restrictions and limits, all there for the sake of backwards compatibility. And we've just kept building on the same platform, so there's no way out.

Re:x86 is a fossil

[Anonymous Coward]

Old it is. Convoluted? Is that such a simple asm program that's giving you headaches? That's REALLY REALLY simple stuff for anyone who's ever done asm at all (on x86). Plain 386 instructions, nothing 32bit (no "db 66" or e?x registers even). No interrupt handlers, no direct I/O to the hardware, etc. If you find this complex, it just goes to say you've never coded asm, perhaps on any platform (I've coded in asm on several - motorola, PICs, Atmels, etc - and all programs were typically more complicated than this little "bootloader"). I find it's one of the easiest platforms when it comes down to machine code, coding things like a simple PLC emulator (that runs ladder) in asm, interfaced over the parallel port is pretty much trivial (did that in a weekend as a teen, just for fun). Extensive documentation was easy to find (schematic of the XT boards - whose chips were easy to understand and control, ralf brown's interrupt list, etc). Interfacing was easy too, just throw a 8255 on a "blank" ISA card and you were pretty much set (24 IOs at TTL level), although PICs have changed this whole thing quite a bit. I still miss these days - I loved coding in asm. Those were the best days of my life (as a geek/programmer/computer hobbyist), things just aren't the same anymore, and programming lacks that little something...

We're "stuck" on x86 mostly by choice - backward compatibility is a great thing. You can buy a dual core high end machine and install anything from say, dos 3.3 to XP (and every version between) and most of it will run just fine (and other OS'es). That's very important to a lot of people. Decent hardware for the platform is quite cheap really, readily avaialble virtually everywhere, and you can get parts that fits your needs no matter what (slow or fast, be it a CPU, video card or whatever). You can assemble what you exactly need. There's tons of reasons to stay on x86, I can't see myself switching to something else anytime soon.

Re:x86 is a fossil

[Anonymous Coward]

You're damn right about that. What the hell is this rubbish even doing on slashdot - assembler crap went out with the dodo - it's got nothing to do with how sleek iPods (all of them) are or how innovative iTunes is, so what gives???

Re:x86 is a fossil

[John Nowak]

This actually doesn't look that bad at all. I mean, I'm technically a *design* student, but I've wrote assembly of similar length for things like atmel avr microcontrollers before. I can actually follow a good portion of this, and being as I'm not a computer scientists, I'd say that makes it fairly easy (in the grand scheme of things -- there is a lot of C++ code that just goes WAY over my head).

Re:x86 is a fossil

[stevey]

And yet despite the fact that there are lots of existing bootloaders out there (lilo, grub, chos, etc) and writing a new one is fairly simple nearly every week there will be a new post asking for help in writing a bootloader on alt.os.development.

I can only assume it is because people first start writing an operating system by coding the bootloader. Still given the number of people who have problems with that it is no wonder that few actually write something functional.

Re:x86 is a fossil

[RAMMS EIN]

I agree. And the thing that puzzled me most is why it isn't ditched even when there is a chance to do so. For example, when AMD designed x86-64, which is incompatible with x86 by nature, they still made i t almost like x86, shortcomings and all. It has few registers (although they recognized the problem and gave x86-64 more registers), and the instructions are variable length, complex, irregular in format, and with various side effects, making them hard for compilers to target and hard for the CPU to decode.

Re:x86 is a fossil

[Anonymous Coward]

And now it's a slashdotted fossil.

Re:x86 is a fossil

[Anonymous Coward]

You are not a nerd, you an ignorant idiot. Begone from this website! Shoo! Get thee hence!

Re:x86 is a fossil

[ultranova]

I agree. And the thing that puzzled me most is why it isn't ditched even when there is a chance to do so. For example, when AMD designed x86-64, which is incompatible with x86 by nature, they still made i t almost like x86, shortcomings and all.

Propably because the 64-bit AMD processors need to support 32-bit mode too, and support it well. The 64-bit and 32-bit mode likely share as many components as possible, to keep the total amount of components as small as possible, and this is of course easier if the modes are as alike as possible.

Besides, the x86 architechture has proven itself capable of being extended to a ridiculous degree, while still maintaining backwards compatibility. Why ditch it for a new architechture, for which such extension capabilites have not been proven ?

Re:x86 is a fossil

[RAMMS EIN]

``Propably because the 64-bit AMD processors need to support 32-bit mode too, and support it well.''

AFAIK, the first thing that happens to x86 instructions is that they are decoded to micro-ops, which are much like RISC instructions. The rest of the CPU than operates on these micro-ops. To me, it seems these micro-ops could just as well come from a different decoder unit, so AMD could as well have designed a proper RISC architecture for the 64-bit mode.

``Besides, the x86 architechture has proven itself capable of being extended to a ridiculous degree, while still maintaining backwards compatibility. Why ditch it for a new architechture, for which such extension capabilites have not been proven?''

You mean they've added extra instructions to it? There is absolutely no reason that couldn't be done with other instruction sets. In fact, the MIPS instruction set is designed with extensions in mind, and they are used, for example, for floating point functions. PowerPC has been extended with AltiVec, just like x86's MMX, 3DNow! and SSE. I don't think extensibility is anywhere close to a good reason to stick with x86.

Re:x86 is a fossil

[Jerry Coffin]

I agree. And the thing that puzzled me most is why it isn't ditched even when there is a chance to do so. For example, when AMD designed x86-64, which is incompatible with x86 by nature, they still made i t almost like x86, shortcomings and all. It has few registers (although they recognized the problem and gave x86-64 more registers), and the instructions are variable length, complex, irregular in format, and with various side effects, making them hard for compilers to target and hard for the CPU to decode.

Intel has tried to replace the x86 repeatedly -- the iAPX432, the i860 and the Itanic^Hum have all been pushed as possible replacements. Unfortunately, when you get down to it, they've all had problems that rendered them inferior to the mainstream x86 for most real use.

First of all, x86 isn't really all that hard to target -- and after umpteen years of writing compilers for it, people have gotten good enough at it, that nearly every "shortcoming" has been put to good use sometime or other. x86 is certainly trivial to target compared to VLIW/EPIC architectures.

Decoding isn't really all that big of a deal either: they (or it, in the case of P4) consume(s) less than 1% of the area of a modern chip. In exchange for that, you get an instruction set that's considerably more dense than (for example) most so-called RISCs.

This has two substantial payoffs. First, the memory bandwidth consumed to read instructions is reduced. Second, the effectiveness of the L2 cache is increased because it can store more instructions.

This is why (for example) an x86 uses only about 40-50% of the chip area for cache, where a RISC often has to devote around 60-75% of its area to cache. The investment of 1% into the decoders pays a dividend of 10-25% in the L2 cache. Now, in fairness, even though it's a small area, the decoder design is relatively difficult. Even though it's bigger, the cache area is much easier to design.

The result is that the x86 costs more to design, but (substantially) less to manufacture (assuming similar speed, manufacturing technology, etc.) This favors the x86 in relatively high volume situations, where the manufacturing cost dominates, and RISCs in lower volume where the design cost dominates.

RISC was originally developed when most CPUs had no on-board cache and most CPUs typically didn't use nearly all the available memory bandwidth. The decoders, microinstruction ROMs and sequencers, etc., consumed a substantial part of the chip, and often limited the clock speed as well. Under those circumstances, eliminating microinstructions and simplifying decoders made a lot of sense.

Those days are gone though: now, CPU clock speeds outrun bandwidth to memory by a wide margin so nearly every CPU devotes substantial chip area to caching. As a percentage of overall size, the decoder(s) and microinstruction ROMs/sequencers have shrunk nearly to insignificance. Decoding has been pipelined to the point that among production processors, x86's have higher clock speeds than RISCs, by a fairly wide margin.

Registers are a bit of a tradeoff as well. While more registers make it easier to write nice code, they also make context switches slower. On modern versions of the SPARC (for one example) they have quite a complex mechanism to handle register swapping lazily, so the registers are basically treated like an L0 cache. Data is swapped into the registers on demand, rather than all registers being loaded before a context switch is considered complete. I don't know for sure, but it wouldn't surprise me if this consumes more chip area than the decoders on an Athlon (for example).

At the same time, the L1 cache minimizes the penalty for repeated access to a small amount of data, so the gain from having the most frequently-used data in registers is much smaller than used to be common.

Anyway, the bottom line is that while most of the considerations you've raised are valid, most of them are really much more tradeoffs than a clearly inferior design.

--
The universe is a figment of its own imagination.

Takes me back to my youth...

[HotNeedleOfInquiry]

When I modified the boot sector of some floppy disks and spread them around at work. If you tried to boot them, you'd see the following message:

You can't boot this floppy. Pull the floppy out of the drive and your head out of your ass and try again

Re:Takes me back to my youth...

[An ominous Cow art]

Back in High School, I used to have a modified version of Apple ][ DOS on all my floppies that would ask for a password during the boot process. Every wrong character you entered would result in one more character from "You are a rotting, slime-covered filth." displaying. Ah, those were the good olde dayes... My copy of "Beneath Apple DOS" saw a lot of use... Basically there were some unused chunks of memory in Apple DOS, where you could stick some of your own code and call it at the right point in the boot process.

Meh, why not just look at LinuxBIOS?

[DeathPenguin]

Here's a bootstrapper that's written in over 99% C, boots Linux off several commercial mainboards, and isn't limited to just x86.

I guess TFA is a good technical resource, but it's also a good reminder as to why big wads of assembly suck.

Re:Meh, why not just look at LinuxBIOS?

[Anonymous Coward]

Yes, but does it run Linux?

Re:Meh, why not just look at LinuxBIOS?

[Anonymous Coward]

Why not look at LinuxBIOS? Perhaps because it is totally irrelevant? It is a replacement of traditional proprietary BIOS systems, not a tutorial on the post-BIOS x86 bootstrapping process. I know this is Slashdot and that anything with Tux is free moderation, but Jesus Christ, this has NO relevance whatsoever.

Re:Meh, why not just look at LinuxBIOS?

[Horus1664]

I agree with the sentiments here, that bootstrap code, or any code within reason, is easier to maintain and generally to understand if written in a high level language (such as C). But I come from a mainframe background where bootstrap (or IPL) code has to use machine-specific instructions. Perhaps one day we'll get to the stage of 'standardising' on hardware level controls so that all functionality could be controlled without the need to 'resort' to assembler.

Until then I'm keeping my assembler reference books handy.

Woohoo!

[Deliveranc3]

Slashdot has an Article JUST FOR ME!

I agree with the other guy

[Kawahee]

Finally! A /. article just for me! I've written an OS before, a small one. It didn't do much. But I didn't understand shit about the ASM involved, or even why 0x700h or whatever was the correct place to put the stack. So kudos to /.. Kudos.

Ummmm..... let's write a new OS!

[teewurstmann]

That's a very insightful article. You know, I'm always amazed by people that sit down and start writing a new OS. I was impressed by BeOS and am very impressed by ReactOS. I'm using a Mac, and I only run Win98 inside Qemu, but as soon as ReactOS can run everything I need, I'll switch for sure! I just wish I had time to contribute to such wonderful projects!

Re:Ummmm..... let's write a new OS!

[Rod Beauvex]

I though ReactOS was cool too, even though it would never install right on any of my machines, not even my new one. It's too bad by the time it comes to some semblence of usability, 64 bit computing slowly begins coming into play.

A good, fast Windows compatible 64 bit OS would rock so hard. :D

Re:Ummmm..... let's write a new OS!

[QBRADQ]

"but as soon as ReactOS can run everything I need, I'll switch for sure!"

I said the same thing not but eight months ago. Then, about two months ago, I installed a Linux distro on my home desktop. I've installed plenty of them before, but they never stuck. Even the one I'm using right now (Gentoo) has been on this system at least half-a-dozen times. However, this time it stuck.

"And why's that?", you may ask. Well, over the past few years I've been migrating to Open Source Software! Replacing Microsoft Office with Open Office, replacing Photoshop with The GIMP, and of course replacing Internet Explorer with Firefox.

Anyway, once I got comforatible with the administration aspects of a Unix-like system, the software all fell into place. Most of the programs I use now on my NIX system are the same programs I used to use on my Windows install. And the best part is, if you enjoy fidling with stuff and don't mind recompiling loads of code, a Gentoo system (or just about any other non-bloated NIX distro) is at least twice as fast as a Win system.

Anyway, just saying Mac guy, if you want to take the plunge, go ahead and do it. You'll be pleasently supprised.

That's my $0.02.
QBRADQ

Re:Ummmm..... let's write a new OS!

[teewurstmann]

With "then I'll switch for sure" I meant replacing the Win98 in QEMU with Reactos. I know and I love Gentoo, I've been using it for over 3 years now, at home and at work. But I wanted a system that "just works" at home, and that's why I bought a Mac mini earlier this year and I love it, Mac OS X is a great OS. It's based on BSD, so you can do all your UNIX stuff on it (in fact, I just got a book called "Mac OS X Tiger for Unix Geeks"). So I share your love for Linux, esp. Gentoo, to a large degree. But some things seem like a waste of time to me using Linux. I want to program and do intersting stuff while I'm on the computer and not spend hours getting a printer driver to work with CUPS or the system to recognize my USB stick or something. I think Ununtu is going the right direction in that aspect... So once again: Windows hasn't been my primary OS for years, and neither it nor ReactOs will be in the future. But for running those one or two Windows Apps, ReactOS in qemu could be an alternative.

Re:Ummmm..... let's write a new OS!

[QBRADQ]

"...I wanted a system that "just works" at home..."

I hear you on that :P As for myself, I'm neck-deep in the Windows world every day at work and at school, and I'm a MCP. So I wanted something different at home.

"...I meant replacing the Win98 in QEMU with Reactos."

Ahh, misunderstanding. Still, got to drop a line about Linux every now and then, eh?

Thanks for the feedback,
QBRADQ

Fossil is a bad analogy. Try "modern mammal".

[Julian Morrison]

To you it might look old and convoluted. To me, it looks like a design sharpened by natural selection. The old 16-bit modes stay, because they have vestigial uses and they aren't sufficiently problematic to make the chip "evolutionarily unfit".

Who cares?

[Moken]

I'm part of the OS dev community and this has been detailed time and time again, what makes this so special?

Re:Who cares?

[Anonymous Coward]

slashdot