Slashdot Mirror
FreeBSD VM Design
Over at DaemonNews is an
excellent explanation of
the FreeBSD VM design, from Matt Dillon, who's been doing a great
deal of work on it recently. It's rare to see good descriptions of the internals (or parts of the internals) of any OS OS (that's "Open Source Operating System") so this is particularly welcome. As is customary, there are a number of
other excellent articles over at DaemonNews, including a new Darby Daemon adventure.
Being very familiar with Matt's work I am certain he has performed analysis and is able to measure the lagging performance of NT. Interesting that Microsoft thinks enough of his work to be using a very large array of FreeBSD systems at HotMail. Replacing them with Win00 doesn't appear to be a practical alternative, according to their own analysis. Matt certainly contributes a lot more that is useful than you have. He is a true professional.
Not only did he write a C compiler, he's also dating Winona Ryder. The man is a living god.
I.e., Windows Terminal Server is "a UNIX way to approach the design of NT"?
I read up on the Solaris VM model a few days ago while trying to debug some Solaris boxen at work. It's an interesting read and covers the basic VM model and system diagnostic tools, including helpful detail of vmstat(1). Not as straightforward as Mr. Dillon's excellent FreeBSD VM article, but worth the look for comparison purposes.
The link to the PDF file is the first one on this page; check out the others as well if you're interested in finding out about more Solaris internals:
http://www.sun.com/sun-on-net/perfo rmance/
----
Jack of all trades, master of none: http://whole.net/~pup/
Just an example: try to untar the qt source on NT.
Szo
Red Leader Standing By!
Although some elements could have been a bit clearer (you have to figure out just what "BSS" is from context unless you already know the appropriate Unixese), Matt has the most lucid explanation of page coloring I've ever read. Skip down to the bottom of the article to read it. It's a good example of why even wholly compute-bound programs can differ in performance under different OSes with the exact same hardware and compiler.
Separate documentation is a waste of time because it *rarely* gets updated along with the code, and thus gets out of date very quickly. Sad, but true.
The only documentation worth starting therefore is in-line source code comments because future programmers can be made to update the comments as they go along. And even then in some cases the project manager has to beat them with a stick to make them do it.
Consciousness is not what it thinks it is
Thought exists only as an abstraction
Am I mis-reading you, ralphclark or are you actually oppsed to documentation?
Ha! Not at all - when I can get it. Or when I'm allowed the time to produce any. But in my experience, although a brand new project will usually be written up with the best of intentions, *very* few software development shops will allocate enough time&resources to keep documentation up to date and in step with bugfixes and enhancements. It's a bit of a vicious circle because the argument can always be made that no-one ever has time to read it anyway...you have to just jump straight into the code and get on with it.
What about documention embedded in the source, like kdoc, javadoc, perl pod, and the like? Do those solve your synchronisation complaints?
This is what I was trying to get at. The cost of maintaining in-line source code comments is far lower as the text is only ever a "Page Up" key away from the code you're working on. Plus then the documentation is right where it's needed when you or another programmer needs to refer to it.
I'm not familiar with the specific examples you named but I'm guessing they are kind of implementations of Donald Knuth's "Literate Programming" concept, where the master copy of the code is actually embedded in the formatted text document which describes its function in natural language. To compile the code you first have to strip it out from the document.
IMHO this is a very good idea...with just one proviso; programmers spend enough time prettying up their code (and reformatting other peoples!). The last thing you'd want is for them to be farting about reformatting paragraphs, selecting heading styles etc. for the comments sections.
Plain unformatted text suitable for viewing on an 80-col-wide dumb terminal is plenty good enough and doesn't waste time. I admit that the ability to add the odd illustration without resorting to "ASCII art" could be useful but this too can be a waste of time and shouldn't be overused.
Finally, apart from source comments there is only one type of application documentation that should ALWAYS be produced for applications, a User/Operations Guide.
Anything else, such as API documentation for programmers can always be most economically done in the body of the source code.
Consciousness is not what it thinks it is
Thought exists only as an abstraction
yes. I forgot the chip numbers, so you'll have to excuse me, but:
The 68000 could not use an MMU or FPU
The 68020 could use an external MMU and FPU
The 68030 had an integrated MMU and could use and external FPU
The 68040 integrated both, with the exception of
The 68LC040 had no FPU but a working MMU.
Back then amiga's were around, which could take advantage of MMU's, but I think most people saw them in Apple's machines. Their first ones were called 32-bit dirt, meaning that there were errors in the ROMs that didn't permit them to use 32 bit addressing, until (it think) connectix released Mode32. The result of that was the system SAW the memory above 8 MB, but wouldn't let you use it. It would just report all the extra memory as being allocated to the system.
From what I hear, Dave Cutler took some time off from Microsoft to play with his racecars. Now he is developing Win64, the 64-bit Windows 2000 vaporware.
cpeterso
"The enemies of the White race would like to pretend that George Lincoln Rockwell never existed."
Herr Rockwell is irrelevant. Period.
A Government Is a Body of People, Usually Notably Ungoverned
NT has a single-user, kernel mode GUI. Since nearly everything you can do in NT needs the GUI, that means only one person can use an NT machine at once. People writing apps today may assume only one person is using the computer at once. So any future multi-user version of Windows may have severe difficulties running today's apps.
perl -e 'fork||print for split//,"hahahaha"'
Don't get me wrong, though, higher level descriptions are good and neccessary. It's just that they're not essential when dealing with Open Source Operating Systems. The reason there are so many "Windows Internals" type books is because those systems are closed, and worse, those higher level descriptions are often the only ones you can get of the system.
This comes perilously close to saying "we don't need documentation, that's what the code is for!" A truly dangerous path to stray down.
The code is good at one thing: telling the machine what to do. It is often a pretty poor mechanism for documenting what you wanted it to do. It is also a poor mechanism for documenting what your concerns were, what you tried but found didn't work very well, what used to be implemented here, where your inspiration came from, etc... etc....
Unfortunately, many projects (open source projects included, but of course not exclusively) suffer from a combination of snobbery (if you really knew what you were doing you'd read the code and figure it out) or worse yet, a lack of design itself (it just works okay). It's too easy to hide lack of design and poor design in this way.
The FreeBSD VM article was exactly the kind of article that open source authors should be working toward. Document what you think. Document what you try. Document what the code does. There is more to programming than just knowing what C programs do.
There is much pleasure to be gained in useless knowledge.
For those interested, the UVM web page describes the UVM virtual memory system used in NetBSD.
-AC
MacOS has had "virtual memory" since System 7, but it blows. It basically spills over to disk when you run out of physical RAM, though it's been improving very much with each OS release. OS X of course, is based on BSD, and will have real VM.
Ditto plus more.
Erroneous or out-dated documentation is even worse than no documentation.
Mr Dillon knows more about FreeBSD than anyone outside Microsoft can know about NT
do these things relate? Back to the topic: if NT is packed with errors solved in Unix a decade ago, relating this to the actual topic of the article, are there designflaws in NT's VM handling? Don't come out now with the words 'we don't know! we don't have the source!!', because that's bollocks. There are numerous documents written by a lot of people about NT's internal structure and also about the VM handling and specifically connected with threads (the buildingblocks of NT's multitasking, Unix uses processes as building blocks). So, in the light of the article, what is so incredibly wrong? Mr. Dillon makes the, IMHO not so clever, remark that NT contains flaws unix doesn't have. So I say: which? because he MAKES the remark, he KNOWS what's wrong, right? otherwise how can he say THAT there is something wrong... that was my point. Perhaps offtopic, whatever, and already moderated down because it's perhaps hurting for the OS OS people here, but I just want to know.
Ah well....
Never underestimate the relief of true separation of Religion and State.
Virtual memory (VM) is how you implement processes with separate memory spaces; with a page table, and a page fault handler in the kernel. It's how your swap space gets used on Linux.
Certainly you could have protection without paging; there were machines in the 60's that did that if I recall correctly. It's kind of silly, though, because you have to either always use position-independent code, or have a loader which relocates your code on the fly. Or you can do weird segment register things. A page table (and hardware assist via a TLB) is a much more flexible solution, considering it lets you do things like paging to disk (instead of swapping entire processes to disk, which you had to do before you had VM).
Are there any mainstream OSes out there which don't do virtual memory? I think MacOS is weird this way somehow, but I'm not sure exactly what it does and doesn't do (just wait for MacOS X.)
Interesting notes about page coloring... I didn't know FreeBSD had this capability.
Linus has stated that he probably will never add page coloring to the Linux kernel. Apparently he doesn't believe it will benefit enough architectures to be worthwhile.
On an Alpha 21164 many binaries run faster on Tru64 Unix than on Linux. Static linking rules out differences in the compilers or libraries. Page coloring (a feature of Tru64) is almost certainly the reason.
At last I feel compelled to go try FreeBSD...