Slashdot Mirror

Have you actually read anything about the case? Be had an agreement with some OEMs to ship dual-boot Windows and BeOS computers, providing the BeOS to those OEMs for free. 3 agreed. Microsoft lawyers visited each, reminding them of the terms of their license - the OEMs cannot sell dual boot computers and get Windows at a competitive price at the same time. It's not a matter of single boot PCs, in which case your arguement has merit. It is a matter of Microsoft preventing OEMs from selling dual boot PCs. Notice the title of articles on the topic include "He who controls the bootloader." This is not about marketing, it is about Microsoft telling OEMs that they can ship either single boot computers or go out of business. Be may or may not have survived if the OEMs had been able to ship the dual boot computers (only one of the three did, but it had to be enabled by the end user, who had to modify the bootloader, making it much more difficult - most users didn't even know the BeOS partition was there). The former CEO of Be, Jean Louis Gassée, originally was in favour of cooperation with Microsoft (he did later change his opinion). The problem isn't that BeOS isn't the most widely used OS. The problem is that Microsoft abused its position as a monopoly to further damage BeOS's chances. At this point, it's not about gaining advantage over Microsoft. Be is dead. It got bought by Palm, who refuse to license out the source or continue developing it for the desktop. Check your facts please. The bootloader article is available at several sites, including here The only thing that Be has left is one employee, 4.9 million dollars, and the right to sue under antitrust laws. Be isn't suing over having their time used. They aren't sueing to get a better place in the market. They barely exist, and no longer have a product. Which bit of this did you manage to remain unaware of? They are claiming it is an antitrust case because 3 companies that agreed to ship BeOS were rendered unable to do so due to agreements with Microsoft. As you pointed out, it makes no sense for a computer company, such as Hitachi, to sell only BeOS instead of Windows. Microsoft gave them that "choice" I suppose. I fail to see how it is not an abuse of monopoly power to refuse to let OEMs dual boot computers that they ship. I'm not saying BeOS would have survived if the OEMs had been allowed to ship the dual boot computers. However, it had an impact, however small. They studied, but someone else who wanted their scholarship threatened the teacher with loosing his/her job.

http://www.theregister.co.uk/content/7/24134.html

http://www.theregister.co.uk/content/4/21410.html The register's summary of this Byte article:

http://www.byte.com/documents/s=1115/byt20010824s0 001/0827_hacker.html Byte's take on the bootloader issue.

daniel teske

RTFA. This one.

The problem wasn't that OEM's couldn't offer another OS, it was the fine print that said if you offered Windows, you couldn't make any changes to the boot loader. Hitachi had a PC, the Flora Prius, that had BeOS installed on a seperate partition, but in order to use it, there were instructions you could get online to make it bootable. Good article on it here.

Admittedly, Blu-ray isn't on the market yet, but the only real technological step being taken is the use of a higher-frequency laser. The rest of the technology is built upon proven DVD reader/burner techniques.

FMD is still total vapor. Those guys have been hyping that vapor for over two years now, but they still have nothing two show or sell. I smell a rat. I think that they've hit some technical hurdles that will keep them from producing anything.

The reason people are making a fuss is that the wrong antitrust argument was presented to the courts. I've given up on this whole thing. The real argument and UNDENIABLE proof of abusing the monopoly (remember kids: monopolies aren't illegal. Abusing your monopoly power is.) power against competitors was the bootloader issue.

The OEM license agreements were the proof and the smoking gun.

"Despite claims to the contrary, Windows is pretty well documented."

You may find this article interesting. It shows how a change in the undefined behavior of one Win32 function crashed his application between Windows versions.

Of course, for some perspective on the nature of processor speculation, I point you to nearly any issue in Byte's print archive.

How can you take a guy seriously when he writes a book in which science fiction writers are guys who save earth from an alien invasion.

His knowledge of history is woeful as he does not appear to have heard of the Marshall plan. Without that, we would have had WWIII already, and with one after WWI, maybe no WWII.

How can you take a guy seriously when he writes a book in which science fiction writers are guys who save earth from an alien invasion.

His knowledge of history is woeful as he does not appear to have heard of the Marshall plan. Without that, we would have had WWIII already, and with one after WWI, maybe no WWII.

A few years ago, when everybody was downloading Mars pictures from the Rover (is the horse still there, by the way?), Jerry Pournelle (see www.byte.com) was suggesting an interesting way to fund these probes: ask everyone to donate 0.1 cents per downloaded picture (voluntarily). At 10 million downloads a day, this could add up to serious money.

> I can produce evidence that he publicly denies saying that infamous quote. You say "He did, in 1981." Where? When?

The History of Computing Foundation was my first source. The fact that Mr. Gates denies having said this carries little weight with me since I can also present definitive proof that he lies when it suits him.

Virg

P.S. I was born in 1968. Oh, and fuck you for the attitude.

Real bloat comes from libraries like STL...
First let me say that I have not used the STL much at all. However, I remembered reading an article about the design and goals of the STL which claimed that STL code, when compiled, was almost as efficient as a version written in assembly. Can any more experienced STL/C++ programmers comment on the validity of the claims? I dug up what I think is the article in the byte archives. Its by Alexander Stepanov. Here's an interesting quote from the article:
"I found that efficiency and generality were not mutually exclusive. In fact, quite the reverse is true. If a component is not efficient enough, it usually means that it's not abstract enough. This is because efficiency and abstractness both require a clean, orthogonal design. A similar phenomenon occurs in mathematics: Making a proof more abstract makes it more concise and elegant."

Real bloat comes from libraries like STL...
First let me say that I have not used the STL much at all. However, I remembered reading an article about the design and goals of the STL which claimed that STL code, when compiled, was almost as efficient as a version written in assembly. Can any more experienced STL/C++ programmers comment on the validity of the claims? I dug up what I think is the article in the byte archives. Its by Alexander Stepanov. Here's an interesting quote from the article:
"I found that efficiency and generality were not mutually exclusive. In fact, quite the reverse is true. If a component is not efficient enough, it usually means that it's not abstract enough. This is because efficiency and abstractness both require a clean, orthogonal design. A similar phenomenon occurs in mathematics: Making a proof more abstract makes it more concise and elegant."

Companies didn't have a chance or choice to adopt it. Scot Hacker wrote an excellent and insightful piece on why....

With a little more polish (multi-user, better networking) it coulda been a contender.

Frankly, no. For a system with such a small user base and development team as BeOS, the product was *highly* polished. It contained virtually every feature of a modern operating system with outstanding features ranging from the kernel (true multithreaded processing) to the interface (the textual "move to" and "copy to" options are some of the most ingenious interface considerations in a long time). Tet it's obvious that BeOS wasn't a finished product, but it was definitely going places quite fast-- and if the company was actually able to get money, the rate and quality of development would have been quite impressive. Ever hear of BONE or BeOpenGL? Besides, does an OS really need to have "polish" to market? Think of a little company in Redmond and define "polished".

The real reason BeOS wasn't "a contender" is because Microsoft screwed Be over with the fine print in its OEM contract. I suggest that you read this article by Scot Hacker with an accurate description of Be's demise.

> You'll have to come up with better than a little heat from the
> marketing guys to prove there was coercion involved. Citing a
> single instance where Microsoft followed through on their threats would
> be a help. However, nobody presents evidence of a follow-through.
> So you're all blowing hot air.

Never say all. How about DR-DOS? Here's a link where the settlement is mentioned, and I leave it to you to dig into the details of the case if you dare. In several instances (two in Europe and one in the U.S.) Microsoft stated in contracts that the companies in question were not allowed to sell a computer without paying them for a license for MS-DOS, even if they didn't actually install MS-DOS. When the company in the U.S. complained, they were told that if they sold a computer without paying the royalty, they'd lose the right to sell MS-DOS at all. This made computers with DR-DOS more expensive than computers with MS-DOS only, and since these companies couldn't afford not to sell MS-DOS, they had to knuckle under. This pressure came in the form of legal documents from Microsoft's attorneys, not "a little heat from the marketing guys" as you put it. This is one of the parts of the case that has Microsoft in such hot water in Europe.

On a more personal note, I recently got a PC from Dell. It came with Windows Millenium preinstalled, and I could not buy a PC without some Microsoft OS installed. I decided that I didn't really want it, so when I got the computer I clicked "I Do Not Agree" to the EULA. It told me to contact my PC vendor for refund information. I called Dell, and they said they wouldn't refund my money, and that I'd have to contact Microsoft if I wanted my refund. I contacted Microsoft and they told me they wouldn't refund my money either. I reminded them that Dell said they'd pay me back, and they said, "take it up with Dell." I called Dell back and they said they can't give me a refund because they can't get the money back from Microsoft.

Now would you like to tell me about hot air? Or perhaps you'd like to refute my points? Or maybe you'll give me back the money that Microsoft won't for a product I don't want and didn't have a choice about buying? Especially since they lied in their license about being willing to refund it if I didn't agree to their ridiculous EULA?

Didn't think so.

Virg

Was looking at the screenies, and I noticed something.

Take a look at this one: http://www.byte.com/documents/s=1783/byt20011112s0 001/msbob.gif

See the dog in the bottomright corner? The dog actually shows up again in XP, in the Search for Files and Folders dialog.
Thought it was kinda neat.

On Byte.com dated April 11, 2001.

So yes, AGP sort-of does some of this stuff. But there is still the issue of going on different busses to get to the CPU and RAM from the card, instead of a direct physical link between them. As far as the Xbox is concerned, the GPU is another CPU but is only sent instructions for doing graphics. I guess you could say it's kinda like SMP.

http://www.byte.com/documents/s=1436/byt20011024s0 002/1029_moshe.html

Could it be that the "Wild-West/Darwinist development methodology" of Linux has created a VM that is on par with or maybe surpassing FreeBSD's VM ??? If that is the case, I bet Matt Dillon would want to use his clue-bat on himself.

Ah, excellent question.

I read my post again and you are correct, in part, about the hot air.

I would have elaborated, however, I had to take care of a corrupted user account on an NT domain being run by a SAMBA server. Redhat 6.X distro that was set up very well, but has its quirks, as do some of the NT boxes.

So, as I said, you are correct that I did not support my argument. I apologise and wish I could have edited the comment for later, but hit submit instead. Rather stupid of me, I suppose.

The answer to your question is:
The campus I work at is moving to Oracle.
My boss said to get it, despite my minor protests that support for Microsoft's SQL server has been/will be dropped on campus.

The GIS apps we use:
a) work only with Microsoft's browser's scripting
b) the SQL server I am building in the future only works (if I remember correctly) with Microsoft's SQL server (or works best with, I'd have to re-read the documentation, again).

I suppose forced is rather strong and inaccurate.
Steered like a piece of cattle is a little more apropos. (Moo?)

Man, I hate you...sheeshe...forcing me to *think for myself* it's so...*difficult* sometimes!
(sheepish grin...that was slightly sarcastic, too, wasn't it? I gotta stop doing that).

As far as not giving examples in my own case, thanks for calling me on it.

But, no example of Microsoft leveraging its dominant market positing, erecting barriers to entry, seemingly inhibiting competition at every turn, creating contracts that lock out competitors, the "bootloader incident", the "extensions incident" and a host of other things brought up (and left out of the trial)?

All I can say is you play an excellent devil's advocate or have no knowledge of what the trial was all about.

Cheers, Zico, you gave me a much needed "mental excercise".

For some odd reason I kept thinking of the phrase I heard in a training seminar: "common sense is seldom common practice".
I give myself as an example of a unix geek running an NT lab and the DOJ snatching defeat from the jaws of victory at the last second.

It all makes perfect sense when you keep that phrase in mind.

Respectfully,

GISboy

Linux Kernel Pillow Talk

(Linux Kernel Pillow Talk: Page1of1)
By Moshe Bar

October 29, 2001

And you thought the netherworlds of dry kernel engineering were free of politics, egos, and prima-donnas? Guess again. The events of the last four to six weeks and the e-mails flying to and from the Linux kernel mailing list show how Byzantine and complex the dynamics of decision finding, features design, and implementations can be. Go to http://www.tux.org/lkml/ to subscribe to the kernel mailing list, but be careful: This is a very high-traffic list. Subscribe only if you really want to follow every single detail of the Linux kernel, or instead read the weekly digest at Linux Kernel Cousin at http://kt.zork.net/kernel-tra ffic.

Sure, the lively debates have always existed. In the past there have been disputes about the Linux firewalling code, networking code, scheduler, installer, driver model, and many more. One recurrent theme has always been the Virtual Memory (VM) manager. Nothing determines the peculiar behavior, the feel -- even the ultimate success or failure of an operating system -- like its virtual memory design. Sometime during the development cycle leading up to the Linux 2.4.0 kernel, in other words in 2.3.xx times, Rik Van Riel (http://www.surriel.com), a Dutch kernel hacker working for Brazil-based Conectiva (one of the smaller Linux distributions), introduced a radically new VM code. It was based on what seemed to be new and advanced algorithms for efficient finding, allocation, and disposal of virtual memory pages requested by programs. Rik later introduced an interesting new kernel feature called the "OOM killer." OOM stands for Out Of Memory. The OOM killer attempts to locate a killable process when memory runs out in the system. Without such a feature the whole machine can go nuts or enter a vicious cycle of swapping out a few pages, realizing immediately after that those pages are needed, and searching again for swappable page candidates, keeping the kernel busy doing only this instead of letting user processes run.

Rik is a gifted hacker, and among other things he has been trying to improve the efficiency and speed of maintenance of those lists in the kernel responsible for managing all the virtual memory pages in the system. One of the main questions to address in every operating system VM code is: "How do you choose which page to steal next when there is a RAM shortage?"

In the 2.4.0 release, the Linux kernel scans the process page and decides which page to remove. The problem with this approach is that sometimes a lot of process tables have to be scanned to free just one page, or very few pages. Also, this approach does not guarantee that the pages stolen are only those that will not be needed again very soon. Some UNIXes introduced the notion of the working set; that is, the minimum amount required by a process to function efficiently. This solution is, however, limited to per-process pages only and does not consider other kinds of pages, such as filesystem caching. Stealing from these pages might in some cases even prove counter-productive. Very often in VM theory, a solution to one problem can worsen another; that's why kernel programming is difficult.

Rik van Riel and I have variously discussed another approach, called "reverse mapping," which implements a reverse-lookup between the page and process table. Once you have reverse-mapped pages, the VM can simply scan the pages for the ones to be freed. Naturally, some extra fields need to be added to the appropriate control tables to allow this reverse mapping. My own implementation has an overhead of 14 bytes and is therefore certainly a lesser solution than Rik's -- his overhead is just 8 bytes.

Other extremely talented kernel hackers such as Marcelo Tosati and Ben LaHaise have made other important contributions to the Linux VM.

However, even though all these intelligent people tried hard to make the Linux VM fast, efficient, and powerful, user reports since the 2.4.0 release indicated poor Linux kernel performance and erratic and unstable behaviors. Up to kernel 2.4.7, for instance, on machines with small memory footprints (less than 40-MB RAM), sudden swap storms could erupt which would virtually freeze the system while it inexplicably started swapping pages in out and like crazy. In some cases, the aforementioned OOM Killer would choose the wrong process to kill; I have seen the all-important init process killed erroneously. Many fringe kernel projects, like my own Mosix project or others such as Win4Lin, suffered because users accused these projects of unstable operations, assuming that a released kernel like 2.4.0 must be free of such nasty bugs. Even though the kernel had gradually evolved from 2.4.0 to 2.4.9, it was evident that the VM design was more of a liability than an advantage.

Linus himself said in a recent kernel list mailing that he wasn't happy yet with the VM. These problems were enough for many Linux shops to resist the migration to the 2.4 kernels and instead continue using the 2.2.19 kind of kernels. Obviously, compared to 2.4., the 2.2. series has many shortcomings -- like no zero-copy networking, the division of page cache and buffer-cache in filesystem operations, big spinlocks (serializations of kernel execution paths for computers with more than one CPU) for many parts of the kernel, and so on.

A simple C program like the one below shows how kernels up to 2.4.9 had problems dealing with stress workloads on the VM system. If, after running this program, you turned the swap partition off with swapoff, your server or workstation would become totally unresponsive for up to 15 minutes.

/* based on a code originally proposed by Andrew Tanenbaum, later by Derek Glidden and many others since */ #include void main(void) { /* in the next line we allocate 200MB, but since the virtual memory page is not actually allocated by the kernel until we use it, we also have to create an access to. The amount of allocated pages should reflect the total RAM on your computer. This test runs well with machines of, say, 256MB */ void *p = (void *)calloc(50000000, sizeof(int)) ; /* In the next line we let the system calm down a bit after allocating pages*/ sleep(12); /* and now re release it all again */ free(p); }

Back in February 2001, I ran an informal and unscientific benchmark comparing FreeBSD 4.1.1 to kernel 2.4.0 (visit http://ww w.byte.com/documents/s=558/byt20010130s0010/) on exactly the same hardware and with exactly the same subsystems versions (MySQL, Sendmail, Apache, and others). The results clearly showed that, indeed, there were major problems with the efficiency and speed of the early 2.4 kernels. A New VM

Then, on September 24, with the kernel standing at version 2.4.9, everything suddenly changed. Andrea Arcangeli, an Italian kernel hacker (read my interview with him two years ago at http://ww w.byte.com/documents/s=287/byt20000229s0008/) and a very prolific contributor, decided that enough was enough. He sat down and in one of those marathon hacking bouts completely rebuilt the VM from scratch. In short succession he sent to Linus Torvalds over 150 patches to the 2.4.9 kernel, to implement a new VM engine. This is an extremely remarkable feat. A VM is a major piece of software and by nature very complex. One needs to satisfy many opposed objectives: Simultaneously efficient handling for server-type loads and interactive-type loads; ease of implementation and at the same time, optimized use of every last and small feature of the CPU. The VM must also be able to run well on Intel CPUs spanning 4 or 5 generations, as well as on AMD chips, Alphas, MIPSes, Sparcs, ARMs, and what have you. Andrea, by the way, does all his development on a Compaq AlphaServer with 2 500-MHz CPUs and 3-GB RAM.

Out of the blue, Linus accepted the new VM and incorporated it into the official Linux kernel tree.

Recently, I spent two days with Andrea giving speeches. During the two days, over many bottles of beer, we had plenty of time to discuss his new VM. I was mainly interested in how the new VM affects Mosix. Because Mosix must migrate virtual memory pages belonging to the program's address spaces between cluster nodes, it is important to correctly understand the VM and interface efficiently to it.

Specifically, Andrea took exception to the following problems in the 2.4 VM:

• kswapd looping forever on DMA or NORMAL class-zones.
• swap+ram will be almost all available address space (modulo when the swap cache serves to avoid swapin of shared anonymous memory after a fork).
• swapout storms.
• benchmarks, when run repeatedly, gradually slow down.

The new VM is much simpler and faster. Let me explain how it works.

The old 2.4 VM had a major design problem that manifested itself mainly when freeing physically dirty pages (remember dirty pages are the frames of 4-KB memory in the RAM whose contents have been modified by one of the virtual memory pages residing in it). The last owner of the page (usually the VM, except in swapoff) has to clear the dirty flag before freeing the page. When being swapped off in swapoff it may be a little more complicated -- we may need to grab the pagecache_lock to ensure nobody starts using the page while we clear it.

So, Andrea went and did the following: All physical pages are now divided into active and inactive pages. These two are further divided into dirty and clean for both active and inactive. When the active dirty pages become about 66 percent of the total number of pages, the VM starts to scan them for the oldest ones to be put into inactive dirty and then, later still, from there to the swap when memory becomes tight. This part is very central to the new VM and its simplicity is...well, simply stunning.

This elegant mechanism totally changes the behavior of the 2.4.10 kernel under heavy load and also makes for much better predictability of the system. Another very important change is that the swap is now additional to the RAM, just like in 2.2 times. All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap, similar to some versions of Solaris or other UNIXes.

Finally, the page scanner doesn't page scan if there are theoretically no freeable pages, whereas before it did. Oh, and the OOM killer never really worked, so Andrea disabled it, as I did for all my kernels. In 2.4.12 it is enabled again; this time, however, it works much better. Try it with the above program to see it in action.

Arcangeli's VM is stable, acts predictably -- something that the old VM never really achieved -- and it makes the swap space look like it did in 2.2 days. Additionally, the design is much simpler and easier to understand. People will catch up fast with it.

However, many kernel hackers disagree. Upon the release of kernel 2.4.10, a virulent and sometimes aggressive debate flamed up, with many people trying to show why one of the two was a good VM and the other not. Some comments got a bit out of control, and only in the last two weeks or so has some calm been restored.

However, one nasty side effect stays. Alan Cox, the number two man after Linus Torvalds, does not yet like the new VM and in his own kernel tree (called the "ac tree") he still continues to use and patch the old VM. As a consequence, users and system administrators now find themselves facing two very different kernel trees to choose from: the official Linux tree and the Alan Cox tree. Quite often, latest patches to drivers and new features are only in Alan Cox's tree. Those who want to go with the official Linux source code may find themselves unable to apply the patches due to the different VM code all over. It is acceptable for the two trees to be different for a few days on such important subsystems like VM, but it is not acceptable to have them different for months and across many kernel versions.

Nobody has yet dared to speak of a Linux source fork, but this is dangerously close to one.

It became obvious that the VM up to 2.4.10 was a design liability. You can try to fix something that was designed badly, but it will never become a beauty. I think Linus' decision to scrap the old VM and go with the Arcangeli VM was courageous and right. Having a functioning and stable Linux box should not be deferred to 2.5 when we can do it already with 2.4. Kernel Preemption

But apart from the VM issues, there are other lively debates in the kernel community. There was an interesting interview at h ttp://kerneltrap.com/article.php?sid=328&mode=thre ad&order=0 with Robert Love, who is leading one of two projects trying to make the Linux kernel fully preemptible. Making the kernel preemptible means making it possible to interrupt whatever the kernel is doing (say, executing a system call) to process some other outstanding task and then return to its original task. Linux, as a multiprocessing OS, obviously always did that for user-land processes. However, many, just like Robert Love, feel that the fact that Linux up to now would not let itself be interrupted contributed to poor latency. Latency describes how quickly you can expect a response from your kernel when you actually need something from it. Note that Linux is not designed as a real-time OS (though there is at least one Linux real-time implementation somewhere), and therefore does not explicitly guarantee latency. User-land programs must be aware of this as, especially with kernel preemption, latencies can be very unpredictable.

Theoretically, an OS will answer faster if it can be interrupted. What does suffer from kernel preemption is the global throughput. If you have a task that gets n seconds within the kernel to complete (let's say executing a given system call takes 0.005 seconds), then all the interruptions add some overhead to switch from one kernel task to another. So, finishing the execution of that system call (in our example) will finally require n+op where p is the frequency of switching and o the static overhead for one switching operation. Notice that kernel context switching does not invalidate the CPU cache, and is therefore not as expensive as process switching. However, kernel preemption will surely lead to a higher rate of switching from kernel space to user space, because upon preemption the scheduler might decide to give higher priority to a user process.

In other words, kernel preemption does decrease latency but slows down overall throughput. It's the math: nothing to be done against it.

Furthermore, in his interview, Robert Love heavily criticized Linus Torvalds for adopting Andrea Arcangeli's new VM in 2.4.10 and dropping the old van Riel VM.

Well, I did try the patch with kernel 2.4.12 and with pre13. While accurate measurement (which Robert Love provides with the preemption kernel patches) does indeed report an improvement in latency, for the life of me I have not noticed it on an empirical basis.

I really do appreciate Love's work, but I do not fully agree with some of his comments in the interview. First, as Linus himself said, if latency sucks in the kernel then we should check why it sucks, with or without preemptive scheduling. If the latency is bad in the stock kernel, then it should be fixed anyway.

The preemptive kernel 2.4.12 worked fine on my laptops and on my SGI 550 workstation where I do interactive work. The MP3 player very rarely skipped beats when doing heavy background work such as kernel compiling or opening large files in the editor. But for my servers and clusters, the decrease in performance and the unpredictability of latency is a problem. Also, some important patches will not apply to a Love-patched kernel. Mosix, the clustering kernel extension, does not patch correctly, and neither do some versions of the LIDS intrusion detection system.

It is up to each individual user to decide whether or not to use the patch, but is important to understand the implications of using it. Linux and FreeBSD Revisited

Upon returning home the other week after meeting with Andrea, I went to my lab and searched for the disk images of the server comparison I ran back in January of this year (of FreeBSD 4.1.1 versus Linux 2.4.0). I took the Compaq ML500 server I have been reviewing (2x 1-GHz CPUs, 2-GB RAM) and upgraded both the FreeBSD disk image to 4.4-Stable and the Linux version to 2.4.12. Then, I changed the memory down to 192-MB RAM so as to stress the VM system more. I also upgraded to the latest stable versions of Sendmail (8.12.1) and MySQL (version 3.23.42). Finally, I compiled everything with the latest version of gcc, 3.0.2, and tuned the two instances to the best of my knowledge (softupdates and increased maxusers for FreeBSD, and untouched default values for Linux).

The results were very interesting indeed. Since this benchmark is too much to be handled in this article, Byte.com will post it here soon for you to read.

The story of this article is that the 2.4 kernel has finally grown up with the 2.4.10 release. Not many users outside the relatively small kernel community realize that. Now you know about it, too. Spread the good news and immediately install 2.4.12 on your busy server. The server will thank you for it.

Moshe Bar is a systems administrator and OS researcher who started learning UNIX on a PDP-11 with AT&T UNIX Release 6, back in 1981. Moshe has a M.Sc and a Ph.D. in computer science and writes UNIX-related books.

For more of Moshe's columns, visit the Serving With LinuxIndex Page . Page1of1

Linux Kernel Pillow Talk

(Linux Kernel Pillow Talk: Page1of1)
By Moshe Bar

October 29, 2001

And you thought the netherworlds of dry kernel engineering were free of politics, egos, and prima-donnas? Guess again. The events of the last four to six weeks and the e-mails flying to and from the Linux kernel mailing list show how Byzantine and complex the dynamics of decision finding, features design, and implementations can be. Go to http://www.tux.org/lkml/ to subscribe to the kernel mailing list, but be careful: This is a very high-traffic list. Subscribe only if you really want to follow every single detail of the Linux kernel, or instead read the weekly digest at Linux Kernel Cousin at http://kt.zork.net/kernel-tra ffic.

Sure, the lively debates have always existed. In the past there have been disputes about the Linux firewalling code, networking code, scheduler, installer, driver model, and many more. One recurrent theme has always been the Virtual Memory (VM) manager. Nothing determines the peculiar behavior, the feel -- even the ultimate success or failure of an operating system -- like its virtual memory design. Sometime during the development cycle leading up to the Linux 2.4.0 kernel, in other words in 2.3.xx times, Rik Van Riel (http://www.surriel.com), a Dutch kernel hacker working for Brazil-based Conectiva (one of the smaller Linux distributions), introduced a radically new VM code. It was based on what seemed to be new and advanced algorithms for efficient finding, allocation, and disposal of virtual memory pages requested by programs. Rik later introduced an interesting new kernel feature called the "OOM killer." OOM stands for Out Of Memory. The OOM killer attempts to locate a killable process when memory runs out in the system. Without such a feature the whole machine can go nuts or enter a vicious cycle of swapping out a few pages, realizing immediately after that those pages are needed, and searching again for swappable page candidates, keeping the kernel busy doing only this instead of letting user processes run.

Rik is a gifted hacker, and among other things he has been trying to improve the efficiency and speed of maintenance of those lists in the kernel responsible for managing all the virtual memory pages in the system. One of the main questions to address in every operating system VM code is: "How do you choose which page to steal next when there is a RAM shortage?"

In the 2.4.0 release, the Linux kernel scans the process page and decides which page to remove. The problem with this approach is that sometimes a lot of process tables have to be scanned to free just one page, or very few pages. Also, this approach does not guarantee that the pages stolen are only those that will not be needed again very soon. Some UNIXes introduced the notion of the working set; that is, the minimum amount required by a process to function efficiently. This solution is, however, limited to per-process pages only and does not consider other kinds of pages, such as filesystem caching. Stealing from these pages might in some cases even prove counter-productive. Very often in VM theory, a solution to one problem can worsen another; that's why kernel programming is difficult.

Rik van Riel and I have variously discussed another approach, called "reverse mapping," which implements a reverse-lookup between the page and process table. Once you have reverse-mapped pages, the VM can simply scan the pages for the ones to be freed. Naturally, some extra fields need to be added to the appropriate control tables to allow this reverse mapping. My own implementation has an overhead of 14 bytes and is therefore certainly a lesser solution than Rik's -- his overhead is just 8 bytes.

Other extremely talented kernel hackers such as Marcelo Tosati and Ben LaHaise have made other important contributions to the Linux VM.

However, even though all these intelligent people tried hard to make the Linux VM fast, efficient, and powerful, user reports since the 2.4.0 release indicated poor Linux kernel performance and erratic and unstable behaviors. Up to kernel 2.4.7, for instance, on machines with small memory footprints (less than 40-MB RAM), sudden swap storms could erupt which would virtually freeze the system while it inexplicably started swapping pages in out and like crazy. In some cases, the aforementioned OOM Killer would choose the wrong process to kill; I have seen the all-important init process killed erroneously. Many fringe kernel projects, like my own Mosix project or others such as Win4Lin, suffered because users accused these projects of unstable operations, assuming that a released kernel like 2.4.0 must be free of such nasty bugs. Even though the kernel had gradually evolved from 2.4.0 to 2.4.9, it was evident that the VM design was more of a liability than an advantage.

Linus himself said in a recent kernel list mailing that he wasn't happy yet with the VM. These problems were enough for many Linux shops to resist the migration to the 2.4 kernels and instead continue using the 2.2.19 kind of kernels. Obviously, compared to 2.4., the 2.2. series has many shortcomings -- like no zero-copy networking, the division of page cache and buffer-cache in filesystem operations, big spinlocks (serializations of kernel execution paths for computers with more than one CPU) for many parts of the kernel, and so on.

A simple C program like the one below shows how kernels up to 2.4.9 had problems dealing with stress workloads on the VM system. If, after running this program, you turned the swap partition off with swapoff, your server or workstation would become totally unresponsive for up to 15 minutes.

/* based on a code originally proposed by Andrew Tanenbaum, later by Derek Glidden and many others since */ #include void main(void) { /* in the next line we allocate 200MB, but since the virtual memory page is not actually allocated by the kernel until we use it, we also have to create an access to. The amount of allocated pages should reflect the total RAM on your computer. This test runs well with machines of, say, 256MB */ void *p = (void *)calloc(50000000, sizeof(int)) ; /* In the next line we let the system calm down a bit after allocating pages*/ sleep(12); /* and now re release it all again */ free(p); }

Back in February 2001, I ran an informal and unscientific benchmark comparing FreeBSD 4.1.1 to kernel 2.4.0 (visit http://ww w.byte.com/documents/s=558/byt20010130s0010/) on exactly the same hardware and with exactly the same subsystems versions (MySQL, Sendmail, Apache, and others). The results clearly showed that, indeed, there were major problems with the efficiency and speed of the early 2.4 kernels. A New VM

Then, on September 24, with the kernel standing at version 2.4.9, everything suddenly changed. Andrea Arcangeli, an Italian kernel hacker (read my interview with him two years ago at http://ww w.byte.com/documents/s=287/byt20000229s0008/) and a very prolific contributor, decided that enough was enough. He sat down and in one of those marathon hacking bouts completely rebuilt the VM from scratch. In short succession he sent to Linus Torvalds over 150 patches to the 2.4.9 kernel, to implement a new VM engine. This is an extremely remarkable feat. A VM is a major piece of software and by nature very complex. One needs to satisfy many opposed objectives: Simultaneously efficient handling for server-type loads and interactive-type loads; ease of implementation and at the same time, optimized use of every last and small feature of the CPU. The VM must also be able to run well on Intel CPUs spanning 4 or 5 generations, as well as on AMD chips, Alphas, MIPSes, Sparcs, ARMs, and what have you. Andrea, by the way, does all his development on a Compaq AlphaServer with 2 500-MHz CPUs and 3-GB RAM.

Out of the blue, Linus accepted the new VM and incorporated it into the official Linux kernel tree.

Recently, I spent two days with Andrea giving speeches. During the two days, over many bottles of beer, we had plenty of time to discuss his new VM. I was mainly interested in how the new VM affects Mosix. Because Mosix must migrate virtual memory pages belonging to the program's address spaces between cluster nodes, it is important to correctly understand the VM and interface efficiently to it.

Specifically, Andrea took exception to the following problems in the 2.4 VM:

• kswapd looping forever on DMA or NORMAL class-zones.
• swap+ram will be almost all available address space (modulo when the swap cache serves to avoid swapin of shared anonymous memory after a fork).
• swapout storms.
• benchmarks, when run repeatedly, gradually slow down.

The new VM is much simpler and faster. Let me explain how it works.

The old 2.4 VM had a major design problem that manifested itself mainly when freeing physically dirty pages (remember dirty pages are the frames of 4-KB memory in the RAM whose contents have been modified by one of the virtual memory pages residing in it). The last owner of the page (usually the VM, except in swapoff) has to clear the dirty flag before freeing the page. When being swapped off in swapoff it may be a little more complicated -- we may need to grab the pagecache_lock to ensure nobody starts using the page while we clear it.

So, Andrea went and did the following: All physical pages are now divided into active and inactive pages. These two are further divided into dirty and clean for both active and inactive. When the active dirty pages become about 66 percent of the total number of pages, the VM starts to scan them for the oldest ones to be put into inactive dirty and then, later still, from there to the swap when memory becomes tight. This part is very central to the new VM and its simplicity is...well, simply stunning.

This elegant mechanism totally changes the behavior of the 2.4.10 kernel under heavy load and also makes for much better predictability of the system. Another very important change is that the swap is now additional to the RAM, just like in 2.2 times. All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap, similar to some versions of Solaris or other UNIXes.

Finally, the page scanner doesn't page scan if there are theoretically no freeable pages, whereas before it did. Oh, and the OOM killer never really worked, so Andrea disabled it, as I did for all my kernels. In 2.4.12 it is enabled again; this time, however, it works much better. Try it with the above program to see it in action.

Arcangeli's VM is stable, acts predictably -- something that the old VM never really achieved -- and it makes the swap space look like it did in 2.2 days. Additionally, the design is much simpler and easier to understand. People will catch up fast with it.

However, many kernel hackers disagree. Upon the release of kernel 2.4.10, a virulent and sometimes aggressive debate flamed up, with many people trying to show why one of the two was a good VM and the other not. Some comments got a bit out of control, and only in the last two weeks or so has some calm been restored.

However, one nasty side effect stays. Alan Cox, the number two man after Linus Torvalds, does not yet like the new VM and in his own kernel tree (called the "ac tree") he still continues to use and patch the old VM. As a consequence, users and system administrators now find themselves facing two very different kernel trees to choose from: the official Linux tree and the Alan Cox tree. Quite often, latest patches to drivers and new features are only in Alan Cox's tree. Those who want to go with the official Linux source code may find themselves unable to apply the patches due to the different VM code all over. It is acceptable for the two trees to be different for a few days on such important subsystems like VM, but it is not acceptable to have them different for months and across many kernel versions.

Nobody has yet dared to speak of a Linux source fork, but this is dangerously close to one.

It became obvious that the VM up to 2.4.10 was a design liability. You can try to fix something that was designed badly, but it will never become a beauty. I think Linus' decision to scrap the old VM and go with the Arcangeli VM was courageous and right. Having a functioning and stable Linux box should not be deferred to 2.5 when we can do it already with 2.4. Kernel Preemption

But apart from the VM issues, there are other lively debates in the kernel community. There was an interesting interview at h ttp://kerneltrap.com/article.php?sid=328&mode=thre ad&order=0 with Robert Love, who is leading one of two projects trying to make the Linux kernel fully preemptible. Making the kernel preemptible means making it possible to interrupt whatever the kernel is doing (say, executing a system call) to process some other outstanding task and then return to its original task. Linux, as a multiprocessing OS, obviously always did that for user-land processes. However, many, just like Robert Love, feel that the fact that Linux up to now would not let itself be interrupted contributed to poor latency. Latency describes how quickly you can expect a response from your kernel when you actually need something from it. Note that Linux is not designed as a real-time OS (though there is at least one Linux real-time implementation somewhere), and therefore does not explicitly guarantee latency. User-land programs must be aware of this as, especially with kernel preemption, latencies can be very unpredictable.

Theoretically, an OS will answer faster if it can be interrupted. What does suffer from kernel preemption is the global throughput. If you have a task that gets n seconds within the kernel to complete (let's say executing a given system call takes 0.005 seconds), then all the interruptions add some overhead to switch from one kernel task to another. So, finishing the execution of that system call (in our example) will finally require n+op where p is the frequency of switching and o the static overhead for one switching operation. Notice that kernel context switching does not invalidate the CPU cache, and is therefore not as expensive as process switching. However, kernel preemption will surely lead to a higher rate of switching from kernel space to user space, because upon preemption the scheduler might decide to give higher priority to a user process.

In other words, kernel preemption does decrease latency but slows down overall throughput. It's the math: nothing to be done against it.

Furthermore, in his interview, Robert Love heavily criticized Linus Torvalds for adopting Andrea Arcangeli's new VM in 2.4.10 and dropping the old van Riel VM.

Well, I did try the patch with kernel 2.4.12 and with pre13. While accurate measurement (which Robert Love provides with the preemption kernel patches) does indeed report an improvement in latency, for the life of me I have not noticed it on an empirical basis.

I really do appreciate Love's work, but I do not fully agree with some of his comments in the interview. First, as Linus himself said, if latency sucks in the kernel then we should check why it sucks, with or without preemptive scheduling. If the latency is bad in the stock kernel, then it should be fixed anyway.

The preemptive kernel 2.4.12 worked fine on my laptops and on my SGI 550 workstation where I do interactive work. The MP3 player very rarely skipped beats when doing heavy background work such as kernel compiling or opening large files in the editor. But for my servers and clusters, the decrease in performance and the unpredictability of latency is a problem. Also, some important patches will not apply to a Love-patched kernel. Mosix, the clustering kernel extension, does not patch correctly, and neither do some versions of the LIDS intrusion detection system.

It is up to each individual user to decide whether or not to use the patch, but is important to understand the implications of using it. Linux and FreeBSD Revisited

Upon returning home the other week after meeting with Andrea, I went to my lab and searched for the disk images of the server comparison I ran back in January of this year (of FreeBSD 4.1.1 versus Linux 2.4.0). I took the Compaq ML500 server I have been reviewing (2x 1-GHz CPUs, 2-GB RAM) and upgraded both the FreeBSD disk image to 4.4-Stable and the Linux version to 2.4.12. Then, I changed the memory down to 192-MB RAM so as to stress the VM system more. I also upgraded to the latest stable versions of Sendmail (8.12.1) and MySQL (version 3.23.42). Finally, I compiled everything with the latest version of gcc, 3.0.2, and tuned the two instances to the best of my knowledge (softupdates and increased maxusers for FreeBSD, and untouched default values for Linux).

The results were very interesting indeed. Since this benchmark is too much to be handled in this article, Byte.com will post it here soon for you to read.

The story of this article is that the 2.4 kernel has finally grown up with the 2.4.10 release. Not many users outside the relatively small kernel community realize that. Now you know about it, too. Spread the good news and immediately install 2.4.12 on your busy server. The server will thank you for it.

Moshe Bar is a systems administrator and OS researcher who started learning UNIX on a PDP-11 with AT&T UNIX Release 6, back in 1981. Moshe has a M.Sc and a Ph.D. in computer science and writes UNIX-related books.

For more of Moshe's columns, visit the Serving With LinuxIndex Page . Page1of1

Linux Kernel Pillow Talk

(Linux Kernel Pillow Talk: Page1of1)
By Moshe Bar

October 29, 2001

And you thought the netherworlds of dry kernel engineering were free of politics, egos, and prima-donnas? Guess again. The events of the last four to six weeks and the e-mails flying to and from the Linux kernel mailing list show how Byzantine and complex the dynamics of decision finding, features design, and implementations can be. Go to http://www.tux.org/lkml/ to subscribe to the kernel mailing list, but be careful: This is a very high-traffic list. Subscribe only if you really want to follow every single detail of the Linux kernel, or instead read the weekly digest at Linux Kernel Cousin at http://kt.zork.net/kernel-tra ffic.

Sure, the lively debates have always existed. In the past there have been disputes about the Linux firewalling code, networking code, scheduler, installer, driver model, and many more. One recurrent theme has always been the Virtual Memory (VM) manager. Nothing determines the peculiar behavior, the feel -- even the ultimate success or failure of an operating system -- like its virtual memory design. Sometime during the development cycle leading up to the Linux 2.4.0 kernel, in other words in 2.3.xx times, Rik Van Riel (http://www.surriel.com), a Dutch kernel hacker working for Brazil-based Conectiva (one of the smaller Linux distributions), introduced a radically new VM code. It was based on what seemed to be new and advanced algorithms for efficient finding, allocation, and disposal of virtual memory pages requested by programs. Rik later introduced an interesting new kernel feature called the "OOM killer." OOM stands for Out Of Memory. The OOM killer attempts to locate a killable process when memory runs out in the system. Without such a feature the whole machine can go nuts or enter a vicious cycle of swapping out a few pages, realizing immediately after that those pages are needed, and searching again for swappable page candidates, keeping the kernel busy doing only this instead of letting user processes run.

Rik is a gifted hacker, and among other things he has been trying to improve the efficiency and speed of maintenance of those lists in the kernel responsible for managing all the virtual memory pages in the system. One of the main questions to address in every operating system VM code is: "How do you choose which page to steal next when there is a RAM shortage?"

In the 2.4.0 release, the Linux kernel scans the process page and decides which page to remove. The problem with this approach is that sometimes a lot of process tables have to be scanned to free just one page, or very few pages. Also, this approach does not guarantee that the pages stolen are only those that will not be needed again very soon. Some UNIXes introduced the notion of the working set; that is, the minimum amount required by a process to function efficiently. This solution is, however, limited to per-process pages only and does not consider other kinds of pages, such as filesystem caching. Stealing from these pages might in some cases even prove counter-productive. Very often in VM theory, a solution to one problem can worsen another; that's why kernel programming is difficult.

Rik van Riel and I have variously discussed another approach, called "reverse mapping," which implements a reverse-lookup between the page and process table. Once you have reverse-mapped pages, the VM can simply scan the pages for the ones to be freed. Naturally, some extra fields need to be added to the appropriate control tables to allow this reverse mapping. My own implementation has an overhead of 14 bytes and is therefore certainly a lesser solution than Rik's -- his overhead is just 8 bytes.

Other extremely talented kernel hackers such as Marcelo Tosati and Ben LaHaise have made other important contributions to the Linux VM.

However, even though all these intelligent people tried hard to make the Linux VM fast, efficient, and powerful, user reports since the 2.4.0 release indicated poor Linux kernel performance and erratic and unstable behaviors. Up to kernel 2.4.7, for instance, on machines with small memory footprints (less than 40-MB RAM), sudden swap storms could erupt which would virtually freeze the system while it inexplicably started swapping pages in out and like crazy. In some cases, the aforementioned OOM Killer would choose the wrong process to kill; I have seen the all-important init process killed erroneously. Many fringe kernel projects, like my own Mosix project or others such as Win4Lin, suffered because users accused these projects of unstable operations, assuming that a released kernel like 2.4.0 must be free of such nasty bugs. Even though the kernel had gradually evolved from 2.4.0 to 2.4.9, it was evident that the VM design was more of a liability than an advantage.

Linus himself said in a recent kernel list mailing that he wasn't happy yet with the VM. These problems were enough for many Linux shops to resist the migration to the 2.4 kernels and instead continue using the 2.2.19 kind of kernels. Obviously, compared to 2.4., the 2.2. series has many shortcomings -- like no zero-copy networking, the division of page cache and buffer-cache in filesystem operations, big spinlocks (serializations of kernel execution paths for computers with more than one CPU) for many parts of the kernel, and so on.

A simple C program like the one below shows how kernels up to 2.4.9 had problems dealing with stress workloads on the VM system. If, after running this program, you turned the swap partition off with swapoff, your server or workstation would become totally unresponsive for up to 15 minutes.

/* based on a code originally proposed by Andrew Tanenbaum, later by Derek Glidden and many others since */ #include void main(void) { /* in the next line we allocate 200MB, but since the virtual memory page is not actually allocated by the kernel until we use it, we also have to create an access to. The amount of allocated pages should reflect the total RAM on your computer. This test runs well with machines of, say, 256MB */ void *p = (void *)calloc(50000000, sizeof(int)) ; /* In the next line we let the system calm down a bit after allocating pages*/ sleep(12); /* and now re release it all again */ free(p); }

Back in February 2001, I ran an informal and unscientific benchmark comparing FreeBSD 4.1.1 to kernel 2.4.0 (visit http://ww w.byte.com/documents/s=558/byt20010130s0010/) on exactly the same hardware and with exactly the same subsystems versions (MySQL, Sendmail, Apache, and others). The results clearly showed that, indeed, there were major problems with the efficiency and speed of the early 2.4 kernels. A New VM

Then, on September 24, with the kernel standing at version 2.4.9, everything suddenly changed. Andrea Arcangeli, an Italian kernel hacker (read my interview with him two years ago at http://ww w.byte.com/documents/s=287/byt20000229s0008/) and a very prolific contributor, decided that enough was enough. He sat down and in one of those marathon hacking bouts completely rebuilt the VM from scratch. In short succession he sent to Linus Torvalds over 150 patches to the 2.4.9 kernel, to implement a new VM engine. This is an extremely remarkable feat. A VM is a major piece of software and by nature very complex. One needs to satisfy many opposed objectives: Simultaneously efficient handling for server-type loads and interactive-type loads; ease of implementation and at the same time, optimized use of every last and small feature of the CPU. The VM must also be able to run well on Intel CPUs spanning 4 or 5 generations, as well as on AMD chips, Alphas, MIPSes, Sparcs, ARMs, and what have you. Andrea, by the way, does all his development on a Compaq AlphaServer with 2 500-MHz CPUs and 3-GB RAM.

Out of the blue, Linus accepted the new VM and incorporated it into the official Linux kernel tree.

Recently, I spent two days with Andrea giving speeches. During the two days, over many bottles of beer, we had plenty of time to discuss his new VM. I was mainly interested in how the new VM affects Mosix. Because Mosix must migrate virtual memory pages belonging to the program's address spaces between cluster nodes, it is important to correctly understand the VM and interface efficiently to it.

Specifically, Andrea took exception to the following problems in the 2.4 VM:

• kswapd looping forever on DMA or NORMAL class-zones.
• swap+ram will be almost all available address space (modulo when the swap cache serves to avoid swapin of shared anonymous memory after a fork).
• swapout storms.
• benchmarks, when run repeatedly, gradually slow down.

The new VM is much simpler and faster. Let me explain how it works.

The old 2.4 VM had a major design problem that manifested itself mainly when freeing physically dirty pages (remember dirty pages are the frames of 4-KB memory in the RAM whose contents have been modified by one of the virtual memory pages residing in it). The last owner of the page (usually the VM, except in swapoff) has to clear the dirty flag before freeing the page. When being swapped off in swapoff it may be a little more complicated -- we may need to grab the pagecache_lock to ensure nobody starts using the page while we clear it.

So, Andrea went and did the following: All physical pages are now divided into active and inactive pages. These two are further divided into dirty and clean for both active and inactive. When the active dirty pages become about 66 percent of the total number of pages, the VM starts to scan them for the oldest ones to be put into inactive dirty and then, later still, from there to the swap when memory becomes tight. This part is very central to the new VM and its simplicity is...well, simply stunning.

This elegant mechanism totally changes the behavior of the 2.4.10 kernel under heavy load and also makes for much better predictability of the system. Another very important change is that the swap is now additional to the RAM, just like in 2.2 times. All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap, similar to some versions of Solaris or other UNIXes.

Finally, the page scanner doesn't page scan if there are theoretically no freeable pages, whereas before it did. Oh, and the OOM killer never really worked, so Andrea disabled it, as I did for all my kernels. In 2.4.12 it is enabled again; this time, however, it works much better. Try it with the above program to see it in action.

Arcangeli's VM is stable, acts predictably -- something that the old VM never really achieved -- and it makes the swap space look like it did in 2.2 days. Additionally, the design is much simpler and easier to understand. People will catch up fast with it.

However, many kernel hackers disagree. Upon the release of kernel 2.4.10, a virulent and sometimes aggressive debate flamed up, with many people trying to show why one of the two was a good VM and the other not. Some comments got a bit out of control, and only in the last two weeks or so has some calm been restored.

However, one nasty side effect stays. Alan Cox, the number two man after Linus Torvalds, does not yet like the new VM and in his own kernel tree (called the "ac tree") he still continues to use and patch the old VM. As a consequence, users and system administrators now find themselves facing two very different kernel trees to choose from: the official Linux tree and the Alan Cox tree. Quite often, latest patches to drivers and new features are only in Alan Cox's tree. Those who want to go with the official Linux source code may find themselves unable to apply the patches due to the different VM code all over. It is acceptable for the two trees to be different for a few days on such important subsystems like VM, but it is not acceptable to have them different for months and across many kernel versions.

Nobody has yet dared to speak of a Linux source fork, but this is dangerously close to one.

It became obvious that the VM up to 2.4.10 was a design liability. You can try to fix something that was designed badly, but it will never become a beauty. I think Linus' decision to scrap the old VM and go with the Arcangeli VM was courageous and right. Having a functioning and stable Linux box should not be deferred to 2.5 when we can do it already with 2.4. Kernel Preemption

But apart from the VM issues, there are other lively debates in the kernel community. There was an interesting interview at h ttp://kerneltrap.com/article.php?sid=328&mode=thre ad&order=0 with Robert Love, who is leading one of two projects trying to make the Linux kernel fully preemptible. Making the kernel preemptible means making it possible to interrupt whatever the kernel is doing (say, executing a system call) to process some other outstanding task and then return to its original task. Linux, as a multiprocessing OS, obviously always did that for user-land processes. However, many, just like Robert Love, feel that the fact that Linux up to now would not let itself be interrupted contributed to poor latency. Latency describes how quickly you can expect a response from your kernel when you actually need something from it. Note that Linux is not designed as a real-time OS (though there is at least one Linux real-time implementation somewhere), and therefore does not explicitly guarantee latency. User-land programs must be aware of this as, especially with kernel preemption, latencies can be very unpredictable.

Theoretically, an OS will answer faster if it can be interrupted. What does suffer from kernel preemption is the global throughput. If you have a task that gets n seconds within the kernel to complete (let's say executing a given system call takes 0.005 seconds), then all the interruptions add some overhead to switch from one kernel task to another. So, finishing the execution of that system call (in our example) will finally require n+op where p is the frequency of switching and o the static overhead for one switching operation. Notice that kernel context switching does not invalidate the CPU cache, and is therefore not as expensive as process switching. However, kernel preemption will surely lead to a higher rate of switching from kernel space to user space, because upon preemption the scheduler might decide to give higher priority to a user process.

In other words, kernel preemption does decrease latency but slows down overall throughput. It's the math: nothing to be done against it.

Furthermore, in his interview, Robert Love heavily criticized Linus Torvalds for adopting Andrea Arcangeli's new VM in 2.4.10 and dropping the old van Riel VM.

Well, I did try the patch with kernel 2.4.12 and with pre13. While accurate measurement (which Robert Love provides with the preemption kernel patches) does indeed report an improvement in latency, for the life of me I have not noticed it on an empirical basis.

I really do appreciate Love's work, but I do not fully agree with some of his comments in the interview. First, as Linus himself said, if latency sucks in the kernel then we should check why it sucks, with or without preemptive scheduling. If the latency is bad in the stock kernel, then it should be fixed anyway.

The preemptive kernel 2.4.12 worked fine on my laptops and on my SGI 550 workstation where I do interactive work. The MP3 player very rarely skipped beats when doing heavy background work such as kernel compiling or opening large files in the editor. But for my servers and clusters, the decrease in performance and the unpredictability of latency is a problem. Also, some important patches will not apply to a Love-patched kernel. Mosix, the clustering kernel extension, does not patch correctly, and neither do some versions of the LIDS intrusion detection system.

It is up to each individual user to decide whether or not to use the patch, but is important to understand the implications of using it. Linux and FreeBSD Revisited

Upon returning home the other week after meeting with Andrea, I went to my lab and searched for the disk images of the server comparison I ran back in January of this year (of FreeBSD 4.1.1 versus Linux 2.4.0). I took the Compaq ML500 server I have been reviewing (2x 1-GHz CPUs, 2-GB RAM) and upgraded both the FreeBSD disk image to 4.4-Stable and the Linux version to 2.4.12. Then, I changed the memory down to 192-MB RAM so as to stress the VM system more. I also upgraded to the latest stable versions of Sendmail (8.12.1) and MySQL (version 3.23.42). Finally, I compiled everything with the latest version of gcc, 3.0.2, and tuned the two instances to the best of my knowledge (softupdates and increased maxusers for FreeBSD, and untouched default values for Linux).

The results were very interesting indeed. Since this benchmark is too much to be handled in this article, Byte.com will post it here soon for you to read.

The story of this article is that the 2.4 kernel has finally grown up with the 2.4.10 release. Not many users outside the relatively small kernel community realize that. Now you know about it, too. Spread the good news and immediately install 2.4.12 on your busy server. The server will thank you for it.

Moshe Bar is a systems administrator and OS researcher who started learning UNIX on a PDP-11 with AT&T UNIX Release 6, back in 1981. Moshe has a M.Sc and a Ph.D. in computer science and writes UNIX-related books.

For more of Moshe's columns, visit the Serving With LinuxIndex Page . Page1of1

Although it will be comparing a moving target (Linux 2.4.x) to a moving target (FreeBSD 4.x), the results will be interesting. AFAIK, there weren't any major changes (I mean like VM changes :) in FreeBSD, so comparing the old and new benchmarks would give a good indication on how much Arcangeli's VM improves things.

Is it too much to ask the slashdot editors to check things like this before posting ? This troll is not even worthy of inadequacy.org

Becuase it's against the licence for OEMs to make any other OS available besides Windows. See this.

This article describes how the HZ value of 100 is obsolete for modern processors and should be increased to 1024. But the problem is that most user-level code is broken in that it assumes the Linux HZ constant is, well, constantly 100.

We think the USDOJ should stop Microsoft from undermining dual boot PCs.

Does this fly in the face of the "innocent until proven guilty" policy? Definitely. But these new laws aren't there for the citizens' benefit - they're there for the snoops, and the snoops don't care if you're sent to jail for 20 years because you couldn't prove you weren't using PGP.

-sting3r

District Court Judge Colleen Kollar-Kotelly
United States District Court for the District of Columbia
333 Constitution Avenue, N.W.
Washington, D.C. 20001

Judge Kollar-Kotelly:

Congratulations on being appointed to the Microsoft case (United States of America v. Microsoft Corporation and State of New York, et al. v. Microsoft Corporation, Civil Action Nos. 98-1232 and 98-1233). This is a landmark case that will affect everyone who uses a personal computer.

I have been a victim of Microsoft's illegal monopolistic practices for over ten years. In that time, Microsoft has made it very difficult for me to use alternatives to their products. In every other market, I have complete freedom of choice. Ford doesn't prevent me from buying a Honda automobile. Toshiba doesn't do anything to hinder my purchase of a Sony TV. Delta Airlines doesn't prevent Continental from flying to any airport. Yet after all this time, I still cannot purchase any IBM-compatible computer I want without Windows pre-installed.

The Department of Justice claims that the tying of Internet Explorer (Microsoft's web browser) to Windows is the most egregious violation of their monopoly. I disagree. The real problem is the secret agreements between Microsoft and the computer vendors (e.g. Dell, Compaq, Gateway, etc.). Vendors enter into these agreements so that they can purchase Windows at a competitive price. Unfortunately, Microsoft includes special conditions in these agreements, and it is my opinion that these conditions are what make Microsoft an illegal monopoly.

If you can, please pull up your web browser and go to

http://www.byte.com/documents/s=1115/byt20010824s0 001/

There, you can read an article about the boot loader issue. The boot loader is a piece of software that determines which operating system your computer loads when it is turned on. On most computers, only one operating system is installed, and the boot loader automatically and transparently loads it. However, it is possible to divide your computer's hard drive into two or more sections and have a different operating system installed in each section. In such cases, the boot loader will ask the user to choose which operating system he wants to load. This happens every time the computer is turned on.

Microsoft's agreements with the vendors prevent them from selling a computer which provides this feature. As you can tell from the article, the company Be was willing to let vendors include their operating system, BeOS, free of charge. Computer vendors today are having a very difficult time competing against each other because their products are all very similar: IBM-compatible personal computers with the Windows operating system. Many vendors would like to ship a computer that has two operating systems, because this would help differentiate their products. A computer that had both Windows and BeOS (or Linux, which is another free operating system) would allow the customer to continue using Windows, but would also allow him to try BeOS, Linux, etc. Microsoft understands that this feature is a serious threat to their monopoly, which is why they disallow it.

When a user purchases a copy of Windows or a computer with Windows installed, that user has to agree to a license. This license, known as an End User License Agreement (EULA), generally allows the purchaser to use his copy of Windows as he sees fit, provided he uses it only one one computer. I believe that this agreement is sufficient for the computer vendors as well, and that computer vendors should not be forced to accept more restrictive agreements just to get a lower price.

I understand that your task in the Microsoft case is to find a remedy to their illegal monopolistic practices. Your remedy will be far more effective if it prohibits these secret agreements. The vendors should be allowed to install Windows however they see fit, with any additional software they want. They should also be allowed to remove whatever parts of Windows they don't want (such as Internet Explorer). This would put Windows (and hence Microsoft) on the same playing field as vendors of other operating systems.

Sincerely,

barring Microsoft from interfering with the way PC makers set up startup screens, the Windows desktop, preferences, and Internet connection wizards.

The story really is News for Nerds. Stuff that matters.

Take a look why.

SJVN commentary on distributed computing and some interviews with various people in the field.

SJVN commentary on distributed computing and some interviews with various people in the field.

This article covers three distributed OS options, with some intro explination of the difficulties. I would think the easiest (not necessarily the best) solution could be to use Mosix (listed last in the article) and thread your application to a logical extent. Mosix won't interfere with your current linux boxes, just add it on. The tasks will automatically be load-balanced among the machines.

is here

. . .

Please don't flame me, as a helpful AC ( in this post ) has already mentioned this machine's existence.

I've been thinking hard about this one. Byte Reviewed the new Libretto L1 here and it sounds awesome. Not only Crusoe based, but has Bluetooth too. Which may not be to your liking, or cause grief on 2.4ghz, depending on your air interface preferences. But hey, I got a Bluetooth GPRS mobile and it's soooo tempting :)

The informed AC gave a very cool reference for Linux info : on Yahoo Groups to which I can only add this picture gallery froma company I found who sells the things properly localised, but, sadly, not with a distro.

Please forgive me if my post already redundant, but this little machine could rock.

Oooh - oo I just saw NEC have some MIPS based things that look like rebadged HP Jornada 720s, only nicer looking. Wonder if anyone can get Linux support on these???

• Though people ask for it continually, people in the know, such as _BeOS Bible_ author Scot Hacker, have repeatedly said that an open source version of BeOS will basically never happen. The system depends on licensed code that Be apparently couldn't give away even if they wanted to. I'd like to see this happen as much as everyone else, but don't count on it ever happening.
• New math department: according to The Register, Be's recent financial reports indicate that revenues are up over 600 percent. Thus proving that 600% of nothing is still, well, nothing.
• Supposedly, somewhere on beosradio.com, a ready to ship copy of BeOS r6 has been presented to CEO Jean Louis Gasseee. Various interesting takes on this one. Supposedly development on the desktop OS had basically halted, with all effort going into the IA version, so it would seem that there isn't enough code to be worth releasing a new version of the desktop OS. This is a shame, because a couple of useful components -- BONE networking, OpenGL graphics, etc -- were apparently under development before the switch to the IA focus, and it isn't clear if these components were then or are now ready for prime time. It could be a move to just get out one last version in whatever state it may be in, or there could actually be some new developments that haven't been publicized.
• Discussion at BeGroovy suggests that, among other things, this Palm press release would indicate that they're the likely buyed, while another commenter suggests, supposedly on good authority, that Sony is the likely buyer and they're already feeling out where they would want to go with Be & its technology. Then again, a a followup to that said that, at least as far as releasing BeOS6, he was full of it, and that the only developer working on BONE has been on an extended vacation anyway. Finally, one commenter noted that the final issue of BeDope ["Be's own Onion" --me.] had anticipated all of this months ago. Hrm....
• Over at BeNews, there was yet another link to the Reg article and a whole lot of discussion, generally going nowhere as these forums are wont to do, throwing out speculation that the buyer -- if there even is one, don't forget that this is still just a rumor -- could be any of Palm (they seem to like that idea; I'm not sure I see it but hey whatever), Gobe (developer of Be software -- seen as unlikely as they probably don't have much more cash than Be does), AOL, Compaq, Sun (now *that* would be a nice Network Computer...), Symbian, QNX (why?), Apple (doubt it), Microsoft (pretty sure that was a joke...) (too bad...), Amiga (ok that was definitely a joke), IBM, Hitachi, Samsung, Nokia, Transmeta, Intel, Red Hat (we're pretty safely into wild speculation territory at this point), SGI (see? completely off the wall, these people have no idea what they're talking about), QSSL (bonkers), DoCoMo (two unprofitable ideas that lose money together!), Wind River (who?), Ericsson, etc. Mostly this is all silliness. Towards the end of the conversation, a commenter notes that over on Yahoo's forums, the rumor has been confirmed (by who?), that the stock price is expected to shoot up (whoa, a whole dollar! golly!), and there will be an after hours announcement. Keep in mind however that, not so long ago, a 15 year old kid had such financial forums in the palm of his hand with his "expert" advice, so take that with the appropriate amount of salt. Still, something to watch for anyway.
• Meanwhile, *checks* yes, Be's own press page hasn't been updated since May 17. No help there...

Hopefully all those links work, if not I apologize. I'm just summarizing the various pages that I've skimmed over the course of today. If there's any truth to the Yahoo rumors, there could be confirmation of this as soon as tonight. Though it would be sad to see the company shut down or swallowed whole, a lot of people have seen this coming for a long time, and it would be nice to have some resolution of the situation. BeOS is some great consumer computing technology, and I hope very much that it has a future. Perhaps we're about to find out if that is the case...

I can't get cable modem here: Adelphia has a monopoly, and while they offer cable modem in many parts of their L.A. franchise area, they haven't seen fit to bring it to Studio City. I can't get DSL because I am a couple of thousand feet too far from the switch.

SNIP

That left Ricochet, a spread-spectrum radio link; and that worked, and is again working. Of course, Ricochet filed for Chapter 11 bankruptcy protection about a week after I got it running, but that doesn't worry me as much as it might. The service works and works well, the physical facilities have been built in a number of cities (alas not in Las Vegas; I'd love to have it for Comdex), and someone will pick up the pieces and keep it operational.

It really is a pity that the maze of regulations of phone and airwaves have strangled competition and prevented good (actually, any) broadband service from being extended to most of the US. The lesson here is that we can't afford to regulate things; regulation may seem to fix a current problem, but it will cause MUCH worse problems down the road, and be politically impossible to remove when it has proven itself disasterously bad (think California electricity).

And, while the Web Edition has some good stuff, especially Udells' columns, it's not as good as the old dead tree version.

Bandwidth wants to be free! Really, it does. :)

I doubt this guy's benchmarks are very accurate. Here's one of the Pentium II vs. PowerPC from 1997 (kind of old, but still very relevant) which proves to be a much closer battle.

there was a test run by byte on freebsd 4.1.1 vs linux 2.4 for various server tasks. Slashdot article is here and a direct link is here . Not sure if it's what you're after but its a start:)

I would agree with the other posters saying bsds are better for server and linux for desktop, although you can exchange them. I don't use bsd for desktop because it doesn't support nvidias drivers (linux only) and netscape is much less stable on fbsd than slackware on my system. All my work computers I installed freebsd on.

I want something like Scot Hacker described in this article (Be's BeIA-based Aura device, aka HARP, aka Home Audio Reference Platform). Aside from doing everything else that's cool, it will either hook up to it's own LCD panel, or to your TV set, to have full visual GUI for navigation.

Plus you gotta have wireless ethernet, so you and your entire family can stream all their MP3's -- AT THE SAME TIME -- to their Wi-Fi "net speakers," located throughout the house and underwater in the swimming pool. Each net speaker set can be controlled by any Wi-Fi webpad or computer system on your network, of course.

And some people think there's no market for IA's... what visionless morons!

A Google search shows that this isn't completely safe:
Byte Article

I'm really glad to see FreeBSD starting to make some traction in SMP server farms. With FreeBSD/Alpha running on 32 processors.. oh wait, that was Linux. I forgot that FreeBSD can hardly juggle two processors.

Excuse me, but I believe you forgot to enclose your comment in <moron> tags. Please don't let it happen in the future.

the evidence