Slashdot Mirror
Microsoft Reports OSS Unix Beats Windows XP
Mortimer.CA writes "In a weblog entry, Paul Murphy mentions a Microsoft report (40 page PDF) that in many instances FreeBSD 5.3 and Linux perform better than Windows XP SP2. The report is about MS' Singularity kernel (which does perform better than the OSS kernels by many of the metrics they use), and some future directions in OS design (as well as examination of the way things have been done in the past)." From the post: "What's noteworthy about it is that Microsoft compared Singularity to FreeBSD and Linux as well as Windows/XP - and almost every result shows Windows losing to the two Unix variants. For example, they show the number of CPU cycles needed to "create and start a process" as 1,032,000 for FreeBSD, 719,000 for Linux, and 5,376,000 for Windows/XP."
Here's an interesting snippet I found while perusing the PDF...thought I'd share.Interesting...Singularity is ostensibly supposed to be about stability, but the 44-page paper has no data on this. Kinda like saying, "Our new bulletproof vest is 40% lighter than our leading competitors, and twice as flexible. How well does it stop bullets, you ask? Sorry...we do not yet have results for that benchmark.".
Wake me when a paper comes out about Microsoft's new stability-oriented OS that actually addresses that particular aspect of the product.
____
~ |rip/\/\aster /\/\onkey
Isn't it telling that the idea of Microsoft telling the truth is considered front page news on /.?
The Russian Mafia will mod you down just to see if the Moderate button works.
hmm funny, the last step is Acceptance. Too bad it seems Microsoft skipped the "bargaining" step.
"Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
Singularity is a very interesting system. But that's not surprising, when you consider some of the brains behind it: Galen Hunt, Wolfram Schulte, Ulfar Erlingsson, Rebecca Isaacs, and many others who are well-known for their research.
In twenty or so years we may look back at Microsoft Research with the same admiration we have for Bell Labs.
Cyric Zndovzny at your service.
Well, that's sort of to be expected. Stability is not as easy to measure as other things, since you need benchmarks over a long period of time. Further, since it's still a research OS, it's likely in constant flux and doesn't have the same kind of stability hardening of a retail OS.
If you need web hosting, you could do worse than here
Microsoft is not Microsoft Research. Microsoft Research folks use and make free software, and in general are not ideologically bound to the parent organisation.
It's not so much about its ability to start thousands of processes. What is important is that it takes Windows XP five times as long as FreeBSD to create a single process, and seven times as long as Linux. That's a significant difference.
Cyric Zndovzny at your service.
I bet the person who put that report on MS's site has been drooling over the severance package... ;-)
- Preferences: Solaris 10 (servers), Ubuntu (desktops), Solaris 11 (personal servers) -
I just have to bow before the guy who can read a 44 page pdf and post an intelligible, coherent comment on it in less than two minutes. I just have to ask - where do you get that kind of caffeine?
Amazing.
The Russian Mafia will mod you down just to see if the Moderate button works.
For one thing, Windows is not slower than Unix in most of the tests. It's slower than Unix in some of the tests and faster in others. For another, these benchmark results are for low-level things like spawning processes and threads. Any programmer who knows anything about Unix and Windows will tell you that threads are cheaper in Windows and processes are cheaper in Unix, because that's how they were designed. So of course Windows is going to be slower than Unix at creating processes, and of course Unix is going to be slower than Windows at creating threads.
The only thing worth reporting about this thing is the performance of Singularity, which looks like it's shaping up to be an excellent modern kernel.
The future: Longhorn will suck far more memory than XP.
They must be in cahoots with the memory makers, alert Rambus!
A feeling of having made the same mistake before: Deja Foobar
see which takes longer
Is "searching the manpages" included in the benchmark time?
*Ducks*
NT (and its latter incarnations like XP and so forth) are desgined to use threads rather than process for multi-processing/concurrency, I understood. Is Win XP less efficient in multi-threading than BSD/Linux? The history of threads in UNIX seems more later bolt-on - UNIX was designed with multi-process model, I think.
No I didn't RTFA.
is also the number of cycles needed to crash a process.
Finally a proper microkernel OS design by Microsoft! prof Tanenbaum would be proud!
/.: Die M$ XP, DIE! *pinky to mouth*
Come on, who cares about statistics? I'm glad they're actually doing something useful: CS research!
Oh wait, this is
This sig is intentionally left blank
This is pretty typical. Microsoft's biggest competitor is their old software, so their new offerings have to look good against it.
Remember Windows 95's marketing? "32-bit memory protection makes it uncrashable!" Remember Windows 98's marketing? "Even more stable than 95!" Remember Windows 2000's marketing? "Based on an NT core, it's more stable than the crash-prone Windows 9x!"
Its revisionist history. The only way to get a somewhat accurate picture is if you compare their current claims with what they've said about new technology in the past.
Bogtha Bogtha Bogtha
This article takes a very interesting report on a reference implementation of some innovative ideas in OS design and reduces it to a couple of entirely peripheral, seat-of-the-pants benchmarks that support the "OSS rulez!" thesis.
Even people like me, who have only a basic knowledge of OS architecture, can tell you that processes are lightweight in Unix and heavyweight in Windows. The lightweight objects in Windows are threads, which is why Windows makes so much use of threads, while Unix spawns processes left and right.
The scenario stated in the slashdot post does show a situation where linux performs better than Windows. ...but after looking through the "performance" section of the whitepaper, it's pretty much the only case where linux is better. Windows appears to beat linux on quite a few other tests (such as memory use of a 'hello world' program, the executable size, and even some of the 'cost of basic operations' tests)
I store my recipes online (the way nature intended)
This is Microsoft Research. They have the same independence as university researchers--that is how Microsoft lures them away from academia. These guys are making honest comparisons to Linux and FreeBSD, because that is what they do as good researchers. Microsoft is enlightened enough not to interfere.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
You clearly don't know much about what makes an operating system stable... Stability depends partly on how much error checking the compiler is capable of doing, partly on how people write software (design) and partly on how well the operating system is designed to separate processes and different parts from each other. Singulary addresses all of these issues: Its mainly writen in a "safe" language which is strongly typed and does lots of compiletime check and it is a microkernel operating system which (at least in theory) prevents your cheezy usb webcam driver from crashing the kernel. Most other unix wannabe systems are writen in the ancient language C :), and run monolithic kernels.
:) (btw, I am a long term happy gnu/linux user, and have no plan of switching...)
But singularity isn't all new, it just implements old ideas: Occam and QNX!
But in my opinion, Singularity just might be the most interessting os to emerge in the last years. It will be interesting to see how long it will take the free software world to come up with something similar
Instead of paying rapt attention to what Microsoft is doing what I would like to see the OSS community do is consciously form more organizations that would as an express purpose chip away at Microsoft's software base. What I mean by this is make sure your program runs on Windows for now. Get people using OSS and used to the idea so that the next time average-joe needs some software he'll search for an OSS program first. Then once that mindshare has been established begin to work towards the more core functions like the OS itself. Who knows, Microsoft might at some point simply open up the source of Windows to counter a loss of control to OSS if they see that their customers are truly ready to abandon ship. And to build that feeling in customers give them options - if all their useful software is OSS then they can swap out the lower levels (like Linux for Windows) without feeling any transition pain at all because their software applications didn't change at all only the plumbing did.
Ballmer's right, it is all about developers. OSS developers can introduce OSS values into the Windows "ecosystem" for lack of a better word and see what happens.
Shh.
Processes in Unix are lightweight objects, and the OS spawns them left and right. Processes in Windows are heavyweight objects, and the OS creates only a handfull of them. The lightweight objects in windows are threads, and you'll notice that Windows thread creation is faster than Unix thread creation. These are just different OS design philosophies.
he gets it right here
Entirely OT, I know, but...
Why is it that some people seem to think that all OS names, when they have a qualifier of some kind attached to the generic term, need a slash to separate them? Just because GNU/Linux is written that way does not mean it's some kind of law, people...
It's Windows XP. That's WINDOWS {SPACE} XP. And Mac OS X. Spaces. No slashes.
...
I don't know why I even bother...
Dan Aris
Fun. Free. Online. RPG. BattleMaster.
Here's the table from the paper, ranked best-worst, W=windows, F=freebsd, L=linux, S=singularity:
Read Cycle Counter: W: 2, F: 6, L: 6, W: 2, S: 8
ABI Call: S:87, L:437, W:627, F:878
Thread Yield: S:394, W:753, L:906, F: 911
2-Thread ping-pong: S:1207, W:1658, L: 4041, F: 4707
2-Message ping-pong: S:1452, L: 5797, W: 6244, F: 13304
Process Creation: S: 300000, L: 719000, F: 1032000, W: 5376000
The only stat in this table that Windows trails on is process creation. And anybody who has ever ported Unix code to Win32 knows exactly why: Windows is thread-oriented, and Windows systems don't tend to use helper programs or demand-forking to get work done. Which might be why Windows beats Unix in the thread benchmarks, but not in the IPC benchmarks. On the more general benchmarks, like cycles to issue a system call, Windows falls smack in the middle --- and, again, Windows has a slightly different take on what is and isn't a system call.
Drawing comparisons between Singularity and normal operating systems here is silly. Singularity doesn't have processes in the conventional sense; since there's no hardware dependencies on "process" creation in Singularity, IPC and forking are much faster.
Which is why this benchmark is reasonable inside the Singularity tech report (they're trying to demonstrate that there's a major performance benefit in rethinking boundaries between programs), but totally unreasonably outside that context: these are micro-benchmarks, like the ones CISC and RISC people throw at each other, and don't describe the amount of time it takes to complete a high-level task. Time to execute a system call is meaningful only in the context of how many system calls it takes to complete the task you're measuring.
While technically, reboots are not required for anything other than kernel patches, there are lots of situations where it's easier to reboot than to restart every application (which might as well be a reboot anyways). For example, glibc updates will require almost every application to be restarted, or you risk exposing vulnerabilities.
If you need web hosting, you could do worse than here
It's a Microsoft OS, and you're saying that they made a mistake when mentioning that one of their goals is increased dependence? Hell yes that's their goal. Vendor lock-in, forced upgrade cycles, dependence - all the same thing, and all the goal of any winning software company. :)
Because a gigahertz is a BILLION hertz, not a MILLION.
With spending like this, exactly what are "conservatives" conserving?
In other words, write the benchtest for the sorts of sub-category of cases that side with you, and you can make any benchmarks show what you want them to show. That's one reason they should always be viewed with a pinch of salt and a dash of vinegar, then served in newspaper.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
init 1
init 3
No reboot required
--fatboy
I bet you don't get a dancing paperclip with Linux, do you?
Deleted
Same difference... you've still shutdown all your network services which to the users means you've had downtime. It's a reboot in all but name.
really... how exactly do you replace a running libc?
Typical distros that support pervasive no-reboot updating (like Debian) don't exactly replace a "running" libc (or any other library), they simply update the on-disk copy. So any programs run after that will get the new libc, but any programs that were started before the update will of course be using the old libc.
Usually this works very well; I suppose for a mega serious security update you might want to restart all your daemons too or something.
We live, as we dream -- alone....
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
Here
"reality has a well-known liberal bias" - Steven Colbert
It's funny that you should mention that Linux is not Unix, as I'd dare say that Linux = Linux is not Unix, or LinUx. :P
You didn't really read it, did you? From TFA(bstract).
The point of the paper is NOT to demonstrate a fully working uber-dependable system, but to validate the practicality of the architecture that is under development, and the new technologies being included. That's why they have the section on performance, with the preface (right above your quote, btw):
That's the point of the paper. I understand, however, that you might have been in too much of a rush to get first post that you didn't understand the point of the paper...
Shouldn't you be doing something useful?
For example, apache and sshd, and various FTPds, can be restarted without anyone possibly noticing, because they simply leave any running children open. You connected before a certain time, you got the old copy, you connected after it, you got the new one.
And, of course, many protocols work fine if you go away for five seconds, like SMB. The client program will just say 'oops, connection hiccup' and reconnect silently, and the end user never notices. Same with IMAP clients. They go 'Hey, the server closed my connection, I better open it again'.
Restarting services on a Linux box is 99% transparent to end users, even ones that are currently directly doing something with the server.
Rebooting is not transparent, even if all the connections are reaqquired automatically, simply because work stopped for the two minute reboot.
If corporations are people, aren't stockholders guilty of slavery?
Hmm, Murphy is a known M$ shill, they must be very desperate to get clicks if he resorts to this kind of admission. Or there is some very dark reason i couldn't even fathom.
Patents Drive Free Software as Hurricanes Drive Construction Industry
mmm... I can see that in a few specific cases, like if you have a lot of users who log on over ssh. Less so for webservers and remote filesystems where you bounce the runlevels fast enough, the interruption will probably never be noticed.
Of course, the context where the Curse Of A Thousand Reboots really bites is for the home computer. I mean, I only have one user on this machine. Rarely I'll have two, never any more than that. So if I cycle runlevels, no-one is going to be put out bar me - and I'm the one doing it.
In General, I find that the people inconvienced by a compulsory reboot are not networked users.
Of course, even if you have remote users, your downtime is going to be a lot less if you don't have to go through POST, bios initialisation, device scanning and all the rest of it. And of course you only have to do it once, becaue you're controlling the process, so you don't get fifteen reboots in a row because windows brute forces everything.
So, I think "all but name" is overstating the case. By rather a lot, actually.
Don't let THEM immanentize the Eschaton!
Yawn, same old stuff - read the rest, Windows is better at thread switching. That makes up for the slow process creation. Windows programmers know that processs creation is slow, and thread creation is quick.
As a result, you get tons of unstable Windows applications because to get any reasonable concurrency you have to throw out the years of hard work that OS designers put into having protected memory.
Threads vs. processes isn't "two different ways of doing the same thing". Barring a massive implementation boondoggle, you make that choice based on whether you want memory protection or not. These numbers highlight a massive boondoggle, which takes the correct choice away from the application author in many cases.
rage, rage against the dying of the light
Actually results are mixed; and they even seem to indicate that Linux seems to excel at nothing but quick process startup (which is cool if you do lots of shell scripting, maybe compiling too)
9 45599
According to the benchmarks published there
- at most OS jobs like threading/process creation, Singularity is at least twice as fast as linux, Linux is very fast at process creation, while XP is good at threads
- in File Operations FreeBSD and Linux beat XP and Singularity at random reads
- in File Operations XP beats Linux and Singularity at sequential reads, with the exception of FreeBSD being fastest if blocksize is high(and very bad for small blocksize)
- linux executable sizes are larger than these of the other OSes, (whatever that means, more good coding, or less bad code SCNR)
Please bear in mind that a benchmark does not tell whether the "slower" OS actually invested more time in doing some smart stuff that pays off in some other way. In particular, I would not be surprised if an experimental OS like Singularity did less.
partial repost from http://slashdot.org/comments.pl?sid=167223&cid=13
I'm still trying to figure out what people mean by 'social skills' here.
My always on XP SP2 machine has not had any spyware in 3 months. Its the dumb users who get infected. There are less dumb people using Linux (due to the learning curve), therefore less problems with unwanted computer activity. An XP machine properly set up with firewall, spyware, and virus scanners/blockers, and used responsibly (no Kazaa) will get a serious virus about as often as a *nix user will get rooted.
Some of the things that Win32 processes do that SFU and native processes don't:
- The Application Compatibility Database, a user mode service has to be contacted to see if the new program needs to have any compatibility shims added. Half of the compatibility that XP has comes from modifying programs as they start, or giving them special treatment. This stage alone causes so much overhead that Windows Server 2003 has a special group policy that lets you turn it off to make starting processes faster.
- The Software Restriction Policies database, a set of registry entries that have allow/deny rules for starting processes based on hash, filename or certificate. To make any actual comparisons, the entire binary has to be hashed and checked for certificates before the program even starts.
- Registering with the Win32 subsystem server (csrss). This involves several out-of-process function calls.
- Load the current locale, including NLS files.
- If enabled, contact the Themes service.
Except for talking to the Themes service, all those steps are done for every new Win32 process, even if it doesn't have a GUI.Insightful? How about "apologetic bull"?
My job is in QA. Your statement says that my job is impossible. Here are a few ways you can test stability:
1) See if the OS comes back online after a power cycle
2) Insert and remove device drivers
3) Send mangled data across the various data busses
4) Run programs that try to allocate all the memory
5) Run programs that try to hog all the CPU
6) Run a program that fills the hardisk/erases the hardisk/refills the hardisk
7) Do all the above all at the same time
The OS is a control point, and should be able to handle all of this and more GRACEFULLY.
(Gracefully being an undefined weasle word indicating that it should fail in a somewhat predictable manner that won't completely piss off the vast majority of administrators.)
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
The fact that the virtual machine is a bit slower isn't the point. The point is that because the virtual machine ensures memory protection, Singularity doesn't need to use hardware memory protection for the kernel. Doing a single system call costs hundreds of clock cycles on a modern CPU, because of the userspace/kernelspace switch. It also necessitates all sorts of complex (and slow) IPC mechanisms that go through the kernel (and invoke the aforementioned switch), all because we're still programming in an antiquated 1970's era language that let's programs randomly write into memory.
Modern CPUs quite be quite a bit faster if they didn't have to support C. Take a look sometime at all the die space an Athlon64 uses for stuff like TLB, etc. Also look how it needs to increase L1 cache latency by 50% (from 2 cycles to 3), just to support the TLB lookup. All of this stuff would be unnecessary if C programs couldn't overwrite whatever memory they wanted.
A deep unwavering belief is a sure sign you're missing something...
Perhaps everyone knows this, but...
The reason you can replace something like libc on a running Unix system, is the result of the way the Unix FS works. If you open a file, in this case libc, the kernel sets a reference count on that inode. If you subsequently unlink() (delete) that file, the kernel doesn't actually remove it until the reference count goes to 0. This means already running processes will be unaffected by this change, while new opens would fail.
In the case of a libc upgrade, one unlinks the old file, and replaces it with the new one. New apps start and link against the new libcxx.so. Old apps work as expected.
Windows doesn't work this way, at least not what i've seen
--
Mu
I didn't mean to say that there aren't some negative consequences to the choice of making threads performant and processes less so. There are, and your post correctly identifies one of them. But I think it's wrong to say that that design decision is therefore across-the-board wrong.
There are 2 seperate issues here
1. Are threads faster than process? Yes, on both Unix and Windows.
2. Are process so slow as to be essentially unusable for concurrency? On Windows, yes for a relatively large problem domain.
(2) is wrong. You force the programmer to give up memory protection in order to use an unrelated feature (concurrency).
There are times when a threaded architecture is appropriate, and that is harder to do in Unix (which is why Apache 1.x spawns processes, even though the 2.x codebase shows that threads are a better idea).
No, Apache used processes because they were a better idea. That's why most sites running 2.x on Unix continue to run it in multi-process mode even though multithreaded mode is available.
Not only does the multiprocess model avoid synchronization issues (see, e.g., http://www.zeuscat.com/andrew/work/aprbench/ for benchmarks showing that pre-fork is faster) but it avoids all the security/stability issues that come along with multithreading.
Would you rather have an obscure bug in some module you're using kill the current connection (and if it's repeatable, make that one page unloadable) or take down the whole web server? Do you want to limit the scope of security breaches?
Now, thread switches on their own are faster than process switches. With careful crafting you could design a threaded server that avoids synchronization issues and have it be faster than the multiproc version by that margin--but that's a very tiny margin that will likely be lost in the noise in any real-world server, and you're talking about increasing development cost dramatically for a return that's marginal at best. You'd be far better served putting that development effort into a state-machine based solution like thttpd or phhttpd if that level of performance is a concern.
That said, threads do have their place. Some problems are best solved with a shared-memory solution where the memory you need to share can't be easily isolated into a few SHM segments. And when that's the case, threads are the right tool for the job.
But the point is that if you don't have a reasonably performant process implementation then you remove the more commonly useful tool and force programmers to accept one feature (memory sharing) that could be very bad in order to use the one they really want (concurrency)--when those two features are really unrelated.
This problem isn't limited to Windows, either; Java similarly suffers from having no good way for programmers to use a multiprocess design.
It is, IMO, the single biggest technical problem with Windows.
rage, rage against the dying of the light
Threads certainly have a place, I never said otherwise. The problem is that the Windows system forces you to accept shared memory to get concurrency, and those two are unrelated. The number of problems that want concurrency and memory protection is large, and eliminating that option is a MAJOR problem.
Having done a fair amount of GUI programming myself, I find a multiprocess solution is often correct (e.g. in something like Photoshop image filters, where you want shared access to one memory segment but don't need to share huge numbers of different memory segments where you can't easily compartmentalize them). For some jobs, though, multiple threads is better. Use the right tool for the job.
rage, rage against the dying of the light
I didn't say your job was impossible, but your job is only testing a small subset of reliability. Reliability is also whether or not the OS stays up for a year at a time, or whether it has long term memory leaks. Reliability also has to do with weird race conditions that only show up after several programs interact for a significant amount of time, etc...
What you're testing is simple stuff, stuff that's easy to identify. There's a whole other class of reliability testing that's far more long term.
If you need web hosting, you could do worse than here
Throw out years of hard work? Give me a break! It almost seems that you are blaming the poor quality of modern threaded applications on Windows! That's rich!
Concurrency is difficult to use correctly no matter what technology you use. Inter-process shared/mapped memory is just as susceptible to race conditions as cross-thread shared memory, and inter-process synchronization logic can deadlock just as easily as thread synchronization logic. And the results are the same: once a process is deadlocked, or corrupts its data due to a race condition, what difference does it make if it's running in its own address space? The software has failed catastrophically either way!
We are ALL well aware that poorly written multi-threaded software is unreliable and that threads can easily trash other threads' data if not written correctly. And yet, for performance-critical applications, programmers still prefer to use threads. Why? It's simple: Because, for MANY applications, the benefits in performance outweigh the risks.
Finally, I'd like to point out one more thing. You claim that to get "reasonable concurrency" on Windows you are FORCED to use threads. I completely disagree. While process startup latency is relatively high on Windows, Windows offers a rich set of interprocess communication mechanisms, and context switching is quite fast. And if your program is so performance-critical that process startup latency is your biggest bottleneck, then switching to thread synchronization seems perfectly reasonable.
UNIX creates a process with fork, which takes no arguments. UNIX runs a new executable with execve, which takes 3 arguments. So in just two system calls with 3 arguments, you launch an app.
Windows has a CreateProcess() function with 10 arguments, many of which are pointers to structs. I call your attention to the absurd "LPSTARTUPINFO lpStartupInfo " argument, which supplies info about the windows style and current desktop.
That's because GNU really has no clue about compatibility. Take a look at any non-GNU libc, and you won't have that problem.
A Government Is a Body of People, Usually Notably Ungoverned
Shame to have to set up like this just to run unreal editor, though. Oh, for you gamers out there, UT runs so much smoother and faster in Ubuntu, it's not funny. UT2k4 (has linux installer on the 1st cd) runs way better in Ubuntu also. You might want to check it out if you have a spare hard drive you can play around with.
Karma: Bad is the liberal way of saying this guy won't drink the kool aid here on slash dot. I wear my Karma with pride