Microsoft Reports OSS Unix Beats Windows XP

Mortimer.CA writes "In a weblog entry, Paul Murphy mentions a Microsoft report (40 page PDF) that in many instances FreeBSD 5.3 and Linux perform better than Windows XP SP2. The report is about MS' Singularity kernel (which does perform better than the OSS kernels by many of the metrics they use), and some future directions in OS design (as well as examination of the way things have been done in the past)." From the post: "What's noteworthy about it is that Microsoft compared Singularity to FreeBSD and Linux as well as Windows/XP - and almost every result shows Windows losing to the two Unix variants. For example, they show the number of CPU cycles needed to "create and start a process" as 1,032,000 for FreeBSD, 719,000 for Linux, and 5,376,000 for Windows/XP."

Comments

44 pages and the main question is still unanswered

[TripMaster Monkey]

Here's an interesting snippet I found while perusing the PDF...thought I'd share.

On the other hand, this paper does not validate our goal of increased dependence. Measuring that aspect of a system is significantly more challenging than performance. We do not yet have results for Singularity.

Interesting...Singularity is ostensibly supposed to be about stability, but the 44-page paper has no data on this. Kinda like saying, "Our new bulletproof vest is 40% lighter than our leading competitors, and twice as flexible. How well does it stop bullets, you ask? Sorry...we do not yet have results for that benchmark.".

Wake me when a paper comes out about Microsoft's new stability-oriented OS that actually addresses that particular aspect of the product.

Too Telling

[teknopagan]

Isn't it telling that the idea of Microsoft telling the truth is considered front page news on /.?

Re:Too Telling

[websaber]

This is the real big threat to the open source community. Once Microsoft becomes honest whith themselves they might start making real progress on the engineering side of their product. Marketing will get you so far when you have no more competition but good engineering can make it stick.

Re:Too Telling

[PickyH3D]

I think it's more telling that the paper shows Linux or FreeBSD as performing better in a few tests, which is the reason it was able to appear on the front page.

I'm happy though that MS may be taking Singularity seriously. Maybe we will see their OS in 2011-2015 based on it? Unless some sort of major shift in its purpose occurs, then I would definitely jump ship from whatever I am on then, to that and I will definitely port/develop my software for the OS.

Re:Too Telling

[Deviate_X]

I don't know what those 5m vs 1m cycles are doing. But what I do know that fundamentally Windows was designed with high-performance threading/wait operations and high-performance asynchronous operations, whereas Unix and its derivates rely on high performance process-creation, blocking I/O for sever applications.

I.e. Apache 1.3x series performs poorly on windows because it was a straight copy of the Unix edition - using processes rather than threads.

Re:Too Telling

[Viper Daimao]

telling for whom? Microsoft or Slashdot?

Re:Too Telling

[arkanes]

Nothing that people didn't already know there - take a look at the numbers for thread operations and note how they're much, much faster than the Linux/BSD numbers. Creating a process on Windows has always been very expensive, and threads have always been fast. It's why Windows applications use threads where equivilent Unix ones fork.

Re:Too Telling

[DavidTC]

Making threads is just as fast as making processes on Linux, you loon, because processes are threads and vis versa. It's literally the same system call, and all that gets specificied different is what 'the same' between things and what should be duplicated. (Although it's not actually duplicated until it changes.)

Re:Too Telling

[fossa]

I thought Linux had somewhat less than stellar thread creation performance for this very reason...?

Re:Too Telling

[DavidTC]

No. Linux's thread creation beats MS's thread creation.

It just doesn't beat it by as the absurd amount as its process creation beats MS's process creation.

Think of it this way:

Linux threads: great
Linux processes: great
Windows threads: good
Windows processes: horrible

Re:Too Telling

[EvanED]

The paper seems to support this claim:

Thread yield cycles:
FreeBSD: 911
Linux: 906
Windows: 753

2 thread wait-set ping-pong:
FreeBSD: 4,707
Linux: 4,041
Windows: 1,658

The latter test measures the cost of swapping between two threads in the same process using synchronization objects.

What I would like to see is thread creation time in Windows vs. process (thread) creation time in Linux.

And also a comparison with Solaris's LWPs.

Re:Too Telling

[EvanED]

Do you have a reference to support this? I don't see benchmarks for Windows thread creation in the paper.

Re:Too Telling

[YU Nicks NE Way]

I see the slashdolts are out moderating today.

Sorry, but what I said above is the truth. Linux and *n*x generally have a fantastic process creation system for command line processes. Windows optimizes for creating GUI processes. It's an engineering tradeoff.

Re:Too Telling

[Compaq_Hater]

Hey, I thought the idea was that more people would like it if Microsoft just fixed all the bugs and Security issues with thier OS ?, Now that they are attempting just that everyone seems to think that is bad.

I think it is the general mind set of the geeks that if Microsfot fixes these things that it will kill Linux acceptance by being just as secure as the linux OS.

while i do not think it will kill Linux, it will in fact in my opion foster new development on both sides of the OS fence as it were by being just another useful tool in an ever changing sea of computer needs for both buissness and Home users as well as sys admins.

but instead of seeing the positive responses this should be bringing, i just see the usual "Hey this can't be good for anyone MS is involved sneaky bastards !, they are up to somthin !".

come on face it, OS R&D is a dog eat dog world and the biggest Dog gets the most Bite.

just my thoughts.

CH

Re:Too Telling

[GooberToo]

I don't have something I can point you at right, however, the information is true. Linux used to have horrible overhead imposed by thread creation. As a result of both the NGTL and NPLT projects, the time needed to create a thread on Linux is tiny...tiny...tiny...some of the well known results from the projects were published... Here's a quote:

"One test mentioned in Ulrich's email - running 100,000 concurrent threads on an IA-32 - generated some interesting discussion. Ingo Molnar explained that with the current stock 2.5 kernel such a test requires roughly 1GB RAM, and the act of starting and stopping all 100,000 threads in parallel takes only 2 seconds. In comparison, with the 2.5.31 kernel (prior to Ingo's recent threading work), such a test would have taken around 15 minutes."

http://kerneltrap.org/node/422
As you can see, the stellar increase in thread performance has been unbelievable. Keep in mind, prior to this effort, Linux's thread creation was no where near the performance delta gained from these projects. Ergo, one can easily deduce that Linux far exceeds (less time) Win's thread creation latencies.

Re:Too Telling

[GooberToo]

http://kerneltrap.org/node/422

Linux was fairly slow at thread creation. Linux is now wicked fast at thread creation. In one benchmark, what used to take 15-minutes for Linux, can now be completed in 2-seconds! No joke!

Re:Too Telling

[PickyH3D]

Why is that a threat?

A great product is a great product regardless of who makes it. I thought OSS was a big deal because it emphasised great engineering with openness. So, if you can't handle the heat, then stay out of the kitchen.

Re:Too Telling

[Korean Elvis]

Yes, correctly said for linuxes!

To each thread uses 'task_struct' which is appearance in the kernel as just one process (but happens to shares resources likes teh address space, for examples). Under in linux, threads and processes aperance of no difference except for threads run in same address spaces.

So when happens for firing the NEW process? Time accounts reasons more for Win/XP?!

Linuxes use 'copy-on-write' modelers - leaves the maps memory sharable to parent process and childs both. When child writes ONLY in shared address space, kernel newly allocates more stores. Lowers much overhead comparisons to different OSs like Win/XP (and so with Unix traditionals!).

From now, firing a new process inside linuxes relates light rain for Win/XP poreing deeply.

Re:Too Telling

[jonabbey]

The problem (and the justification for your troll mod, I suppose) is that process and thread creation are (or should be) wholly orthogonal to GUI issues.

Why should the GUI have any impact whatsoever on low-level process and thread operations?

Re:Too Telling

[P3NIS_CLEAVER]

I think microsoft could definitly sell single use machines with a new, better os on it. Why port sql server to linux, when you can port it to shinybetteros?

Re:Too Telling

[GooberToo]

That used to be true. It is no longer the case. You are also, only counting the overhead imposed by the system call and ignoring all the thread work that goes into user space.

Long story short, creating threads under Linux used to be fairly slow...even compared to a process...though it was faster strictly comparing the system call because the system call had to do less work. These days, creating threads under Linux is very fast. Modern kernels now have thread specific system calls...which means threads *may* (I'm actually not sure) actually be faster than process forking under Linux.

Re:Too Telling

[YU Nicks NE Way]

Why should the GUI have any impact whatsoever on low-level process and thread operations?

There's a cost involved in wiring a process or a thread to a GUI -- data structures to initialize, event handlers to start up, etc. If most threads are going to be GUI-less, then that price shouldn't be payed except for the ones which explicitly ask for it, which is the optimization which *n*x usually makes. Not so for Windows OS's, which assume (correctly) that almost all processes and threads want a rich GUI.

Different goals, different designs.

Re:Too Telling

[jonabbey]

Why should *any* thread have to bear a cost that it does not benefit from?

The threading mechanism should be efficient enough that the optional cost of creating a per-thread event queue or whatever can be set up as a second step, just as a database thread might need to set up thread local storage as a second step, or whatever. If you get the underlying mechanism efficient, everything on top benefits. Limiting the benefits of such efficiency to threads that do some particular limited task is not efficient.

Do you actually have any experience with this under Windows? What sort of optimizations are you alluding to, and how do you imagine they are done better than under Linux?

Re:Too Telling

[jsight]

You are completely wrong.

The Windows GUI (GDI, Winforms, any of it) is not thread-safe. All interaction should be done through a single thread.

Differences in the performance of their GUIs (X vs. Windows) are significant, but not because of threading issues.

Re:Too Telling

[canuck57]

Windows applications use threads where equivilent Unix ones fork.

Unix can do either. Just depends how the programmers want to do it. Both have advantages and disadvantages. You can run threaded Apache 2.x or Apache 1.x process based. Both work but with different traits.

For example, by forking and allowing processes to die your more immune to memory leaks. If the code cores, it does not take then all down. If the code is known to leak memory, fork and subsequent termination cleans things up. Making rapid program development quicker but at the expense of fork() and memory. Another advantage of fork is memory separation. Each process does not share data segments so less errors from bad code. This is a major reason why Unix code tends to be outright fewer problems than Windows code. Other reasons may include prioritization, I can nice down one process over another. You can also set maximum system limits on each process much lower to lead to more stability to abusive user input.

Threads in a "perfect world" are technically more efficient at memory usage and avoid expensive fork operations. Very expensive in Windows, which is why threads are preferred in Windows. And the world is not perfect. Say you want to limit 1 thread to 100MBs of data core, you have to write a crap load of code in a Windows (or UNIX) threaded model to effectively do this. When in a process model you could use a much more stable ulimit like features of the OS. And if you have a memory leak in a threaded model, you have to restart the whole thing, even if it is just one thread causing all the issues.

Which model you choose entirely depends on the needs of the application and the circumstances you anticipate running it.

Re:Too Telling

[r00t]

The main difference for a typical 2.6.xx kernel is that very large processes take longer to fork. This is because Linux has to do some work with the page tables. The very latest kernel (not out yet I guess, or maybe the feature went out in .14 already) can avoid a bunch of this though, so it's about time for a fresh benchmark. Remember to vary the process size when you benchmark.

Re:Too Telling

[YU Nicks NE Way]

The Windows GUI (GDI, Winforms, any of it) is not thread-safe. All interaction should be done through a single thread.

Uh, no. In fact, GDI objects are shared kernel objects, and they are thread safe.

Re:Too Telling

[DavidTC]

What the hell are you yammering about?

Linux is, in fact, slower at creating windows than Windows. (Technically, it is X that is slower, but that makes no difference.)

This has nothing whatsoever to do with starting another thread or process. Threads and processes (That, is a process that duplicates itself.) almost never have anything to do with new windows.

The only time the speed it takes to create a window matter WRT to process creation speed is when a new application launches...and 'loading into memory' is a billion times slower than either of those, so that's rather moot.

We're talking about an existing process launching another copy of itself or a new thread of execution within itself, and while that may indeed happen when a new window opens, that circumstance is almost negligable compared to all the other times a new thread or process starts and the application continues to keep exactly the same number of windows open.

Re:Too Telling

[DavidTC]

Windows does not assume all threads have their own GUI. That's just flat wrong, and it would be a damn stupid assumption if it did. 'Hey, this mail client wants a thread to get email, I better give that thread a GUI.' Luckily for everyone, it does not do that.

In fact, it doesn't 'assume' that all processes have their own GUI. It's just that almost all applications link against the GUI library, so it's loaded at start, which is a good thing.

It is, however, perfectly possible to write console apps that are multi-threaded. I've done it.

This is exactly how it works on Linux, too, except the Linux GUI libraries are varied instead of there just being one, and tend to be heavier to load. And they have to talk with X instead of just doing things.

Re:Too Telling

[YU Nicks NE Way]

Sorry, but that's just not true. There are a set of items created for each thread which prepare it to support a GUI (such as the extra slot assigned to, say, MsgWaitForMultipleObjects, for instance), and they are created whether or not any call is made into a routine inside USER32.dll. There are some things which can be delayed (such as the creation of the actual windows message queue), but there are other things which aren't.

(And, for what it's worth, console applications being multithreaded have nothing whatesoever with the presence of such preparations. The kernel objects are there either way.)

But the key point is this:

the Linux GUI libraries [...] tend to be heavier to load. And they have to talk with X instead of just doing things.

Why do you think those things are "heavier to load"? Because Linux makes different engineering tradeoffs.

Re:Too Telling

[bergeron76]

I think Apple already beat them to the punch there.

At least MSFT is consistent in their corporate strategy: Play catch up, and over-market.

Re:Too Telling

[Varun Soundararajan]

I think M$'s reply would be like this:

We have two versions of the whitepaper. This is a research whitepaper, you should not use this seriously when making commercial decisions. We also have a commercial whitepaper which you should use to make a decision when you want to decide which OS to choose, and we know what OS you should choose ;)

====
this space is now not blank

Re:Too Telling

[DavidTC]

MsgWaitForMultipleObjects doesn't have anything to do with the GUI. It is for message passing. Without it, threads couldn't talk to each other! You can't even do mutexes.

It is also how the GUI tells the application about changes, but it's used for a lot of other stuff, especially in a multithreaded app.

And all OSes have 'a set of objects assigned to each thread'. That's how it can tell the state of the things.

And makes records of them elsewhere. Otherwise it couldn't find them.

Pretending Windows does a great deal more work in this regard, or that this hypothetical large amount of work is somehow an 'advantage' for opening Windows,is just a lie. I've written multithreaded console apps in Windows and Linux, I know exactly how much 'work' gets done.

The advantage in opening windows and creating GUI objects on Windows comes from the fact that it is much faster to open windows and create GUI objects, period. This has nothing to do with threads, it is a matter of the architecture...it is simply slower to connect to X and write to the screen.

Re:Too Telling

[YU Nicks NE Way]

MsgWaitForMultipleObjects doesn't have anything to do with the GUI. It is for message passing.

You should go read MSDN on that -- you're wrong. MsgWaitForMultipleObjects has nothing to do with message passing; it has to do with yielding control.

Re:Too Telling

[Deviate_X]

If you actually read the .pdf file you will see that, thread/wait (synchronization) (which is the most important component of any high-perfromance multi-thread server application) operations are faster than on windows than linux or freeBsd.

Thread Synchronization -- MSDN Magazine, October 2005

Re:Too Telling

[DavidTC]

Can you not look at your own links?

It waits for many different events, including passed messages.

Look at QS_SENDMESSAGE. That's how threads talk to each other.

But, more importantly, without MsgWaitForMultipleObjects (And other ways of doing the same thing.), you cannot have mutexes. You cannot say 'Wait for this resource to become free, so I can lock it', which is a fundamental part of writing multithreaded programs. Look at CreateMutex, it's right there on the screen. There's a reason all those topics are together.

The message queue is an incredibly important part of any Windows program. It is, indeed, how the GUI talks to the program, and it's how everything else talks to it, also. You need it without the GUI. You don't even know when to exit without it, because nothing could tell you to exit.

And Linux sets up something similiar on every program, called a signal handler. For multithreaded programs, it requires more, and for X programs, it requires yet more.

So this is a red herring when talking about speed of starting processes and threads. Windows has a unified message queue, and setting it up is logically a lot easier than the silliness on Linux.

Re:Too Telling

[DavidTC]

I don't agree with the recommendations in that article.

In particular, a spinwait for a resource is about the stupidist thing you can do in a multithreaded app. If you're on a single processor system, you're just wasting time...the lock can't clear until the thread holding it gets time to execute. Thus a spinwait will never lock faster than just trying to grab the mutex.

Maybe in some magical universe where everyone was on hyperthreaded CPUs and always ran exactly one process with three threads, sure, the advice would make sense. No sense in swapping out a thread. However, even pretending we all have mutli-CPUs systems here, 99% of the time, we'd actually like to use our fancy new CPUs to actually be doing work instead of having the processor spin in place so one app looks responses faster at the expense of anything else ever getting a CPU slice. (Hey, if your app starts hogging the CPU, won't it get less slices anyway?)

And if your OS is swapping in and out threads too rapidly, the correct thing to do is not to force it to run your thread longer. That's idiotic. The correct thing to do is to adjust the timeslices.

There's plenty of good advice out there about managing resources in multithreading apps. This is not it.

Re:Too Telling

[jsight]

I was referring to GDI+ in the Dotnet APIs. Look through the API docs for System.Drawing and you will see MANY things that are specified as non-thread safe.

5 Steps of Grieving

[ciroknight]

hmm funny, the last step is Acceptance. Too bad it seems Microsoft skipped the "bargaining" step.

Re:5 Steps of Grieving

[justsomebody]

Too bad it seems Microsoft skipped the "bargaining" step.

And all those times billg or steveb visited some coutry would be called what? Extortion?

Singularity is truly an intriguing system.

[CyricZ]

Singularity is a very interesting system. But that's not surprising, when you consider some of the brains behind it: Galen Hunt, Wolfram Schulte, Ulfar Erlingsson, Rebecca Isaacs, and many others who are well-known for their research.

In twenty or so years we may look back at Microsoft Research with the same admiration we have for Bell Labs.

Re:Singularity is truly an intriguing system.

[Guysmiley777]

In twenty or so years we may look back at Microsoft Research with the same admiration we have for Bell Labs.

I just shot soda out of my nose. You owe me a keyboard.

Re:Singularity is truly an intriguing system.

[CyricZ]

Now that you're done being sarcastic, go look into some of the research that is being done at Microsoft Research. Like it or not, it is top of the line work. They're at the cutting edge, and they're well financed.

Re:Singularity is truly an intriguing system.

[yurnotsoeviltwin]

You definitely have a good point there. Everyone around here bashes Microsoft obviously, and for good reason. Their business practices can get a bit on the shady side sometimes, though they problably aren't deserving of quite the amount of hate they get around these parts. But their programming and research, particularly research, isn't that shabby, and certainly isn't "evil." Remember, M$ doesn't just sell operating systems, it makes them too, and to do that you have to have brains. I think some people around here need to give at least the engineers and researchers in Microsoft a little more respect.

Re:Singularity is truly an intriguing system.

[geomon]

Like it or not, it is top of the line work. They're at the cutting edge, and they're well financed.

Okay, but how many of their innovations (Christ Microsoft loves that word!) actually make it to the outside world?

I think your comparison to Bell Labs is good, however, in that much of what Bell Labs created required others to make into real products. AT&T/Ma Bell sat on every innovation until it nearly suffocated due to lack of capital investment.

Re:Singularity is truly an intriguing system.

[October_30th]

So, if you work for Microsoft Research, there's no way you can be doing cutting edge research?

Re:Singularity is truly an intriguing system.

[Spy der Mann]

In twenty or so years we may look back at Microsoft Research with the same admiration we have for Bell Labs.

Well, they caught up with our hatred for Ma' Bell in NO TIME. That ought to say something :)

Re:Singularity is truly an intriguing system.

[geomon]

So, if you work for Microsoft Research, there's no way you can be doing cutting edge research?

No, but if you work for Microsoft Research it is likely that the results of your research may never see the light of day as products. Unless there is a way for Microsoft to make hoards of cash from your idea, it will be stillborn.

Re:Singularity is truly an intriguing system.

[interiot]

And where did the last great scientific lab come from? A raging monopoly (AT&T), that's where. When did they do their best work? When they were funded by a raging monopoly. I think the comparison to Microsoft Research is quite apt.

Re:Singularity is truly an intriguing system.

[CyricZ]

We will just have to wait and see. Then again, even if a specific project doesn't go commercial, there is always the knowledge that was gained from it. In many cases that is more valuable than the tangibles the project may deliver.

Re:Singularity is truly an intriguing system.

[errxn]

I think some people around here need to give at least the engineers and researchers in Microsoft a little more respect.

Here? On /.? I suspect that day will occur right around...NEVER.

Re:Singularity is truly an intriguing system.

[raxx7]

The ideia behind isn't new. IBM's very successfull AS/400 line used the same concept.
However, I think that now is a good time to take another look at the concept, using the improvements in JIT and applying OOP concepts to the OS.

Re:Singularity is truly an intriguing system.

[October_30th]

I bet Microsoft patents most of its research just like IBM. Hence, most of the research will see the light of day. Of course, if you believe that patenting is a bad idea and equivalent to withholding research, you are right.

Re:Singularity is truly an intriguing system.

[LaughingCoder]

Sad that your comment is modded as funny. In fact what you say is insightful and probably will turn out to be true. It amazes me how most people in this forum refuse to give Microsoft credit for anything they do or have done, but they are more than willing to heap blame upon them. I believe that *overall* Microsoft has in fact been a positive force in the industry. This doesn't mean everything they have done worked out for the "common good", but I think the scale tips in that direction. And don't forget that they continue to spend lots of R&D dollars both on product development and pure research. You would think a technical audience like /. would appreciate that. To me it smacks mostly of envy and jealousy. Can't we all just get along?

Re:Singularity is truly an intriguing system.

[SnprBoB86]

I am even more saddened that your comment was not modded up. I wish had had mod points for you.

Re:Singularity is truly an intriguing system.

[Alioth]

The interesting thing is that MS Research does do a lot of truly interesting pieces of research - but the funny thing is, despite this, MS itself uses few to none of the fruits of this research, preferring instead to just buy up other companies and copy other technologies.

Re:Singularity is truly an intriguing system.

[cloudmaster]

Or we may look back on them with the contempt we feel for Bell Telephone. Either way, I guess. :)

Re:Singularity is truly an intriguing system.

[geomon]

I bet Microsoft patents most of its research just like IBM.

They will probably patent all of the stuff they can.

Hence, most of the research will see the light of day.

You missed my qualifier: "as products". In that context I stand by my statement.

Of course, if you believe that patenting is a bad idea and equivalent to withholding research, you are right.

I don't consider patenting a bad idea, per se, but I do believe defensive and method patents are stupid and should be removed from US patent code.

And no, I do not consider withholding research to equivalent to patenting an idea. I don't know where you got that idea.

Re:Singularity is truly an intriguing system.

[Zerathdune]

I loath microsoft, and I think I'm justified in that. Most of it is because of their business practices. We can't ignore the fact that they are a convicted monopolist, especially after having gotten off with little more than a slap on the wrist. It also annoys me that they buy/copy more than they innovate. they bought DOS, and based their OS's off of that for a while, and they hired out another company to make NT for them. And the constant FUD is yet another perfectly legitamite reason to hate them. The fact this is front page news is proof of that. even in this case, they aren't being entirely truthful. from TFA: So why is this interesting? Because their test methods reflect Windows internals, not Unix kernel design. There are better, faster, ways of doing these things in Unix, but these guys - among the best and brightest programmers working at Microsoft- either didn't know or didn't care. if that's honesty, I'm not surprised nobody gives them any credit.

that said, I agree with you that they get even more shit than they deserve. Not all of their products are terrible. MS Office is actually significantly more efficient than OOo. they've done a very nice job with MSN messenger. I can't comment on Singularity, as I know next to nothing about it. I still think they're satan incarnate, but I'm willing to give credit where credit is due.

Re:Singularity is truly an intriguing system.

[LowneWulf]

I've seen talks and papers that have come out of Microsoft research, and while it may look good as a website summary, the quality of the actual projects and results varies wildly. They may talk big, but in the end, I've only ever seen a couple of projects out of MSR that were even worth talking about, and the research labs of places like IBM, HP, and even Sun do many far more interesting things.

One other big problem from MSR - on the occasional project that's actually good, they somehow manage to kill it, or at least never tech transfer it into products. I cry when I think of some of the awesome dev technologies MSR was working on a few years ago that never made it out.

Re:Singularity is truly an intriguing system.

[rthille]

I have to agree with another reply to your post. If the microsoft engineers are so great why do their products suck so much? I hate microsoft more for their crap products dominating the market than their crap business practices which got them there.

Re:Singularity is truly an intriguing system.

[A nonymous Coward]

(following a long rant about how poor old Microsoft is misunderstood) Can't we all just get along?

Why don't you ask Microsoft that? They have NEVER tried to get along with anybody. Even at the antitrust trial, they tried to hoodwink the judge. Remember that fake video? The one that was supposedly shot in one straight session, and when somebody pointed out that the screen changed, the wall clocks changed, etc, they had some lame excuse.

You tell me once where Microsoft has tried to get along with ANYBODY. I double dog dare you.

Show me any innovation, any REAL innovation.

All Microsoft does is lie, cheat, and steal. They deserve all the flack they get and more.

Re:Singularity is truly an intriguing system.

[amightywind]

I came across this as shining example of some of the pointless projects going on at M$ Labs.

The goal of the "New dimensions in Excel" project is to empower end-user programmers (that is, Excel users who are not professional programmers) to tackle more challenging applications. At the moment, larger and more complex spreadsheets often require the help of a professional programmer to write part of the application in Visual Basic. Our extensions would extend the "reach" of what can be done without leaving the familiar cells-and-formulae paradigm that has proved so successful in practice.

...They're at the cutting edge, and they're well financed.

Guffaw! At least they are well financed.

Re:Singularity is truly an intriguing system.

[ajs]

Absolutely! I've been saying this since the very early days of C# (it had a different name then, and hadn't been Javafied yet). I was stunned by some of the good work that had come out of MS Research. It truly was an independent and fascinating organization. They released many things as open source (real open source, under various OSI-approved licenses), and all of their work was very interesting, hard CS.

Sadly, Microsoft decided that they were getting out of hand, and has done thing like insist that any code released by MSR must be released under MS's shared source license, They've also been taking a much stronger role in constraining the goals of research to match existing Microsoft initiatives.

That's not to say that it's not 10x cooler a place to work than most of the world for a programmer, but it's not as cool as it was.

Re:Singularity is truly an intriguing system.

[flyinwhitey]

"If the microsoft engineers are so great why do their products suck so much?"

Because they have to make the products marketable, which can mean making compromises.

But you knew that, you just wanted to show off your anti-MS bias.

Re:Singularity is truly an intriguing system.

[rumcho]

Can't we all just get along?

How'bout you ask Microsoft this question?! Can't u see these assholes are trying to take over the world?

Re:Singularity is truly an intriguing system.

[P3NIS_CLEAVER]

the quality of the actual projects and results varies wildly
This is what you would expect from basic research. I am sure there are thousands of Bell Labs duds that everyone has forgotten about.

Re:Singularity is truly an intriguing system.

[killjoe]

So all that great research is just useless in the end then? WHy should I have respect for all that research if ends up as a crap operating system that I am forced to use at work?

Re:Singularity is truly an intriguing system.

[Danse]

Guffaw! At least they are well financed.

Actually, giving users abilities that were previously accessible only to programmers is a worthy pursuit, and one that many if not most software projects are working towards, at least in part. Think of all the things that we can do with our computers today that we couldn't do 5 years ago, or 10 years ago. Coming up with good interfaces and studying the way users think about the software they're using is valuable work. It may not be a new field of study, but it's still cutting edge in a way. Software is forever advancing and people's perceptions and expectations are advancing with it. Coming up with new ways to meet and exceed user's expectations with software is something that I wish more companies and development teams would take on. They don't always succeed (e.g. Clippy), but the more study they do, the better things get over the long run.

Re:Singularity is truly an intriguing system.

[norton_I]

Why do you think that is pointless? While it is not the same style of "basic" research as some of their other work (which is, as the original poster said, top notch), it is certainly valuable, at least from the abstract you posted. They also do some cool stuff with compiler optimization, distributed/parallel computing, and all sorts of more respectable forms of research.

I don't know your background, but I have never met anyone with a substantial CS background to say anything bad about MS research, including a number of OSS zealots (such as myself). I suspect that most people who make fun of them are simply not in a position to judge their work.

Re:Singularity is truly an intriguing system.

[Danse]

Their business practices can get a bit on the shady side sometimes, though they problably aren't deserving of quite the amount of hate they get around these parts.

I think they deserve every bit of hate they get due to their business practices, and then some. But I do have to agree that they make some good products, Visual Studio .Net being the one I use the most right now. It's a fantastic IDE. Better than anything else out there. I give Microsoft respect where they deserve it, but I will always be a critic of their business practices as long as they continue their shady and outright criminal ways.

Re:Singularity is truly an intriguing system.

[amightywind]

I don't know your background, but I have never met anyone with a substantial CS background to say anything bad about MS research

Does Cornell qualify as l33t enough?

Re:Singularity is truly an intriguing system.

[LaughingCoder]

Actually it looks to me like Google is trying to take over the world. And Yahoo. As was Sun once upon a time. And IBM. And don't tell me Apple isn't interested. In fact it seems to me that the FOSS movement is trying to take over the world. As any company would. And most countries for that matter ... that hardly separates Microsoft from the pack, except that they are perhaps more successful than the others I mentioned in their pursuit.

Re:Singularity is truly an intriguing system.

[mmmiiikkkeee]

>>They're at the cutting edge corners are edges

Re:Singularity is truly an intriguing system.

[amliebsch]

Knowledge for knowledge's sake, my friend. Not all science must be applied science. Or do you disrespect pure science? That seems silly.

Re:Singularity is truly an intriguing system.

[Geoffreyerffoeg]

C. A. R. f'ing Hoare works for Microsoft Research.

You know, the guy who invented stacks and quicksort and all that?

And...what's your personal opinion of AT&T (the old Ma Bell)? Did it color your opinion of Bell Labs?

Re:Singularity is truly an intriguing system.

[Erik Hollensbe]

Most people here are too young for that.

Re:Singularity is truly an intriguing system.

[larien]

Probably because Bell Labs didn't have the same culture. Bell labs would have said "here, look what we've created, us it, enjoy". MS labs will say "Look what we've made! No, you can't use it yourself unless you pay us money...."

Re:Singularity is truly an intriguing system.

[BitchKapoor]

Does Cornell qualify as l33t enough?

That depends on what you're doing at Cornell. Freshman in agriculture? CS prof? Something in between?

Re:Singularity is truly an intriguing system.

[AArmadillo]

Erm... you can't be a "convicted monopolist." Being a monopoly is not a crime. Saying "convicted monopolist" is like saying "convicted software developer."

Re:Singularity is truly an intriguing system.

[avsed]

RE: Microsoft Research: I've used the Detours research project in many applications - most users (even tech savvy IT support) think I've accomplished something magical using it - it's extremely powerful software.
I am firmly in the "outside world", and these are real products (even if they are internal to my employer, a Fortune 500 corp) - just because you are unaware of the innovation that is happening there (and just because it is funded by Microsoft) doesn't mean you should automatically discount this research. I fully endorse the grandparent post: there is cutting edge work going on there.

- Dan

Re:Singularity is truly an intriguing system.

[Breakfast Pants]

Actually Bell Labs patented all sorts of stuff, tried to sue creators of unix workalikes, and more.

Re:Singularity is truly an intriguing system.

[Breakfast Pants]

*cough*google*cough*. Buying up other companies is not bad in and of itself and infact it can show a willingness to let the best people for the job handle the job.

Re:Singularity is truly an intriguing system.

[killjoe]

This is a corporate R&D arm. Everything they do must go towards a product. Otherwise MS could spend all their R&D dollars trying to cure cancer or something.

Re:Singularity is truly an intriguing system.

[Guy Harris]

and they hired out another company to make NT for them

Indeed? What company might that be? (If the answer you give is "Digital Equipment Corporation", please cite information to support that answer, and note that "DEC sued Microsoft over it and part of the settlement was that they had to {support|keep supporting} NT on Alpha" isn't sufficient without references.)

Re:Singularity is truly an intriguing system.

[Zerathdune]

Microsoft hired a group of developers from Digital Equipment Corporation led by Dave Cutler to build Windows NT, and many elements reflect earlier DEC experience with VMS and RSX-11.

http://en.wikipedia.org/wiki/Windows_NT

Happy now?

Re:Singularity is truly an intriguing system.

[Zerathdune]

Okay, bad choice of words. Keep in mind though, people make that same arguemnt when they say copyright enfringement isn't stealing. Technically they're right, but it doesn't nessicarily equate to copyright infringement isn't wrong just because the term doesn't sound quite as strong. Of course, in reality, the punishment for copyright infringement is usually much worse than theft.

I'm going to stop ranting for now and just finish up with saying that I think my point is still valid; they were [insert equivalant legal language for "convicted" when dealing with a civil case], of something that typically means a company is no longer allowed to exist as is, and they bribed their way out of it.

Re:Singularity is truly an intriguing system.

[Guy Harris]

I.e., "hired out" means "hired people from", not "contracted out to". They might have needed to buy the expertise of folks from DEC, but that doesn't mean that Microsoft didn't develop NT, any more than hiring Charles Simonyi means they didn't develop Word. (Their purchase of Forethought does mean they didn't originally develop PowerPoint.) No, NT isn't hugely innovative - but, then neither are most core OSes, these days; the improvements are largely incremental.

From a quick look at the paper, Singularity is at least an interesting system; some ideas might resemble what's done in other systems, but you'll probably find that with just about any new OS (and I might be a bit suspicious of an OS that isn't influenced by any other system; perhaps such an OS's developers are titanic geniuses who don't need to stand on the shoulders of others, or perhaps they're just legends in their own minds).

Re:Singularity is truly an intriguing system.

[geomon]

RE: Microsoft Research: I've used the Detours research project in many applications - most users (even tech savvy IT support) think I've accomplished something magical using it - it's extremely powerful software.

No doubt. Bell Labs created the transistor. Perhaps you've heard of it. :)

I am firmly in the "outside world", and these are real products (even if they are internal to my employer, a Fortune 500 corp) - just because you are unaware of the innovation that is happening there

I think you are assuming too much. The company I work for is in the innovation business too. We produce scientific solutions for Fortune 500 companies and Microsoft is heavily embedded in our organization. We see plenty of Microsoft's best stuff because we manage USDOE laboratories. My point was, and is, that Microsoft will fund activities in Microsoft Research just like AT&T funded Bell Labs. There was a strong 'inventive' capability in the people they hired (they employed quite a few were Nobel-level fellows), they produced cutting edge technology, and some of what they produced was deployed in the field.

The problem is that much of what they produced/invented never saw the light of day as products. The reason that much of what they produced never became consumer or commercial products is because they didn't know what to do with it. It was perhaps a basic science discovery whose real application did not fit into their business profile. Not necessarily their fault, just an economic reality.

(and just because it is funded by Microsoft) doesn't mean you should automatically discount this research.

My comment shouldn't be interpreted as discounting the research. I have no idea where you got that from my post.

I fully endorse the grandparent post: there is cutting edge work going on there.

Who in this thread said they didn't?

Re:Singularity is truly an intriguing system.

[rthille]

Apple seems to do just fine marketing quality products.
Sure, when Microsoft was just starting out and didn't have the huge cash pool they have now, I can understand the struggle to make good software. But you'd think that starting sometime around 1995 they could have turned their attention to quality instead of producing crap.
And sure I'm biased against Microsoft. I've used their software.

Re:Singularity is truly an intriguing system.

[rthille]

Huh? I don't run linux. And as near as I can tell no one is having trouble giving linux away. They might be having trouble making money giving it away though.

Re:Singularity is truly an intriguing system.

[AArmadillo]

I wasn't trying to get you on a word techincality. A monopoly is not considered wrong by any US law, criminal or civil. The government even establishes monopolies -- for example, the US postal service has a monopoly on domestic mail. What is considered wrong by US law is using a monopoly position to eliminate compitition in a way that harms consumers. Certainly the argument can be made that Microsoft is guilty of this, but there is nothing inherently wrong about Microsoft being a monopoly.

Competition

[Anonymous Coward]

Don't you love competition????

How can you hate microsoft or *nix? We need each other so that we can constantly get better and gauge our progress. I really hope that MS can learn to live with the OSS community; it will benefit everyone, including MS.

Re:Competition

[Master of Transhuman]

But (according to Bill) it won't benefit Bill.

So fergeddaboutit.

Re:Competition

[jejones]

It depends. Do you mean competition based on merit, or competition via FUD and monopoly influence?

5,376,000 cycles for Windows/XP

[Anonymous Coward]

Is that including the spyware/worm/rootkit overhead?

Re:5,376,000 cycles for Windows/XP

[Jozer99]

My always on XP SP2 machine has not had any spyware in 3 months. Its the dumb users who get infected. There are less dumb people using Linux (due to the learning curve), therefore less problems with unwanted computer activity. An XP machine properly set up with firewall, spyware, and virus scanners/blockers, and used responsibly (no Kazaa) will get a serious virus about as often as a *nix user will get rooted.

Re:44 pages and the main question is still unanswe

[man_of_mr_e]

Well, that's sort of to be expected. Stability is not as easy to measure as other things, since you need benchmarks over a long period of time. Further, since it's still a research OS, it's likely in constant flux and doesn't have the same kind of stability hardening of a retail OS.

That explains a lot

[afidel]

I guess this explains why linux boxes do so much better than windows boxes at high load, it takes the windows computer almost 8x as long to start a new process! That's something where a little bit of optimising really helps =)

Re:That explains a lot

[wzzrd]

In regard to your signature: "There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now." You forgot one box. A Linux box. But you probably get that remark all the time ;)

Re:That explains a lot

[misleb]

And even once started, it is slow. I find Windows desktops to be HORRIBLE multitasking environments. Try unzipping a large file in the background and the machine comes to a crawl. On Linux you hardly even notice it unless you try to do something that requires lots of CPU. I mean, obvoiusly you're not going to get 100% CPU for each process, but the desktop shouldn't crawl.

I always found it amusing that Windows and Mac users used to think that multitasking meant just having Word and IE open at the same time. ;-)

-matthew

Re:That explains a lot

[gbjbaanb]

Yawn, same old stuff - read the rest, Windows is better at thread switching. That makes up for the slow process creation. Windows programmers know that processs creation is slow, and thread creation is quick. Using Threads over Processes is not the same model you generally use in the Linux world which prefers Processes to threads.

Even the Blog author makes the same comment: "So why is this interesting? Because their test methods reflect Windows internals, not Unix kernel design." yet he will still draws out the process creation figures as his headline.

Re:That explains a lot

[pthisis]

Yawn, same old stuff - read the rest, Windows is better at thread switching. That makes up for the slow process creation. Windows programmers know that processs creation is slow, and thread creation is quick.

As a result, you get tons of unstable Windows applications because to get any reasonable concurrency you have to throw out the years of hard work that OS designers put into having protected memory.

Threads vs. processes isn't "two different ways of doing the same thing". Barring a massive implementation boondoggle, you make that choice based on whether you want memory protection or not. These numbers highlight a massive boondoggle, which takes the correct choice away from the application author in many cases.

Re:That explains a lot

[MotoMan045yahoo.com]

Process creation time is rarelly a serious bottle neck, but creating effective multi-tasking programs is still very difficult, on any platform. NT provides serveral multi-taksing choices, from threads, fibers ( a lite weight unix like process), processes, app contexts, and with .net, app domains, ( with a power permission assert model). Its been my experience that Windows provides every bit as many viable multi-tasking options as Linix. Windows challenge, and advantage, it there are so many RAD type VB developers creating apps with varying degress of quailty. Thats a refelection on the type of programmers, NOT the OS. The whole point of the MS reasearch project is to determine if they can create an OS that is secure, stable, supports RAD, and is performant enough.

Re:That explains a lot

[MotoMan045yahoo.com]

To imply any modern operating system can't multi-task zip in the back ground is just plain silly.

Re:That explains a lot

[misleb]

It is a question of how well it does it. And my experience is that Windows does it particularly poorly. If your definition of multitasking is simply having two or more programs open at once, fine, but I have higher standards. I expect to be able to browse the web smoothly while I compile a program and burn a DVD in the background.

-matthew

Re:That explains a lot

[MotoMan045yahoo.com]

Seriously how old is your XP computer. I recently bought a mid-level MCE XP pc, and I was concerned that the MCE functionality might slow down the system too much, but it has no effect on the peformance. In fact I've been able to record tv program's, burn a dvd, all while doing real dev work. I'm a dev for Microsoft by the way, but like I said, I'd expect all moderen OS's on reasonable hardware to do the same thing.

Re:That explains a lot

[misleb]

My home computer is a P4 1.6Ghz. I won't run XP. I find it to be unnecessarily heavy. Normally I run Linux, but occasionally I boot Win2k. As far as MCE functions go... do you have hardware MPEG compression? If so, that would explain why it doesn't slow your machine. Try doing something more intensive. What about when you are building a large project? How responsive is your desktop? When you click the startmenu, is there a significant delay?

Maybe you'd expect all modern OS's to perform the same under load, but my experience is that they don't. All too often I get annoyed by the lags and delays when pushing XP and Win2k. Some time it is OK, but others it is just unacceptable. YMMV.

-matthew

Re:That explains a lot

[Quantam]

From what I can gather from Linux Kernel 2.4 Internals, Linux gives elevated priority to threads that haven't run in a while, while NT uses thread priorities literally (a semi-cooperative multitasking system; information taken from Inside Windows NT). However, NT will temporarily boost thread priority for threads waiting on something (in this case I would imagine the desktop would be waiting on the message queue), so it should still give the desktop elevated priority (priority above what the ZIP program would have) and thus a fair chance at the CPU.

It's possible the ZIP program you're using is badly behaved and setting itself to high priority (lots of programs like that have options to set the priority of their worker threads; if yours has one, setting the priority to low will be certain to fix your problem); that would explain the observed slowdown, as NT would give significantly more CPU time to the ZIP program (you'll have to investigate that for yourself, as I've never had that problem, nor do I know exactly what program you're using).

It's also possible that you have your Windows configured to give priority to background tasks (this is a setting you can modify) rather than foreground apps (Windows usually gives a priority boost to whatever application is the foreground app). This could also explain your experience, as again the CPU would be going primarily to the ZIP program running in the background.

Re:That explains a lot

[Quantam]

One more thing I should mention on the subject. Explorer is Gord-awful, as far as threading is concerned; it does WAY too much in too few threads, and consequently hangs a lot as long operations complete. That's old news. It's also known to suffer from retry multiplication problems, which frequently cause it to hang waiting for something that won't work (such as accessing a network drive when the network is down).

None of this is a problem with the NT kernel itself. At this point I cite my knowledgeable friend Continuity (who has done a substantial amount of programming on Linux, as well as several consoles), who is an ardent Linux supporter (and believes Linux is superior to Windows overall) and 'M$' hater. Yet he still admits the NT kernel is well made; it's the lower layers (APIs, drivers, apps) that are buggy and poorly made.

Re:That explains a lot

[misleb]

Also, the 2.6 kernels have an extra "preemptable kernel" option which makes desktop operations even smoother.

All modern OS's have somewhat complex schemes for elevating and reducing thread/process priority dynamically. Usually it revolves around giving I/O bound processes less CPU time and CPU bound processes more all without starving any one process. I've actually gotten down to tuning the Solaris kernel scheduling tables for different types of work loads.

Unzipping a file was only one example
of when I experience poor multitasking performance on Windows. And no, I've never turned on the "give priority to background processes" option in Windows.

Have you ever used a unix desktop extensively? It is really difficult to understand what I am talking about unless you have. It is quite easy to become accustomed to the quirks of one OS and not really notice just how bad it is until you try something else which does certain things much better.

-matthew

Re:That explains a lot

[Quantam]

Also, the 2.6 kernels have an extra "preemptable kernel" option which makes desktop operations even smoother.

Ah, I see. I expected Linux always had that (NT always has).

Have you ever used a unix desktop extensively? It is really difficult to understand what I am talking about unless you have. It is quite easy to become accustomed to the quirks of one OS and not really notice just how bad it is until you try something else which does certain things much better.

I used CDE for a semester, and KDE a couple times. Besides being slower in general than XP/Explorer, I haven't noticed much difference. Anyway, I was trying to define the problem in concrete terms, as something which may be empirically verified, as opposed to "Windows bad!".

Microsoft Research is not Microsoft.

[Kickasso]

Microsoft is not Microsoft Research. Microsoft Research folks use and make free software, and in general are not ideologically bound to the parent organisation.

Re:What's the point of CreateProcess benchmarks?

[CyricZ]

It's not so much about its ability to start thousands of processes. What is important is that it takes Windows XP five times as long as FreeBSD to create a single process, and seven times as long as Linux. That's a significant difference.

You're SO fired!

[Kevin Burtch]

I bet the person who put that report on MS's site has been drooling over the severance package... ;-)

Re:44 pages and the main question is still unanswe

[teknopagan]

I just have to bow before the guy who can read a 44 page pdf and post an intelligible, coherent comment on it in less than two minutes. I just have to ask - where do you get that kind of caffeine?

Amazing.

Give me a fucking break

[Wonko42]

I've been seeing this damn report hailed all over the Internet for the last few days as Microsoft saying Unix is better than Windows, but apparently nobody has actually read the report.

For one thing, Windows is not slower than Unix in most of the tests. It's slower than Unix in some of the tests and faster in others. For another, these benchmark results are for low-level things like spawning processes and threads. Any programmer who knows anything about Unix and Windows will tell you that threads are cheaper in Windows and processes are cheaper in Unix, because that's how they were designed. So of course Windows is going to be slower than Unix at creating processes, and of course Unix is going to be slower than Windows at creating threads.

The only thing worth reporting about this thing is the performance of Singularity, which looks like it's shaping up to be an excellent modern kernel.

Re:Give me a fucking break

[Anonymous Coward]

Any programmer who knows anything about Unix and Windows will tell you that threads are cheaper in Windows and processes are cheaper in Unix, because that's how they were designed.

You, sir, actually know something about computers and are not welcome here. Or, at least, please confine your comments to "I bet Steve Ballmer through a chair at teh flying spagetti monstar!!!!" and the like.

Re:Give me a fucking break

[sammy baby]

Are you saying that Paul Murphy's statement that, "almost every result [in the published report] shows Windows losing to the two Unix variants." is inaccurate?

(I'm not being sarcastic: I haven't yet had time to read the full report, and would genuinely like to know.)

Re:Give me a fucking break

[Wonko42]

Yes. It's the "almost every" that I have an issue with, because it's a blatant exaggeration. I've also seen that phrasing used in several news articles about the report. But when I looked at the actual report, I saw plenty of tests where Windows actually beat Unix. I didn't bother counting, but I'd estimate that the two came out pretty evenly matched, with Unix maybe slightly ahead. In any case, no matter which one beat the other more times, these are very low-level tests. Nobody's going to notice these differences unless they're running a high-traffic server or doing some really heavy-duty computing.

Re:Give me a fucking break

[Jeffrey Baker]

Actually, both thread creation and process creation are much faster on Linux than Windows. However the margin is smaller for threads.

I agree that the report is meaningless for the purposes suggested in this slashdot write-up. If anything, it tells us that something coming out from MS Research has the potential to kick the asses of both Windows and Linux.

Re:Give me a fucking break

[GooberToo]

Any programmer who knows anything about Unix and Windows will tell you that threads are cheaper in Windows and processes are cheaper in Unix, because that's how they were designed.

That used to be true for Linux...but no longer. On Linux, threads are now faster than processes. Keep in mind, I'm only talking about creation latencies...not scalability or metrics of specific application using a specific implementation.

Memory Usage?

[ackthpt]

XP sucks memory.

The future: Longhorn will suck far more memory than XP.

They must be in cahoots with the memory makers, alert Rambus!

Re:Memory Usage?

[dtfinch]

s/ memory//g

It's not so bad. I tried running firefox on a 233mhz debian system with 96mb and it took over a minute to load. I was using icewm, which is supposed to be pretty light. Times are changing I guess. Back in '95, Windows 95 was seen as bloatware. Nowadays, we look at Win2k which wants at least 16-32x as much ram minimum as 95 and think of how light 2k it is compared to XP.

Re:Memory Usage?

[dtfinch]

Microsoft did very well with their boot prefetch optimization in XP. Most distros still run startup programs one at a time, waiting for each to exit before starting the next. It simplifies configuration, only having to specify loading order than specify dependencies for each startup item, but it's really slow, and will continue to be slow unless the distros want to break some backward compatibility. That's my guess anyways.

TFA

[222]

From TFA...

"So why is this interesting? Because their test methods reflect Windows internals, not Unix kernel design. There are better, faster, ways of doing these things in Unix, but these guys - among the best and brightest programmers working at Microsoft- either didn't know or didn't care."

So, Windows still loses at times when using what seems to be a biased (or simply uninformed) testing method? Loelz.

sigh.. dupe.

[Random Destruction]

Gotta love dupes http://slashdot.org/article.pl?sid=05/11/03/174423 0 another chance to get that karma up.

Re:premature optimization

[Anonymous Coward]

see which takes longer

Is "searching the manpages" included in the benchmark time?

*Ducks*

Processes v. threads

[Anonymous Coward]

NT (and its latter incarnations like XP and so forth) are desgined to use threads rather than process for multi-processing/concurrency, I understood. Is Win XP less efficient in multi-threading than BSD/Linux? The history of threads in UNIX seems more later bolt-on - UNIX was designed with multi-process model, I think.

No I didn't RTFA.

Re:Processes v. threads

[Todd Knarr]

Exactly. NT got it's process model from VMS, and process creation was a very heavyweight operation. Unix, by contrast, had a very lightweight process creation operation. Hence NT needed threads to provide a faster alternative to processes, while Unix (whose processes were almost as cheap to create as NT threads) didn't really need threads for anything other than a marketing checklist (about the only thing Unix threads get you that processes don't is fully-shared address space, and I'd argue that's often more a problem than an advantage).

Re:Processes v. threads

[Todd Knarr]

You need to upgrade to a current version of GDB, then. Anything recent (within the last year or two) handles threads fine. I've used it to debug applications with a couple of hundred simultaneously-active threads without any problems.

Re:Processes v. threads

[TummyX]

Hence NT needed threads to provide a faster alternative to processes, while Unix (whose processes were almost as cheap to create as NT threads) didn't really need threads


You make it sound like they added threads cause they couldn't get processes fast enough which is absolutely false. Threading and asynchronous IO were a design *feature* not an after thought.


while Unix (whose processes were almost as cheap to create as NT threads) didn't really need threads for anything other than a marketing checklist (about the only thing Unix threads get you that processes don't is fully-shared address space, and I'd argue that's often more a problem than an advantage).


Threads are suprior to forked processes in many ways. If you don't think having all the process memory shared between threads is important, why bother having memory shared between different functions? The fact is that the need for multiple asynchronous operations to occur within the context of a single memory space is not only useful, but very important in modern day applications. There are cases where isolating memory is good (for security) but there are just as many cases where having isolated memory is a pain because suddenly you've lost the usefulness of the native language (C in this case) and have to resort to using mmap and other system calls just to share and pass data.

Re:Processes v. threads

[Todd Knarr]

Forked processes do share memory (at least until it's modified). What you get with a shared memory space is the requirement for a lot of mutexing to avoid multiple updates, and a massively increased chance of corrupting memory across all threads. Been there, done that, in spades. Having done several massively-multi-threaded applications, my evaluation is that 90% of the time you're probably better off avoiding memory-based communication entirely in favor of message-passing through pipes, sockets or similar IPC mechanisms. That has the advantage of also removing most of the interlocking needed for high-performance shared-memory designs. The exceptions are when you've got a large amount of common configuration data that all tasks need access to. Using mmap() is a better way of handling that than sharing the entire address space, but when you've got pointer-based structures the workarounds for variable base addresses can get hairy enough to make shared-address-space the better option.

Just my opinion, but after 25 years of making machines with bits for brains do things I'm inclined to trust my own judgement.

Re:Processes v. threads

[WilburCobb]

> about the only thing Unix threads get you that processes
> don't is fully-shared address space, and I'd argue that's
> often more a problem than an advantage

What about context switch overhead? I think that fully-shared address space (in threads) is very useful because with it I can easily write a program for a multiprocessed system engaging all processors, a case where fully-shared address space is not a problem but just the right thing. This is just me, though.

Re:Processes v. threads

[TummyX]

Forked processes do share memory (at least until it's modified).


That's not what I meant by shared. That's just copy-on-write memory copied memory and only 'shared' for the purposes of saving memory.

Maybe you're writing a different class of applications, but I find threading extremely important. There are many classes of problems where you want to go off and perform a task asynchronously whilst you continue doing something else. I don't think an animation or maybe some blocking IO should require spawning a another process.

It's not about using memory for data communication, it's about using memory for sharing or resources. Like I said, if using seperate processes were so great, why not just write every *function* in your program as a forked process and use IPC and shared memory?

Re:Processes v. threads

[Todd Knarr]

As for the first part, as I noted a lot of "shared" memory is read-only as far as the child's concerned. The only real issue with seperate processes is the potential for a massive memory split if one process suddenly updates the majority of that "read-only" memory (eg. reloading of configuration data). Where you have lots of memory shared that doesn't involve pointer-based data structures, shared memory will get you what you need without the need for fully-shared address space (which means your various processes are protected from heap corruption by their siblings, among other things). There are times you need the entire address space shared, but IMHO they aren't nearly as common as they're made out to be and the problems of fully-shared address space are well-known and not insignificant.

As far as why you don't make every function a process, for the same reason you don't make every function a thread: the vast majority of a program is not parallelizable. Parallel threads of execution, be they threads or processes, are applicable when the parent can fire off the child and go do something else while the child executes. If the only thing the parent can do at that point is wait for the child to finish, it's more efficient to simply do a call within the same thread of execution.

Re:Processes v. threads

[Todd Knarr]

Even between threads you've got overhead, either true context-switch overhead (on systems with a 1:1 thread-to-KSE mapping) or a similar amount of thread-scheduler overhead (on systems with an N:1 thread-to-KSE mapping). Note that multiple processes will also use all available CPUs in a multi-processor system, and on some systems threads can only> execute on the single CPU their KSE is currently running on, so depending on your threads implementation a multi-threaded app may or may not use all CPUs.

Re:Processes v. threads

[WilburCobb]

> Even between threads you've got overhead, either true context-switch overhead (on systems with a 1:1 thread-to-KSE mapping) or a similar amount of thread-scheduler overhead (on systems with an N:1 thread-to-KSE mapping).

Then in those systems threads are less useful, though as efficient as processes in terms of overheads. Threads in this case are more efficient than pipes between processes as there is less copy of data. Also, threads can be used for asyncronous IO implementation in a much simpler and portable way than with pipe-fork.

> Note that multiple processes will also use all available CPUs in a multi-processor system,

Only in smp capable systems. On the other ones (e.g linux 1.x), processes for paralel computation are less useful. You see, kernels can support smp or not, systems can have true lightweight smp threads or not.

> and on some systems threads can only> execute on the single CPU their KSE is currently running on,

Again, in those systems threads are less useful but not useless.

> so depending on your threads implementation a multi-threaded app may or may not use all CPUs.

This seems academic, as we have good operating systems that address all those questions properly, at least one of them free and multi-plataform.

Re:Processes v. threads

[TummyX]

As for the first part, as I noted a lot of "shared" memory is read-only as far as the child's concerned. The only real issue with seperate processes is the potential for a massive memory split if one process suddenly updates the majority of that "read-only" memory (eg. reloading of configuration data). Where you have lots of memory shared that doesn't involve pointer-based data structures, shared memory will get you what you need without the need for fully-shared address space (which means your various processes are protected from heap corruption by their siblings, among other things). There are times you need the entire address space shared, but IMHO they aren't nearly as common as they're made out to be and the problems of fully-shared address space are well-known and not insignificant.


Again, I totally disagree. I find your idea that 90% of the time, different threads of executation don't need to execute in the context of a shared address space to be rather bizarre. Perhaps it's because you've spent so many years programming with processes rather than threads.


As far as why you don't make every function a process, for the same reason you don't make every function a thread: the vast majority of a program is not parallelizable. Parallel threads of execution, be they threads or processes, are applicable when the parent can fire off the child and go do something else while the child executes. If the only thing the parent can do at that point is wait for the child to finish, it's more efficient to simply do a call within the same thread of execution.


You missed my point which is that functions share the exact same address space because it is useful and there multiple threads of execution have the exact same address space for exactly the same reasons.

I find it astounding that someone can think that 90% of the time, a single address space only needs one thread of execution.

What kind of applications have you been writing the last 25 years?

Re:Processes v. threads

[TummyX]

You're misreading. It's not 90% of the problems out there, it's 90% of the code in a given program that's synchronous.


I really doubt that.


Take, for example, the process of reading data from a single input source and processing it. With no other input sources to look at, and no processing that doesn't require the data you're trying to read, exactly what can the code do while the read's completing?


That's not a typical modern server or end user application.


You incur the overhead of creating a thread, and then the parent simply blocks until the child thread completes. Less overhead to simply do the read in the same thread as a simple function call


Who was suggesting that we should call functions in a seperate thread when the operation is synchronous? My example was an example of how silly it was for you to state that a shared address space between async operations weren't needed.


You incur the overhead of creating a thread, and then the parent simply blocks until the child thread completes.


I would think someone with your experience would know about something as basic as thread-pooling.


Most of a program is like that: the caller can't proceed until the function it's called returns it's results, so running the called function in a seperate thread doesn't actually result in any parallelization


That most programs today aren't GUI applications where the UI should not be *blocked* whilst some request is processed? Even something as basic as IO can easily be made asynchronous as there is almost always some post processing to be done after a file or part of a file is loaded into memory that can be done whilst the next block is read.


Most of a program is like that: the caller can't proceed until the function it's called returns it's results, so running the called function in a seperate thread doesn't actually result in any parallelization


Sounds like you need to be more creative in how you design your applications. Most modern applications aren't top-down flow chart style processes.

Re:Processes v. threads

[Todd Knarr]

That's not a typical modern server or end user application.

You mean a modern network server doesn't have to read data from it's incoming connections? News to me, every one I've worked on has that as one of it's primary tasks. And it can't process the data until it's read it (or at least a sufficient portion of it to be able to figure out what to do next). Now, you can get fancy by trying to multiplex multiple connections in a single thread, but it's much simpler to spawn one thread of execution for each connection. You've got enough complexity already that you don't need to go creating any more for yourself.

That most programs today aren't GUI applications where the UI should not be *blocked* whilst some request is processed?

Modern GUI applications are message-driven. It's the message-processing loop that needs to never block. If you look, you'll find that 90% of the code in that GUI application is plain old traditional synchronous code to handle each type of message. Only a small portion in the message loop actually deals with controlling multiple threads.

As an example, if a GUI event triggers a time-consuming compute process, I can do two things. One, I can spawn off a thread to do the compute job and signal me when it's done. Two, I can write the data block down a pipe and let another thread or process deal with it, after having set a monitor on the pipe so I'll get a message when data's available on it. When the data-available message comes in, I snarf the results back from the pipe and do what needs done with them. Depending on how I implement the compute job there may be a layer to handle multiple incoming requests and fire off compute jobs, but the big function that actually does the crunching will be plain old single-threaded synchronous code and I'll let the system scheduler handle multi-tasking between the various jobs running at any given time. And on a Unix system where fork/exec is relatively cheap, why waste development time on a complex multi-threaded compute program when I can design a simple single-threaded one that reads it's input from standard input (connected to the incoming pipe) and writes it's output to standard output (connected to the outgoing pipe) and is pretty much immune to memory leaks, memory corruption and a host of other problems caused by other threads of execution in the same address space. On a modern Unix this'll probably handle 50 or so forks/second, and I've the option of doing pre-forking and having one instance handle multiple requests before terminating which should net me a hundred or so requests/second easily. By my standards that's hideously slow, but it's more than sufficient for an awful lot of applications. Especially in user-interface applications, because humans simply can't handle hundreds of operations per second.

Re:Processes v. threads

[TummyX]

You mean a modern network server doesn't have to read data from it's incoming connections? News to me, every one I've worked on has that as one of it's primary tasks. And it can't process the data until it's read it (or at least a sufficient portion of it to be able to figure out what to do next). Now, you can get fancy by trying to multiplex multiple connections in a single thread, but it's much simpler to spawn one thread of execution for each connection. You've got enough complexity already that you don't need to go creating any more for yourself.


Duh. The typicaly modern network server *or* end user application doesn't just sit there, read data and have NOTHING ELSE TO DO whilst its waiting for data.


Modern GUI applications are message-driven. It's the message-processing loop that needs to never block. If you look, you'll find that 90% of the code in that GUI application is plain old traditional synchronous code to handle each type of message. Only a small portion in the message loop actually deals with controlling multiple threads.


Duh. That doesn't mean it can't be rewritten to do tasks asynchronously. Many applications in the past have been written poorly but most applications now days will take advantage of multiple threads to make the GUI more responsive.


As an example, if a GUI event triggers a time-consuming compute process, I can do two things. One, I can spawn off a thread to do the compute job and signal me when it's done. Two, I can write the data block down a pipe and let another thread or process deal with it, after having set a monitor on the pipe so I'll get a message when data's available on it. When the data-available message comes in, I snarf the results back from the pipe and do what needs done with them


Oh that's smart. Calling something in another thread is no harder than calling a function and the alternative is to serialize your data and mutable global state, create a RPC protocol and send the task to another process and then read the response back. Yeah, that sounds heaps easier.


And on a Unix system where fork/exec is relatively cheap, why waste development time on a complex multi-threaded compute program when I can design a simple single-threaded one that reads it's input from standard input (connected to the incoming pipe) and writes it's output to standard output (connected to the outgoing pipe) and is pretty much immune to memory leaks, memory corruption and a host of other problems caused by other threads of execution in the same address space


ROFL. Threading (basically calling a function) more complex than writing client/server processes? We're back to asking you why you believe that somehow, different threads of execution have no need to be associated with the address space of another thread of execution even though different functions do.

I'm pretty tired of talking to someone who clearly has no experience with end user apps or modern concurrent and async application design.

Strangely enough, 5,376,000

[Anonymous Coward]

is also the number of cycles needed to crash a process.

No it doesn't.

[Anonymous Coward]

It explains why apache for windows uses threads. Everyone has always known creating processes is slow on windows. Unix has always heavily optimized this since unix programs were so frequently written to fork(). Very few windows programs spawn processes at any rate that would cause you to notice a slowdown.

Wohoo!

[AlXtreme]

Finally a proper microkernel OS design by Microsoft! prof Tanenbaum would be proud!

Come on, who cares about statistics? I'm glad they're actually doing something useful: CS research!

Oh wait, this is /.: Die M$ XP, DIE! *pinky to mouth*

Re:Wohoo!

[Kickasso]

It's not really a mickokernel, as everything (including the userland) is in one big address space.

Re:Wohoo!

[delire]

I'm glad they're actually doing something useful: CS research!

No, the equivalent of 'R&D' on the microsoft campus is R&P, or 'Research and Patent'. Like most of their would-be innovations, they are born into formaldehyde, destined to serve as court room exhibits; the last thing microsoft can afford, is a lively, competitive software industry spurred by brilliant implementations and ideas.

To satisfy your inevitable curiosity, peruse their fine patent collection. Plenty of 'research' and 'innovation' in there.

Re:Wohoo!

[arkanes]

It's still a microkernel, because they define "object spaces" rather than memory spaces. Which is fine, because nothing is allowed to run except .NET code, so nothing has direct memory access, just object references.

Re:Wohoo!

[Nasarius]

So it's a monolithic kernel that tries to behave like a microkernel internally. Could work.

Re:Wohoo!

[arkanes]

Monolithic vs microkernel are *design* concepts. It is designed and implemented as a microkernel. Don't get hung up on the address space issue. Singularity doesn't *have* address spaces.

Re:Wohoo!

[be-fan]

Singularity isn't really a microkernel. It's a "no kernel", in that there is no seperation between kernel space and user space. The protection offered by such a design is much more fine-grained than with a microkernel (individual threads are protected from each other), and doesn't require the same performance hit (no expensive context switches between protection domains).

Typical

[Bogtha]

This is pretty typical. Microsoft's biggest competitor is their old software, so their new offerings have to look good against it.

Remember Windows 95's marketing? "32-bit memory protection makes it uncrashable!" Remember Windows 98's marketing? "Even more stable than 95!" Remember Windows 2000's marketing? "Based on an NT core, it's more stable than the crash-prone Windows 9x!"

Its revisionist history. The only way to get a somewhat accurate picture is if you compare their current claims with what they've said about new technology in the past.

Re:Typical

[AFCArchvile]

So here's the list of application crashes not fatal to general system stability, with OS version. And of course, the newer version had "No more ($errorname)s!", because they were no longer called $errorname.

- Windows 3.0: Uninterruptible Application Error
- Windows 3.1: General Protection Fault
- Windows 95/98/ME: Illegal Operation
- Windows NT/2K/XP: Access Violation

I wonder what the new, re-euphemised name for this in Windows Vista will be. Unapproved Boondoggle Cessation?

Re:Typical

[maxume]

Those bastards! They keep making their software better!

Re:Typical

[mcrbids]

Remember Windows 95's marketing? "32-bit memory protection makes it uncrashable!" Remember Windows 98's marketing? "Even more stable than 95!" Remember Windows 2000's marketing? "Based on an NT core, it's more stable than the crash-prone Windows 9x!"

Heh - I remember when Win95 first came out. I went to the local Circuit City and asked why I'd want to buy a computer with Win95. His first response was that it was so much more stable than previous versions of Windows. He turned to the nearest computer and jiggered the mouse - it was frozen.

Mumbling something he went on to the net computer, to find it in a similar position. Finally, on the 3rd computer, he got it to respond, and then proceeded to show me the cool Start Menu.

I didn't buy it for over a year after that demo.

Article misses the point

[ichin4]

This article takes a very interesting report on a reference implementation of some innovative ideas in OS design and reduces it to a couple of entirely peripheral, seat-of-the-pants benchmarks that support the "OSS rulez!" thesis.

Even people like me, who have only a basic knowledge of OS architecture, can tell you that processes are lightweight in Unix and heavyweight in Windows. The lightweight objects in Windows are threads, which is why Windows makes so much use of threads, while Unix spawns processes left and right.

Typical slashdot post exaggerations

[chris09876]

The scenario stated in the slashdot post does show a situation where linux performs better than Windows. ...but after looking through the "performance" section of the whitepaper, it's pretty much the only case where linux is better. Windows appears to beat linux on quite a few other tests (such as memory use of a 'hello world' program, the executable size, and even some of the 'cost of basic operations' tests)

Re:Typical slashdot post exaggerations

[51mon]

> Windows appears to beat linux on quite a few other tests (such as memory use of a 'hello world' program, the executable size, and even some of the 'cost of basic operations' tests)

The executable thing is a figment of comparing apples and oranges, the Unix code was statically linked, which is basically including all sorts of stuff that you wouldn't in the real world.

Good dynamic linking is something the Unix (and clones) have done for a while (read long before Windows even started on this route), and was why we could run fully featured, X Windows based, Office suites (and Window managers) for umpteen users simultaneously on 125MHz PA-RISC machines with 100 or so MB of memory, and still have memory spare. And you still see this edge if you compare terminal services like deployments.

The cost of operations is less on pretty much everything but thread stuff, but then the cost of creating a process is so low in Linux, threads are more of a convenience for programmers than a structural component, as others have discussed. Although I think there are advantages to good thread handling, cost of creation can usually be mitigated by maintaining pools of threads (as can the cost of process creation in Windows for certain tasks), certainly the typical environments where one uses threads for performance reasons (think Apache2).

The tests basically tell those who understand computers, that they understand them correctly. Windows is faster at creating threads, the Unix crowd is better at pretty much everything else. It is what you'd expect comparing server class OSes against a desktop system. No one really cares if XP can't fork a process efficiently, if you only run a new application every half an hour, but if you fork (or prefork) for every incoming email request, you better do it efficiently.

I don't quite follow the sequential I/O for small block sizes, the default blocksize in the Unix world for the traditional filesystems is typically 8Kb, and no one ever opts for smaller (well some of the new filesystems do), particularly if they expect a lot of sequential I/O when they opt for bigger, so faster sequential I/O for block sizes less than this is I suspect of no relevance in the real world. Probably just indicates code in the Unix systems that assumes a larger block size, and is thus redundant. Kind of like discovering that a human brick layer out performs your wall building machine, if you break all the brick in to pieces before you start, interesting, but mostly shows your wall building machine expected bricks to be, well, brick sized.

Mod Parent Up!!!

[RockClimbingFool]

I wish I has some mod points for you.

This isn't Microsoft

[The Pim]

This is Microsoft Research. They have the same independence as university researchers--that is how Microsoft lures them away from academia. These guys are making honest comparisons to Linux and FreeBSD, because that is what they do as good researchers. Microsoft is enlightened enough not to interfere.

Re:44 pages and the main question is still unanswe

[pingveno]

It sounds like this is a *research* project. That doesn't mean that everything is perfect and optimized yet, but only that new ideas are being tested out. Stability comes far later.

Re:That's why it's called "development"

[geomon]

Funny how memos like "Windows XP still hands Linux its ass in the desktop market -- any questions?" never leak out to the public.

How much of that advantage is performance based and how much is due to monopoly control?

Re:What's the point of CreateProcess benchmarks?

[Just Some Guy]

Did you ever think that the reason there aren't any is that Windows can't handle it well? Seems like pretty circular reasoning.

Re:44 pages and the main question is still unanswe

[FinestLittleSpace]

That is such a troll. Every OS has updates.

this is old news

[P3NIS_CLEAVER]

unix is based on processes vs threads for windows. of course it is faster!

Re:44 pages and the main question is still unanswe

[trondd]

You clearly don't know much about what makes an operating system stable... Stability depends partly on how much error checking the compiler is capable of doing, partly on how people write software (design) and partly on how well the operating system is designed to separate processes and different parts from each other. Singulary addresses all of these issues: Its mainly writen in a "safe" language which is strongly typed and does lots of compiletime check and it is a microkernel operating system which (at least in theory) prevents your cheezy usb webcam driver from crashing the kernel. Most other unix wannabe systems are writen in the ancient language C :), and run monolithic kernels.

But singularity isn't all new, it just implements old ideas: Occam and QNX!

But in my opinion, Singularity just might be the most interessting os to emerge in the last years. It will be interesting to see how long it will take the free software world to come up with something similar :) (btw, I am a long term happy gnu/linux user, and have no plan of switching...)

Waking up?

[headkase]

Instead of paying rapt attention to what Microsoft is doing what I would like to see the OSS community do is consciously form more organizations that would as an express purpose chip away at Microsoft's software base. What I mean by this is make sure your program runs on Windows for now. Get people using OSS and used to the idea so that the next time average-joe needs some software he'll search for an OSS program first. Then once that mindshare has been established begin to work towards the more core functions like the OS itself. Who knows, Microsoft might at some point simply open up the source of Windows to counter a loss of control to OSS if they see that their customers are truly ready to abandon ship. And to build that feeling in customers give them options - if all their useful software is OSS then they can swap out the lower levels (like Linux for Windows) without feeling any transition pain at all because their software applications didn't change at all only the plumbing did.
Ballmer's right, it is all about developers. OSS developers can introduce OSS values into the Windows "ecosystem" for lack of a better word and see what happens.

Re:Waking up?

[killjoe]

We'll get right on that. No, no, don't get up, there is no need for YOU to do anything. It's my pleasure to work to make you happy and not charge you a dime.

Re:Waking up?

[headkase]

I'll get right on that.
;)

Re:Waking up?

[headkase]

You know, I don't know. Isn't that the problem? I know what helped me a lot when I ran SuSE was the table of equivalents that used to be available at: linuxshop.ru/linuxbegin/win-lin-soft-en/table.shtm l but isn't anymore so you can find it on the coralized on the wayback machine here:
http://web.archive.org.nyud.net:8090/web/200402021 11451/http://linuxshop.ru/linuxbegin/win-lin-soft- en/table.shtml
So your right most good software is already available but I off the top of my head still don't know where I could get windows viewers for .djvu files too... Visibility I guess.

Re:44 pages and the main question is still unanswe

[lubricated]

but very few gnu/linux updates require reboots. IOW, It is rare for the kernel itself to need a patch.

Re:What's the point of CreateProcess benchmarks?

[ichin4]

Processes in Unix are lightweight objects, and the OS spawns them left and right. Processes in Windows are heavyweight objects, and the OS creates only a handfull of them. The lightweight objects in windows are threads, and you'll notice that Windows thread creation is faster than Unix thread creation. These are just different OS design philosophies.

not caffeine...

[conJunk]

he gets it right here

Re:44 pages and the main question is still unanswe

[hal2814]

Some OSes don't require a complete reboot when you do an update. Also, some don't require a reboot but recommend one (Solaris comes to mind). In most of the *NIX world, you can just restart a few services instead of rebooting. Personally, I've never been in a situation where I can't take the server down at non-peak times for routine maintenance so it really doesn't matter to me. In fact, a lot of times rebooting is easier than finding which services were patched and restarting them.

Win/XP, MacOS/X, WhatThe/Heck?

[danaris]

Entirely OT, I know, but...

Why is it that some people seem to think that all OS names, when they have a qualifier of some kind attached to the generic term, need a slash to separate them? Just because GNU/Linux is written that way does not mean it's some kind of law, people...

It's Windows XP. That's WINDOWS {SPACE} XP. And Mac OS X. Spaces. No slashes.

...

I don't know why I even bother...

Dan Aris

Re:Win/XP, MacOS/X, WhatThe/Heck?

[Nasarius]

Nobody cares about "GNU/Linux". I believe the slash comes from OS/2.

Re:Win/XP, MacOS/X, WhatThe/Heck?

[danaris]

Ah, that's possible, too.

It's still stupid...clinging to the last vestiges of a long-dead OS...

Dan Aris

Re:Win/XP, MacOS/X, WhatThe/Heck?

[AEton]

I don't understand how this could surprise you; it's a very common virus naming convention.

Re:Win/XP, MacOS/X, WhatThe/Heck?

[Arandir]

Just because GNU/Linux is written that way...

Actually it's not written that way. It's written "Linux". But that's another topic...

Re:Win/XP, MacOS/X, WhatThe/Heck?

[Crag]

Since I work in a unix shell so much I tend to avoid whitespace in the names of things. Therefore, I call the Microsoft operating systems Win95, WinNT, Win2k, WinXP. I call Unix variants by their distribution name: Debian, RedHat, FreeBSD, Solaris. I only use a slash when it really is part of the OS name: OS/2, HP/UX. Whenever possible I avoid using names which would have to be quoted or escaped.

I don't know what other people's excuses are.

Wow is this ever misleading

[tqbf]

Here's the table from the paper, ranked best-worst, W=windows, F=freebsd, L=linux, S=singularity:

Read Cycle Counter: W: 2, F: 6, L: 6, W: 2, S: 8

ABI Call: S:87, L:437, W:627, F:878

Thread Yield: S:394, W:753, L:906, F: 911

2-Thread ping-pong: S:1207, W:1658, L: 4041, F: 4707

2-Message ping-pong: S:1452, L: 5797, W: 6244, F: 13304

Process Creation: S: 300000, L: 719000, F: 1032000, W: 5376000

The only stat in this table that Windows trails on is process creation. And anybody who has ever ported Unix code to Win32 knows exactly why: Windows is thread-oriented, and Windows systems don't tend to use helper programs or demand-forking to get work done. Which might be why Windows beats Unix in the thread benchmarks, but not in the IPC benchmarks. On the more general benchmarks, like cycles to issue a system call, Windows falls smack in the middle --- and, again, Windows has a slightly different take on what is and isn't a system call.

Drawing comparisons between Singularity and normal operating systems here is silly. Singularity doesn't have processes in the conventional sense; since there's no hardware dependencies on "process" creation in Singularity, IPC and forking are much faster.

Which is why this benchmark is reasonable inside the Singularity tech report (they're trying to demonstrate that there's a major performance benefit in rethinking boundaries between programs), but totally unreasonably outside that context: these are micro-benchmarks, like the ones CISC and RISC people throw at each other, and don't describe the amount of time it takes to complete a high-level task. Time to execute a system call is meaningful only in the context of how many system calls it takes to complete the task you're measuring.

Re:44 pages and the main question is still unanswe

[man_of_mr_e]

While technically, reboots are not required for anything other than kernel patches, there are lots of situations where it's easier to reboot than to restart every application (which might as well be a reboot anyways). For example, glibc updates will require almost every application to be restarted, or you risk exposing vulnerabilities.

Re:44 pages and the main question is still unanswe

[misleb]

Didn't Mel Gibson say that in a movie once?

"Ev'ry OS has oopdates, but not ev'ry OS reeequires a reboot!"

Seriously though, very few Linux updates, for example, require a reboot. Most updates occur in user space and can be adequately applied by restarting the applicable services (if any). You just have to be aware of exactly what is being updated and what it affects.

In the (non-Windows) server world, rebooting is a big no-no.

-matthew

But that impacts "stability".

[khasim]

I am reminded of the Win9x bug that caused a failure every 49 days.

If your system is not "up" long enough to trigger a bug, is it "stable"?

If "yes", then at what point does a system become "unstable"? If I have to reboot every year? Month? Week? Day? Hour? Minute?

Or is "stable" defined in terms of "unexpected crash" and discounts any "crash" that is avoided by rebooting the system?

This is one of the reasons I like Linux. Because the system doesn't require reboots except to replace the kernel, it is easier to identify apps that are unstable AND GET THEM FIXED.

dependance or dependability?

[autopr0n]

Are these guys non-native speakers or what? Dependance is a bad thing, dependability is good.

sheesh.

Re:dependance or dependability?

[cloudmaster]

It's a Microsoft OS, and you're saying that they made a mistake when mentioning that one of their goals is increased dependence? Hell yes that's their goal. Vendor lock-in, forced upgrade cycles, dependence - all the same thing, and all the goal of any winning software company. :)

Re:44 pages and the main question is still unanswe

[Anonymous Coward]

All of you OSS d*ldo heads still cannot understand the need for a OSS MS Exchange solution. Back end integration of mail/tasks/calendar and syncing with pdas from the desktop client and vica-versa and sharing calendar info with other and such is the way of the future. Still we have nothing that comes close. The germans keep putting out overly complicated java based crap like openexchange but nothing that is clean and works and en masse like the apache webserver.

Talk about a misleading summary

[autopr0n]

Windows performed better then Free BSD on all the micro-tests but one, and better then Linux on half of them.

Oh well, this is slashdot, can't expect editors to, you know, edit anything or even bother to read what they post.

Re:44 pages and the main question is still unanswe

[EnderWiggnz]

i would *love* to see how someone updates a running glibc...

really... how exactly do you replace a running libc?

Re:Colonel Realtime

[Caspian]

Because a gigahertz is a BILLION hertz, not a MILLION.

How do you bow without a brain to weigh your head?

[autopr0n]

Did it ever occur to you that he might have seen that linked somewhere else, since Slashdot is really really slow?

Re:44 pages and the main question is still unanswe

[WalterODimm]

Will it does say dependence, not dependability. Perhaps that's a typo, but since the article seemed to be about performance, I would think that's why the dependability factor is mysteriously missing. On a side note... selling crack is all about increasing dependence as well.

Where is your fantastic research?

[autopr0n]

I'm a little confused, I don't see a link to all the wonderful, earthshacking research you've been doing. Perhaps you could give us a link?

Re:Where is your fantastic research?

[CaymanIslandCarpedie]

Here

just think about it.

[Kuku_monroe]

reverse psychology is the only way to get linux zealots to shut up, they did the same thing praising the Revolution, and they will soon kiss google feet

Re:44 pages and the main question is still unanswe

[Feyr]

http://www.gnu.org/software/hurd/hurd.html

Re:44 pages and the main question is still unanswe

[dknj]

um, rm old libc, copy over new libc (making sure copy is a statically linked binary). viola, new libc in place. now say app A,B,C,D,E,F,G,H,I,and K are still using old libc (computers 101: an OS copies the app and associated libraries to memory, then executes, so we're still linked to an old libc stored in memory). you need to kill and restart these applications in order to reload a runtime linked library. now if your app dynamically loads libraries (libdl), a simple signal may instruct the application to reload its libraries. however unless you know the behavior of the application, you're better off restarting it. this is why when a major library is updated, it is safer to just reboot the box. i will pick security over uptime, this isn't the 80s anymore

*replace libc with any dynamically loaded library

Depends

[jd]

There's been a lot of work on improving the threading under Unix variants. M:N threading models, zero-copy where data structures are identical, etc. It is entirely possible, if not probable, that some cases of threading will actually be faster under some u*ix-like OS' than Windows. Because there has also been a lot of work on security models under u*ix-like OS' (role-based memory encapsulation, etc) which are inherently slooooow, there will certainly be u*ix-like OS' which are slower at starting new processes under those OS' (where you will have maximal security checks) than Windows.

In other words, write the benchtest for the sorts of sub-category of cases that side with you, and you can make any benchmarks show what you want them to show. That's one reason they should always be viewed with a pinch of salt and a dash of vinegar, then served in newspaper.

Re:Depends

[PsychicX]

M:N threading models Solaris did this at one point. In short, it didn't go well. The system was a bitch to implement, but when it finally worked it was a big point of advertising. Later, when porting to the Itanium, they simply couldn't get it to work with any kind of reasonable performance, so they stripped it out and went for a more conventional approach. It was much faster than the M:N system. A little later, somebody ported the new system with the M:N threading stripped out back to Sparc, only to find that the conventional model beat the crap out of the M:N threader.

Re:44 pages and the main question is still unanswe

[ergo98]

It will be interesting to see how long it will take the free software world to come up with something similar :)

Aren't there quite a few Java Operating System initiatives out there already? (both free and open source, and commercial) Basically the same idea.

OS in C# ???

[sanferrera]

From the singularity home page:
"Singularity: A research OS written in C#"

1) How can you write an OS in C#, if C# is by def. an OS dependent language?
2) Mmmh, that virtual machine stuff doesn't it make it a liiiitle bit slower?

Re:OS in C# ???

[ichin4]

1. I have no idea what you mean by "an OS dependent language". Does that mean you believe that mono, which compiles C# in Linux, uses a little hidden copy of the windows OS? Anyway, that isn't right. C# can be compiled all the way down to the CPU's instruction set, and can therefore be used to write an OS.

2. Yes, the runtime makes things a little slower. As long as it isn't much slower, or the speed of your hardware is such that it doesn't matter for your application, this isn't a problem. (By the way, if you want your app to run really fast, you run it without an OS -- but for most people the convenience of the services the OS provides outweighs its perf hit.)

Re:OS in C# ???

[Anonymous Coward]

1) Written in Sing# not C#
2) The virtual machine is not the same as the .NET virtual machine. They've implemented a much tighter and leaner "VM" and interwoven it with the kernel. In addition, some of the "stuff" in the .NET VM and base class libraries are handled by the Sing# compiler differently than the C# compiler.

Re:OS in C# ???

[managedcode]

Its ridiculous to use managed languages to write OS. Guess I should commit suicide pretty soon.

Re:OS in C# ???

[sanferrera]

1. No, dude. I don't believe mono runs in a hidden copy of windows. What i meant is that both OS implementations require a service which translate to machine language more or less realtime. To boot itself into his own bootstraps using C# is very interesting, but will require compiled stuff at least for the bootloader, the virtual machine and at least the beginning of the process manager. Interesting but complex. 2. You don't seem to realize how many times more instructions you need for a system like this in comparisson to a plain c version. It's not only a bit slower, it's a lot slower. (oh, by the way, try to make a browser and word run at the same on a pc without an OS, and tell me how well you did without disk driver, memory mgr, process mgr, video drivers and other io drivers)

Re:OS in C# ???

[be-fan]

The fact that the virtual machine is a bit slower isn't the point. The point is that because the virtual machine ensures memory protection, Singularity doesn't need to use hardware memory protection for the kernel. Doing a single system call costs hundreds of clock cycles on a modern CPU, because of the userspace/kernelspace switch. It also necessitates all sorts of complex (and slow) IPC mechanisms that go through the kernel (and invoke the aforementioned switch), all because we're still programming in an antiquated 1970's era language that let's programs randomly write into memory.

Modern CPUs quite be quite a bit faster if they didn't have to support C. Take a look sometime at all the die space an Athlon64 uses for stuff like TLB, etc. Also look how it needs to increase L1 cache latency by 50% (from 2 cycles to 3), just to support the TLB lookup. All of this stuff would be unnecessary if C programs couldn't overwrite whatever memory they wanted.

Re:OS in C# ???

[Quantam]

Singularity can only get you so far. I expect Singularity to work well for preventing application crashes, and it should significantly speed up interprocess communication. It may even somewhat reduce the amount of blue-screens (or whatever color the Linux equivalent is); but it won't be able to help with most kernel mode faults, because it still can't protect the software from the hardware. The hardware is still free to stomp all over program, driver, and OS memory, because a driver made some error in sending values to the hardware (which is impossible for the JITer to validate).

A more in-depth version of this post

Re:OS in C# ???

[Guy Harris]

C# is not an OS (Windows) dependent language. It is a platform (ie - Intel x86) dependent language.

I.e., there exists no native CLR for IA-64 (or any other instruction set), and there cannot be one? Are you certain of that?

If you aren't doing any tasks like that which require Windows, the raw machine code (or bytecode, I don't know which is the correct term) can just execute on the x86 processor.

Are you asserting here that the raw machine code into which C# is compiled is x86 machine code? It's not - it's Microsoft Intermediate Language, which gets translated to machine language before the code is run.

Re:44 pages and the main question is still unanswe

[fatboy]

init 1
init 3
No reboot required

Re:44 pages and the main question is still unanswe

[misleb]

(at least in theory)

And in reality it just makes things like disk I/O extremely slow, ala OS X. Personally, I am pretty disappointed by OS X as a server. Both in stability and speed. If that is a good example of what a microkernel can do in the real world...

-matthew

Umm... yeah, sure... but...

[Colin Smith]

I bet you don't get a dancing paperclip with Linux, do you?

Re:Umm... yeah, sure... but...

[JustNiz]

Yeah... thats another reason I use Linux...

Re:Umm... yeah, sure... but...

[nilram]

If you really want one try Vigor.

Re:44 pages and the main question is still unanswe

[Tony Hoyle]

Same difference... you've still shutdown all your network services which to the users means you've had downtime. It's a reboot in all but name.

Re:44 pages and the main question is still unanswe

[Anonymous Coward]

He did not read the paper at all, as far as I can tell.
The entire paper detailed the steps they were taking to make dependability the core goal of Singularity. That there are few reasonable metrics more measuring something as subjective as 'dependability', is not a criticism of this paper.
Grandparent is an idiot, most likely. Submitter of the entire article is dishonest in his quoting of metrics out of context and ignoring the point of the paper.

I read probably 60% of the paper. In short, they're using better languages to write a familiar operating system with some formalisms in place of add-on hacks of old (like shm, sem, threads)

Cache

[Mark_MF-WN]

Creating thousands of threads seems like a horribly inacurrate way to gauge performance. Creating a process is something that only goes on every once in a while -- threads or state machines are what are used when high levels of concurrency are required. The problem is that any code being executed repeatedly will be cached and often optomized by modern processors. P4s are especially aggressive about this thanks to their microcode cache. And different code will be optomized and perform from cache differently. So you wont get a very realistic idea about how long single process creations will take.

I could be wrong -- I'm hardly an OS or computer hardware expert. But the above is what my limited knowledge of those two areas would suggest.

Re:Cache

[EvanED]

"Creating a process is something that only goes on every once in a while -- threads or state machines are what are used when high levels of concurrency are required"

Except that in the Linux world, a kernel level thread IS a process. So creating a thread == creating a process. You're running Apache and a request comes in, it forks a process to handle it.

Re:Cache

[pthisis]

Creating thousands of threads seems like a horribly inacurrate way to gauge performance. Creating a process is something that only goes on every once in a while -- threads or state machines are what are used when high levels of concurrency are required.

That's true on Windows but generally not on other platforms, and the only reason it's true on Windows is because of the horribly broken process implementation. Memory protection is a good thing, and there's no reason that you should throw it away because you need concurrency--Windows forces you to do that by using threads instead of processes, and the result is a host of security and stability issues.

Choosing threads vs. processes ought to be a decision over whether you want to share all your memory or not. In real OSes, that's the case. In Windows, it isn't--you have to choose the wrong tool for the job much of the time.

Basically, you're arguing that "Windows programmers use this horrible workaround that introduces lots of problems, but avoids this speed issue--so the speed issue is irrelevant."

In fact, this is highlighting that Windows' process performance is so horrible that programmers really are forced to use a miserable workaround that causes a lot of other problems.

Re:Cache

[Mark_MF-WN]

Suck. I thought that lightweight threads had been added to the Linux kernel in the last couple years?

Re:Cache

[Mark_MF-WN]

Aren't there techniques for providing software memory protection? Software threading does have advantages. Of course, I think what we're really debating here is precisely which mistakes Microsoft made. There certainly no doubt that concurrency in Windows suffers from ... issues.

Re:Cache

[pthisis]

Aren't there techniques for providing software memory protection? Software threading does have advantages.

The theoretical difference between threads and processes is essentially that threads share their memory while process do not*. So really using threads is exactly throwing out memory protection between the threads.

That's good and useful in some limited cases, but it's very bad in many cases. Unfortunately, a system without good process support requires you to choose "concurrent but no protection" or "no/crappy concurrency with protection".

IOW, by not having a good process implementation you force the programmer to accept an unrelated (and potentially very harmful) feature in order to use the feature they really want.

Not only that, but you make it impossible (or crappy) to use well-designed applications on your system. Witness Apache 1.x performance sucking rocks on Windows, and 2.x having a rewrite to use threads basically as a massive workaround for the crap performance of Windows processes--and that cascades down, even once Apache was modified many modules that (with good reason) relied on having seperate address spaces don't work in 2.x or need to be rewritten.

It's a massive problem. IMO, it's probably the single biggest technical problem with Windows (and Java too).

Think about this: suppose you have a helper process that your web server calls to go do something needing concurrency. And then a bug arises in that process. The results could be pretty bad, a remote overflow that can crash your machine or kill that helper. But you can limit the damage--the helper can run as a different user without access to the main httpd, limited fs permissions, etc.

Now put it into a thread instead of a process. Not only can it crash as before, but remote exploits could grovel around to find open database connections and wipe out data on the remote DB server. The web server could be brought down entirely. Etc. By losing compartementalization, you open a host of security issues.

And independent of security, a bug in that helper could wind up clobbering the DB or httpd accidentally (most likely a stray pointer access crashing the httpd and bringing down the whole site instead of just, say, the streaming videos or whatever the helper was used for).

*A few other resources may also be shared or not, depending on the system, but the core distiction is memory sharing.

Re:Colonel Realtime

[Doc Ruby]

Right. Damn orders of magnitude getting out of control here :).

Re:44 pages and the main question is still unanswe

[macshit]

really... how exactly do you replace a running libc?

Typical distros that support pervasive no-reboot updating (like Debian) don't exactly replace a "running" libc (or any other library), they simply update the on-disk copy. So any programs run after that will get the new libc, but any programs that were started before the update will of course be using the old libc.

Usually this works very well; I suppose for a mega serious security update you might want to restart all your daemons too or something.

Wierd numbers

[redelm]

I'm an asm pgmr, and do lots of cycle counting. These numbers are screwey. First, RDTSC (read Cycle counter)_takes 16-40 clocks, depending on CPU. ABI/OS calls & yieldsound about right, mostly because every OS has the Ring3-r0-R3 transitions to do, except Singularity with forgoes this protection.

The real screwy number is Proc creation. I haven't timed recently, lbut I'd expect vfork() or exec() from RAM buffers to take

Re:Wierd numbers

[be-fan]

To be more accurate, that should read "except for Singularity, which doesn't need this protection".

Re:44 pages and the main question is still unanswe

[catalupus]

I just have to bow before the guy who can read a 44 page pdf and post an intelligible, coherent comment on it in less than two minutes. I just have to ask - where do you get that kind of caffeine?

The cops in England? http://www.theregister.co.uk/2005/11/11/drugs_mix_ up/

supporting quote

[The Pim]

Quote from a Microsoft researcher:

It's very nice working for an outfit that lets you do full-time research, doing pretty much what you want to do. Microsoft generally has fairly bad press, but I think that this is something that Microsoft should really brag about, because they pay lots of people to do essentially very freely directed research. They don't correct our papers, they let us go to whatever conferences we want to. I'm publishing at a higher rate than I did at the university.

(Simon Peyton Jones, 2001)

Wow...

[Khyber]

Microsoft admits that they're gettng beaten in several areas. I'm shocked. Ballmer must be getting senile.

Re:Colonel Realtime

[Doc Ruby]

And 1s != 1ms. But I think I get it.

Re:premature optimization

[mopslik]

install python, wxpython, pythoncard, egenix python extensions, java, jpype, & a few other python modules in linux. do the same in windows. see which takes longer.

You could do that, but unless you install each of these things on a regular basis instead of once or twice, then it's a fairly useless "benchmark". Something can take 1 hour to install vs. 30 minutes, but if it performs 15 seconds faster on a daily basis, where you use it 4 times a day... well, you onle need to use it for 30 days before the slower install pays off in terms of "performance".

Re:Linux is NOT a Unix Variant

[Laebshade]

It's funny that you should mention that Linux is not Unix, as I'd dare say that Linux = Linux is not Unix, or LinUx. :P

Re:44 pages and the main question is still unanswe

[arkanes]

The benchmarks shown (you DID read the article, right?) show that the MS OS easily keeps up with modern disk speeds for IO, but I wonder how things like graphics cards and even CD/DVD-ROM drives will work. Even on modern systems, you need to enable DMA for these sorts of devices to get acceptable performance, and the Singularity model seems like it will preclude this.

CPU cycles needed to "create and start a process"

[MaGGuN]

..I honestly cannot see how this is a benchmark worthy quality, especially for mainstream Windows XP users, or mainstream x86(-64) computer users for that matter. Anyone?

I use and value FreeBSD personally, but I surely consider other advantages more prevailing than how many cpu cycles a process creation and startup requires. Little can I recollect of other people, articles etc., that mention this at all.

Dependence or dependability?

[tomcres]

Please let it not be increased dependence... To be more dependent on Windows is probably the last thing we need!

Did you actually read it?

[cyclopropene]

Interesting...Singularity is ostensibly supposed to be about stability, but the 44-page paper has no data on this. Kinda like saying, "Our new bulletproof vest is 40% lighter than our leading competitors, and twice as flexible. How well does it stop bullets, you ask? Sorry...we do not yet have results for that benchmark.".

You didn't really read it, did you? From TFA(bstract).

...Singularity demonstrates the practicality of new technologies and architectural decisions, which should lead to the construction of more robust and dependable systems.

The point of the paper is NOT to demonstrate a fully working uber-dependable system, but to validate the practicality of the architecture that is under development, and the new technologies being included. That's why they have the section on performance, with the preface (right above your quote, btw):

If Singularity's goal is more dependable systems, why does this report include performance measurements? The answer is simple: these numbers demonstrate that [the] architecture that we proposed not only does not incur a performance penalty, but is often as fast as or faster than more conventional architecture[s]. In other words, it is a practical basis on which to build a system.

That's the point of the paper. I understand, however, that you might have been in too much of a rush to get first post that you didn't understand the point of the paper...

Re:44 pages and the main question is still unanswe

[DavidTC]

Not really.

For example, apache and sshd, and various FTPds, can be restarted without anyone possibly noticing, because they simply leave any running children open. You connected before a certain time, you got the old copy, you connected after it, you got the new one.

And, of course, many protocols work fine if you go away for five seconds, like SMB. The client program will just say 'oops, connection hiccup' and reconnect silently, and the end user never notices. Same with IMAP clients. They go 'Hey, the server closed my connection, I better open it again'.

Restarting services on a Linux box is 99% transparent to end users, even ones that are currently directly doing something with the server.

Rebooting is not transparent, even if all the connections are reaqquired automatically, simply because work stopped for the two minute reboot.

Nope...

[Skiron]

You get hot-babe

hot-babe

Much better than a paper clip...

Re:44 pages and the main question is still unanswe

[drinkypoo]

Well, usually you delete the existing symlink, and use a static tool (perhaps a static busybox binary) to make the new one. This is best done in single-user mode, though. I shouldn't have to tell you that when you delete a file on a Unix system, it doesn't go away until all references to it are closed, but you can still put a new file down in the same place and new open() or whatever-open() calls will open the new file.

Re:44 pages and the main question is still unanswe

[electroniceric]

I'd be curious to know how people think Singularity compares to the Hurd. IIRC, the Hurd looks at a lot of these ideas and takes the opposite tack from Singularity: rather than run type-checked, managed processes in kernel space, push everything out to userspace, and broker interaction with the kernel via various server processes. Has anyone done a comparison of the merits of these two approaches?

statistics and more statistics

[external400kdiskette]

realistically you can do any tests to say anything you want and all OS's are going to perform differently depending on the test. with an organisation as big as MS 99% are always going to say windows unequivically performs better but I Guess a few slip out sometimes.

Video of Singularity architects talking about it

[notaprguy]

If you have time, this is a pretty interesting video of two of the inventors of Singularity. They go into some detail on their motivations for doing the work, what they're learning from the prototype etc.

http://channel9.msdn.com/ShowPost.aspx?PostID=6830 2

Re:44 pages and the main question is still unanswe

[lubricated]

most patches are to one application only. If you are running a server then after upgrading libc you only have to restart the services exposed to the network and/or depending on the nature of the exploit log in and out.

Re:44 pages and the main question is still unanswe

[kimvette]

Troll? Bullshit.

I was making a funny, as in ha-ha. You guys are worse than Fark mods.

Re:premature optimization

[corsec67]

I run gentoo on all of my main computers.

If you want to complain about install times, look at OO.org, where it takes me a minimum of 10 hours to install OO.org from source.

High-level systems languages

[shmlco]

Actually, if the uber-geeks of the world had used a higher level language like Object Pascal or Modula for systems work, we wouldn't be in nearly the mess we're in now. Both languages used byte-length prefixed strings, and usually forced array bounds checking, which would have killed almost all of the current buffer-overflow errors and exploits in existence.

But no, we choose C, a cast-anything-to-anything-no-checks-whatsoever-poi nters-run-wild language PURELY for speed reasons, and ignored all of the other issues it created.

In today's 3GHz plus dual-core world, I'd happily give up, say, 5% worth of performance in exchange for a completely stable, secure system.

Re:High-level systems languages

[Thundersnatch]

In today's 3GHz plus dual-core world, I'd happily give up, say, 5% worth of performance in exchange for a completely stable, secure system.

The problem is, buffer overruns are not the only type of security vulnerability. There are actually a lot of good tools to find simple buffer overflows in C/C++ source code. But a lot of vulnerabilities are far more subtle than buffer overruns. Type-safe, bounds-checked languages are not "magically perfectly secure"; they are simply tools to prevent the programmer from having to worry about certain issues.

Buffer overruns are merely the low-hanging fruit in the vulnerability space, because of widespread poor C/C++ coding practices. But even with perfect input validation, there are still situations (like intra-application privilege escalation, or race conditions) that are not easy to find in code nor uncover in a beta test.

Re:High-level systems languages

[Nasarius]

In today's 3GHz plus dual-core world, I'd happily give up, say, 5% worth of performance in exchange for a completely stable, secure system.

Says someone who probably hasn't done much serious programming. Crashes and security problems are possible in any language. If you can write an infinite loop, you can have a bug that locks up your OS.

Re:High-level systems languages

[bani]

You are obviously not a programmer. Or at best, a novice one.

Buffer overflows are only one small class of exploit.

The latest xmlrpc attacks going around have absolutely nothing to do with buffer overflows; the fact they are even possible at all is due precisely to xmlrpc being "higher level language" than C.

In fact virtually no php attack is a buffer overflow. They are mostly script injection attacks enabled solely because php is such a high level language.

SQL/script injection attacks also have nothing to do with buffer overflows. In fact they are enabled precisely because they are high level languages.

To get a "completely stable, secure system" you'll have to give up far more than 5%. Try 50%-60%.

Re:High-level systems languages

[be-fan]

Obviously you're not a security guy or a compiler guy. The estimates show that 50% of security bugs are the result of some sort of memory-related hole. Also, array-range checking, with a compiler (native-code, not JIT) that does range-propagation and bounds check elimination, is about 3-5% on a modern superscaler processor.

Re:High-level systems languages

[bani]

You obviously have no clue what you're talking about. Or you simply can't read.

Bounds checking might be "cheap", but all the other checking you have to do in order to have a "completely stable, secure system" - as parent poster referred to - is very costly.

But please, do go ahead and build your "completely stable, secure system" by only using bounds checks. You are obviously so confident that bounds checking will save you from everything, you really deserve what you get ;)

Re:High-level systems languages

[shmlco]

Bugs are bugs. But bugs where some "serious" programer, who, in his need for speed and "efficiency", blithely strcpy'ed every argument in sight to a buffer whose size was "known" beforehand could have been entirely avoided.

Re:High-level systems languages

[shmlco]

Yep, hundreds of thousands of home computers have been compromised and turned into spam remailers due to SQL/script injection attacks.

Thanks for sharing.

Re:High-level systems languages

[bani]

moron.

Re:High-level systems languages

[Tyler Durden]

In that case the programmer used a strcpy when he should have used strncpy. In other words, the problem is with the programmer and not the programming language.

And that's the whole flippin' point. The same people who will make stupid mistakes writing system software in C will make mistakes in some higher-level language.

Re:High-level systems languages

[shmlco]

"And that's the whole flippin' point. The same people who will make stupid mistakes writing system software in C will make mistakes in some higher-level language."

Assuming it let's them do so. When a string assignment is s1 := s2; there's not much you can do to screw it up.

Re:High-level systems languages

[Tyler Durden]

When I was a CS undergrad, the language of choice to learn was Pascal. And let me tell you something, writing kernels in that language (especially in the state it was in then) would be a laughable proposition at best. To give it the flexibility required to create code made to run at such a low level would result in so many convoluted structures (or if I remember correctly from the time 'records') and function (and procedure) wrappers that it wouldn't be worth it. It would be a huge, unmaintainable mess of code because the amount of abstractions necessary to pound that square peg language into the round hole of system development would make it more confusing than necessary.

Better to use a language like C that's made for the task and depend on the programmer's own know-how to create something stable and maintainable than depend on a language to put said programmer in chains and force him to hack at the thing when they prevent him from doing something he needs. System development just requires more flexiblity than application development does; it's just the nature of the beast. It's something that you have to actually try for yourself to see how that is.

Look, those people who created Unix are revered for their choices of design to bring a combination of flexibility, efficiency, portability, maintainability, and yes, stability to OS development. Did it ever occur to you that perhaps they knew what they were doing when they chose/created a programming language to do the job? Maybe there's more to the fact that 99.9% of the people doing kernel development use C with maybe some C++ than a need to stroke their egos or use existing code?

Re:44 pages and the main question is still unanswe

[oever]

You just replace it with the new glibc, deleting the old version. The running programs will continue to use the old version. Even though it was deleted/replaced, the old version is still available to running programs using it. Only when the last program using the old version is deleted, will the last reference to the lib disappear and the file truly be deleted.

So as a previous comment mentioned, you will need to restart all applications using the affected library, if you are concerned that the security problem may affect them. In practice this means that you'll probably want to restart the server programs you're running.

Re:44 pages and the main question is still unanswe

[jargoone]

use a static tool (perhaps a static busybox binary) to make the new one.

Or "sln", which is intended for this very purpose. Interestingly enough, I learned about this right here on /. just a couple of months ago.

ZDNet must be very desperate...

[someone1234]

Hmm, Murphy is a known M$ shill, they must be very desperate to get clicks if he resorts to this kind of admission. Or there is some very dark reason i couldn't even fathom.

Re:ZDNet must be very desperate...

[someone1234]

Uhm, yeah. I read lots of his notes. One comes like this:

"Although most of the community disagrees with me on this, I think the basic SCO claim is a no brainer: strip away the absurdities perpretrated by lawyers using their client's money (and corporate goodwill) to go after their billion dollar contingency, and the basic claim is just that some IBM people allowed some contractually protected Unix system code to leak into the Linux development tree"
Translated: While SCO's claims might be overblown, basically they are right.

He also wrote that SCO isn't about to destroy Linux, but IBM is.

Re:44 pages and the main question is still unanswe

[Mancat]

The sad thins is that Singularity is probably more usable than Hurd is right now, after decades of Hurd development.

Re:What's the point of CreateProcess benchmarks?

[shmlco]

BS. I start MAYBE a dozen or so programs a day on my computer. So how significant is my difference? I would have saved what on Linux, a fraction of a half of a second?

Perhaps if you're running a server AND using some stone-age CGI executable that's called millions of times the differences may start to pile up... but probably not. Most such things on Windows are created as services that are always running, don't need process creation to do their jobs, and use threads that are faster than Linux versions.

"Significant difference" my a**.

Re:44 pages and the main question is still unanswe

[NickFortune]

Same difference... you've still shutdown all your network services which to the users means you've had downtime. It's a reboot in all but name.

mmm... I can see that in a few specific cases, like if you have a lot of users who log on over ssh. Less so for webservers and remote filesystems where you bounce the runlevels fast enough, the interruption will probably never be noticed.

Of course, the context where the Curse Of A Thousand Reboots really bites is for the home computer. I mean, I only have one user on this machine. Rarely I'll have two, never any more than that. So if I cycle runlevels, no-one is going to be put out bar me - and I'm the one doing it.

In General, I find that the people inconvienced by a compulsory reboot are not networked users.

Of course, even if you have remote users, your downtime is going to be a lot less if you don't have to go through POST, bios initialisation, device scanning and all the rest of it. And of course you only have to do it once, becaue you're controlling the process, so you don't get fifteen reboots in a row because windows brute forces everything.

So, I think "all but name" is overstating the case. By rather a lot, actually.

Re:44 pages and the main question is still unanswe

[man_of_mr_e]

This kind of selective security is a serious problem in the Linux/Unix world. The assumption is that "Oh, that service isn't exposed to the internet, so I don't need to worry about it. I don't give anyone shell access".

This ignores the possibility of exposure by a different (non-root) vulnerability.

Suppose you decide not to update your kernel for a local root vulnerability. Now, suppose a new vulnerability is discovered in an internet facing daemon like Apache or SSL. Even though those services may run as an unpriviledged user, a vulnerability in those services could allow an attacker to utilize the local root vulnerability in the kernel that you neglected to patch (becuase you thought it didn't matter) to gain root access.

Simply put, there's no such thing as a security patch that doesn't apply (other than for software that isn't installed). You're fooling yourself if you think otherwise.

Probably doesn't work on current implementations..

[hackwrench]

You start a new process linked to the new glibc for every old process then pass all the data the old processes contain to the new ones.

Re:44 pages and the main question is still unanswe

[edwdig]

Typical distros that support pervasive no-reboot updating (like Debian) don't exactly replace a "running" libc (or any other library), they simply update the on-disk copy. So any programs run after that will get the new libc, but any programs that were started before the update will of course be using the old libc.

I remember upgrading from the last libc5 Debian to the first libc6 one. The output of "who" and related commands was messed up until I rebooted because the file that contains that info had a different format between libc5 and libc6. It got messed up a bit from having both running. In this particular case it wasn't something that really mattered, but it certainly shows the potential for problems.

Re:44 pages and the main question is still unanswe

[kimvette]

No kidding, yet I get marked a troll for making a FUNNY-yet-true remark, and your parents gets modded +2? WTF!

Re:44 pages and the main question is still unanswe

[lubricated]

> Simply put, there's no such thing as a security patch that doesn't apply (other than for software that isn't installed).

Yeah, you should apply all security patches. However after updating libc you really don't need a reboot. Restarting all relevant processes is sufficient. (e.g. if it's an exploit in how libc creates temp files, you don't need to restart a process that doesn't create temp files), stupid example I know, but that is what I meant.

Re:44 pages and the main question is still unanswe

[WMD_88]

The question was, "How long will it take for the free software community to come up with something like this?" Judging by your response, the answer is "16 years with no 1.0 release in sight."

Which really disappoints me, because I think the HURD would be a great system to have around if it got enough developer attention to actually be released.

IPC

[fossa]

As a young and naive wannabe programmer, I have a question about IPC: From what little I understand of fork() and shared memory, it only allows the sharing of continuous blocks of memory. Is this correct? In my programs, I typically use threads and a glib GAsyncQueue to push pointers to memory structures between threads. For me this works rather well, because I essentially keep the threads completely separate (like processes), but I can still share a complex memory structure full of pointers. How are things like this accomplished in fork()/IPC land? Thanks.

Re:IPC

[vidarh]

The short answer:

man ipc
man socket
man mmap
man pipe

The long answer:

That depends entirely on the characteristics you need. Shared memory is typically in large contiguous chunks, yes, but that doesn't mean you can't implement a shared memory queue - you just need to allocate the memory for the individual messages from the shared memory.

However what you'll typically see is that heavily message oriented programs will either use System V IPC, or serialize it's messages through pipes or sockets. The advantage of using sockets/pipes is that it's easy to switch the application from being multi process on the same machine to splitting it out over several machines, which can provide a significant scalability advantage. It also eases synchronisation and safety (no chance of the processes failing to acquire locks or clobbering parts of shared memory).

mmap() or system V shared memory are typically used mainly when processes need to work on large amounts of shared data.

The various methods are often combined - you'll find apps using mmap() for shared memory but uses pipes to synchronise access to it, for instance.

Personally I see using threads as a premature optimization - I've yet to come across a case where the overhead of process oriented IPC was sufficient to warrant switching to use threading instead of multiple process (of course they exist - they're just exceedingly rare in the types of fields I've worked in). In a well structured application the switch, if needed, should involve only minor changes, and by using a multi process approach from the start you end up being forced to structure things in a way that minimize communication which will only benefit you if you switch to threads too.

Re:44 pages and the main question is still unanswe

[man_of_mr_e]

Most people are really not going to know which processes are affected by a given fix. Most people may not even know WHY there's a new update to a file on their apt/you/portage/whatever. They're just going to apply it.

The prudent thing to do, unless you want to spend a ton of time researching the fixes, and what they effect, and what programs are using those features, is simply to reboot.

So, yeah, you don't HAVE to reboot, but I always will. I've got better things to do with my time.

Re:44 pages and the main question is still unanswe

[xcomm]

>>Stability depends partly on systems are [NOT] writen in the ancient language C >>:), and run monolithic kernels.

This is truth following the theory, but as with most things on the planet, in real life it seems to be such the opposite.

Re:What's the point of CreateProcess benchmarks?

[pthisis]

Processes in Unix are lightweight objects, and the OS spawns them left and right. Processes in Windows are heavyweight objects, and the OS creates only a handfull of them. The lightweight objects in windows are threads, and you'll notice that Windows thread creation is faster than Unix thread creation. These are just different OS design philosophies.

No, they're not just different OS design philosophies. Windows forces you to choose between reasonably well-performing concurrency and memory protection. The poor performance of Windows processes is a root cause of many security holes and stability issues. OS designers spent years working on memory protection so that you could have reasonable isolation in the system and not have a bug in one area affect the whole system. When you're forced to use threads to get good concurrency, you effectively throw out a lot of that work and

Re:44 pages and the main question is still unanswe

[fuzzix]

Death to all Fanatics!

Seen this?

Off topic: glibc updating.

[Nevyn]

Typical distros that support pervasive no-reboot updating (like Debian) don't exactly replace a "running" libc (or any other library), they simply update the on-disk copy.
...
Usually this works very well

Actually ... not so much. nss is the biggest problem, but a bunch of glibc interfaces aren't guaranteed to work across updates like this, including anything using iconv and locales. Pretty much all distros. that "support" large updates without a reboot just ignore this problem in the general case (debian has hacks that help, if all the stars align).

But then a similar problem is the real ABI of glibc changing, and programs going from working to not working because of it (this is against the theoretical ABI that "is compatible") ... all of the distros. also ignore this problem.

Re:Off topic: glibc updating.

[Arandir]

That's because GNU really has no clue about compatibility. Take a look at any non-GNU libc, and you won't have that problem.

That explains...

[Syberghost]

...why their FTP sites were on BSD for so many years. It still remains to be proven that they actually switched them to NT, not just set up their BSD servers to act like NT.

Be afraid

[nurb432]

Something smells fishy here. Im sure there is an angle they are working on that we dont know yet.

Microsoft admits nothing. Why start now?

This is not true

[RedLaggedTeut]

Actually results are mixed; and they even seem to indicate that Linux seems to excel at nothing but quick process startup (which is cool if you do lots of shell scripting, maybe compiling too)

According to the benchmarks published there

    - at most OS jobs like threading/process creation, Singularity is at least twice as fast as linux, Linux is very fast at process creation, while XP is good at threads

    - in File Operations FreeBSD and Linux beat XP and Singularity at random reads

    - in File Operations XP beats Linux and Singularity at sequential reads, with the exception of FreeBSD being fastest if blocksize is high(and very bad for small blocksize)

    - linux executable sizes are larger than these of the other OSes, (whatever that means, more good coding, or less bad code SCNR)

Please bear in mind that a benchmark does not tell whether the "slower" OS actually invested more time in doing some smart stuff that pays off in some other way. In particular, I would not be surprised if an experimental OS like Singularity did less.

partial repost from http://slashdot.org/comments.pl?sid=167223&cid=139 45599

Re:This is not true

[wprowe]

linux executable sizes are larger than these of the other OSes, (whatever that means, more good coding, or less bad code SCNR)

I imagine this is because Windows includes everything but the kitchen sink in DLLs in the kernel whereas *nix variants do not. I bet if you include the size of all the DLLs required to actually make a Windows application run, it is as large or larger than a *nix application. This probably also explains why it takes more than 5 times the number of kernel cycles to start a process in Windows as it does on *nix variants.

Re:44 pages and the main question is still unanswe

[abradsn]

Not all windows updates require a reboot. Why does everyone think that they do? Maybe they all still run windows 98?

Re:44 pages and the main question is still unanswe

[misleb]

I don't know if they "require" a reboot, but most updates ask to reboot. I run Win2k mostly.

-matthew

Creating and starting a process?

[Anonymous Coward]

...For example, they show the number of CPU cycles needed to "create and start a process" as 1,032,000 for FreeBSD, 719,000 for Linux, and 5,376,000 for Windows/XP.

That's about as useful a metric as total elapsed urination time.

What's the real delta?

Lab Technology transfer

[steve_l]

As someone who works at a research lab (HP's) I dont want to pick on MS. Technology transfer is a problem for all R&D labs. The bits of the company that build things are in short-term firefighting mode and with senior management changing direction, it is hard to guess what the long term areas to research are.

Some of the stuff that MSR have done have been kind of interesting, in a CS-hard way. But that doesnt mean relevant. The other thing is that they've hired some great names, but that doesnt map into great stuff. It works in universities, because those names supervise the PhD students that do the real work...

Returning to tech transfer, the problem that MSR has is that their main pipeline for apps has been MS products, which are invariably behind schedule. And when things are behind schedule what happens to the leading edge stuff? Yes, it gets dropped. Whereas if the people in google-labs are doing research, they can try it out on a server or two in another 'beta program': the pipeline from research to product is shorter.

For me, OSS has transformed our pipeline. We can improve products without having to go through the normal inertia of the product process because if the patch is good, it goes in. Indeed, the core runtime of the I work on -deployment- is actually up online Smartfrog. This means that we can get stuff out there being used within days of being coded, and evolve it in the field. OSS and corporate R&D labs go hand in hand. This is somewhere where MSR are at a disadvantage.

Re:Wow is this ever misleading - yes you are

[molo]

Yes, what you are saying is misleading.

Linux ALSO beat Windows on: ABI call (437 vs 627) and 2-message ping-pong (5797 vs 6244). Its right there in your post. Read it.

-molo

Re:44 pages and the main question is still unanswe

[photon317]

Mostly he's referring to minor libc updates that happen regularly. libc5 -> libc6 was a massive change that in some cases involved a lot more than running an updater and rebooting for some distros. libc5 -> libc6 was one of those rare few and far between changes where everything gets broken. There will always be major marks in time like that.

could be...

[emseabrown]

but it is equally likely that he saw this comment over week ago,
http://slashdot.org/comments.pl?sid=167223&thresho ld=3&commentsort=0&tid=190&tid=109&mode=nested&cid =13944438

Re:44 pages and the main question is still unanswe

[no reason to be here]

It will be interesting to see how long it will take the free software world to come up with something similar

Well, Hurd is a microkernel, so OSS already has that covered...the Mach kernel in OSX is opensource and a microkernel. Is it written in Objective C (which would be strongly typed, right?) or is it just the GUI that is ObjC?

The BeOS had a microkernel, written in a mix of C and C++; the Haiku project is rewriting it from scratch in C++, which can be strongly typed, if you are willing to put in the effort; I'm hoping that they are.

Seems like the OSS world is ahead of the curve in a lot of ways.

Win32 vs Posix processes

[Foolhardy]

First of all, the Windows processes created in this example are Win32 processes, which have a lot more baggage than the posixy processes that FreeBSD, Linux and Windows's Posix subsystem use. I'd like to see added to that list the number of cycles to create a native Windows process and a SFU process (they're going to be a lot shorter), and also a WINE process under Linux.

Some of the things that Win32 processes do that SFU and native processes don't:

• The Application Compatibility Database, a user mode service has to be contacted to see if the new program needs to have any compatibility shims added. Half of the compatibility that XP has comes from modifying programs as they start, or giving them special treatment. This stage alone causes so much overhead that Windows Server 2003 has a special group policy that lets you turn it off to make starting processes faster.
• The Software Restriction Policies database, a set of registry entries that have allow/deny rules for starting processes based on hash, filename or certificate. To make any actual comparisons, the entire binary has to be hashed and checked for certificates before the program even starts.
• Registering with the Win32 subsystem server (csrss). This involves several out-of-process function calls.
• Load the current locale, including NLS files.
• If enabled, contact the Themes service.

Except for talking to the Themes service, all those steps are done for every new Win32 process, even if it doesn't have a GUI.

Repost

[sstidman]

I believe this topic was already covered on SlashDot recently.

Re:Repost

[sstidman]

And not to be an attention whore, but I did a partial summary of the interesting parts of the M$ document, if that's helpful to anyone.

Re:44 pages and the main question is still unanswe

[Shotgun]

Insightful? How about "apologetic bull"?

My job is in QA. Your statement says that my job is impossible. Here are a few ways you can test stability:

1) See if the OS comes back online after a power cycle
2) Insert and remove device drivers
3) Send mangled data across the various data busses
4) Run programs that try to allocate all the memory
5) Run programs that try to hog all the CPU
6) Run a program that fills the hardisk/erases the hardisk/refills the hardisk
7) Do all the above all at the same time

The OS is a control point, and should be able to handle all of this and more GRACEFULLY.

(Gracefully being an undefined weasle word indicating that it should fail in a somewhat predictable manner that won't completely piss off the vast majority of administrators.)

Re:44 pages and the main question is still unanswe

[hunterx11]

There is a difference between an OS still going under revolutionary updates, and one only undergoing evolutionary updates. Consider that Mac OS X didn't even finalize on launchd or a kernel API until Tiger.

Re:44 pages and the main question is still unanswe

[muixA]

Perhaps everyone knows this, but...

The reason you can replace something like libc on a running Unix system, is the result of the way the Unix FS works. If you open a file, in this case libc, the kernel sets a reference count on that inode. If you subsequently unlink() (delete) that file, the kernel doesn't actually remove it until the reference count goes to 0. This means already running processes will be unaffected by this change, while new opens would fail.

In the case of a libc upgrade, one unlinks the old file, and replaces it with the new one. New apps start and link against the new libcxx.so. Old apps work as expected.

Windows doesn't work this way, at least not what i've seen
--
Mu

Re:What's the point of CreateProcess benchmarks?

[ichin4]

I didn't mean to say that there aren't some negative consequences to the choice of making threads performant and processes less so. There are, and your post correctly identifies one of them. But I think it's wrong to say that that design decision is therefore across-the-board wrong. There are times when a threaded architecture is appropriate, and that is harder to do in Unix (which is why Apache 1.x spawns processes, even though the 2.x codebase shows that threads are a better idea).

Re:What's the point of CreateProcess benchmarks?

[ichin4]

That's not quite right. It is true that Linux provides a very innovative and interesting forking model that allows you to say at fork-time which resouces are shared between a child and parent and which are not. It's true that you use this make the fork a "thread" or a "process", depending on which resources you choose to share. (And the theads of apps that use this look, confusingly, like a bunch of processes with the same PID and working set when you run ps.) But that doesn't mean that Linux beats the pants of Window's thread creation perf, because Windows creates threads faster than Linux does forks.

T(Windows thread) < T(Linux thread) ~ T(Linux process) < T(Windows process)

Apt name?

[drbhoneydew]

One has to wonder whether any stability issues are caused by divide by zero errors.

Re:What's the point of CreateProcess benchmarks?

[pthisis]

I didn't mean to say that there aren't some negative consequences to the choice of making threads performant and processes less so. There are, and your post correctly identifies one of them. But I think it's wrong to say that that design decision is therefore across-the-board wrong.

There are 2 seperate issues here
1. Are threads faster than process? Yes, on both Unix and Windows.
2. Are process so slow as to be essentially unusable for concurrency? On Windows, yes for a relatively large problem domain.

(2) is wrong. You force the programmer to give up memory protection in order to use an unrelated feature (concurrency).

There are times when a threaded architecture is appropriate, and that is harder to do in Unix (which is why Apache 1.x spawns processes, even though the 2.x codebase shows that threads are a better idea).

No, Apache used processes because they were a better idea. That's why most sites running 2.x on Unix continue to run it in multi-process mode even though multithreaded mode is available.

Not only does the multiprocess model avoid synchronization issues (see, e.g., http://www.zeuscat.com/andrew/work/aprbench/ for benchmarks showing that pre-fork is faster) but it avoids all the security/stability issues that come along with multithreading.

Would you rather have an obscure bug in some module you're using kill the current connection (and if it's repeatable, make that one page unloadable) or take down the whole web server? Do you want to limit the scope of security breaches?

Now, thread switches on their own are faster than process switches. With careful crafting you could design a threaded server that avoids synchronization issues and have it be faster than the multiproc version by that margin--but that's a very tiny margin that will likely be lost in the noise in any real-world server, and you're talking about increasing development cost dramatically for a return that's marginal at best. You'd be far better served putting that development effort into a state-machine based solution like thttpd or phhttpd if that level of performance is a concern.

That said, threads do have their place. Some problems are best solved with a shared-memory solution where the memory you need to share can't be easily isolated into a few SHM segments. And when that's the case, threads are the right tool for the job.

But the point is that if you don't have a reasonably performant process implementation then you remove the more commonly useful tool and force programmers to accept one feature (memory sharing) that could be very bad in order to use the one they really want (concurrency)--when those two features are really unrelated.

This problem isn't limited to Windows, either; Java similarly suffers from having no good way for programmers to use a multiprocess design.

It is, IMO, the single biggest technical problem with Windows.

Re:44 pages and the main question is still unanswe

[xTantrum]

1) i'd like to see how MINIX compares 2) since MINIX is under a BSD style liscence who is to say M$ didn't steal code or concepts from them, not like they haven't before. HOTMAIL, DOS etc..

I was hoping nobody would say anything...

[divisionbyzero]

about the performance numbers. Those numbers are going disappear as soon as some marketing monkey from Microsoft sees this article. Now Microsoft will never release real numbers again. No good deed goes unpunished, I suppose.

Re:44 pages and the main question is still unanswe

[AaronGTurner]

If the current Linux kernel is outperforming Windows' kernel (the dominant desktop OS) then there is little incentive for people to swap (or be swapped, if you are an IBM, SGI, Red Hat, etc employee) from Linux kernel development to HURD development. At the point when a rival looks to be close then it is entirely possible that those companies will invest in a switch to HURD or something else. Mind you, the Linux kernel at 2.2 wasn't a great choice for interactive multimedia due to preemption issues, but that is much better.

But how...?

[Makarakalax]

Fraankly, as a Linux and Windows GUI programmer, I can't see how you can do a lot of mulit-threaded GUI work with separate processes instead of threads. Maybe I'm just too n00B.

Re:But how...?

[pthisis]

Threads certainly have a place, I never said otherwise. The problem is that the Windows system forces you to accept shared memory to get concurrency, and those two are unrelated. The number of problems that want concurrency and memory protection is large, and eliminating that option is a MAJOR problem.

Having done a fair amount of GUI programming myself, I find a multiprocess solution is often correct (e.g. in something like Photoshop image filters, where you want shared access to one memory segment but don't need to share huge numbers of different memory segments where you can't easily compartmentalize them). For some jobs, though, multiple threads is better. Use the right tool for the job.

Re:44 pages and the main question is still unanswe

[Quantam]

I've made a couple of posts on my blog listing the technical pros and cons of Singularity as a real-life OS (the latter I just wrote and put up today).

Re:44 pages and the main question is still unanswe

[jsight]

In the (non-Windows) server world, rebooting is a big no-no.

I used to think this was true... and then I ran into real world UNIX sites. It's amazing how many reboot once per week as a matter of policy!

Re:44 pages and the main question is still unanswe

[illtud]

Same difference... you've still shutdown all your network services which to the users means you've had downtime. It's a reboot in all but name.

mmm... I can see that in a few specific cases, like if you have a lot of users who log on over ssh.

ssh sessions survive a linux network reset, unless you've some seriously funky network hardware. Try it!

Re:44 pages and the main question is still unanswe

[man_of_mr_e]

I didn't say your job was impossible, but your job is only testing a small subset of reliability. Reliability is also whether or not the OS stays up for a year at a time, or whether it has long term memory leaks. Reliability also has to do with weird race conditions that only show up after several programs interact for a significant amount of time, etc...

What you're testing is simple stuff, stuff that's easy to identify. There's a whole other class of reliability testing that's far more long term.

I did it, by hand even.

[r00t]

1. install the new libc with *.new names (both file and symlink)
2. optionally, hard link the old libc files and symlinks to *.old names
3. move the new libc to the proper names (unlike "cp", "mv" is atomic)
4. restart most background tasks via "kill -HUP the-pid-number"
5. restart init via the telinit program
6. restart your shell via "exec -l /bin/bash"
7. use fuser or lsof to find anything still using the old libc, restarting as needed

Re:44 pages and the main question is still unanswe

[udippel]

Ah, yeah ! The best comment in here I have seen so far.

Noticed the same. No Andrew T. in the references.
If they did their job *somewhat* properly, they'd have to refer to Minix. And I am sure they did; they still have to be politically correct; just look at the common roots.
Could Microsoft confess that they screwed up years back with their go-on-your-own DOS ? Not economically; that made kind of sense on an 8086; but from a visionary aspect ? Except of new blood from DEC they dug themselves into program loaders and their current architecture (I haven't seen the source code, sorry, based on hearsay) is only rudimentary more intelligent.

Thanks for pointing us to Minix and an evident omission in the MS report !

Re:44 pages and the main question is still unanswe

[heinousjay]

I have to know how your examples (which, for your edification, are two things Microsoft bought) are examples of 'stealing' BSD licensed code.

Incidentally, it is my understanding that the BSD license means that covered code is freely available, and is, as such, unstealable.

Or have I been trolled? Probably. Still, with the general knowledge level of slashdot being what it is, you can never tell...

Process concurrency is hardly the panacea

[kylef]

As a result, you get tons of unstable Windows applications because to get any reasonable concurrency you have to throw out the years of hard work that OS designers put into having protected memory.

Throw out years of hard work? Give me a break! It almost seems that you are blaming the poor quality of modern threaded applications on Windows! That's rich!

Concurrency is difficult to use correctly no matter what technology you use. Inter-process shared/mapped memory is just as susceptible to race conditions as cross-thread shared memory, and inter-process synchronization logic can deadlock just as easily as thread synchronization logic. And the results are the same: once a process is deadlocked, or corrupts its data due to a race condition, what difference does it make if it's running in its own address space? The software has failed catastrophically either way!

We are ALL well aware that poorly written multi-threaded software is unreliable and that threads can easily trash other threads' data if not written correctly. And yet, for performance-critical applications, programmers still prefer to use threads. Why? It's simple: Because, for MANY applications, the benefits in performance outweigh the risks.

Finally, I'd like to point out one more thing. You claim that to get "reasonable concurrency" on Windows you are FORCED to use threads. I completely disagree. While process startup latency is relatively high on Windows, Windows offers a rich set of interprocess communication mechanisms, and context switching is quite fast. And if your program is so performance-critical that process startup latency is your biggest bottleneck, then switching to thread synchronization seems perfectly reasonable.

Re:Process concurrency is hardly the panacea

[pthisis]

As a result, you get tons of unstable Windows applications because to get any reasonable concurrency you have to throw out the years of hard work that OS designers put into having protected memory.
Throw out years of hard work? Give me a break! It almost seems that you are blaming the poor quality of modern threaded applications on Windows! That's rich!

No, I'm blaming Windows for the prevalence of multithreaded apps that don't need to be multithreaded. And yes, using threads throws out memory protection within the application; to an extent, it's like old unprotected Mac apps grovelling around in system memory. It requires a lot of discipline, and bugs are more able to bring the whole system down--and it's a lot less secure. That said, it's still sometimes the best option. But you shouldn't be forced into it.

Concurrency is difficult to use correctly no matter what technology you use.

Absolutely. Why compound that by throwing out shared memory as well?

Inter-process shared/mapped memory is just as susceptible to race conditions as cross-thread shared memory, and inter-process synchronization logic can deadlock just as easily as thread synchronization logic. And the results are the same: once a process is deadlocked, or corrupts its data due to a race condition, what difference does it make if it's running in its own address space? The software has failed catastrophically either way!

Example:
You have a helper function that does something asynchronously in your web server. Your helper segfaults when trying to read a file that's too large. Would you rather have the web server go down completely, or just have connections that try to load large files get closed?

In both cases the software has failed, but one is far more catastrophic. Obviously if you can write bug-free code then either way will work--one of the major reasons for having memory protection is that code is _not_ bug free, and when it fails you'd like to isolate the damage.

What's more, with an IPC solution you know explicitly what you're sharing; with a multithreaded solution it's a lot easier to not realize when another thread is accessing the same memory as you. It's a lot easier to introduce bugs that way, because you're sharing everything and not just the things you need to share.

Threads are certainly right for a lot of tasks, but that choice ought to be dictated by whether you need shared memory or not.

And it's not just a Windows problem. Java's worse in a lot of ways; at least, as you point out, there are ways to work around the problem in Windows. Java doesn't have any real multiprocess solution short of native extensions.

Re:Process concurrency is hardly the panacea

[Quantam]

Are you familiar with Windows Structured Exception Handling (SEH)? While I have learned some about signals in an educational context, I have nowhere near the experience with error handling on Unix as I do on Windows (and thus don't know if Unix has anything similar to SEH). SEH is, more or less literally, C++ exceptions that are thrown for things like access violations (the kind of stuff that would generate serious signals). You can catch these exceptions in catch blocks in the thread that caused the violation (or rethrow it if you can't deal with it, and let a higher up handler catch it; stack classes can also be destructed as an exception gets propogated upward), and actually recover from the error. This method is quite slow (can take hundreds of thousands of cycles for a catch block to get executed after an exception is thrown), but that's incomparably superior to having the thread (or the process) die due to some access violation. I should also add that the overhead of SEH blocks (when exceptions are not thrown) is very low - only 1-2 dozen cycles per function that has a catch block. This is a far more elegant solution to exception handling than anything I know of on Unix (but again, I could be unaware of something on Unix).

Re:Process concurrency is hardly the panacea

[pthisis]

In Unix/C you can install a signal handler to catch SIGSEGV. I don't do C++ so I don't know if there's an exception typically defined for that there.

Re:44 pages and the main question is still unanswe

[(negative video)]

Even on modern systems, you need to enable DMA for these sorts of devices to get acceptable performance, and the Singularity model seems like it will preclude this.

In theory at least, an IOMMU can protect system memory from DMA-capable peripherals under the control of hostile code. I don't know if any existing machines have a good enough IOMMU, though.

every Win32 process gets GUI crap at start-up

[r00t]

I know this is going to be hard to believe, but...

UNIX creates a process with fork, which takes no arguments. UNIX runs a new executable with execve, which takes 3 arguments. So in just two system calls with 3 arguments, you launch an app.

Windows has a CreateProcess() function with 10 arguments, many of which are pointers to structs. I call your attention to the absurd "LPSTARTUPINFO lpStartupInfo " argument, which supplies info about the windows style and current desktop.

Re:44 pages and the main question is still unanswe

[Arandir]

Maybe you can hold off the reboot until after midnight? It's really annoying when admins reboot the servers in the middle of the day when I'm trying to get some productive work done.

Re:What's the point of CreateProcess benchmarks?

[flosofl]

BS. I start MAYBE a dozen or so programs a day on my computer. So how significant is my difference? I would have saved what on Linux, a fraction of a half of a second? Perhaps if you're running a server AND using some stone-age CGI executable that's called millions of times the differences may start to pile up... but probably not. Most such things on Windows are created as services that are always running, don't need process creation to do their jobs, and use threads that are faster than Linux versions. Ahh... And I was wondering when the clueless would start to chime in. Your workstation != Real world servers under high load

Re:What's the point of CreateProcess benchmarks?

[flosofl]

Man that's annoying... let's try that again :)

BS. I start MAYBE a dozen or so programs a day on my computer. So how significant is my difference? I would have saved what on Linux, a fraction of a half of a second?

Perhaps if you're running a server AND using some stone-age CGI executable that's called millions of times the differences may start to pile up... but probably not. Most such things on Windows are created as services that are always running, don't need process creation to do their jobs, and use threads that are faster than Linux versions.

Ahh... And I was wondering when the clueless would start to chime in.

Your workstation != Real world servers under high load

Read some of the above comments to see exactly why using threads are not the optimal solution for high availability server applications.

Not to mention...

[chaves]

... Brain and Pink... actually, I think they started this trend. Amazing what big corporations can learn from highly intellectually developed mice.

Re:44 pages and the main question is still unanswe

[niteice]

The kernel (XNU, a combination of Mach and BSD) is almost entirely C, drivers (I/O Kit) are in a subset of C++, userland is in C, high-level userland is in Objective-C++ (combination of Objective-C and C++, there's bound to be some plain old C for good measure).

Re:44 pages and the main question is still unanswe

[amRadioHed]

It's pretty much just the Cocoa software written in ObjC. The Mach kernel was developed independant of NeXTSTEP or OS X, so it would be very surprising to see it written in ObjC. I'm pretty sure it is written in plain c.

Re:44 pages and the main question is still unanswe

[drsmithy]

In the (non-Windows) server world, rebooting is a big no-no.

There are several reasons why regular (though not necessarily frequent) reboots are a good idea. Not least to make sure the machine will actually restart into a usable (and accessible) state.

If you have a 24/7 environment, and a proper architecture to support that requirement, individual machine restarts are irrelevant.

If you don't have a 24/7 environment, machine restarts can be scheduled outside of production hours - daily if need be - again making them irrelevant.

If you're a home user, chances are extremely high your PC is turned off at least once a week, if not every night. Even if it isn't, rebooting occasionally is hardly a problem.

Reboots only really matter to two types of people - a) those whose ego depends on the number returned by uptime(1) and b) those who are trying to run a 24/7 environment where they shouldn't be.

Re:44 pages and the main question is still unanswe

[drsmithy]

If that is a good example of what a microkernel can do in the real world...

OS X isn't a microkernel. It's the same sort of "microkernel-based" thing that Windows NT is. Which is to say it's probably architecturally designed mostly like a microkernel, but since everything runs in kernel space there's no stability benefit.

Re:44 pages and the main question is still unanswe

[misleb]

The only people i've ever known to reboot servers regularly were just really bad sys admins who didn't have the time or skill to deal with persistent problems. Rather than deal with the configuration problems or software bugs, they simply choose to reboot regularly.

There is a third type of person to whom rebooting matters: one who actually cares about resolving problems If you're running a cluster of machines, maybe it isn't worth it to resolve aproblem with an individual node, but if you rely on just one or two servers performing a function, it is usually worthwhile in the long run to solve theproblem. If nothing else, going for long uptimes is a good way to test the over all health of the system. If you reboot regularly, you might not be aware of things like memory leaks which could come and bite you in the ass between reboots.

-matthew

Re:What's the point of CreateProcess benchmarks?

[MotoMan045yahoo.com]

You still haven't awnsered the question, why are you implying you have to start a process for each request when creating services applicaitons? IIS is a process, that starts once, and has different proction models for incoming requests request. Is it really that much an advantage for processes to start faster on Linix, then they do on NT?

Windows is faster in Ubuntu

[Ticklemonster]

I am a recent convert to Ubuntu, but I do Unreal Tournament mapping which can't be done in Ubuntu, so I had a dilemma. I was about to learn how to dual boot when I found out about VMware player, and setting up a virtual machine to run XP. I set it up, and honestly, XP runs faster this way than it ever did on a regular install. No, I can't install stuff like video drivers I need, but the drivers that install with XP work well enough to run the unreal editor. I wonder if someone could test XP in VMware in Ubuntu against XP on a hard drive and see what kind of difference there is. I sure seems like XP is way faster than it ever was.

Shame to have to set up like this just to run unreal editor, though. Oh, for you gamers out there, UT runs so much smoother and faster in Ubuntu, it's not funny. UT2k4 (has linux installer on the 1st cd) runs way better in Ubuntu also. You might want to check it out if you have a spare hard drive you can play around with.

Re:44 pages and the main question is still unanswe

[Quantam]

Systems like that are usually fairly stable, but quite slow, as the overhead of interprocess communication is high. If the information I've received is accurate (I don't have any first-hand knowledge on the matter) OS X is structured kinda like that, and that's part of the reason it's sluggish (at least it some aspects). Singularity takes a radically different (and perhaps misguided) approach to stability.

Re:44 pages and the main question is still unanswe

[Quantam]

Oops. There was a mistake in the hyperlink in that post (making it not show up). Should have been 'misguided'

Hey, MS research is VERY useful

[BerntB]

So all that great research is just useless in the end then?

No, no. It is like giving lots of money for AIDS research. Good PR is never useless!

Never wondered about all dictators patting children's heads?

A response

[speed_of_light]

In response to tests confirming that XP ran more slowly than open source alternatives, Bill Gates was quoted as saying: "Hmm, maybe we shouldn't charge anything for our OS. . . HA HA HA HA!!!"

How is file system performance these days?

[BerntB]

How are Msoft on the file system performance, these days?

Have they gotten better since last I looked?

remember Ada?

[NuShrike]

Remember Ada? Almost every government computer system was running that because it was a safe and very strongly-typed language, that was easy to check correctness. I think ground-based systems to jets ran stuff written in Ada.

What are they running today? Stuff written in C/C++ because it's just more practical.

Re:remember Ada?

[Vintermann]

Ada is still in use in safety-critical systems. If you want a really safety-critical real time system, like fighter aircraft or space code, you use Ada, or more likely one of the subsets of Ada (SPARK, Ravenscar). No one right in his head uses C++ to control the cooling systems of a nuclear reactor, say.
By the time you make a subset of C/C++ small enough to be safe in these kinds of environments, it's more of a PITA to use than Ada anyway.
But you are right that C (and to a lesser degree C++) is used more: this is because it's really hard to get enough qualified Ada programmers. Also, although Ada is designed with low-level bit manipulation in mind, extremely few people have experience with that compared to the number of people who have experience with that sort of thing in C. For the contractor, it's just easier to find one of the top 0.1% C device driver programmers.

Re:What's the point of CreateProcess benchmarks?

[shmlco]

Huh. One of the people I do work for has a IIS / ColdFusion MX / SQL Server installation that's doing about five million page views a day on a CMS-based dynamic site. Excuse me while I call the CTO up and tell him his system isn't "real world", and how his load-balanced clusters aren't "high availability". I'm sure he'll be interested in your views and experience.

Re:44 pages and the main question is still unanswe

[mw]

At least on Linux KEXEC might be the answer to this problem. Booting a kernel without really doing a real boot.

Re:44 pages and the main question is still unanswe

[NickFortune]

ssh sessions survive a linux network reset, unless you've some seriously funky network hardware.

Coo. I've seen ssh session cope with me restarting local network connections, but I didn't think they'd survive cycling the remote sshd process. I suppose it might depend on what runlevels you have sshd running in.

Like you say, I'll have to try it and see...

Re:44 pages and the main question is still unanswe

[illtud]

Coo. I've seen ssh session cope with me restarting local network connections, but I didn't think they'd survive cycling the remote sshd process. I suppose it might depend on what runlevels you have sshd running in.

Don't listen to me - that'll teach me to post late at night. I missed the context of an runlevel change - they'll survive restarting the network service, but I'm unsure about ssh... let me try...

Oh yes, it does work. I can init 3 & init 5 and my ssh session doesn't blink.

Re:44 pages and the main question is still unanswe

[cyphercell]

It's simple really, stability comes from donning Window's brand new chainmail +5 vs. malware. http://www.microsoft.com/athome/security/default.m spx?Ad=SAH.Abby.00044&id=A/

Re:44 pages and the main question is still unanswe

[DavidTC]

I was being conservative because I just knew people would pop up and say 'My server reboots in two minutes!'. ;)

Re:44 pages and the main question is still unanswe

[WhiplashII]

Ah yes, but can you log in and drop all iptables input chains (with a default deny rule)?

How many of us have done that remotely? (2000 miles away?)