"other things OSS as a class could stand to do better"
You mention documentation a lot. The only to do that right for the end user is to have expert users write the documentation, not the developers. The end users are usually the lazy bunch of an OSS projects, hence the end-user documentation sucks. To 'fix' that, the expert users of the software of the project need to chip in and make documentation. If there are many expert users, and no small group is willing/able to do the whole thing, then a Wiki can be of tremendous use.
Similar with testing and release. OSS developers don't usually take much time to do that, because they are busy developing (whining about that won't change that, this is OSS and the developers are volunteers). Especially because of the usability and end-user experience arguments, testing and release is more an expert-user task than a developer task.
I made that statement about the documentation, because for the developers a tremendously large portion of the documentation load is carries by the 'Source' in 'Open Source', or as more commonly said: UTSL... "Use the Source Luke!". I've even looked at things like the libc source to find out exactly what is what, where, and why. Documentation doesn't get more precise and up-to-date than that. UTSL works, if you speak the language of a developer.
"They report less than a 5% slowdown on x86 kernels, which is below the realm of significance in science (10%+ is significant)."
Whoa. Absolutely not true. there is nothing in science that causes an arbitrary number such as '10% or less' to be called insignificant. If you make measurements that you have shown to have 1% measurement error max, then it is significant if you measure 1.01% difference. There are many examples in science where a noted difference of much less than 10% resulted in a new discovery.
Besides that, a 5% slowdown for Linux is a pretty significant slowdown for me and many people like me.
So... besides that: l4linux has existed for a while. And... is it widely used because of its obvious/great advantages? No, it's barely used because it's advantages are debatable and its disadvantage is significant.
"I'd like to see the measurements that are based on modern microkernels like l4, coyotos or QNX"
Each of them have very limited application and have their own unique disadvantages. Some have unique advantages that given them a litle niche to play in.
What the microkernel theorists want you to believe is that microkernels are not just for niche-applications or just of academic interest. They want everybody to believe that microkernels are fundamentally a lot better than what we have today. With such a theoretical promise of microkernels, there should be a serious Linux/Unix contender real-soon-now.
I've heard the microkernel theorist say that a couple of times before, each time with pretty much the same argumentation along these lines: "no this time it really is better, all previous times we said it and it didn't turnout, we found we did something wrong, the old stuff is crap this new thing will rule everything".
So... I'll believe it when I see it, not when people keep repeating the same un-convincing arguments of why they believe it's better.
The microkernel theorists have sandblasted us with their theory again this time around. After posing a theory, the next scientific step is to find proof of the theory by observing/measuring. For microkernels, that means: make it and show that it's what your theory promised.
No microkernel that exists today proves the theory.
"Minix 3 was written in scratch over the last year by three people, and it is a functional OS that can be used right now"
First Linux source 'intended to be read not run' was released Sept 1991, and in Oct 1991 thre was the fist usable linux. And in Dec 1992, it was self hosted.
First Point is: making something that can be used a bit is not a feat unique to microkernels.
Secont Point is: Re-starting from scratch doesn't reset the 'development time' counter. It simply means that you're trying again because the previous tries failed.
Well, call people a liar. If that's what you want, fine, go hate people who don't have trouble being effective with mutexes and semaphones. If that makes you feel better, go ahead.
If you think that everything what source analysis tools such as coverity return are bugs, then you don't know how source analysis tools work and how they should be used.
Otherwise, I will explain here why I said that mutexes and semaphores are not inherently difficult to get absolutely right.
PP of the thread suggested that using threads is bad, he/she said never to use threads and to always separate it in processes. That is just a completely ill-advised assertion
The level of synchronization and communication bandwidth you _can_ get for processes is much less than what you can get for threads, and it doesn't even come close to the requirements for the in-kernel synchronization issues.
In the kernel, the people want to get the absolute best performance with as much parallelism as possible. You're talking about multiple kernels (SMP/NUMA) talking to the same hardware from totally different contexts where is it very important to keep the locks as short as possible to keep latencies low.
The deadlocks and races that you refer to didn't exist until after they moved from a global kernel lock to very fine grained locking.
Point is: It's as complicated as you make it, and for very decent synchronization and communication performance at a level much higher than what you can get between processes, you don't have to make it even close to as complicated as in-kernel synchronization. Using mutexes and semaphores in threads is actually very simple.
For threads, the locking/synchronization requirements are such that using some very simple synchronization techniques, it is very simple to get great performance that beats any IPC method that you can do between processes (except shared memory).
I have done it, seen it, used it and it works really well.
People who suggest not using mutexes and semaphores because they consider them difficult simply avoid learning how to be effective using them.
And when I point that out, I get called a liar? Go figure.
"Some things are simply hard to use. Hard to use means hard to get correct."
It's like math. Math is hard to use, but it's actually the only thing in existance you can absolutely guarantee correctness for. If you study math and excercise math, then you will learn how to use math and use it correctly.
Mutexes and semaphores are not hard when you understand how to use them. The problem with mutexes and semaphores is that a lot of people (try to) use them (often incorrectly) without really understanding them.
If once in your career you take the time to learn them and really understand what they do and what they don't and what are good ways to use them, they are not hard anymore.
"I think that in this case reference counter mechanism can be used."
You can do that, and it will probably work after you spend quite considerable time on making sure it works... You either need redundant hardware like you said, or you need to write the driver such that it can share the hardware with another driver (in the latter case, a bug in one of the drivers can still put the hardware in an unusable state). With for example a filesystem or block device driver, it becomes even more complicated because the 'state' of the hardware is the harddisk, a pretty big state to copy around to your redundant hardware (even if you do it 'live', it's like syncing a software raid0, will slow down your disk speed by quite a bit). Point is, it's not automagic because it has a microkernel, it still costs time to architect, implement, and test all such features.
Nothing is stopping Linux from having the exact same capability when unloading/reloading modules. In fact, clustering addons such as Linux-HA/Linux-vserver and openssi/openmosix already have a lot of those capabilities even across nodes.
You can add lots of stuff to the microkernel. If you add it all up, it won't be so micro anymore, it will get bigger and bigger and the 'proven correct' argument will go up in smoke.
"You answered the question you wanted to hear, not the one I asked. It's a neat dance you did, but you should save it for someone that won't call you on it."
I did not answer your question for a reason: I don't post here to answer your questions, and I explained to you why your question was irrelevant to the discussion.
The animosity is toward the theorists who keep insisting microkernels are such a good idea, but are void of evidence. Science is posing theory and then testing it with observations, a.k.a. measurements. The available measurements don't look good for the microkernel theory.
Maybe the microkernel theorists should quit posting here and work on some evidence of their theory.
Reloading an essential driver such as, say, the block device driver, or in a telephone system, the line card driver, is pretty much the same thing as a reboot. You end up with a service interruption. There should be more to a microkernel than 'it can reboot fast because you reboot only parts of it' for it to be convincingly better.
A bug in your filesystem driver will still corrupt the disks, even if you can reload the driver.
ps: Linux has a method to load a new kernel without needing a reboot.
You list only two advantages and a long list of disadvantages. And for the "Drivers in user space have real advantages", you don't need a microkernel (e.g. FUSE and USB drivers in Linux), and how it's done in Linux, the 'bloat' (if any) is no worse than that of a driver on a microkernel.
The other advantage "If you get it right, the kernel doesn't change much over time." holds for driver interfaces in unified kernels too, and for both the problem is in the 'if you get it right'.
The "In the mainframe world, the kernel can put program in direct communication with the hardware without giving it the ability to write all over memory." would equally help unified kernels.
Microkernels may be very interesting academically, but it really does not offer anything that can't be done equally well with a unified kernel.
You would be surprised how few kernel developers have worked on the core of Linux that is functionally equivalent of what is in a microkernel (memory management, scheduler, etc).
Besides that, the question is irrelevant. Minix existed way before Linux, yet is very far behind in development even on basic functionality such as memory management. If Microkernels were really so much better, with things supposedly being so much easier to get 'proven correct', then it should have developed faster than a unified kernel like Linux, not slower.
"Threading is bad. Don't use it. When you have to use code that uses it, refactor the code to use processes or a state machine. It can be done. Don't whine. But don't use threads either."
Pfft. Quit wasting cycles and memory. No form of ipc has better bandwidth than shared memory. Threading is the easiest way to share a heap. Splitting a threaded process to processes usually costs you memory (because a single word of memory write causes the whole page to be copied (copy-on-write)).
"so you need to have mutexes, semaphores, and all of that synchronization crap that makes for buggy code."
Don't hate it because you don't know how to use it. Mutexes and semaphores doth not make a program buggy.
"Try doing what I do with Minix3: run it in VMWare, allocate it 4GB of RAM, and let VMWare do your virtual memory manegement." "(Yes, I know it's an ugly hack. But it means I don't worry about giving Bash 120mb, and cc some enormous number...)"
Warning: Flames below. Ignore if not fire resistant...
It's not only ugly. It shows the Minix3 'microkernel' is not even a kernel, if you need to run something UNDERNEATH it to make it even remotely usable... Besides that, I fail to see how this memory management problem is more secure than Linux against, umm let's see, how about Denial of Service attacks? Btw, is this an example or the proof of how easy it is to make a microkernel 'proven correct' too?
Remember microkernel-loving theorists out there, we're talking about Minix, something quite alot _older_ than Linux. It has had much more time to develop, and if the theory is correct, it should have been much easier to develop to an advanced level than the Linux kernel has been... It should have at least basic, basic, basic functionality such as fully functional memory management today...
"99% of the time when using shm, you use a semaphore, another SysV IPC primitive."
This is my opinion, but I had to say it: I personally don't like SysV. There are various ways to synchronize, and each method has advantages and disadvantages, but SysV is at the bottom of the pack if you ask me.
process-shared pthread mutexes and conditions are much faster than SysV, because they usually don't make a system call. A disadvantage of the SysV ipc that process-shared pthread mutexes have too is demonstrated by the existance of a program called 'ipcrm': "to delete resources that were left by irresponsible processes or process crashes": When a process crashes (or gets a kill -9, or a similar rude interruption), it may keep a lock causing the other contenders to wait forever until somebody cleans up the mess.
That is why fcntl()-based locks can be the right thing for your app, even though it's a syscall every time too like SysV. fcntl()-based locks are cleared as soon as you exit or close the filedescriptor (you can use the same fd as the one you used to mmap() the shared memory). That last behaviour, in turn, is a disadvantage of fcntl locks, because you lose the lock too if you close another filedescriptor referring to the same file (grunt)... There is also the flock(), which can hold only one lock per file (and I'm not sure, may have the same limitation as the fcntl() locks).
None of these are 'perfect', but SysV has never been my choice yet... I have good hopes for the new 'robust futexes' in the very lastest 2.6 kernels, which should combine the flexibility and efficiency of pthread mutexes with the robustness of flock()...
Oh, and yes: you can place a pthread mutex or condition in shared memory between processes (pthread_ZZZattr_setpshared(attr, PTHREAD_PROCESS_SHARED), it's not very well documented but it has been in the nptl for a while now and it works great).
"This is ironic because supposedly Linus used Tanenbaum's Minix as a starting point and influence for Linux."
It's actually ironic the other way around, because Linus created Linux because he was unhappy with the fact that Minix had too many unpractical shortcomings/limitations, among which lack of support for the new 32-bit 80386 chips in 32-bits mode.
And still today, Tanenbaum's kernels are lagging significantly in functionality.
The status-quo is a clear QED from Linus. If Tanenbaum claims microkernels are so much easier to make fully functional and rock-solid stable, then reality shows he's wrong until he proves it to the world by shoing us a 'kick-ass kernel'. Linus came late to the game, relatively compared to Unix and Minix, and it's way ahead now on so many fronts it's not even funny.
Seriously, the best marketing for Microsoft is to "leak" one of its products.
Take a leak on its products? -> First thought in my mind after reading the headline "New Windows Media Player Leaks" was that it still needs potty training.
If that was their intention, then, well, mission accomplished!
Sure, 'the client can'... so it may be 'fixable' by adding code to the nfs client, but that's a bunch of extra code for something you don't need with UDP. Why do things the hard way?
If something is fixable, that alone doesn't mean it's the right way to go.
The retransmit rates I see for nfs (nfsstats) on even the busiest clients are less than 0.05 percent. At rates like that, whatever TCP gains by doing smaller retransmits cannot weigh up agains the additional latency of SYN/ACK/RST TCP packets. Switching nfs to tcp will slow things down.
I can see why Slowlaris would default to TCP, after their NFS UDP checksumming-bypass debable in SunOS: Their users are still afraid of UDP (I've seen the corruptions myself).
"when I was developing the NFS-over-TCP support for the NetApp servers"
Your own colleage shows that TCP is about 5% slower than UDP. Where he gets that that is not an issue is a mistery though... The rest of the slides just make my point of TCP being the hard way to do nfs.
Ok. I'm with samba for Slowlaris and HPYucks. Maybe it's me, but I have not yet seen a fast Slowlaris or HPYuck NFS client or server.
"Because the first problem they mention:"
That actually is a problem with using tcp not udp... the server is stateless with UDP, hence if a server crashes the clients simply wait until the server comes back, after which everything resumes as if nothing happened. With TCP, the server doesn't know about the connection (the state) when it comes back, with UDP it doesn't need to know, so when the server comes back it can do the retransmit and the client continues as if nothing happened.
That is on top of the overhead... both things you don't want, so go for udp unless you have a very bad network (for example, bad switches, overloaded routed links between client and server, or very long distances)
> "For me, nfs is faster than any tcp socket, including samba." >> "Including NFS, too? (Were using NFS-over-UDP or NFS-over-TCP?)"
nfs defaults to udp and if you have decent switches, it doesn't need any of the things that tcp offers (see below). Why add the overhead and problems (see below) of tcp to an nfs mount?
A new feature, available for both 2.4 and 2.5 kernels but not yet integrated into the mainstream kernel at the time of this writing, is NFS over TCP. Using TCP has a distinct advantage and a distinct disadvantage over UDP. The advantage is that it works far better than UDP on lossy networks. When using TCP, a single dropped packet can be retransmitted, without the retransmission of the entire RPC request, resulting in better performance on lossy networks. In addition, TCP will handle network speed differences better than UDP, due to the underlying flow control at the network level.
The disadvantage of using TCP is that it is not a stateless protocol like UDP. If your server crashes in the middle of a packet transmission, the client will hang and any shares will need to be unmounted and remounted.
The overhead incurred by the TCP protocol will result in somewhat slower performance than UDP under ideal network conditions, but the cost is not severe, and is often not noticable without careful measurement. If you are using gigabit ethernet from end to end, you might also investigate the usage of jumbo frames, since the high speed network may allow the larger frame sizes without encountering increased collision rates, particularly if you have set the network to full duplex.
"NFS is poop. You actually get better transfer rates using samba between two Unix systems than you do using NFS."
You're either trolling, or there is something very wrong with your network or your nfs setups. I can easily fully saturate the network cards using nfs. For me, nfs is faster than any tcp socket, including samba.
> "Did you notice that for each definable segment, there is a separate register? That is because only registers are fast enough."
>> "Not really. Registers are caches for segment indices. The segment information is stored either the global descriptor or local descriptor table."
Nope. CS/DS/SS/ES are not caches they are actual registers. Registers are never caches, registers are (simple) flip-flops. Caches are (mostly) SRAM. Totally different beasts.
"No, it will not. We are talking 32 bits more per call. That's negligible,"
It's not.
"That's negligible, since the stack areas will already be in cache anyway."
Besides adding cycles to basic instructions such as a call, you're adding significantly to the memory bandwidth.
"And what you don't understand is that these checks are already performed at each call. And not only there is a segment limit check, but since most calls will call the kernel, there is also a jump from higher to lower ring as well (and that is costly!)."
Nope, 'the checks' are a result of being able to know 'page faults', and each page is 4kbyte (or more for hugetlb), and if page boundaries are not crossed, no checks are needed so none are done. They actually added hugetlb support to the chips to not only reduce the table sizes, but also to reduce the number of page boundary crossings, to improve the performance.
"It seems to me you have no idea how 80x86 works."
Seems to me that you are a software person who thinks he knows hardware.
Slashdot Mirror
"other things OSS as a class could stand to do better"
You mention documentation a lot. The only to do that right for the end user is to have expert users write the documentation, not the developers. The end users are usually the lazy bunch of an OSS projects, hence the end-user documentation sucks. To 'fix' that, the expert users of the software of the project need to chip in and make documentation. If there are many expert users, and no small group is willing/able to do the whole thing, then a Wiki can be of tremendous use.
Similar with testing and release. OSS developers don't usually take much time to do that, because they are busy developing (whining about that won't change that, this is OSS and the developers are volunteers). Especially because of the usability and end-user experience arguments, testing and release is more an expert-user task than a developer task.
I made that statement about the documentation, because for the developers a tremendously large portion of the documentation load is carries by the 'Source' in 'Open Source', or as more commonly said: UTSL... "Use the Source Luke!". I've even looked at things like the libc source to find out exactly what is what, where, and why. Documentation doesn't get more precise and up-to-date than that. UTSL works, if you speak the language of a developer.
Maybe /. should have built-in wikipedia linking...
http://en.wikipedia.org/wiki/Kool-Aid
Isall splained right there bro.
If this minix is something totally not linked to the other minix, then why call it minix?
"We call it Minix because it has nothing to do with Minix".
Yeah. Ok.
Fine. I'm done posting in this thread. Nobody benefits from us disagreeing here.
"They report less than a 5% slowdown on x86 kernels, which is below the realm of significance in science (10%+ is significant)."
Whoa. Absolutely not true. there is nothing in science that causes an arbitrary number such as '10% or less' to be called insignificant. If you make measurements that you have shown to have 1% measurement error max, then it is significant if you measure 1.01% difference. There are many examples in science where a noted difference of much less than 10% resulted in a new discovery.
Besides that, a 5% slowdown for Linux is a pretty significant slowdown for me and many people like me.
So... besides that: l4linux has existed for a while. And... is it widely used because of its obvious/great advantages? No, it's barely used because it's advantages are debatable and its disadvantage is significant.
"I'd like to see the measurements that are based on modern microkernels like l4, coyotos or QNX"
Each of them have very limited application and have their own unique disadvantages. Some have unique advantages that given them a litle niche to play in.
What the microkernel theorists want you to believe is that microkernels are not just for niche-applications or just of academic interest. They want everybody to believe that microkernels are fundamentally a lot better than what we have today. With such a theoretical promise of microkernels, there should be a serious Linux/Unix contender real-soon-now.
I've heard the microkernel theorist say that a couple of times before, each time with pretty much the same argumentation along these lines: "no this time it really is better, all previous times we said it and it didn't turnout, we found we did something wrong, the old stuff is crap this new thing will rule everything".
So... I'll believe it when I see it, not when people keep repeating the same un-convincing arguments of why they believe it's better.
The microkernel theorists have sandblasted us with their theory again this time around. After posing a theory, the next scientific step is to find proof of the theory by observing/measuring. For microkernels, that means: make it and show that it's what your theory promised.
No microkernel that exists today proves the theory.
"Minix 3 was written in scratch over the last year by three people, and it is a functional OS that can be used right now"
First Linux source 'intended to be read not run' was released Sept 1991, and in Oct 1991 thre was the fist usable linux. And in Dec 1992, it was self hosted.
First Point is: making something that can be used a bit is not a feat unique to microkernels.
Secont Point is: Re-starting from scratch doesn't reset the 'development time' counter. It simply means that you're trying again because the previous tries failed.
http://lug.mtu.edu/slides-fall04/history.html
Well, call people a liar. If that's what you want, fine, go hate people who don't have trouble being effective with mutexes and semaphones. If that makes you feel better, go ahead.
If you think that everything what source analysis tools such as coverity return are bugs, then you don't know how source analysis tools work and how they should be used.
Otherwise, I will explain here why I said that mutexes and semaphores are not inherently difficult to get absolutely right.
PP of the thread suggested that using threads is bad, he/she said never to use threads and to always separate it in processes. That is just a completely ill-advised assertion
The level of synchronization and communication bandwidth you _can_ get for processes is much less than what you can get for threads, and it doesn't even come close to the requirements for the in-kernel synchronization issues.
In the kernel, the people want to get the absolute best performance with as much parallelism as possible. You're talking about multiple kernels (SMP/NUMA) talking to the same hardware from totally different contexts where is it very important to keep the locks as short as possible to keep latencies low.
The deadlocks and races that you refer to didn't exist until after they moved from a global kernel lock to very fine grained locking.
Point is: It's as complicated as you make it, and for very decent synchronization and communication performance at a level much higher than what you can get between processes, you don't have to make it even close to as complicated as in-kernel synchronization. Using mutexes and semaphores in threads is actually very simple.
For threads, the locking/synchronization requirements are such that using some very simple synchronization techniques, it is very simple to get great performance that beats any IPC method that you can do between processes (except shared memory).
I have done it, seen it, used it and it works really well.
People who suggest not using mutexes and semaphores because they consider them difficult simply avoid learning how to be effective using them.
And when I point that out, I get called a liar? Go figure.
"Some things are simply hard to use. Hard to use means hard to get correct."
It's like math. Math is hard to use, but it's actually the only thing in existance you can absolutely guarantee correctness for. If you study math and excercise math, then you will learn how to use math and use it correctly.
Mutexes and semaphores are not hard when you understand how to use them. The problem with mutexes and semaphores is that a lot of people (try to) use them (often incorrectly) without really understanding them.
If once in your career you take the time to learn them and really understand what they do and what they don't and what are good ways to use them, they are not hard anymore.
"I think that in this case reference counter mechanism can be used."
You can do that, and it will probably work after you spend quite considerable time on making sure it works... You either need redundant hardware like you said, or you need to write the driver such that it can share the hardware with another driver (in the latter case, a bug in one of the drivers can still put the hardware in an unusable state). With for example a filesystem or block device driver, it becomes even more complicated because the 'state' of the hardware is the harddisk, a pretty big state to copy around to your redundant hardware (even if you do it 'live', it's like syncing a software raid0, will slow down your disk speed by quite a bit). Point is, it's not automagic because it has a microkernel, it still costs time to architect, implement, and test all such features.
Nothing is stopping Linux from having the exact same capability when unloading/reloading modules. In fact, clustering addons such as Linux-HA/Linux-vserver and openssi/openmosix already have a lot of those capabilities even across nodes.
You can add lots of stuff to the microkernel. If you add it all up, it won't be so micro anymore, it will get bigger and bigger and the 'proven correct' argument will go up in smoke.
"You answered the question you wanted to hear, not the one I asked. It's a neat dance you did, but you should save it for someone that won't call you on it."
I did not answer your question for a reason: I don't post here to answer your questions, and I explained to you why your question was irrelevant to the discussion.
The animosity is toward the theorists who keep insisting microkernels are such a good idea, but are void of evidence. Science is posing theory and then testing it with observations, a.k.a. measurements. The available measurements don't look good for the microkernel theory.
Maybe the microkernel theorists should quit posting here and work on some evidence of their theory.
"What if I don't ever want to reboot."
Reloading an essential driver such as, say, the block device driver, or in a telephone system, the line card driver, is pretty much the same thing as a reboot. You end up with a service interruption. There should be more to a microkernel than 'it can reboot fast because you reboot only parts of it' for it to be convincingly better.
A bug in your filesystem driver will still corrupt the disks, even if you can reload the driver.
ps: Linux has a method to load a new kernel without needing a reboot.
You list only two advantages and a long list of disadvantages. And for the "Drivers in user space have real advantages", you don't need a microkernel (e.g. FUSE and USB drivers in Linux), and how it's done in Linux, the 'bloat' (if any) is no worse than that of a driver on a microkernel.
The other advantage "If you get it right, the kernel doesn't change much over time." holds for driver interfaces in unified kernels too, and for both the problem is in the 'if you get it right'.
The "In the mainframe world, the kernel can put program in direct communication with the hardware without giving it the ability to write all over memory." would equally help unified kernels.
Microkernels may be very interesting academically, but it really does not offer anything that can't be done equally well with a unified kernel.
"Soooo, I think it's a better idea to focus on how Minix might be made a viable OS rather than arguing the same nonsense all over again."
Exacltly. Start architecting/programming/coding. That is the only way to prove us wrong.
"As several of the posters here have already proven, they're not reading Tanenbaum's arguments anyway."
It's all theory until proven by observations (measurements). Until that, I stick with my opinion about the theory of microkernels.
Show us the killer kernel and we'll be convinced.
You would be surprised how few kernel developers have worked on the core of Linux that is functionally equivalent of what is in a microkernel (memory management, scheduler, etc).
Besides that, the question is irrelevant. Minix existed way before Linux, yet is very far behind in development even on basic functionality such as memory management. If Microkernels were really so much better, with things supposedly being so much easier to get 'proven correct', then it should have developed faster than a unified kernel like Linux, not slower.
"Threading is bad. Don't use it. When you have to use code that uses it, refactor the code to use processes or a state machine. It can be done. Don't whine. But don't use threads either."
Pfft. Quit wasting cycles and memory. No form of ipc has better bandwidth than shared memory. Threading is the easiest way to share a heap. Splitting a threaded process to processes usually costs you memory (because a single word of memory write causes the whole page to be copied (copy-on-write)).
"so you need to have mutexes, semaphores, and all of that synchronization crap that makes for buggy code."
Don't hate it because you don't know how to use it. Mutexes and semaphores doth not make a program buggy.
Stop whining yourself.
"Try doing what I do with Minix3: run it in VMWare, allocate it 4GB of RAM, and let VMWare do your virtual memory manegement." "(Yes, I know it's an ugly hack. But it means I don't worry about giving Bash 120mb, and cc some enormous number...)"
Warning: Flames below. Ignore if not fire resistant...
It's not only ugly. It shows the Minix3 'microkernel' is not even a kernel, if you need to run something UNDERNEATH it to make it even remotely usable... Besides that, I fail to see how this memory management problem is more secure than Linux against, umm let's see, how about Denial of Service attacks? Btw, is this an example or the proof of how easy it is to make a microkernel 'proven correct' too?
Remember microkernel-loving theorists out there, we're talking about Minix, something quite alot _older_ than Linux. It has had much more time to develop, and if the theory is correct, it should have been much easier to develop to an advanced level than the Linux kernel has been... It should have at least basic, basic, basic functionality such as fully functional memory management today...
"99% of the time when using shm, you use a semaphore, another SysV IPC primitive."
This is my opinion, but I had to say it: I personally don't like SysV. There are various ways to synchronize, and each method has advantages and disadvantages, but SysV is at the bottom of the pack if you ask me.
process-shared pthread mutexes and conditions are much faster than SysV, because they usually don't make a system call. A disadvantage of the SysV ipc that process-shared pthread mutexes have too is demonstrated by the existance of a program called 'ipcrm': "to delete resources that were left by irresponsible processes or process crashes": When a process crashes (or gets a kill -9, or a similar rude interruption), it may keep a lock causing the other contenders to wait forever until somebody cleans up the mess.
That is why fcntl()-based locks can be the right thing for your app, even though it's a syscall every time too like SysV. fcntl()-based locks are cleared as soon as you exit or close the filedescriptor (you can use the same fd as the one you used to mmap() the shared memory). That last behaviour, in turn, is a disadvantage of fcntl locks, because you lose the lock too if you close another filedescriptor referring to the same file (grunt)... There is also the flock(), which can hold only one lock per file (and I'm not sure, may have the same limitation as the fcntl() locks).
None of these are 'perfect', but SysV has never been my choice yet... I have good hopes for the new 'robust futexes' in the very lastest 2.6 kernels, which should combine the flexibility and efficiency of pthread mutexes with the robustness of flock()...
Oh, and yes: you can place a pthread mutex or condition in shared memory between processes (pthread_ZZZattr_setpshared(attr, PTHREAD_PROCESS_SHARED), it's not very well documented but it has been in the nptl for a while now and it works great).
"This is ironic because supposedly Linus used Tanenbaum's Minix as a starting point and influence for Linux."
It's actually ironic the other way around, because Linus created Linux because he was unhappy with the fact that Minix had too many unpractical shortcomings/limitations, among which lack of support for the new 32-bit 80386 chips in 32-bits mode.
And still today, Tanenbaum's kernels are lagging significantly in functionality.
The status-quo is a clear QED from Linus. If Tanenbaum claims microkernels are so much easier to make fully functional and rock-solid stable, then reality shows he's wrong until he proves it to the world by shoing us a 'kick-ass kernel'. Linus came late to the game, relatively compared to Unix and Minix, and it's way ahead now on so many fronts it's not even funny.
Seriously, the best marketing for Microsoft is to "leak" one of its products.
Take a leak on its products? -> First thought in my mind after reading the headline "New Windows Media Player Leaks" was that it still needs potty training.
If that was their intention, then, well, mission accomplished!
"Tanenbaum had a doctorate before Linus was potty trained."
And tanenbaum now comes up with a kernel on which programs have problems doing malloc for dynamic memory allocation???
Goes to show who moves like a glacier and who swims around its icebergs in circles... like a penguin.
Sure, 'the client can'... so it may be 'fixable' by adding code to the nfs client, but that's a bunch of extra code for something you don't need with UDP. Why do things the hard way?
m l)
If something is fixable, that alone doesn't mean it's the right way to go.
The retransmit rates I see for nfs (nfsstats) on even the busiest clients are less than 0.05 percent. At rates like that, whatever TCP gains by doing smaller retransmits cannot weigh up agains the additional latency of SYN/ACK/RST TCP packets. Switching nfs to tcp will slow things down.
I can see why Slowlaris would default to TCP, after their NFS UDP checksumming-bypass debable in SunOS: Their users are still afraid of UDP (I've seen the corruptions myself).
(http://www.sunmanagers.org/archives/1992/1589.ht
"when I was developing the NFS-over-TCP support for the NetApp servers"
Your own colleage shows that TCP is about 5% slower than UDP. Where he gets that that is not an issue is a mistery though... The rest of the slides just make my point of TCP being the hard way to do nfs.
http://www.connectathon.org/talks06/eisler.pdf
Ok. I'm with samba for Slowlaris and HPYucks. Maybe it's me, but I have not yet seen a fast Slowlaris or HPYuck NFS client or server.
"Because the first problem they mention:"
That actually is a problem with using tcp not udp... the server is stateless with UDP, hence if a server crashes the clients simply wait until the server comes back, after which everything resumes as if nothing happened. With TCP, the server doesn't know about the connection (the state) when it comes back, with UDP it doesn't need to know, so when the server comes back it can do the retransmit and the client continues as if nothing happened.
That is on top of the overhead... both things you don't want, so go for udp unless you have a very bad network (for example, bad switches, overloaded routed links between client and server, or very long distances)
> "For me, nfs is faster than any tcp socket, including samba."
h tml
>> "Including NFS, too? (Were using NFS-over-UDP or NFS-over-TCP?)"
nfs defaults to udp and if you have decent switches, it doesn't need any of the things that tcp offers (see below). Why add the overhead and problems (see below) of tcp to an nfs mount?
http://nfs.sourceforge.net/nfs-howto/performance.
5.4. NFS over TCP
A new feature, available for both 2.4 and 2.5 kernels but not yet integrated into the mainstream kernel at the time of this writing, is NFS over TCP. Using TCP has a distinct advantage and a distinct disadvantage over UDP. The advantage is that it works far better than UDP on lossy networks. When using TCP, a single dropped packet can be retransmitted, without the retransmission of the entire RPC request, resulting in better performance on lossy networks. In addition, TCP will handle network speed differences better than UDP, due to the underlying flow control at the network level.
The disadvantage of using TCP is that it is not a stateless protocol like UDP. If your server crashes in the middle of a packet transmission, the client will hang and any shares will need to be unmounted and remounted.
The overhead incurred by the TCP protocol will result in somewhat slower performance than UDP under ideal network conditions, but the cost is not severe, and is often not noticable without careful measurement. If you are using gigabit ethernet from end to end, you might also investigate the usage of jumbo frames, since the high speed network may allow the larger frame sizes without encountering increased collision rates, particularly if you have set the network to full duplex.
"NFS is poop. You actually get better transfer rates using samba between two Unix systems than you do using NFS."
You're either trolling, or there is something very wrong with your network or your nfs setups. I can easily fully saturate the network cards using nfs. For me, nfs is faster than any tcp socket, including samba.
> "Did you notice that for each definable segment, there is a separate register? That is because only registers are fast enough."
>> "Not really. Registers are caches for segment indices. The segment information is stored either the global descriptor or local descriptor table."
Nope. CS/DS/SS/ES are not caches they are actual registers. Registers are never caches, registers are (simple) flip-flops. Caches are (mostly) SRAM. Totally different beasts.
"No, it will not. We are talking 32 bits more per call. That's negligible,"
It's not.
"That's negligible, since the stack areas will already be in cache anyway."
Besides adding cycles to basic instructions such as a call, you're adding significantly to the memory bandwidth.
"And what you don't understand is that these checks are already performed at each call. And not only there is a segment limit check, but since most calls will call the kernel, there is also a jump from higher to lower ring as well (and that is costly!)."
Nope, 'the checks' are a result of being able to know 'page faults', and each page is 4kbyte (or more for hugetlb), and if page boundaries are not crossed, no checks are needed so none are done. They actually added hugetlb support to the chips to not only reduce the table sizes, but also to reduce the number of page boundary crossings, to improve the performance.
"It seems to me you have no idea how 80x86 works."
Seems to me that you are a software person who thinks he knows hardware.