Slashdot Mirror
Torvalds Has Harsh Words For FreeBSD Devs
An anonymous reader writes "In a relatively technical discussion about the merits of Copy On Write (COW) versus a very new Linux kernel system call named vmsplice(), Linux creator Linus Torvalds had some harsh words for Mach and FreeBSD developers that utilize COW: 'I claim that Mach people (and apparently FreeBSD) are incompetent idiots. Playing games with VM is bad. memory copies are _also_ bad, but quite frankly, memory copies often have _less_ downside than VM games, and bigger caches will only continue to drive that point home.' The discussion goes on to explain how the new vmsplice() avoids this extra overhead."
Do I have that right?
If so, I'm not really seeing his issue. Or at least not as hard-line as he sees it. The issue of memory copy performance is a tricky one, especially since CPU cycles are not the be-all to end-all of performance. Does the exception generated really cost that much more than he believes, or is it often eclipsed by the cost of the extra memory read/writes and CPU waits that are normally generated by a copy? Is it really feasible to expect program developers to do manual memory management in a day in age when programs easily weigh in at hundreds of megs?
I'm just not sure that Torvalds is really looking at all sides of this. He may be right, but I'd like to hear more discussion between the *BSD guys and Torvalds before we put this matter to rest. And preferrably without the insults this time.
Links:
Copy on Write as explained by Wikipedia
FreeBSD page on Zero Copy Patches
Duke Uni Research
Javascript + Nintendo DSi = DSiCade
It sounded like the opening volley of the second great Unix war, only this time instead of pitting proprietary Unix vendors and systems against each other... it is two open source ones.
It will be interesting to see what weapon the BSD crowd will retaliate with.
Help Brendan pay off his student loans
Maybe it's the stress, dunno but this guy is developing a chip on his shoulder that needs to be knocked off.
In the spirit of open source community development, he can't make statements like this and expect to be a role model for the open source community.
What is with this guy slamming on people. I am sure he's bright and all but would it hurt him to try and be constructive instead of just being rude. He could simply provide a example of what problems will occur and then stop.
Solaris 10/11 and openSolaris are looking better each and everyday
Are the *BSD people are nicer? Or at least more tactful?
No. Thats why there is more than one BSD. Issues come up, and booom crash goes the fork. Pity.
And in other news...
Grass is green;
Oil is overpriced;
Absolute power corrupts absolutely.
Here we go again, imposing "role model" status. Linus is just a guy. Sometimes he gets his buttons pushed, sometimes he's doing the pushing. BFD. Maybe you'd be a little pissy too if Slashdot posted a story every time you did or said something. Linus Prefers Gas-X, Says Bean-o Is For Douchebags. Who cares? (BTW, Linus didn't really say that, I made it up. Don't wanna get the Bean-o people on his case too.)
As far as this whole VM thing goes, time and testing will tell the true story. Meanwhile, maybe we could try NOT deifying Linus (any more)?
I saw it on Slashdot, it must be true!
"I claim that Mach people (and apparently FreeBSD) are incompetent idiots."
Linus, who's becoming more outspoken as he ages, needs to find that line between anonymous forum geek and software spokesperson...and then not cross it. Calling anyone an incompetent idiot is both non-constructive if you're hoping to improve a situation, and just plain unfriendly in an area where cooperation amongst developers is so crucial (open source).
The basic idea is to fake some memory to memory copying operations by using the virtual memory hardware. More specificially, the idea is that when you do a big "write", the space just written becomes read-only to the writing process, rather than being actually copied. When the write is complete, read-only mode is turned off. This eliminates one copy.
The way CoW works is that when a process copies something already in memory, the kernel has the MMU map those same memory pages to a new location in the process' address space and mark them as read-only, after which the kernel returns the address of the "copied" memory to the process. When any of the processes using that memory try to write to it, the MMU generates an exception (because the pages are marked read-only). The kernel intercepts the exception, allocates additional memory and copies the pages being written to into it, has the MMU remap that process' address space to point to those pages, and then proceeds with the write.
Linus, you may be right and you may be very smart, but you should try a little tact. Here's a good definition for it that I learned from a drill sergeant: "Tact is the ability to tell someone to go to hell and look forward to the trip."
Being nice and respectful doesn't mean you can't tell it like it is.
Do what is right and let the consequence follow
Why not focus on the real content instead of write tabloid style articles.
Linus has frequently called people idiots, and ignored patches, and done stuff his own way for a very long time now. He's quite successful at it. Perhaps what most people need to realize is that he is that good, that he can. The average read-Slashdot-during-work-while-coding Slashdotter is not in his league, so decrying his adhominem attacks, or "I would do X instead" arguments just dont hold much water.
I want to delete my account but Slashdot doesn't allow it.
Funny you should say that Linus, seeing how much of a fucking disaster was changing the VM in the middle of the 2.4 kernel branch that is supposed to be STABLE.
More likely he had some really bad acid the previous night.
After all he did more than 6 revisions of the Linux VM using CopyOnWrite before this latest fad.
Possibly more.
Off the top of my head that is at least 1 in the 1.2 tree, 1 in the 2.0 tree, 1 in the 2.2 tree, 2 in the 2.4 tree and more than 2 in the 2.6 tree, all of which being CopyOnWrite and at least some of which has been hailed as the next best thing after hot bread.
As far as the technical point he is possibly correct for x86 where COW goes through the fault mechanism and causes some TLB and cache abuse which is really bad on modern CPUs. I am not sure as far as other architectures are concerned, because IIRC (I may be wrong) the memory mapper hardware on the old Sparc was designed for COW in first place.
Anyway, before calling somebody else an idiot for something you have happily done for 10+ years till yesterday it may be nice if you look at yourself in the mirror. Because I never remember any branch of FreeBSD reaching the point where you can do a find /usr -exec cat {} > /dev/null \; to hang the system. That is 2.6.16 at your service (from rc4 onward) on at least two x86 subarchitectures where I had the time to test it. That is besides the unkillable processes in [S] state on an nfs flock in 2.6.14 (yep, that is a gem which no other unix has managed so far), besides the OOM idiocies in 2.6.10, besides deliberately making it absolutely impossible to backtrack any more interesting patch to a previous kernel without employing a team of kernel developers because the VM and locking is not compatible across any kernel version since 2.6.9 and even when it is something else is changed like the tty layer, besides.... Aarghh.....
Baker's Law: Misery no longer loves company. Nowadays it insists on it
http://www.sigsegv.cx/
I can certainly see the value in explicit notification of page usage, but I have to wonder if this isn't attacking the problem at the wrong level. It seems that these problems are caused by the semantics of read() and write() calls, requiring data to be read/written to an arbitrarily aligned userspace buffers.
Zero copy can definitely make things complex, and in the current implementations, the value is arguable. (and being argued...) Still, memory copies have an associated cost. While they may be better than COW with explicit notification, it is still a performance hack, and represents a non-optimal way of dealing with data transfers. (It could be the easiest and best hack to be made, I can't say. In any case, Linus is acting like a git with his name calling here.)
Perhaps more consideration should be given to the API instead. Using zero copy is obviously a good goal, and it is primarily hindered by the ancient API and protocols. Something where the buffer management is explicit, and the devices themselves actually own the them. (After all, they are the only entities which know what the buffer requirements are.) Arranging it so that the user applications have access to the actual network buffers would be far preferable to playing any of these "games".
Unfortunately, Ethernet and the IP protocols are not particularly conducive to such an optimal implementation. With enough intelligence in the network adapters though, many of the issues should be manageable, and allow for a good zero copy implementation with a suitable API. It may be more trouble for the application, but if you need the performance, it is a small price to pay.
I just don't seem what the big deal is with Linus's comment. Of course I'm an OpenBSD person, and can hardly consider a mere 'incompetent idiots' to be a serious disparagement.
Andy went out and said that he thought the Linux approach was wrong, and archaic, and that people should go and wait for GNU.
Linus said that he felt this was wrong, and that being a prof is no excuse for Minix being the mess it was (and Minix was a mess in the late 1980s/early 1990s). He also apologized if he came off as too harsh for his writing about how people should be able to throw away an old design in favour of a new one anyway, etc.
It was very polite compared to some of the non-Andy/Linux replies.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
I'm not too surprised, as he seems to be somewhat of a visionary: he see things as he thinks they should be... and explodes when they aren't. ;)
What I don't get is why he choose to use incompetent to describe a group of people who are not implementing something he is just now implementing himself.
-=- ThE DaRK MaN oF tHe ObScURiTY -=-
Thank you. I've read more than 30 high-modded posts in this article, and yours is the best explanation of the issue by far.
So the big question is, what happens if user mode breaks the promise, either intentionally or through lousy programming? If the program fucks up, well, then, I'd rather have FreeBSD's model (actually, I'd rather have someone come up with a thread-safe wrapper function, and keep I/O the way it's supposed ot be, i.e., atomic).
I'm sure someone said the same thing about the total size of segmented ICMP packets.