Slashdot Mirror
Does Linux "Fail To Think Across Layers?"
John Siracusa writes a brief article at Ars Technica pointing out an exchange between Andrew Morton, a lead developer of the Linux kernel, and a ZFS developer. Morton accused ZFS of being a "rampant layering violation." Siracusa states that this attitude of refusing to think holistically ("across layers") is responsible for all of the current failings of Linux — desktop adoption, user-friendliness, consumer software, and gaming. ZFS is effective because it crosses the lines set by conventional wisdom. Siracusa ultimately believes that the ability to achieve such a break is more likely to emerge within an authoritative, top-down organization than from a grass-roots, fractious community such as Linux.
This is like comparing a monarchy with anarchy, without acknowledging that there are in-between solutions that have advantages of their own. Democracy (and representative democracy) being one example.
Not saying the linux development community should be a democracy with everything voted on or whatnot, just saying that there may be creative approaches that have yet to be explored. You'd think smart people with a penchant for game theory would be working on it.
Food for thought.
Personally, I think the Linux kernel manages these issues quite well, if (by conventional standards) rather inefficiently.
The practice, as I see it is: "The current rules (layering, etc.) are enforced rigourously (at least in Linus' tree) but radical rewrites
of the rules take place relatively often"
So if ZFS really does achieve wonderful things by violating the current layering it WON'T be accepted for Linux's kernel, but, if Linus can be convinced (via an appropriate chain of lieutenants, usually) that the layering is really an obstacle to achieving these things, we might see a completely new layering appear in 2.6.25 or somewhere, into which ZFS can fit. The inefficiency
comes from the number of substantial pieces of work that get dropped because they don't fit in, or were misconceived. A more economically rational system would try to kill them sooner. Also, inefficiency arises from the fact that changing the filesystem layering would require every existing filesystem to be rewritten. Linux is notoriously unfazed by this, but in a commercial world, I suspect this would be too hard to swallow and you'd end up with all your filesystems fitting into the model except one, from whence come bugs and code cruft.
Linux will "support gaming" once games are supported for Linux. Linux has OpenGL, OpenAL, all the illusionary walls are market-made. Linux is a platform to build on without the fear of being obsolete in 2 years. DOS games nowadays run on DosBox, as do early Windows games. Even XP needs tweaks to run Win9X games. How is targeting a moving sucky platform preferable to one that is open? Easy. Games sell for 6 months tops. You get the initial sales, you get the money. After that it's tough shit if it won't work after next Windows Update(tm). I have used Linux since 1994, but work in the IT industry. I am constantly amazed by the amount of BULLSHIT the windows folks put up with. For weird quirks "shit happens" is the most common reply.
'Once scientists, even the dim-witted social scientists, get muzzled, the Western Civilization is finished.' - oldhack
With Democracies, you end up with the tyranny of the majority, regardless of whether the minority opinion is the correct one. Under a Republic form, a large enough minority can plug up the works and force negotiation with the majority before a final solution is agreed upon.
The Linux Development community needs representative decision making, there are too many voters, hence, almost no direction or real progress towards a cohesive goal. Nothing will change without true leadership, and sadly, accountability.
You cant measure progress without accountability for failure. Socialism has not worked in ANY form, and it wont work for Linux either.
Reiser4 introduced us to all sorts of interesting capabilities never before seen in a file system (at the time) but I believe this same "layering violation" attitude pretty much put a stop to any of it getting into the kernel. The Reiser guys were forced to pretty much cripple their file system feature wise if they were to have any hope of getting it included in the kernel.
See Reiser4 Pseudo Files as one example.
I can understand that in certain cases "layering violations" are bad, but Linux kernel developers don't even seem to be willing to experiment or think outside the box at all.
Both sides have valid arguments... I don't think there is any easy solution, but it would be nice to see more forward thinking in the community.
Open Source Time and Attendance, Job Costing a
Layers might not be ideal, but they're consistent. The filesystem does its part, RAID/LVM does its own, etc.
ZFS seems to want to take all over the disk subsystem. Why? Is there a reason why it needs its own snapshot capabilities, instead of just using LVM?
These sorts of things always smell fishy to me, due to a feeling that once you start using it, it locks you in more and more until you're doing it all in this new wonderful way that's incompatible with everything else. Even though it's open source, it's still inconvenient.
This approach reminds me a lot of DJB's software: If you try to get djbdns you'll be also strongly suggested to use daemontools as well. The resulting system is rather unlike anything else, and a reason why many people avoid DJB's software.
Lovely biting sarcasm aside, to be honest, our storage layering in Linux leaves much to be desired. As witness the slow pace of improvement of the volume manager in recent years. This does not prove that layering is bad, but it suggests that our current conception of layering sucks pretty badly. For example, we are burdened with a ridiculously complex interface between application programs and kernel-level volume management support. Managed volumes live off in their own name space. Why can't I say "/dev/hda, you are now snapshotted, shazam"?. No, instead I have change my system over to use /dev/mapper/snapshotted-hda or some such nonsense. Similarly, we are unable to manage all block devices using the same administration interface. No higher level raid integrated with the volume manager, instead this is a separate subsystem that fights a lot with LVM. Partition support hopelessly misfactored and broken. It goes on and on. Nothing unfixable but lots of unfixed brokenness. Compared to this mess, Sun's massive layering violations seem like a breath of fresh air.
But the thing to do is fix our broken implementation of layering and not be fooled into thinking that layers are bad. What is bad is exactly as the author here claims: it is bad to have no powerful capability to cross layer boundaries so that applications see a simple, powerful model instead of the current situation, where one's face is constantly rubbed in the minutae of layering administrivia. ZFS actually has layering, it just bypasses some traditional Unix subsystems and takes care of the functionality itself. But is wrong to conclude that this must therefore be the optimal approach just because it improves on the mess that preceded it. If ZFS internal interfaces are worth using, then they are worth using as core interfaces, not ZFS-only interfaces. Translated into Linux terms, the implication is that it is high time to get busy and rectify some of the serious deficiencies in our storage model. Not by mashing all the layers together, but by teaching them how to get along more efficiently and powerfully, and not be so layered that important things don't even work.
Note: perhaps the biggest design distinction between Linux and other Unixen is that, internally Linux is all just one big flat function space where anything can call anything else and share any data. This is said to be a reason why Linux is more efficient than, say, the Mach kernel with its microkernel layering. If being all one big hairball of functions is good for memory management, vfs, scheduling and so on, then why is it not also good for volume management? I don't know the answer to this, but I do know that we have plenty of bogus layering in our storage stack that has really slowed progress in recent years and needs a good dunging out. Any nonbogus layering can stay.
Have you got your LWN subscription yet?
I've heard from people looking to port ZFS to the BSDs that it is a very difficult port, due to how tangled up it is with the operating system (Solaris) itself, and how much ZFS does for itself (volume managment, RAID, etc).
On one hand, this gives it some serious advantages when run on Solaris 10. But it also makes it difficult to port. I wonder if that is partially responsible for delaying OS X Leopard?
Uh, Unices aren't that easy to hack if you set it to block connection attempts from outside sources that get the root password wrong say..5 times. You can't password crack in that time. Then again, Macs and Ubuntu disable root by default so they have to guess username AND password. So, no, Vista's probably not more secure than Unices. Linux and Vista have the SAME problem. Drivers for both are in limited supply.
look! it's a bird, it's a plane, it's....a girl? yes, a girl browsing Slashdot on Linux
I maintain a Linux file system which is typically used across various kernel versions, including 2.4.x. Yes folk's 2.4.x is still used to ship new products. The changing interface makes for Fun-And-Games.
The VFS to file system is not particularly clean as you need to do pretty ugly things like increment page counts etc within the file system. Much of this is done to enhance performance, but could probably have been done better (ie. preserving a clean interface without real performance compromises).
Engineering is the art of compromise.
The Rise and Fall of Online Community
"1. Fonts, they are simply not as good as Windows."
Maybe. But just now I'm writing this on a Linux desktop while a Windows XP laptop is near left. I really can't apreciate any significative difference. Maybe I lack good taste or something, but I can say that no people that saw my Linux desktop (neither this one at home nor the one at work -basically the same) said "oh, those fonts are so ugly!"
"2. Ease of use. Nobody has sat first time users in front of a linux desktop and watch them puzzle over what those multiple desktops do, or how to switch between them."
On the table to the left (that's left from my laptop) stays my girfriend's laptop I gifted her. She's musician and graduated on History (go figure her techy abilities) and she was simply amazed when she turned on her new computer and saw the pretty login screen (while eventually I teached her how to change it, and she in fact tried some variants she returned to the "original" one since "it's so cute"). And I still remember she was just in heaven when she discovered she had "four computers in one" (four desktops) instead of just one like those poor windows users ("they'll be green with envvy" were her words more or less). Obviously she doesn't use the command line for anything and still manages quite well on her day-to-day tasks and asks my help no more (well, much *less* in fact) than she did when she used Windows on her previous computer.
"3. Basic styling problems. Needless flickery redraws of desktops. Uneven and asymetric layouts, huge icons in some places, tiny icons in others. Isometric icons"
Again, my girlfriend doesn't seem to be so much affected by this (and, by the way, what the heck is an "isometric" icon? The only thing I can imagine is a family of icons that maintain their aspect ratio disregarding scale, but this do seem to me an advantage much more than a defect), nor does my on-her-sixties mother which is so happy with her Linux box (I doubt very much she's even concious she's using Linux -she knows there exists some "gadgets" known as "computers" and barely anything else) and her ability to play some card games and talk to her daughter few thousand miles away through IM.
"4. Lack of help, I try to save, it fails, where's the link to the help that tells me..."
True enough. But if you are trying to imply that other operative systems' help is of any use -cough- Windows -cough- I'll say that in my more than a decade of professionally dealing with computers (while I never did "pure" helpdesk I've been always more or less near to end users) I *never* saw the case that the including help was of any aid to any user to solve their problems. Not a single time. Never.
"5. I am not interested in your philosophy, assemble me a bundle of software that fits my needs regardless of whether than software fits your philosophy."
Good luck using any version of a Microsoft solution and its "bundle of software" fitting your needs. Do you really have all your needs covered out of Notepad and Minesweeper?
"If there is one thing I would suggest, get Ubuntu played with by ordinary grandma"
I *already* did the experiment out to the letter (you see, my mother doubles as a grandma too). And yes, she was quite confused at the begining even pointing and clicking with the mouse (it's not such an easy task for an old woman, did you know?). On one hand I can't see how pointing-and-clicking would have been any easier on Windows; on the other she manages now to do with the computer anything she was interested at (playing some easy games, using IM, ordering her photo collections, writing an odd e-mail...) and I honestly feel that her using Windows wouldn't make her experience any easier. And please, pay attention that I'm not even living in the same city so I really can't "babysit" her on the computer but the odd weekends I visit her. Oh! and the OS isn't even one of those "easy Ubuntus": she must be a hardcore hacker, since she's using Debian "Sarge". Yeah, she must be a hardcore hacker since now I remember her computer has been free of virus and spyware all this time, not as some friends of her that doesn't happen to use the same OS.
Linux supports multiple "personalities" for different types of executable. It has for a very long time. NT made a bigger deal about its implementation of that feature because Microsoft wanted to (a) hurry up and kill OS/2 and (b) make frivolous claims of standards compliance. Even at the time, NT's approach was neither particularly novel nor particularly useful -- how many programs have you ever run that use the OS/2 or the Posix subsystem rather than the Win32 subsystem?
The vast majority of games run on Windows because that is where the market has been. Performance is largely irrelevant in choice of market; gaming performance will follow the money.
Why can one not seriously argue that running large parts of GDI in the kernel was a security problem? New categories of problems cropped up in the wake of that transition. It seems to me that the only basis for saying it was not "a massive introduction of insecurity" is that Windows *already* had rather large security problems.
One legitimate complaint is the poor state of integrated RAID support in the linux LVM. Yes, the LVM can mirror logical volumes now, but it is very klunky (I miss AIX LVM). Creating PVs on top of md devices is tedious and error prone (because you often need to split physical devices into multiple partitions to avoid resyncing the world). The LVM should at least support RAID 0 and 1.
Another complaint is the lack of consistent high level utilities for file system admin. On AIX, one command allocates a logical volume and creates a filesystem on it (or you can do the steps separately for greater control). Linux makes you create the LV, then create the filesystem. Worse, linux used to offer e2fsadm to resize a filesystem and the underlying LV. Now, you have to run lvextend THEN resize2fs - or the reverse if shrinking (and you had better type those sizes right). While it is nice to have access to raw LVs for some things, I really don't want to have to manually compute volume sizes - with potential data destruction in case of mistakes.
Yes, yes, yes. Given NT's connection with VMS I would expect the architecture to be sound and well thought out. Furthermore, I don't think anyone (in this thread at least) has said anything that sounded like "Windows totally devoid of all worth"
That said, the GP does have a point. The problem with NT family operating systems is not that it lacks layers. The problem as I understand it is Microsoft keep making technical decisions driven by political and/or marketing concerns which have the effect of short circuiting the protection that these layers would otherwise offer.
A recent example would be Vista's treatment of install.exe and setup.exe. Both of these trigger UAC elevations, since it is assumed that any .exe with such a filename is an installer. However, recent reports suggest that the UAC elevation is both silent and unstoppable. If true,
this opens an obvious attack vector - a classic case of MS weakening their security model to make things easier for the naive user.
This is the down side of "thinking across layers". Once you accept that it's ok to violate the layer model, then there will always be one more special case. And if your decisions are driven by marketing or by (say) anti-trust concerns, then some of those layer violations are probably not going to be very well thought out from a technical viewpoint.
Don't let THEM immanentize the Eschaton!