Slashdot Mirror
Does Linux "Fail To Think Across Layers?"
John Siracusa writes a brief article at Ars Technica pointing out an exchange between Andrew Morton, a lead developer of the Linux kernel, and a ZFS developer. Morton accused ZFS of being a "rampant layering violation." Siracusa states that this attitude of refusing to think holistically ("across layers") is responsible for all of the current failings of Linux — desktop adoption, user-friendliness, consumer software, and gaming. ZFS is effective because it crosses the lines set by conventional wisdom. Siracusa ultimately believes that the ability to achieve such a break is more likely to emerge within an authoritative, top-down organization than from a grass-roots, fractious community such as Linux.
Personally, I think the Linux kernel manages these issues quite well, if (by conventional standards) rather inefficiently.
The practice, as I see it is: "The current rules (layering, etc.) are enforced rigourously (at least in Linus' tree) but radical rewrites
of the rules take place relatively often"
So if ZFS really does achieve wonderful things by violating the current layering it WON'T be accepted for Linux's kernel, but, if Linus can be convinced (via an appropriate chain of lieutenants, usually) that the layering is really an obstacle to achieving these things, we might see a completely new layering appear in 2.6.25 or somewhere, into which ZFS can fit. The inefficiency
comes from the number of substantial pieces of work that get dropped because they don't fit in, or were misconceived. A more economically rational system would try to kill them sooner. Also, inefficiency arises from the fact that changing the filesystem layering would require every existing filesystem to be rewritten. Linux is notoriously unfazed by this, but in a commercial world, I suspect this would be too hard to swallow and you'd end up with all your filesystems fitting into the model except one, from whence come bugs and code cruft.
Linux will "support gaming" once games are supported for Linux. Linux has OpenGL, OpenAL, all the illusionary walls are market-made. Linux is a platform to build on without the fear of being obsolete in 2 years. DOS games nowadays run on DosBox, as do early Windows games. Even XP needs tweaks to run Win9X games. How is targeting a moving sucky platform preferable to one that is open? Easy. Games sell for 6 months tops. You get the initial sales, you get the money. After that it's tough shit if it won't work after next Windows Update(tm). I have used Linux since 1994, but work in the IT industry. I am constantly amazed by the amount of BULLSHIT the windows folks put up with. For weird quirks "shit happens" is the most common reply.
'Once scientists, even the dim-witted social scientists, get muzzled, the Western Civilization is finished.' - oldhack
Alternativ approaches to implementing subsystems of the Linux kernel are often developed concurrently, in parallel, and there's a system you can compare to darwinistic evolution that decides (in most cases) which one of a given set of workalikes makes it into the mainline tree in the end. That's why the Linux kernel itself incorporates, or tries to adhere to, a UNIX-like philosophy - make a large system consist of small interchangeable parts that work well together and do one task as close to perfect as possible.
That's why there are so many generic solutions to crucial things - like "md", a subsystem providing RAID-levels for any given blockdevice, or lvm, providing volume management for any given blockdevice. Once those parts are in place, you can easily mingle their functions together - md works very nice on top of lvm, and even so vice versa, since all block devices you "treat" with one of lvm's or md's functions/features, again, result in a block device. You can format one of these blockdevices with a filesystem of choice (even ZFS would be perfectly possible, I suppose), and then incorporate this filesystem by mounting to whereever you happen to feel like it.
There are other concepts deep down in there in the kernel's inner workings that closely resemble this pattern of adaptability, like, for example, the vfs-layer, which defines a set of reuqirements every file-system has to adhere and comply to. This ensures a minimal set of viable functionality for any given filesystem, makes sure those crucial parts of the code are well-tested and optimized (since everyone _has_ to use them), and also makes it easier to implement new ideas (or filesystems, in this sepcific case).
Now, zfs provides at least two of those already existing and very well working facilites, namely md and lvm, completely on its own. That's what's called "code-duplication" (or rather "feature-duplication" - I suppose that's more appropriate here), and it's generally known as a bad thing.
I do notice that zfs happens to be very well-engineered, but this somewhat monolithic architecture still bears the probability of failure: suppose there's a crucial flaw found somewhere deep down in this complex system zfs inevitably is - chances are you've got to overhaul all of its interconnecting parts massivley.
Suppose there's a filesystem developed in the future that's even better than zfs, or at least better suited to given tasks or workloads - wouldn't it be a shame if it had to implement mirroring, striping and volume-management again on its own?
Take an approach like md and lvm, and that's not even worth wasting a single thought on. The systems are already there, and they're working fantastically (I'm an avid user of md and lvm for years by now, and I frankly cannot imagine anything doing these jobs noticeably better). I'd say that this system of interchangeable functional equivalents, and the philosophy of "one tool doing one job" is absolutely ideal for a distributed development model like Linux'.
It seems to be working since the early nineties. There must be something right about it, I suppose.
:%s/Open Source/Free Software/g
YTARY!
Reiser4 introduced us to all sorts of interesting capabilities never before seen in a file system (at the time) but I believe this same "layering violation" attitude pretty much put a stop to any of it getting into the kernel. The Reiser guys were forced to pretty much cripple their file system feature wise if they were to have any hope of getting it included in the kernel.
See Reiser4 Pseudo Files as one example.
I can understand that in certain cases "layering violations" are bad, but Linux kernel developers don't even seem to be willing to experiment or think outside the box at all.
Both sides have valid arguments... I don't think there is any easy solution, but it would be nice to see more forward thinking in the community.
Open Source Time and Attendance, Job Costing a
I totally agree. I think the main problem with Linux based systems (Fedora, Ubuntu, etc.) is that there are so many of them. Diversity is wonderful for free speech, but in the open source community, we have 100 solutions for every 1 problem.
The best solution would be for the Linux Kernel project to say, "Open source developers can do as they please, but we here at the Kernel project encourage developers to contribute to THESE specific projects: Gnome, Open Office, etc...
The open source community is massive, but development will take an eternity until a majority of the community starts to support ONE software solution over it's alternatives.
ZFS is a file system developed by Sun over the past several years. But the important thing is, in this context, that the ZFS design philosophy (never mind the actual design, which isn't what this discussion is about) differs from that of ordinary file system design. Most file systems make strong assumptions about reliability of the underlying block storage facility: there's some gizmo down there, whether it be a disk (for itsy-bitsy systems), a RAID set (for not so bitsy systems), or a SAN, that reliably stores and retrieves blocks with reasonable performance. ZFS doesn't do this. It manages many details of the storage layers -- it does RAID its own way (to get around problems that conventional RAID doesn't solve), and does volume management itself as well.
From the point of view of a UNIX/Linux file system person, this seems very weird. However, these ideas are not really new or revolutionary (there are new things in ZFS, but this philosophy isn't one of them). It pretty much describes how network storage vendors (NetApp, EMC, etc) have been building things all along.
Am I part of the core demographic for Swedish Fish?
Layers are both easier to code, to understand, and to test. Layers/boundaries between software are your friend. To some degree that is why the Internet, based upon a layered network model (TCP on top of IP on top of Ethernet) is so diverse.
Layering is what keeps things manageable. One you start getting your software tentacles into several layers you make a mess of things for both yourself and others. Its a tradeoff--complexity/speed vs simplicity/maintainability/interoperability.
Indeed. Whatever downsides layers have, they keep things sane. If you're going to make a mess of things, at least with layers you have an organized mess. There's a reason that Linux is more secure than Windows.
The World Wide Web is dying. Soon, we shall have only the Internet.
Open source software gets better because new people want new features to which they contribute. You can't blame Andrew Morton for disliking what ZFS is going to do, this is just how people work. This is why they say you can't teach an old dog new tricks.
That said ZFS is one of the coolest things to happen to your files in a long time. The current disk block device usage is basically the same from the beginning of computing, it is ancient and actually quite stupid. Over decades layers keep getting added to it to make it more robust, but really it's a monstrosity. Partitions are dumb, LVM is dumb, disk block RAID is dumb, monolithic filesystems are dumb. All the current linux filesystems should be thrown out.
I don't want to care how big my partitions are, what level parity protection my disks have, or any of that junk. I want to add or remove storage hardware whenever I want, and I want my files bit-exact, and I want to choose at will for each file what the speed vs protection from hardware failure is. Why shouldn't one file be mirrored, the next be stripped, and the next have parity or double parity protection? Why can't very, very important files have two or three mirrors?
From the current status of ZFS however I think this could be quickly built using GPL 2+ by one or two determined people, and it would involve gutting the linux file systems.
"I don't know that atheists should be considered citizens, nor should they be considered patriots." George HW Bush
I have spent the last three days teaching someone how to use windows XP when all they used to use was windows 98. Every interface is different. Stop teaching interfaces and start teaching ideas. Stop teaching MSFT word, start teaching word processing. Teach spreadsheets not excel.
I can sit down in front of any computer and begin to figure it out. i wasn't taught windows, I learned about windows from windows. I learned about OS X from OS X. and I figured out how to make a custom kde setup from KDE.
You want to know what I find short comings in them all. They are tied to one group, one development process. I want an OS that has the ease of use of OS X, with the multi-platform binaries of java, and the remote windowing of X. I want to carry my home directory files on an encrypted thumb drive, and load up my files, whether or not the OS is OS X, linux, windows, solaris, plan 9, or what ever else the future may bring.
we have the knowledge and technology to do that today.
i thought once I was found, but it was only a dream.
Linux and other open source projects are getting a harsh lesson in what it is like to ship consumer grade software products.
um, you do know that linux has been the operating system of choice for supercomputers, webservers, special effects production, scientific computing etc. for a number of years now, don't you? because you seem to think that linux, freebsd, openbsd or whatever just suddenly turned up yesterday or something. are you also aware of the fact that a lot of people who write free and open-source software get paid good money to do so?
Lovely biting sarcasm aside, to be honest, our storage layering in Linux leaves much to be desired. As witness the slow pace of improvement of the volume manager in recent years. This does not prove that layering is bad, but it suggests that our current conception of layering sucks pretty badly. For example, we are burdened with a ridiculously complex interface between application programs and kernel-level volume management support. Managed volumes live off in their own name space. Why can't I say "/dev/hda, you are now snapshotted, shazam"?. No, instead I have change my system over to use /dev/mapper/snapshotted-hda or some such nonsense. Similarly, we are unable to manage all block devices using the same administration interface. No higher level raid integrated with the volume manager, instead this is a separate subsystem that fights a lot with LVM. Partition support hopelessly misfactored and broken. It goes on and on. Nothing unfixable but lots of unfixed brokenness. Compared to this mess, Sun's massive layering violations seem like a breath of fresh air.
But the thing to do is fix our broken implementation of layering and not be fooled into thinking that layers are bad. What is bad is exactly as the author here claims: it is bad to have no powerful capability to cross layer boundaries so that applications see a simple, powerful model instead of the current situation, where one's face is constantly rubbed in the minutae of layering administrivia. ZFS actually has layering, it just bypasses some traditional Unix subsystems and takes care of the functionality itself. But is wrong to conclude that this must therefore be the optimal approach just because it improves on the mess that preceded it. If ZFS internal interfaces are worth using, then they are worth using as core interfaces, not ZFS-only interfaces. Translated into Linux terms, the implication is that it is high time to get busy and rectify some of the serious deficiencies in our storage model. Not by mashing all the layers together, but by teaching them how to get along more efficiently and powerfully, and not be so layered that important things don't even work.
Note: perhaps the biggest design distinction between Linux and other Unixen is that, internally Linux is all just one big flat function space where anything can call anything else and share any data. This is said to be a reason why Linux is more efficient than, say, the Mach kernel with its microkernel layering. If being all one big hairball of functions is good for memory management, vfs, scheduling and so on, then why is it not also good for volume management? I don't know the answer to this, but I do know that we have plenty of bogus layering in our storage stack that has really slowed progress in recent years and needs a good dunging out. Any nonbogus layering can stay.
Have you got your LWN subscription yet?
Socialism worked pretty well in the democratic western countries, that's why people aren't dying of cholera/typhoid/starvation in slums anymore.
"Ease of use. Nobody has sat first time users in front of a linux desktop and watch them puzzle over what those multiple desktops do, or how to switch between them.......If there is one thing I would suggest, get Ubuntu played with by ordinary grandma's so you can see how they get confused."
Just because your grandma is a little slow (okay, ALOT SLOW) does not mean all of them are.
My grandmother WAS sat in front of an Ubuntu box for the first time, and after 5 minutes, she asked me why her windows PC did not have Desktop switching, as it only makes sense, rather than constantly minimizing countless windows. Since she already has Firefox on her PC, there was no great hunt for the Big Blue "E" aka "the internet", and after a short explanation about how she, as a user, has her own little piece of the computer called a HOME FOLDER, and can save all her stuff there, she was set.
I am so tired of this myth that only people with a Mensa I.Q. are capable of understanding how to use a non-windows based system. Granted, she wont be editing config files or writing code, but how many outside the IT industry do that on a regular basis?
Mod me insightful (or fraking obvious, take your pick)
But the OSI layers are guidelines that help design things, not rigid levels that must be maintained. They are mixed up ALL the time. As a simple example, see Layer-3 switches. These are devices that simultaneously work at Layer 2 and 3 when it comes to dealing with traffic. They break down the traditional view of a separate router and switch, and they are good at what they do. There's plenty of stuff at the high end that's similar. Many things that are part of the presentation layer are actually done by the application (like encryption in SSH) and so on.
There's nothing wrong with having a layered design philosophy as it can help people decide what their product needs to do, and what it needs to talk to. For example if I am designing an application that works over TCP/IP, I really don't need to worry about anything under layer 4 or 5. However it shouldn't be this rigid thing that each layer must remain separate, and anything that combines them is bad. I don't need to, and shouldn't, take the idea that my app can't do anything that would technically be Layer 6 itself. Likewise in other situations I might find that TCP just doesn't work and I need to use UDP instead, but still have a session which I handle in my own code (games often do this).
Had we stuck to the OSI model as a maximum, rather than a guiding principle, with the Internet, it probably wouldn't have scaled to the point we have now.
"I mean, although ZFS is a rampant layering violation and we can do a lot of
the things in there (without doing it all in the fs!) I don't think we can
do all of it." http://lkml.org/lkml/2006/6/9/409
It sounds like his main point was pointing out problems with the current file system, rather than saying ZFS is bad. I bet he simply thinks they should try to implement a much better file system than ext3 without breaking the current layering scheme. I don't see why this is so bad. Why not try it, and if it fails miserably, ZFS is already here.
I think the author of the article took everything out of context and was just looking for some ammo against Linux. His blog post sucked. He just says the same crap that everyone always says. I'm not saying there are no problems, but I don't see how any of the problems relate to Andrew Morton saying the Linux file systems need to be upgraded/replaced.
You're right, that's why nobody is using Linux for real systems.</sarcasm>
The problem with a "traditional" layered model is that the file system has to assume that the underlying storage device is a single consistent unit of storage, where a single write either succeeds, or it fails (in which case the data you wrote may or may not have been written). This all sounds very good and file systems like ext2 are written based on this assumtion.
However, if the underlying storage system is RAID5, and there is a power loss during the write, the entire stripe can become corrupt (read the Wikipedia article on the subject for more information). The file system can't solve this problem because it has no knowledge about the underlying storage stucture.
ZFS solves this problem in two ways, both of which reuires the storage model to be part of the filesystem:
- Each physical write never overwrites "live" data on the disk. It writes the stripe to a new location, and once it's been completely committed to disk the old data is marked as free.
- ZFS uses variable stripe width, so that it does not have to write larger stripes than nescessary. In other words, a large write can be directly translated to a write to a large stripe on the sotrage system, and a smaller write can use a smaller stripe width. This can improve performance since it can reduce the amount of data written.
There are plenty of other areas where this integration is needed, including snapshotting, but I hope the above explanation explains that the layered model is not always good.You don't seem to understand snapshots.
A snapshot works by creating a copy of the device, with the contents it had when the snapshot was created. If you make a snapshot of
Why would you change anything over? Snapshots are temporary. You snapshot your drive, use the snapshot to create a consistent backup (or whatever), then destroy it.
Normally you won't keep a snapshot around for long, as they're maintained by keeping copies of modified blocks, and that takes space. Unless you have enough space for fully duplicating the device you made a snapshot of, you won't be able to keep it around forever.
I think you'll find that it is you that doesn't understand what a snapshot could be. Take a look at ZFS, try it, and see if you think of snapshots the same way again. In ZFS, a snapshot can be promoted to a clone, which is a writeable copy of the original filesystem, sharing unmodified blocks using a copy-on-write algorithm.
This is increadibly powerful and useful. For example, a single master 'image' volume can have customizations added for specific purposes. This is useful in desktop deployment, iSCSI or NFS network boot, etc...
Would you expect a 'first class' writeable clone to have a name like 'dev/mapper/snapshotted-hda' or 'dev/hda.1'? Which one makes more sense? Why would the original have a special name, when the clone is identical?
It's this kind of narrow 'snapshots are throwaway' thinking that causes artifical limitations in APIs and operating system design that serve no real purpose.
Most of all, to me, I am astonished that almost everyone talks 'virtualisation', VM, QEMU, Xen.
When it comes to filesystems, suddenly many seem to want to do everything on their own, on physical platters: partition, volumes/RAID, format. ZFS is a virtual filesystem, where none of such is physically needed. There is a nice http://www.opensolaris.org/os/community/zfs/demos
Of course, filesystem should be a black box, an object, instead of the user having to do low-level work. ZFS provides this, and more relevant: of course it needs to be cross-layered therefore.
Snapshots ought to be available easily, at any moment in time, without taking much space. ZFS does so, by only storing the changes and sharing the unmodified data. If you want to do so, you need an abstraction of the hardware. That is, crossing layers. Not to mention writeable snapshots.
Adding new drives without partitioning, slicing, formatting. Just adding to the existing pool. Inclusive striping being adapted automagically. This needs a cross-layer interface, right ?
The transactional filesystem guarantees uncorrupted data at power failures and OS crashes. If you do this across a pool of physical platters, you need operations across layers.
There is an interesting blog on the usage of ZFS for home users. It contains some good arguments, why ZFS is useful for Linux' Desktop Stride. You find it here: http://uadmin.blogspot.com/2006/05/why-zfs-for-ho
Last ot least, the online checking of all your data ('scrubbing' and 'resilvering') is a valuable feature for Linux (and the home user) as well.
To me it looks like, as of today, that about everyone liked the features of ZFS. Now, as it requires to break some old habits, suddenly we resist change and rather stick to older concepts.
As if GPLv2 vs GPLv3 was not enough of a threat to Linux, now we unashamedly permit a new-from-the-bottom-up filesystem to overtake us as well ?