Slashdot Mirror
Does Linux "Fail To Think Across Layers?"
John Siracusa writes a brief article at Ars Technica pointing out an exchange between Andrew Morton, a lead developer of the Linux kernel, and a ZFS developer. Morton accused ZFS of being a "rampant layering violation." Siracusa states that this attitude of refusing to think holistically ("across layers") is responsible for all of the current failings of Linux — desktop adoption, user-friendliness, consumer software, and gaming. ZFS is effective because it crosses the lines set by conventional wisdom. Siracusa ultimately believes that the ability to achieve such a break is more likely to emerge within an authoritative, top-down organization than from a grass-roots, fractious community such as Linux.
This is like comparing a monarchy with anarchy, without acknowledging that there are in-between solutions that have advantages of their own. Democracy (and representative democracy) being one example.
Not saying the linux development community should be a democracy with everything voted on or whatnot, just saying that there may be creative approaches that have yet to be explored. You'd think smart people with a penchant for game theory would be working on it.
Food for thought.
Personally, I think the Linux kernel manages these issues quite well, if (by conventional standards) rather inefficiently.
The practice, as I see it is: "The current rules (layering, etc.) are enforced rigourously (at least in Linus' tree) but radical rewrites
of the rules take place relatively often"
So if ZFS really does achieve wonderful things by violating the current layering it WON'T be accepted for Linux's kernel, but, if Linus can be convinced (via an appropriate chain of lieutenants, usually) that the layering is really an obstacle to achieving these things, we might see a completely new layering appear in 2.6.25 or somewhere, into which ZFS can fit. The inefficiency
comes from the number of substantial pieces of work that get dropped because they don't fit in, or were misconceived. A more economically rational system would try to kill them sooner. Also, inefficiency arises from the fact that changing the filesystem layering would require every existing filesystem to be rewritten. Linux is notoriously unfazed by this, but in a commercial world, I suspect this would be too hard to swallow and you'd end up with all your filesystems fitting into the model except one, from whence come bugs and code cruft.
Linux will "support gaming" once games are supported for Linux. Linux has OpenGL, OpenAL, all the illusionary walls are market-made. Linux is a platform to build on without the fear of being obsolete in 2 years. DOS games nowadays run on DosBox, as do early Windows games. Even XP needs tweaks to run Win9X games. How is targeting a moving sucky platform preferable to one that is open? Easy. Games sell for 6 months tops. You get the initial sales, you get the money. After that it's tough shit if it won't work after next Windows Update(tm). I have used Linux since 1994, but work in the IT industry. I am constantly amazed by the amount of BULLSHIT the windows folks put up with. For weird quirks "shit happens" is the most common reply.
'Once scientists, even the dim-witted social scientists, get muzzled, the Western Civilization is finished.' - oldhack
1. Fonts, they are simply not as good as Windows.
2. Ease of use. Nobody has sat first time users in front of a linux desktop and watch them puzzle over what those multiple desktops do, or how to switch between them.
3. Basic styling problems. Needless flickery redraws of desktops. Uneven and asymetric layouts, huge icons in some places, tiny icons in others. Isometric icons( a classic sign of a programmer drawing an icon instead of an artist drawning icons).
4. Lack of help, I try to save, it fails, where's the link to the help that tells me that this is a security feature and I can only save in some places.
5. I am not interested in your philosophy, assemble me a bundle of software that fits my needs regardless of whether than software fits your philosophy.
If there is one thing I would suggest, get Ubuntu played with by ordinary grandma's so you can see how they get confused. Then get the Firefox guys to look at it, because to me, it's uneven styling, sometimes big crayola friendly styling aimed at kids, sometimes business like.
Knoppix for example, you start it up and look at the "Windo..." icon and wonder why the fuck they chose suck a large font and such a small icon spacing. So big it can't even display the words 'Windows'.
With Democracies, you end up with the tyranny of the majority, regardless of whether the minority opinion is the correct one. Under a Republic form, a large enough minority can plug up the works and force negotiation with the majority before a final solution is agreed upon.
The Linux Development community needs representative decision making, there are too many voters, hence, almost no direction or real progress towards a cohesive goal. Nothing will change without true leadership, and sadly, accountability.
You cant measure progress without accountability for failure. Socialism has not worked in ANY form, and it wont work for Linux either.
Reiser4 introduced us to all sorts of interesting capabilities never before seen in a file system (at the time) but I believe this same "layering violation" attitude pretty much put a stop to any of it getting into the kernel. The Reiser guys were forced to pretty much cripple their file system feature wise if they were to have any hope of getting it included in the kernel.
See Reiser4 Pseudo Files as one example.
I can understand that in certain cases "layering violations" are bad, but Linux kernel developers don't even seem to be willing to experiment or think outside the box at all.
Both sides have valid arguments... I don't think there is any easy solution, but it would be nice to see more forward thinking in the community.
Open Source Time and Attendance, Job Costing a
Layers might not be ideal, but they're consistent. The filesystem does its part, RAID/LVM does its own, etc.
ZFS seems to want to take all over the disk subsystem. Why? Is there a reason why it needs its own snapshot capabilities, instead of just using LVM?
These sorts of things always smell fishy to me, due to a feeling that once you start using it, it locks you in more and more until you're doing it all in this new wonderful way that's incompatible with everything else. Even though it's open source, it's still inconvenient.
This approach reminds me a lot of DJB's software: If you try to get djbdns you'll be also strongly suggested to use daemontools as well. The resulting system is rather unlike anything else, and a reason why many people avoid DJB's software.
Lovely biting sarcasm aside, to be honest, our storage layering in Linux leaves much to be desired. As witness the slow pace of improvement of the volume manager in recent years. This does not prove that layering is bad, but it suggests that our current conception of layering sucks pretty badly. For example, we are burdened with a ridiculously complex interface between application programs and kernel-level volume management support. Managed volumes live off in their own name space. Why can't I say "/dev/hda, you are now snapshotted, shazam"?. No, instead I have change my system over to use /dev/mapper/snapshotted-hda or some such nonsense. Similarly, we are unable to manage all block devices using the same administration interface. No higher level raid integrated with the volume manager, instead this is a separate subsystem that fights a lot with LVM. Partition support hopelessly misfactored and broken. It goes on and on. Nothing unfixable but lots of unfixed brokenness. Compared to this mess, Sun's massive layering violations seem like a breath of fresh air.
But the thing to do is fix our broken implementation of layering and not be fooled into thinking that layers are bad. What is bad is exactly as the author here claims: it is bad to have no powerful capability to cross layer boundaries so that applications see a simple, powerful model instead of the current situation, where one's face is constantly rubbed in the minutae of layering administrivia. ZFS actually has layering, it just bypasses some traditional Unix subsystems and takes care of the functionality itself. But is wrong to conclude that this must therefore be the optimal approach just because it improves on the mess that preceded it. If ZFS internal interfaces are worth using, then they are worth using as core interfaces, not ZFS-only interfaces. Translated into Linux terms, the implication is that it is high time to get busy and rectify some of the serious deficiencies in our storage model. Not by mashing all the layers together, but by teaching them how to get along more efficiently and powerfully, and not be so layered that important things don't even work.
Note: perhaps the biggest design distinction between Linux and other Unixen is that, internally Linux is all just one big flat function space where anything can call anything else and share any data. This is said to be a reason why Linux is more efficient than, say, the Mach kernel with its microkernel layering. If being all one big hairball of functions is good for memory management, vfs, scheduling and so on, then why is it not also good for volume management? I don't know the answer to this, but I do know that we have plenty of bogus layering in our storage stack that has really slowed progress in recent years and needs a good dunging out. Any nonbogus layering can stay.
Have you got your LWN subscription yet?
UNIX and Linux design is quite holistic: features are often added at various levels of the system in order to make a whole work out. For example, desktop search support had both user and kernel space components, Beryl/Compiz-style interfaces have triggered changes in Gnome, X11, and the kernel, etc.
UNIX and Linux have been careful about avoiding simplistic designs. ZFS is a simple, obvious answer to a problem: just pack all the functionality into one big codeball and start hacking. Microsoft does a lot of the same thing in Windows and Apple in OS X. It gives companies great time-to-market and long, impressive feature lists. It's also creates a mess. Microsoft, Apple, and Sun each have several iterations under their belts where they start off like that, then the system bloats, and finally collapses, causing the company to start over again from scratch or just abandon the market altogether.
Thanks, we know about the kind of "holistic" that these people are implementing, and we don't want it.
(And, frankly, I think Sun isn't really a UNIX company anymore; their system may still be UNIX compatible, but they stopped following the UNIX philosophy long ago.)
...you look at the grease that makes the world go round, and that is money, and games make lotsa money and garner a huge interest with hundreds of millions of people. Moolah, loot, it's there for the taking.
Personally, I don't game, but it would be naive of me to not notice how much interest there is, and how much it has pushed new tech like advanced video cards, new processors like the cell, etc.
In fact, outside the corporate desktop with the emphasis on business apps, it's games that drive all the other desktops, they are one of the holy triumvirate-games, messenging and media playback that home PCs get used for a lot.
Granted, we have consoles for games, but I don't think games on the PC are quite dead yet.
I'll tell ya. I've been using linux for around 12-13 years. over time, there have been quite a few little items that have been a problem. Drivers, video mode, random hardware pieces, etc.
Most recently, it has been the package management. I have been all but forced to use the "commercial" RedHat up at work, and I still cannot believe that Redhat uses a lame package manager that requires you to "solve your own" dependencies. What the shit is that? Anyways, i'm a die-hard debian user, so I am used to just apt-getting every little package, and the dependencies figure themselves out. I could care less that libfuckingrandom from germany could be installed or not, I just want my program to be installed and running. I think one more step is if when I did a 'gimp' if it saw that I didn't have it installed, installed it, and fucking ran it. The newest ubuntu tells you how to install it using apt-get, but really, do it for me. I don't have time to type in another command.
Redhat is another problem. rpm doesn't have the smarts to do anything for you. If you want any kind of 'immediate' commands, you have to 'yum' them. This isn't acceptable in a corporate environment. yum is a bastard that is excluded from RedHat so they can maximize acceptable up2date profits. I could really care less if RedHat goes out business or not. Debian is at least 1 full generation ahead of RedHat. Redhat Enterprise is still redhat 9 with updates.
Anyways, for linux to beat the hype, they need to add the spit and polish that Ubuntu puts into the system. I applaud them for using Debian. Debian isn't perfect, but it is at least usable and their politics should be something people strive for. Freedom should be free.
End of story.
Why read the article when I can just make up a snap judgement?
I've heard from people looking to port ZFS to the BSDs that it is a very difficult port, due to how tangled up it is with the operating system (Solaris) itself, and how much ZFS does for itself (volume managment, RAID, etc).
On one hand, this gives it some serious advantages when run on Solaris 10. But it also makes it difficult to port. I wonder if that is partially responsible for delaying OS X Leopard?
Uh, Unices aren't that easy to hack if you set it to block connection attempts from outside sources that get the root password wrong say..5 times. You can't password crack in that time. Then again, Macs and Ubuntu disable root by default so they have to guess username AND password. So, no, Vista's probably not more secure than Unices. Linux and Vista have the SAME problem. Drivers for both are in limited supply.
look! it's a bird, it's a plane, it's....a girl? yes, a girl browsing Slashdot on Linux
I maintain a Linux file system which is typically used across various kernel versions, including 2.4.x. Yes folk's 2.4.x is still used to ship new products. The changing interface makes for Fun-And-Games.
The VFS to file system is not particularly clean as you need to do pretty ugly things like increment page counts etc within the file system. Much of this is done to enhance performance, but could probably have been done better (ie. preserving a clean interface without real performance compromises).
Engineering is the art of compromise.
The Rise and Fall of Online Community
This hits the nail right on the head - and here's another example:
There has been somewhat of a "battle" (maybe that's a strong word) over support for TOEs (TCP Offload Engines) - or any network stack offload engine - within the kernel.
A TOE would bypass all/most of the network stack to allow sockets to talk directly to hardware [for example] to achieve an array of performance advantages.
The Linux guys will not allow for any support of TOEs in the Kernel. (I/O-AT is not an offload engine!) A summary of primary reasons include "security issues" and "mixture of non-GPL (the card's firmware) into the Kernel", though one could argue you could apply these same issues to *any* card you plug into your PC, that runs its own firmware and can bus-master.
The point being - users of the kernel (end-users, developers, commercial and OEMs) and NIC vendors now have to scramble around to find ways to try to appease the Linux community - comply with GPL - and/or hack around the kernel, stack, hardware and user-apps - all to still get the performance that the Linux stack is sorley lacking.
You can argue the merits of stack-design, layers, etc. - but in the end, the performance isn't there - and the technologies that are actually *available* to make it - but people don't *want* to include them.
This seems to me as more of a political argument than a technical one.
So what's Linux's "competitor's" take on all this? Microsoft's "Chimney" - an API to allow exactly that.
At least someone is thinking "outside-the-box".
Linux supports multiple "personalities" for different types of executable. It has for a very long time. NT made a bigger deal about its implementation of that feature because Microsoft wanted to (a) hurry up and kill OS/2 and (b) make frivolous claims of standards compliance. Even at the time, NT's approach was neither particularly novel nor particularly useful -- how many programs have you ever run that use the OS/2 or the Posix subsystem rather than the Win32 subsystem?
The vast majority of games run on Windows because that is where the market has been. Performance is largely irrelevant in choice of market; gaming performance will follow the money.
Why can one not seriously argue that running large parts of GDI in the kernel was a security problem? New categories of problems cropped up in the wake of that transition. It seems to me that the only basis for saying it was not "a massive introduction of insecurity" is that Windows *already* had rather large security problems.
One legitimate complaint is the poor state of integrated RAID support in the linux LVM. Yes, the LVM can mirror logical volumes now, but it is very klunky (I miss AIX LVM). Creating PVs on top of md devices is tedious and error prone (because you often need to split physical devices into multiple partitions to avoid resyncing the world). The LVM should at least support RAID 0 and 1.
Another complaint is the lack of consistent high level utilities for file system admin. On AIX, one command allocates a logical volume and creates a filesystem on it (or you can do the steps separately for greater control). Linux makes you create the LV, then create the filesystem. Worse, linux used to offer e2fsadm to resize a filesystem and the underlying LV. Now, you have to run lvextend THEN resize2fs - or the reverse if shrinking (and you had better type those sizes right). While it is nice to have access to raw LVs for some things, I really don't want to have to manually compute volume sizes - with potential data destruction in case of mistakes.
Yes, yes, yes. Given NT's connection with VMS I would expect the architecture to be sound and well thought out. Furthermore, I don't think anyone (in this thread at least) has said anything that sounded like "Windows totally devoid of all worth"
That said, the GP does have a point. The problem with NT family operating systems is not that it lacks layers. The problem as I understand it is Microsoft keep making technical decisions driven by political and/or marketing concerns which have the effect of short circuiting the protection that these layers would otherwise offer.
A recent example would be Vista's treatment of install.exe and setup.exe. Both of these trigger UAC elevations, since it is assumed that any .exe with such a filename is an installer. However, recent reports suggest that the UAC elevation is both silent and unstoppable. If true,
this opens an obvious attack vector - a classic case of MS weakening their security model to make things easier for the naive user.
This is the down side of "thinking across layers". Once you accept that it's ok to violate the layer model, then there will always be one more special case. And if your decisions are driven by marketing or by (say) anti-trust concerns, then some of those layer violations are probably not going to be very well thought out from a technical viewpoint.
Don't let THEM immanentize the Eschaton!