Slashdot Mirror
Does Linux "Fail To Think Across Layers?"
John Siracusa writes a brief article at Ars Technica pointing out an exchange between Andrew Morton, a lead developer of the Linux kernel, and a ZFS developer. Morton accused ZFS of being a "rampant layering violation." Siracusa states that this attitude of refusing to think holistically ("across layers") is responsible for all of the current failings of Linux — desktop adoption, user-friendliness, consumer software, and gaming. ZFS is effective because it crosses the lines set by conventional wisdom. Siracusa ultimately believes that the ability to achieve such a break is more likely to emerge within an authoritative, top-down organization than from a grass-roots, fractious community such as Linux.
There is some merit to what Siracusa is saying, at least on gaming and multimedia fronts.
Windows was a hamstrung peformer for graphics until NT 4.0 saw rearchitecture which placed key portions of the OS (including 3rd-party graphics drivers) at a much lower level.
This is like comparing a monarchy with anarchy, without acknowledging that there are in-between solutions that have advantages of their own. Democracy (and representative democracy) being one example.
Not saying the linux development community should be a democracy with everything voted on or whatnot, just saying that there may be creative approaches that have yet to be explored. You'd think smart people with a penchant for game theory would be working on it.
Food for thought.
If someone could elaborate on what ZFS is, I believe our discussion may prove to stay more on-topic.
Personally, I think the Linux kernel manages these issues quite well, if (by conventional standards) rather inefficiently.
The practice, as I see it is: "The current rules (layering, etc.) are enforced rigourously (at least in Linus' tree) but radical rewrites
of the rules take place relatively often"
So if ZFS really does achieve wonderful things by violating the current layering it WON'T be accepted for Linux's kernel, but, if Linus can be convinced (via an appropriate chain of lieutenants, usually) that the layering is really an obstacle to achieving these things, we might see a completely new layering appear in 2.6.25 or somewhere, into which ZFS can fit. The inefficiency
comes from the number of substantial pieces of work that get dropped because they don't fit in, or were misconceived. A more economically rational system would try to kill them sooner. Also, inefficiency arises from the fact that changing the filesystem layering would require every existing filesystem to be rewritten. Linux is notoriously unfazed by this, but in a commercial world, I suspect this would be too hard to swallow and you'd end up with all your filesystems fitting into the model except one, from whence come bugs and code cruft.
Linux will "support gaming" once games are supported for Linux. Linux has OpenGL, OpenAL, all the illusionary walls are market-made. Linux is a platform to build on without the fear of being obsolete in 2 years. DOS games nowadays run on DosBox, as do early Windows games. Even XP needs tweaks to run Win9X games. How is targeting a moving sucky platform preferable to one that is open? Easy. Games sell for 6 months tops. You get the initial sales, you get the money. After that it's tough shit if it won't work after next Windows Update(tm). I have used Linux since 1994, but work in the IT industry. I am constantly amazed by the amount of BULLSHIT the windows folks put up with. For weird quirks "shit happens" is the most common reply.
'Once scientists, even the dim-witted social scientists, get muzzled, the Western Civilization is finished.' - oldhack
Linux and other open source projects are getting a harsh lesson in what it is like to ship consumer grade software products. No more RTFM! No more 'did you submit a bug report???' No more this bug/problem is not our fault since we don't control such and such library we use.
Project vs Product
Everyone is impressed with how far you've progressed when you are working on a project.
Everyone is pissed off with how much you've left undone when you are working on a product.
Welcome to reality open source developers. Before long you will all be saying "Damn, if I have to work this hard to make a consumer grade software product I might as well be getting paid to do so"
http://en.wikipedia.org/wiki/ZFS
1. Fonts, they are simply not as good as Windows.
2. Ease of use. Nobody has sat first time users in front of a linux desktop and watch them puzzle over what those multiple desktops do, or how to switch between them.
3. Basic styling problems. Needless flickery redraws of desktops. Uneven and asymetric layouts, huge icons in some places, tiny icons in others. Isometric icons( a classic sign of a programmer drawing an icon instead of an artist drawning icons).
4. Lack of help, I try to save, it fails, where's the link to the help that tells me that this is a security feature and I can only save in some places.
5. I am not interested in your philosophy, assemble me a bundle of software that fits my needs regardless of whether than software fits your philosophy.
If there is one thing I would suggest, get Ubuntu played with by ordinary grandma's so you can see how they get confused. Then get the Firefox guys to look at it, because to me, it's uneven styling, sometimes big crayola friendly styling aimed at kids, sometimes business like.
Knoppix for example, you start it up and look at the "Windo..." icon and wonder why the fuck they chose suck a large font and such a small icon spacing. So big it can't even display the words 'Windows'.
With Democracies, you end up with the tyranny of the majority, regardless of whether the minority opinion is the correct one. Under a Republic form, a large enough minority can plug up the works and force negotiation with the majority before a final solution is agreed upon.
The Linux Development community needs representative decision making, there are too many voters, hence, almost no direction or real progress towards a cohesive goal. Nothing will change without true leadership, and sadly, accountability.
You cant measure progress without accountability for failure. Socialism has not worked in ANY form, and it wont work for Linux either.
Alternativ approaches to implementing subsystems of the Linux kernel are often developed concurrently, in parallel, and there's a system you can compare to darwinistic evolution that decides (in most cases) which one of a given set of workalikes makes it into the mainline tree in the end. That's why the Linux kernel itself incorporates, or tries to adhere to, a UNIX-like philosophy - make a large system consist of small interchangeable parts that work well together and do one task as close to perfect as possible.
That's why there are so many generic solutions to crucial things - like "md", a subsystem providing RAID-levels for any given blockdevice, or lvm, providing volume management for any given blockdevice. Once those parts are in place, you can easily mingle their functions together - md works very nice on top of lvm, and even so vice versa, since all block devices you "treat" with one of lvm's or md's functions/features, again, result in a block device. You can format one of these blockdevices with a filesystem of choice (even ZFS would be perfectly possible, I suppose), and then incorporate this filesystem by mounting to whereever you happen to feel like it.
There are other concepts deep down in there in the kernel's inner workings that closely resemble this pattern of adaptability, like, for example, the vfs-layer, which defines a set of reuqirements every file-system has to adhere and comply to. This ensures a minimal set of viable functionality for any given filesystem, makes sure those crucial parts of the code are well-tested and optimized (since everyone _has_ to use them), and also makes it easier to implement new ideas (or filesystems, in this sepcific case).
Now, zfs provides at least two of those already existing and very well working facilites, namely md and lvm, completely on its own. That's what's called "code-duplication" (or rather "feature-duplication" - I suppose that's more appropriate here), and it's generally known as a bad thing.
I do notice that zfs happens to be very well-engineered, but this somewhat monolithic architecture still bears the probability of failure: suppose there's a crucial flaw found somewhere deep down in this complex system zfs inevitably is - chances are you've got to overhaul all of its interconnecting parts massivley.
Suppose there's a filesystem developed in the future that's even better than zfs, or at least better suited to given tasks or workloads - wouldn't it be a shame if it had to implement mirroring, striping and volume-management again on its own?
Take an approach like md and lvm, and that's not even worth wasting a single thought on. The systems are already there, and they're working fantastically (I'm an avid user of md and lvm for years by now, and I frankly cannot imagine anything doing these jobs noticeably better). I'd say that this system of interchangeable functional equivalents, and the philosophy of "one tool doing one job" is absolutely ideal for a distributed development model like Linux'.
It seems to be working since the early nineties. There must be something right about it, I suppose.
:%s/Open Source/Free Software/g
YTARY!
Linux kernel development is an absolute, shameful ad-hoc mess. This is typified by the fact that the kernel devs are dogmatic about refusing to chose a stable abi and stick to it.
In short, Linux wants all the 'perks' of a professional enterprise OS, without accepting any of the responsibilities (for example, stable interfaces which third party developers are able to count on developing in the next version).
Far too many people take linux seriously in the professional world, far more than it deserves. Linux has no direction, no goals, no compatibility (I can run solaris 2.4 binaries on opensolaris; you can't even run FC3 binaries on FC4!) and, in all honesty; half the time the fucking kernels don't even work (this is particularly true of the 2.6 series)!
Linux is a pile of crap; sadly it's a case of 'people adopt the worst technologies'; there are far better systems out there (shit, other than osx or windows any other system would be better); but it's linux which has the mindshare
No wonder our IT industry is complete shit.
It's not just Andrew Morton, it's basically every core linux kernel hacker that has spoken on the issue.
It's pretty obvious; I don't think that even the ZFS developers will deny it. They'll just say "it's a layering that was worth breaking".
Reiser4 introduced us to all sorts of interesting capabilities never before seen in a file system (at the time) but I believe this same "layering violation" attitude pretty much put a stop to any of it getting into the kernel. The Reiser guys were forced to pretty much cripple their file system feature wise if they were to have any hope of getting it included in the kernel.
See Reiser4 Pseudo Files as one example.
I can understand that in certain cases "layering violations" are bad, but Linux kernel developers don't even seem to be willing to experiment or think outside the box at all.
Both sides have valid arguments... I don't think there is any easy solution, but it would be nice to see more forward thinking in the community.
Open Source Time and Attendance, Job Costing a
Linux as a cli is robust, mature. I work every day via ssh or terminal and I manage a number of servers this way, it's a pleasure.
But when I look at Linux as a viable desktop alternative for the non compsci crowd I tend to cringe. The patchwork that can make Linux so flexible, that *really* puts *you* in charge is the exact thing that makes Linux so unfriendly. Most people don't want tonnes of choice, not because their stupid, but because they don't want to spend a lot of time fussing with their computer. They want one way to do things and they want it to be well thought out and seamless. You still can't get that with a Linux distro. Instead you get choice and/or pieces of the patchwork.
Quack, quack.
. How is targeting a moving sucky platform preferable to one that is open?
The moving sucky one has ninety plus percent of the home desktop market. Linux has less than one percent, and I've never seen any credible figures suggesting otherwise. Why target a tiny niche market when you can target a huge one?
And bear in mind, the proportion of linux users who are serious about gaming and do not have access to a windows machine is probably one percent of Linux users. So even if you target windows, ninety nine percent of Linux gamers can play your games anyway.
"I realise this is not a very popular opinion but it's the truth, and there for needs to be said" -Bill Hicks
I don't know anything about ZFS, but I think his general point may have merit. Consider the problem of speeding up the boot process. This would require interaction between desktop hackers, init hackers, filesystem hackers, etc. etc. Many possible speedups might require layering violations (desktop application making requests about desired file layout on the disk etc.) Due to the technical, social, political structure of Linux this is just unlikely to happen (unless a single distro has enough resources to throw at this particular problem). Consider this rant (that may be too strong a word) by KDE developer Lubos Lunak: Why does Linux need defragmenting?
From Sun:
"If you're willing to take on the entire software stack, there's a lot of innovation possible."
Jeff Bonwick
Distinguished Engineer
Chief Architect of ZFS
Sun Microsystems, Inc.
You mean like DragonFly BSD, where they're already porting it. More and more I love that project.
Layers are both easier to code, to understand, and to test. Layers/boundaries between software are your friend. To some degree that is why the Internet, based upon a layered network model (TCP on top of IP on top of Ethernet) is so diverse.
Layering is what keeps things manageable. One you start getting your software tentacles into several layers you make a mess of things for both yourself and others. Its a tradeoff--complexity/speed vs simplicity/maintainability/interoperability.
It's really hard to argue with the claim that Linux is a fundamentally flawed failure of an operating system. It's a nice free tech toy, sure, but when it comes to being an accepted and realistic product, there are a great many reasons to look elsewhere.
Efforts like Ubuntu, while admirable, are really just polishing a turd. No matter how much paint you slap on it, it's basic nature will not change.
Someone has to come up with an alternative to Lunis's "work on the kernel and let everything else go to hell" development strategy. Redhat or SUSE could have done so much, but ultimately failed once they started pandering to "the community".
Linux is like sending a herd of Roombas into the Louvre and hoping that statistically, most of the floor is covered. They need somebody to run the whole thing and say stuff like, "USB drivers, get your act together. Now!" "We're using Qt. GNOME, shut up." "This is what you're gonna use for HTML rendering. Tough luck." "You guys have five days to make a decent UI before I firebomb your house."
Open source software gets better because new people want new features to which they contribute. You can't blame Andrew Morton for disliking what ZFS is going to do, this is just how people work. This is why they say you can't teach an old dog new tricks.
That said ZFS is one of the coolest things to happen to your files in a long time. The current disk block device usage is basically the same from the beginning of computing, it is ancient and actually quite stupid. Over decades layers keep getting added to it to make it more robust, but really it's a monstrosity. Partitions are dumb, LVM is dumb, disk block RAID is dumb, monolithic filesystems are dumb. All the current linux filesystems should be thrown out.
I don't want to care how big my partitions are, what level parity protection my disks have, or any of that junk. I want to add or remove storage hardware whenever I want, and I want my files bit-exact, and I want to choose at will for each file what the speed vs protection from hardware failure is. Why shouldn't one file be mirrored, the next be stripped, and the next have parity or double parity protection? Why can't very, very important files have two or three mirrors?
From the current status of ZFS however I think this could be quickly built using GPL 2+ by one or two determined people, and it would involve gutting the linux file systems.
"I don't know that atheists should be considered citizens, nor should they be considered patriots." George HW Bush
" Siracusa ultimately believes that the ability to achieve such a break is more likely to emerge within an authoritative, top-down organization than from a grass-roots, fractious community such as Linux"
ie. Apple
Layers might not be ideal, but they're consistent. The filesystem does its part, RAID/LVM does its own, etc.
ZFS seems to want to take all over the disk subsystem. Why? Is there a reason why it needs its own snapshot capabilities, instead of just using LVM?
These sorts of things always smell fishy to me, due to a feeling that once you start using it, it locks you in more and more until you're doing it all in this new wonderful way that's incompatible with everything else. Even though it's open source, it's still inconvenient.
This approach reminds me a lot of DJB's software: If you try to get djbdns you'll be also strongly suggested to use daemontools as well. The resulting system is rather unlike anything else, and a reason why many people avoid DJB's software.
Please remove yourself from the internet.
"God deliver us from our friends, we can handle the enemy." -Patton
Lovely biting sarcasm aside, to be honest, our storage layering in Linux leaves much to be desired. As witness the slow pace of improvement of the volume manager in recent years. This does not prove that layering is bad, but it suggests that our current conception of layering sucks pretty badly. For example, we are burdened with a ridiculously complex interface between application programs and kernel-level volume management support. Managed volumes live off in their own name space. Why can't I say "/dev/hda, you are now snapshotted, shazam"?. No, instead I have change my system over to use /dev/mapper/snapshotted-hda or some such nonsense. Similarly, we are unable to manage all block devices using the same administration interface. No higher level raid integrated with the volume manager, instead this is a separate subsystem that fights a lot with LVM. Partition support hopelessly misfactored and broken. It goes on and on. Nothing unfixable but lots of unfixed brokenness. Compared to this mess, Sun's massive layering violations seem like a breath of fresh air.
But the thing to do is fix our broken implementation of layering and not be fooled into thinking that layers are bad. What is bad is exactly as the author here claims: it is bad to have no powerful capability to cross layer boundaries so that applications see a simple, powerful model instead of the current situation, where one's face is constantly rubbed in the minutae of layering administrivia. ZFS actually has layering, it just bypasses some traditional Unix subsystems and takes care of the functionality itself. But is wrong to conclude that this must therefore be the optimal approach just because it improves on the mess that preceded it. If ZFS internal interfaces are worth using, then they are worth using as core interfaces, not ZFS-only interfaces. Translated into Linux terms, the implication is that it is high time to get busy and rectify some of the serious deficiencies in our storage model. Not by mashing all the layers together, but by teaching them how to get along more efficiently and powerfully, and not be so layered that important things don't even work.
Note: perhaps the biggest design distinction between Linux and other Unixen is that, internally Linux is all just one big flat function space where anything can call anything else and share any data. This is said to be a reason why Linux is more efficient than, say, the Mach kernel with its microkernel layering. If being all one big hairball of functions is good for memory management, vfs, scheduling and so on, then why is it not also good for volume management? I don't know the answer to this, but I do know that we have plenty of bogus layering in our storage stack that has really slowed progress in recent years and needs a good dunging out. Any nonbogus layering can stay.
Have you got your LWN subscription yet?
UNIX and Linux design is quite holistic: features are often added at various levels of the system in order to make a whole work out. For example, desktop search support had both user and kernel space components, Beryl/Compiz-style interfaces have triggered changes in Gnome, X11, and the kernel, etc.
UNIX and Linux have been careful about avoiding simplistic designs. ZFS is a simple, obvious answer to a problem: just pack all the functionality into one big codeball and start hacking. Microsoft does a lot of the same thing in Windows and Apple in OS X. It gives companies great time-to-market and long, impressive feature lists. It's also creates a mess. Microsoft, Apple, and Sun each have several iterations under their belts where they start off like that, then the system bloats, and finally collapses, causing the company to start over again from scratch or just abandon the market altogether.
Thanks, we know about the kind of "holistic" that these people are implementing, and we don't want it.
(And, frankly, I think Sun isn't really a UNIX company anymore; their system may still be UNIX compatible, but they stopped following the UNIX philosophy long ago.)
>
Without being able to reach across layer, linux will never be able to accept viruii/worms like the other well known OS and will therefore remain a niche player. If your browser can infect the kernel the guys in RU just ain't gonna let it happen.
But the OSI layers are guidelines that help design things, not rigid levels that must be maintained. They are mixed up ALL the time. As a simple example, see Layer-3 switches. These are devices that simultaneously work at Layer 2 and 3 when it comes to dealing with traffic. They break down the traditional view of a separate router and switch, and they are good at what they do. There's plenty of stuff at the high end that's similar. Many things that are part of the presentation layer are actually done by the application (like encryption in SSH) and so on.
There's nothing wrong with having a layered design philosophy as it can help people decide what their product needs to do, and what it needs to talk to. For example if I am designing an application that works over TCP/IP, I really don't need to worry about anything under layer 4 or 5. However it shouldn't be this rigid thing that each layer must remain separate, and anything that combines them is bad. I don't need to, and shouldn't, take the idea that my app can't do anything that would technically be Layer 6 itself. Likewise in other situations I might find that TCP just doesn't work and I need to use UDP instead, but still have a session which I handle in my own code (games often do this).
Had we stuck to the OSI model as a maximum, rather than a guiding principle, with the Internet, it probably wouldn't have scaled to the point we have now.
With a name like Siracusa, one might think so.
"I mean, although ZFS is a rampant layering violation and we can do a lot of
the things in there (without doing it all in the fs!) I don't think we can
do all of it." http://lkml.org/lkml/2006/6/9/409
It sounds like his main point was pointing out problems with the current file system, rather than saying ZFS is bad. I bet he simply thinks they should try to implement a much better file system than ext3 without breaking the current layering scheme. I don't see why this is so bad. Why not try it, and if it fails miserably, ZFS is already here.
I think the author of the article took everything out of context and was just looking for some ammo against Linux. His blog post sucked. He just says the same crap that everyone always says. I'm not saying there are no problems, but I don't see how any of the problems relate to Andrew Morton saying the Linux file systems need to be upgraded/replaced.
You have no clue at all what this article is about.
Try Ubuntu/Kubuntu.
They don't strip away your choices, but they sure dumb them down so unless you really want to - you are not aware of having to make any decision.
Linux is ready for the desktop - and it is already in many desktops.
Many people don't use it not because its not ready, but because they don't know how to burn a CD, how to boot from it, what "installing an OS" means, and because they are afraid. Not because Linux "is not ready" and any of that nonsense.
Its not 1998. Try Ubuntu/Kubuntu out.
You don't need to know anything to get those running.
Their installation is far easier than the Windows installation, and most of the common things people do "just work". Those rare occasions that don't "just work" have very simple step-by-step howto's all over the place.
The problem with a "traditional" layered model is that the file system has to assume that the underlying storage device is a single consistent unit of storage, where a single write either succeeds, or it fails (in which case the data you wrote may or may not have been written). This all sounds very good and file systems like ext2 are written based on this assumtion.
However, if the underlying storage system is RAID5, and there is a power loss during the write, the entire stripe can become corrupt (read the Wikipedia article on the subject for more information). The file system can't solve this problem because it has no knowledge about the underlying storage stucture.
ZFS solves this problem in two ways, both of which reuires the storage model to be part of the filesystem:
- Each physical write never overwrites "live" data on the disk. It writes the stripe to a new location, and once it's been completely committed to disk the old data is marked as free.
- ZFS uses variable stripe width, so that it does not have to write larger stripes than nescessary. In other words, a large write can be directly translated to a write to a large stripe on the sotrage system, and a smaller write can use a smaller stripe width. This can improve performance since it can reduce the amount of data written.
There are plenty of other areas where this integration is needed, including snapshotting, but I hope the above explanation explains that the layered model is not always good....you look at the grease that makes the world go round, and that is money, and games make lotsa money and garner a huge interest with hundreds of millions of people. Moolah, loot, it's there for the taking.
Personally, I don't game, but it would be naive of me to not notice how much interest there is, and how much it has pushed new tech like advanced video cards, new processors like the cell, etc.
In fact, outside the corporate desktop with the emphasis on business apps, it's games that drive all the other desktops, they are one of the holy triumvirate-games, messenging and media playback that home PCs get used for a lot.
Granted, we have consoles for games, but I don't think games on the PC are quite dead yet.
Nothing stops an "authoritative, top-down organization" from taking all the open-source work done on Linux, and applying its own methodology to driving it forward; if that's more effective than what everyone else in the Linux community is doing, users will be more interested in adopting what they do with it (and, heck, once the transition occurs, the less-centralized portions of the community will probably follow along and start working on the "Neo-Linux" thus produced.)
Its true that revolutionary, rather than evolutionary, change is probably best driven by a narrow committed group with a shared vision and the skills to realize than a disorganized community. But there is no barrier to that within Linux; and between the occasional revolutionary changes, the evolutionary changes that the community is very good at will still remain important. With open source, you don't have to choose: you can have a top-down narrow group working on revolutionary changes (you can have many of them working on different competing visions of revolutionary changes, which, given the risk involved in revolutionary change, is a good thing), all while the community at large continues plugging away on evolutionary changes to the base system—and if once one of the revolutionary variants attracts attention, begins working on evolutionary improvements to that, too.
while i was waiting for the year of linux on the desktop, ZFS has actually taken over on the desktop instead?
did i miss something?
In Soviet Russia, Layers think of Linux!
It has the ability to strip it's self across more than one disk to make it pleasant for when migrating live systems to a new set of storage devices. /me also thinks Linux devs would do good to look into porting GEOM from FreeBSD to Linux.
I'll tell ya. I've been using linux for around 12-13 years. over time, there have been quite a few little items that have been a problem. Drivers, video mode, random hardware pieces, etc.
Most recently, it has been the package management. I have been all but forced to use the "commercial" RedHat up at work, and I still cannot believe that Redhat uses a lame package manager that requires you to "solve your own" dependencies. What the shit is that? Anyways, i'm a die-hard debian user, so I am used to just apt-getting every little package, and the dependencies figure themselves out. I could care less that libfuckingrandom from germany could be installed or not, I just want my program to be installed and running. I think one more step is if when I did a 'gimp' if it saw that I didn't have it installed, installed it, and fucking ran it. The newest ubuntu tells you how to install it using apt-get, but really, do it for me. I don't have time to type in another command.
Redhat is another problem. rpm doesn't have the smarts to do anything for you. If you want any kind of 'immediate' commands, you have to 'yum' them. This isn't acceptable in a corporate environment. yum is a bastard that is excluded from RedHat so they can maximize acceptable up2date profits. I could really care less if RedHat goes out business or not. Debian is at least 1 full generation ahead of RedHat. Redhat Enterprise is still redhat 9 with updates.
Anyways, for linux to beat the hype, they need to add the spit and polish that Ubuntu puts into the system. I applaud them for using Debian. Debian isn't perfect, but it is at least usable and their politics should be something people strive for. Freedom should be free.
End of story.
Why read the article when I can just make up a snap judgement?
ZFS or atleast a somewhat similar filesystem is the way we're heading sooner or later. *ALOT* of people are interested or planning to migrate into (Open)Solaris just to get ZFS, myself included. Even my friends that are casual users who don't care that much about Linux and are content about their Windows or whatever, have demonstrated great interest in ZFS after seeing the demos and reading about it. It is unfortunate how so many of the kernel people seem to reject ZFS, albeit there are some licensing issues to sort out too. You can test ZFS by running it on FUSE, but that's alpha status.
Why stop at violating layering? We should reach across the subsystems and modules! We could have every part of the system involved in performing every system task! Once no one understands how our system works anymore, we could mark it as proprietary and have our own monopoly on GNU+linux maintenance!
So am I to assume that ZFS a .. er.. file system has high desktop adoption, user-friendliness, and good gaming support?
Let me make a quick address to why Linux has trouble in those areas, at least at this point. Some of the /.'ers hit the nail on the head, but mumbling something about software layers being independent doesn't make it valid criticism. This whole internet thing runs on layered TCP/IP software.. seems to be working well so far.
So yea. The problem with individual layers of the software not working is just that. If the software written to edit something or work as a middle layer between something doesn't work well, that's because no one's working on it enough, not because layering is flawed. The person working on some adjunct in the kernel isn't suddenly going to start coding you up a nice GUI to access their "layer" of software because their isn't one. Chances are, *they* don't need it, or give a rats arse about it.
There might be problems with particular soft spots in the software stack on Linux.. but it's because they're soft spots. They wouldn't be solved by the programmer working on the underlying layer stopping. The whole argument in the context of a ZFS in-kernel stack having too much "stuff" in it is fallacious. Like pointing and yelling about someone's hair when they're making a point on world politics.
You're reading Slashdot. Of course you like Linux and pc hardware
That sounds like the wrong way to solve the problem. I'd rather solve it in the RAID layer, so that anything using the disk can take advantage.
For instance, vmware can use a block device for a virtual disk. Why should it have to implement its own version of that when it could be implemented in a single place?
Same goes for snapshotting. Why does the FS have to support it? With LVM I can snapshot vfat if need to for whatever reason.
I've heard from people looking to port ZFS to the BSDs that it is a very difficult port, due to how tangled up it is with the operating system (Solaris) itself, and how much ZFS does for itself (volume managment, RAID, etc).
On one hand, this gives it some serious advantages when run on Solaris 10. But it also makes it difficult to port. I wonder if that is partially responsible for delaying OS X Leopard?
Siracusa states that this attitude of refusing to think holistically ("across layers") is responsible for all of the current failings of Linux
In related news, a lack of synergy was responsible for all of the bankruptcies at the end of the bubble.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
You don't seem to understand snapshots.
A snapshot works by creating a copy of the device, with the contents it had when the snapshot was created. If you make a snapshot of
Why would you change anything over? Snapshots are temporary. You snapshot your drive, use the snapshot to create a consistent backup (or whatever), then destroy it.
Normally you won't keep a snapshot around for long, as they're maintained by keeping copies of modified blocks, and that takes space. Unless you have enough space for fully duplicating the device you made a snapshot of, you won't be able to keep it around forever.
Maybe my entire view of things is wrong, but isn't strict layering responsible for most of the things that set Linux apart from less-efficient and less-secure operating systems? Isn't layering what allows the same exact operating system be a high-performance server or a normal work station? I point out the latter because without layering, things like Windows can "disable" a lot of things that are all tied together, but never really remove many things completely since ... whatever.
I guess what I'm saying/asking can be summarized: isn't layering one of those really-important things to Linux that shouldn't be violated? Reading the summary sounds like "In order to make Linux more Windows-like, we need to violate the things that makes Linux much more of a proper layered OS than Windows is."
The Windows installer also talks about partitions. It asks you to [c]reate, [r]emove, or format a partition.
It actually offers more options with more terminology than the Ubuntu/Kubuntu installer.
The Ubuntu installer offers you install options:
"Simple - use free space"
"Simple - overwrite whole disk"
"Advanced - Setup your own partition table"
Ofcourse most users can choose one of the simple options. The advanced one has a nice GUI to resize partitions and basically do everything from a GUI.
In Windows its a bit more complicated than that, as explained above.
So nice try, but even partition-wise, Ubuntu/Kubuntu have easier installers.
Snapshots in the filesystem itself make much more sense than the LVM. For one thing, the filesystem can be designed to have this functionality in mind and can be much more efficient about it rather than LVM which just takes a generic snapshot approach.
Having used snapshots where I work via a NetAPP server I can attest at their usefulness. I just go to ~/.snapshot and go back in hours, days, weeks or months.
With ZFS it can be fully automatic so if, say, I'm editing a file several times and screw it up, I can easily go back to a previous snapshot, and the snapshot could even be from a few seconds back. Since the filesystem is aware of snapshotting, it can be faster and more efficient and more flexible than what LVM can do.
It could almost be like an automatic SVN where I can automatically, for example, go to the state of the filesystem on April 28, 11:42:13am. I believe (I may be wrong here) that LVM requires the user to specify when a snapshot should take place.
I'm not sure that ZFS is the end-all be-all filesystem for Linux, but Linux has a long way to go before it can compete with filesystems like what NetAPP offers.
Another problem with LVM that could be solved with a filesystem is shrinking a logical volume. If I want to remove a drive I can notify the filesystem and since it knows what files are on that drive it can move them off of it. Or if you have one drive that's a lot faster than the others you could tell the file system which files are performance critical so it will place those files on that drive and put non-critical files on other drives.
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
for tasks it does well (serving files, pages, etc). Linux lacks games, fun content, refinement, and apps. I fails the common user's needs.
You hit the nail on the head: 'something being worked on' (project) is NOT the same as 'something ready for use' (product). Note to mods: parent deserves credit for making this point.
However, there is a large overlap between 'project' and 'product'. In-development-systems can be quite useful, and when are products really 'done'? Look around in your average household store. Many simple items (like paperclips) may have evolved, but their basic form is the same as the 1st day they where conceived. Read: the very first incarnation of a paperclip may have been rude, but no less useful than the matured, modern-day version. For complex products: try and find one where every aspect of its use, from production, marketing, distribution, everyday use, to how it is discarded/recycled, is well-thought through and 'just works'. Good luck.
Basically: things that are both complex and well-engineered (in every aspect) are rare. 'Project' or 'Product' is not the same, but only a label. Which one applies, depends on your point of view.
Take some projects, add packaging, marketing, support, ask money for the whole, and voila: a product. See IBM, Red Hat, SUSE for example. Whether a Linux kernel or a Perl binary shipped by them is a project or a product, only depends on how you use it. If you're a developer working on it, it's your project. When you're selling DVD's that include these in bulk, the same thing becomes a product.
I've used Linux for different purposes starting around '94/'95 (normal desktop use these days, Windows is history for me), and I can assure you: it far exceeds what one might call 'consumer grade product'. That Linux has maybe around 1% market share among ordinary PC users, has nothing to do with quality or technical limitations, but everything with marketing, industry inertia and historical reasons.
Same old anti-Linux FUD decorated with made-up existential-sounding gibberish, turned in at a minute before deadline so the columnist could dash out the door for the weekend. Excuse to dangle the words "failings, desktop, user-friendliness, consumer, and gaming" in front of your eyes. Author's probably laughing his ass off: "I BSed that out in ten seconds and they posted it on Slashdot!"
He's either a troll or an idiot. Either way, he doesn't deserve your replies.
vi ~/.emacs # I'm probably going to Hell for this.
I think you'll find that it is you that doesn't understand what a snapshot could be. Take a look at ZFS, try it, and see if you think of snapshots the same way again. In ZFS, a snapshot can be promoted to a clone, which is a writeable copy of the original filesystem, sharing unmodified blocks using a copy-on-write algorithm.
This is increadibly powerful and useful. For example, a single master 'image' volume can have customizations added for specific purposes. This is useful in desktop deployment, iSCSI or NFS network boot, etc...
Would you expect a 'first class' writeable clone to have a name like 'dev/mapper/snapshotted-hda' or 'dev/hda.1'? Which one makes more sense? Why would the original have a special name, when the clone is identical?
It's this kind of narrow 'snapshots are throwaway' thinking that causes artifical limitations in APIs and operating system design that serve no real purpose.
I maintain a Linux file system which is typically used across various kernel versions, including 2.4.x. Yes folk's 2.4.x is still used to ship new products. The changing interface makes for Fun-And-Games.
The VFS to file system is not particularly clean as you need to do pretty ugly things like increment page counts etc within the file system. Much of this is done to enhance performance, but could probably have been done better (ie. preserving a clean interface without real performance compromises).
Engineering is the art of compromise.
Ok, if I understand this correctly, then LVM and ZFS snapshots have different uses.
For me, the main point of LVM snapshots is consistent backups. LVM can freeze any block device in time. The nice thing is that it doesn't have to be a known filesystem. It could be for example a device used by vmware, or a database as-is, with no filesystem on it. A neat application of this is data recovery. LVM can take a writable snapshot of a corrupted filesystem. Then you can try the latest experimental fsck tool on it, without worrying about it breaking something, and not needing to keep a full copy of the data.
ZFS is certainly really neat, but it can't fully replace LVM snapshots, nor LVM can provide what ZFS can.
a reason why many people avoid DJB's software.
Why, it's the second most popular MTA on the Internet! Surely you are joking.
Don't piss off The Angry Economist
LVM has this already. CONFIG_DM_SNAPSHOT in the kernel config.
If you use LVM, then all devices you put a filesystem on are in
But they aren't identical. LVM works with block devices, it doesn't know about the filesystem. If you do a bit-by-bit comparison of the original device with its snapshot, if the original changed, then there will be differences. The snapshot contains the data it would if you unmounted the FS and make a copy of the device.
I hope the above explanation explains that the layered model is not always good.
Or simply that they've just drawn the lines in the wrong places this time.
The direct quote is "I've long seen the Linux community's inability to design, plan, and act in a holistic manner as its greatest weakness."
You can see the meaning has been completely changed in the summary from one of positive criticism to one of arrogant condemnation.
Through this change, we can see the posters true feelings, feelings that are shared by many in the Linux community. That is to respond immaturely and get all bent out of shape if somebody builds anything that doesn't follow the "Linux philosophy".
The Truth. Both Linux in general, and ZFS are amazing, and powerful tool. One of best philosophy I've encountered is "use the right tool for the job".
Nobody is forcing Linux devs to port ZFS, or even use, or even think about it. The only reason this is an issue, is because many in the Linux community realize how powerful ZFS is, and they're subconsciously pissed off that they can't have it. So they respond like a 3rd grade bully by attacking it in a self defeating attempt to minimize its importance.
You don't seem to understand snapshots
:-)
/dev/hda at 12:15, then you'll get /dev/mapper/snapshotted-hda as it was at 12:15, while /dev/hda will continue being possible to modify... Why would you change anything over?
/dev/hda directly when it is snapshotted. You must access /dev/hda through some other device and that some other device must located in the /dev/mapper directory. No wonder you apparently mixed up what is a snapshot and what is being snapshotted - the way we currently do this in Linux is quite unnatural and is a wide open invitation to such confusion, not to mention a pointless makework project for system administrators.
If you say so
A snapshot works by creating a copy of the device, with the contents it had when the snapshot was created. If you make a snapshot of
Because with the incumbent volume management strategy you may not continue to use
Have you got your LWN subscription yet?
Second? A bit of googling suggests qmail was third in 2003: http://www.credentia.cc/research/surveys/smtp/2003 04/0 701/mxsurvey.html
And this survey suggests it's falled to 22nd in 2007: http://www.securityspace.com/s_survey/data/man.20
qmail seems to have been abandoned by DJB, and his licensing requirements mean other people can't fork development, they can only distribute patches for it.
No, you may not continue to use
If you're using LVM, you don't mount
Can we get him to take this dupe with him when he leaves?
Sort of make sense to start hiding the hardware details of the storage devices. Most users don't know and don't care other than when they try to do something and get told, "insufficient space" or a disk crashes and their data is gone. A file system that prevents/hides such nasties will probably appeal to people who want a computing appliance (most of the world).
Cheers,
dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
You can't snapshot anything that isn't a LVM logical volume
True, and what kind of sense does that make? It is purely an artifact of the incumbent low level LVM model. Please go back and read the original post and notice how you misinterpreted it.
Have you got your LWN subscription yet?
Ok, under LVM, /dev/hda is a physical volume. PVs get grouped into volume groups, then get subdivided into logical volumes.
/dev/hda (which maps to a single hardware device) because nothing stops you from creating a logical volume spanning 4 hard disks. What exactly would a snapshot of /dev/hda in that case be? As far as LVM is concerned, your data is somewhere in volume group, which is a virtual entity made of a number of physical volumes. Your logical volume may span one or several PVs, and doesn't have a fixed location either. You can at any time tell LVM to free up a physical volume and disconnect it.
It doesn't make sense to talk of a snapshot of
I'd love to used Linux as a desktop. I just need a little help.
I even enjoy spending time tweaking my desktop computer, from back in the days when memory came in 16k chips, IRQs had to be tediously managed, and squeezing every drop out of 640k was fun. But try as I might I have yet to get a stable, visually appealing, or useful version of linux on any of my previous 3 computers. Why? Because I can't even get a minimally functional system running, and give up before I get to the tweaking stage.
Major problems I encountered which I spend more than 1/2 an hour working on each: picking a distro, much harder than you think for the non-initiate. KDE vs Gnome? Utterly crappy (ie Mac 6) video support without special do-it-get-it-complile-it yourself drivers. Can't install video drivers, I didn't install gcc (silly me). Can't install video drivers, I'm missing some contingencies. Can't install video drivers, I didn't install the source code for the kernel (silly me). Multiple conflicting versions of drivers and conflicting advice about which one to use. Multiple conflicting instructions on how to install said video drivers. Video driver installer has reams of text output, some of which are error messages. Based on more advice, appearantly these error messages may or may not be normal and may or may not be why I never got good video output. My sound card stopped working. I still don't know why.
Valuing my time at a paltry $50 an hour, I could have easily bought a newer better system with WinXP on it and then taken my wife out to dinner with the remainder.
If anyone can recommend a distro that will run, out-of-box, on my Dell e1505 with an ATI x1400 graphics card and Creative Audigy soundcard, then I promise you I will excitedly hunt it down and intall it, I really do want to switch to linux, the visuals I've seen other users have is incredible.
Unfortunately the fact that I have to ask such a question really shows how linux in general is completly unprepared for the desktop market. Prove me wrong and recommend a distro.
PS - please, no berating, calling-of-noob'ing, or general fun making at my expense. I really honestly do want help, and Linux people have tried to help me in these ways before. (they haven't proven helpful yet)
companies would be smart to make CDs or DVDs that boot into the game.
I prefer the "u" in honour as it seems to be missing these days.
I raised the same question to ESR at Linucon in 2006. There are certain advantages to working with a large organization. One of those advantages is to apply resources to accomplishing unpleasant tasks. Tasks such as fixing unpleasant bugs and fixing performance issues. Much more attention will get paid to very unpleasant problems if an organization attaches social and financial capitol with finding a solution.
They use C, a language from the same period, not C++.
Thank God for that. C++ is an abomination. It's not good at OO, it's not strictly procedural. Hell, it's not even clean.
They use an interface that literally emulates an ancient teletype.
Hey! Don't talk about GNOME like that!
Microsoft is to software what Budweiser is to beer.
This hits the nail right on the head - and here's another example:
There has been somewhat of a "battle" (maybe that's a strong word) over support for TOEs (TCP Offload Engines) - or any network stack offload engine - within the kernel.
A TOE would bypass all/most of the network stack to allow sockets to talk directly to hardware [for example] to achieve an array of performance advantages.
The Linux guys will not allow for any support of TOEs in the Kernel. (I/O-AT is not an offload engine!) A summary of primary reasons include "security issues" and "mixture of non-GPL (the card's firmware) into the Kernel", though one could argue you could apply these same issues to *any* card you plug into your PC, that runs its own firmware and can bus-master.
The point being - users of the kernel (end-users, developers, commercial and OEMs) and NIC vendors now have to scramble around to find ways to try to appease the Linux community - comply with GPL - and/or hack around the kernel, stack, hardware and user-apps - all to still get the performance that the Linux stack is sorley lacking.
You can argue the merits of stack-design, layers, etc. - but in the end, the performance isn't there - and the technologies that are actually *available* to make it - but people don't *want* to include them.
This seems to me as more of a political argument than a technical one.
So what's Linux's "competitor's" take on all this? Microsoft's "Chimney" - an API to allow exactly that.
At least someone is thinking "outside-the-box".
Because the first thing you thought of when you read my tag line was to vehemently defend Linux against any criticisms. 1.) So I can ping from the GUI but I can't even install a program such as a Firefox nightly build unless I go skinny dipping in the command console? 2.) Why is Linux forcing me to use the AC97 onboard audio in place of my Creative Sound Blaster Audigy 2 ZS Platinum audio card... and then gives me no GUI option to switch from between the devices across the several flavors of Linux I've tried out? 3.) Consistency of design, example: key combinations need to work include Win+D, CTRL+ESC, ALT+TAB, CTRL+ALT+DEL, among many others. Kudos to Xandros for making headway on this. 4.) "YAST" is a Linux developer's preferred way of referring to something as a control panel. If Linux were to replace Windows normal users would be looking for their "preferences", "settings", or "options". 5.) Get rid of useless junk: virtual memory (and a forced virtual memory partition on top of that). There are fifty programs to do a task such as FTP, but not catagory called "FTP" so how would these programs see the light of day by normal users? If you disagree you've probably got a high end processor (instead of getting a slower one and then overclocking) but only have half the physical memory you actually use. 6.) Stop turning my hard drive in to swiss cheese. One drive one partition. It's not just about the code, it's what you do with the code (or don't).
- John
http://www.jabcreations.com/
If you're so unsatisfied with the current Linux filesystem offerings, develop something better yourself.
You use Linux and you criticize what you have to use concerning the filesystem. Considering all the effort already placed in all the different alternatives you have concerning filesystems with linux, all I see in this thread are people that are ingrateful. Can't you just appreciate the work and thank God all these filesytem alternatives exist for Linux?
The fact remains there are many choices here: fat, ntfs, ext2, ext3, reiser, ext4, and others if I choose to look harder.
You talk about other kinds of filesystems that version the entire filesytem in a snapshot. Interesting and all for some, but most people don't have overkill storage space to afford this luxury for the time being let alone having enough computers for all households in all countries. OLPC(one laptop per child) is an admirable attempt for helping kids learn about computers all over the planet. That is a great priority to have. Why don't criticize the fact that there are still kids on the planet that don't have computers instead of critizing the lack of versioning filesystem api's in Linux?
Sure Linux kernel/filesystem developers have a thick skin and take criticism, but I figured it would be nice to see someone give praise for all the efforts the Linux kernel/filesystem developers have made instead of simply telling them "It is hightime they fix stuff in blah module" in an ingrateful manner.
Most of all, to me, I am astonished that almost everyone talks 'virtualisation', VM, QEMU, Xen.
When it comes to filesystems, suddenly many seem to want to do everything on their own, on physical platters: partition, volumes/RAID, format. ZFS is a virtual filesystem, where none of such is physically needed. There is a nice http://www.opensolaris.org/os/community/zfs/demos
Of course, filesystem should be a black box, an object, instead of the user having to do low-level work. ZFS provides this, and more relevant: of course it needs to be cross-layered therefore.
Snapshots ought to be available easily, at any moment in time, without taking much space. ZFS does so, by only storing the changes and sharing the unmodified data. If you want to do so, you need an abstraction of the hardware. That is, crossing layers. Not to mention writeable snapshots.
Adding new drives without partitioning, slicing, formatting. Just adding to the existing pool. Inclusive striping being adapted automagically. This needs a cross-layer interface, right ?
The transactional filesystem guarantees uncorrupted data at power failures and OS crashes. If you do this across a pool of physical platters, you need operations across layers.
There is an interesting blog on the usage of ZFS for home users. It contains some good arguments, why ZFS is useful for Linux' Desktop Stride. You find it here: http://uadmin.blogspot.com/2006/05/why-zfs-for-ho
Last ot least, the online checking of all your data ('scrubbing' and 'resilvering') is a valuable feature for Linux (and the home user) as well.
To me it looks like, as of today, that about everyone liked the features of ZFS. Now, as it requires to break some old habits, suddenly we resist change and rather stick to older concepts.
As if GPLv2 vs GPLv3 was not enough of a threat to Linux, now we unashamedly permit a new-from-the-bottom-up filesystem to overtake us as well ?
I'm sure I'll get modded down for pointing this out, but the truth is that X-windows sucks. Linux desperately needs a replacement for X. Once that happens, Linux will finally be ready for the desktop.
Don't get me wrong: I love using Linux on the command line, and I think Linux is vastly superior to Microsoft-Windows for server environments. I just can't stand the wretched excuse for a windowing system known as X.
Ok, under LVM, /dev/hda is a physical volume...
And that is exactly my point: there is no need for this restriction. It is a stupid restriction that results from a stupid design. OK, at the time the design didn't seem so stupid because it could do things we could not do without it. But now, nearly ten years later, fossilized and frozen in time, it is obviously stupid and limiting.
Have you got your LWN subscription yet?
I think that the beauty of linux is the fact that it is not limited in its expression like other operating systems. Linux can be what ever you want it to be. The lack of cross integration actually lends itself to adaption. The paradigm of hardware and software companies creating markets and then selling them will not be as important as costumers wanting purpose driven machines and linux delivering.
Also,
Consider that your raid on the lower level cannot rebuild a single filesystem block or piece of data in flight to the application. ZFS can do this because it has raid knowledge and filesystem knowledge and can retrieve or calculate a repaired piece of data from RAID info if it detects that it got corrupted. Conventional RAID systems will repair a drive if you yank the whole thing and that's al lit can do because the layers don't communicate with each other.
I will be rich and famous when i find a way to stab people in the face over the internet - http://bash.org/?4281
That way I can remove them.
[ Now picture Jon Stewart saying this while :-)
impersonating either Sen. Ted Stevens or President Bush. ]
It must have been something you assimilated. . . .
But this is where journalling file systems such as reiserfs win. Its not exactly an old-style file system either, its more like a database. (AN ACID DATABASE) ...Atomic, Consistent, Isolated and durable... If you don't shut down happily, the file system is safe because before writing (or trying to write) the file, it made a journal entry. It then does the write, then removes the journal entry. If there is a power outage during a write, it checks the journal to see if the file system is clean (no journal entries and a clear file system bit). If its not clean, it cleans it before mounting it, and it does that by checking files against journal entries (and cleaning the journal as the file is either fixed, or just checked (all result in journal entries removed). With (also) all of the special cases, just like a database. Reiser4 also does snapshots (as does plan9). The only thing ZFS had over any other was size (and Reiser4 was big enough so that all the hard disks in the US could not exceed what Reiser4 could consider as 1 single filesystem). There are things in some of the other file systems that are not in ZFS (no tail-packing for example, which lets you put 5% more data on an average hard disk, and considering the drive sizes today, 5% can be a lot). Last I heard, ZFS was incompatible with the GPL, another valid reason for not including it in Linux.
No, it is not stupid. You seem to be suggesting that snapshotting should be something that can be done on an arbitrary block device. To do this, you would have to have this supported in the block device handling, and this could be *very dangerous* in some environments (GFS, for example, or other DLM-based clustered filesystems) and misleadingly useless in others (snapshot one volume in a RAID 5 array). To make this work, I think it is best to handle snapshotting on the bottom-half of the filesystem (the inode layer->block device interface). However, there is some use in having it handled by the LVM (in that it allows you snapshot logical block devices rather than filesystems).
/dev/ location of LVM volumes to /dev/lvm/... Makes for less typing and more transparency. Makes the entire picture both cleaner and clearer.
I am not saying that the current system is perfect (no system is, and one should always strive for improvements), but I don't think doing snapshots of arbitrary block devices is a good way to handle it.
My list of things to improve include:
1) Change the
2) Make sure that *all* the newbie Linux documentation covers LVM.
LedgerSMB: Open source Accounting/ERP
Miguel de Icaza got married recently and I'm pretty sure his wife was the type of Brazilian woman who was born that way. I'm not sure who Jason Reynolds is, but Joel Spolsky is a big Microsoft guy according to Wikipedia.
When Argumentum ad Hominem falls short, try Argumentum ad Matrem
One legitimate complaint is the poor state of integrated RAID support in the linux LVM. Yes, the LVM can mirror logical volumes now, but it is very klunky (I miss AIX LVM). Creating PVs on top of md devices is tedious and error prone (because you often need to split physical devices into multiple partitions to avoid resyncing the world). The LVM should at least support RAID 0 and 1.
Another complaint is the lack of consistent high level utilities for file system admin. On AIX, one command allocates a logical volume and creates a filesystem on it (or you can do the steps separately for greater control). Linux makes you create the LV, then create the filesystem. Worse, linux used to offer e2fsadm to resize a filesystem and the underlying LV. Now, you have to run lvextend THEN resize2fs - or the reverse if shrinking (and you had better type those sizes right). While it is nice to have access to raw LVs for some things, I really don't want to have to manually compute volume sizes - with potential data destruction in case of mistakes.
The article you linked to suggested nothing about intelligence and only compared Dell users to Mac users. I don't own any Dell computers and they all (with one exception of a Windows Server 2003 system I keep for testing purposes only) run Linux. I suppose I could argue that makes me statistically likely to be smarter than John ;-)
Secondly, while I like the Steve Jobs quote, I would point out that in my largest open source project (LedgerSMB, see my sig), our core team is composed of six people, and only two have CS degrees. One hasn't finished her degree yet, and one of the degree holders double-majored in music. Me? I am a historian. We haven't got the best results yet but that is because the codebase we inherited is, politely put, a mess. Hopefully within a year, we will have the best darned business accounting package out there and it will be open source. Part of the reason is because there are so many contributors who are from other fields (art, music, science, history, etc).
There is a lot of great open source software out there. I don't understand what you mean by a failure to think holisticly.
LedgerSMB: Open source Accounting/ERP
Andrew Morton basically said, "Yeah it's great, but the API sucks." The blogger responds about Sun products are designed by ninth djinn feng shui masters.
Woop-di-doo. And good for you.
You seem to be suggesting that snapshotting should be something that can be done on an arbitrary block device. To do this, you would have to have this supported in the block device handling
Well duh. What do you think an LVM virtual device is? Right, an instance of a block driver. Now what do you think a "physical" block device is? Right again, an instance of a block driver. Spot the difference? No, neither can I. Get my point yet? Ah, darn.
Have you got your LWN subscription yet?
Well duh. What do you think an LVM virtual device is? Right, an instance of a block driver. Now what do you think a "physical" block device is? Right again, an instance of a block driver. Spot the difference? No, neither can I. Get my point yet? Ah, darn.
:-)
'358' is an instance of an 'integer.' '35800' is an instance of an integer. Spot the difference? Neither can I but I guess the IRS does
My point is that physical and logical devices are used in fundamentally different ways. Taking a snapshot of a networked device using GFS and then promoting it to read-write would be downright dangerous and could result in severe data corruption. Similarly taking a snapshot of volume 1 in a 10-volume raid-5 array would not do you much good.
It seems better to allow this only on logical volumes which are specifically managed by LVM.
LedgerSMB: Open source Accounting/ERP
You really know nothing about ZFS then if you think that's all ZFS has over the competition including Reiser. Time for you to do some Googling so as not to sound ignorant.
This isn't that new. In fact, it's rather like Tandem's file system from the 1980s. It's good that people are thinking about this. Traditional UNIX file system semantics are incredibly dumb; no locking, no transactions, and no guarantees. Really, if you're looking into reliability, find out how Tandem did it. Tandem systems routinely ran for years, in some cases decades, without crashing. Even during reconfigurations and hardware upgrades.
To me, the real "layering" issue is that the file system is in the kernel. If it were simply an application one talks to by message passing, as it is in QNX, replacing the file system wouldn't be a big deal.
Just try this in a kubuntu machine: log-in and as soon as the desktop apears, try to open some instances of konqueror... frecuently konqueror will frezee for a few seconds while opening, and after that the file system will be read-only; in the next reboot you will be forzed to do a fsck.
And impatient coworker manages to do this frecuently... never had lost data like this; but it sucks, so he is starting to despise kubuntu
I have only seen Linux kernel panic once since 1999 because of a setting I did with the bios under kernel 2.0 that was my fault. Windows on the other hand crashes alot more often and has more security issues because things run in the kernel or ring0 which should not. Its very will layered. The apps on my linux distros are bleeding edge and suck compared to most commercial unix counterparts and even Windows but its well layers. Ubuntu, Debian, and RHES are the exceptions.
.... and of course the FreeBSD manual that came with the box is the best unix resource I ever read. Everything is just integrated and works and feels like one unified system.
I have switched to FreeBSD for awhile because I felt Linux lacked quality and a lack of design since it tends to evolve compared to the BSD family of operating systems which are designed
For the background here I started to use Linux around 1998 or 1999 and switched to FreeBSD in 2002 - 2005. I then switched to Ubuntu and then back to Windows but still run Ubuntu under the free VMWare Server on my laptop. I needed to learn Java for my MIS program and java reaks under any BSD so I switched back to Windows and then linux again.
Ubuntu is the only OS that feels BSDish in the way that everything is integrated and well tested with look and feel and quality.
Anyway I feel its very mature though the apps could be improved which is an argument for commercial unixies like Solaris on mission critical servers, but with distros like Ubuntu that problem is going away thanks to integrated bug reporting.d
http://saveie6.com/
My point is that physical and logical devices are used in fundamentally different ways.
/dev/hda, a perfectly reasonable thing to do, I should be able to continue to access the snapshotted device as /dev/hda, which will then be running a different driver that knows about the snapshot). There is no technical reason why this cannot be done. Bold is in the hope that I do not have to say "oh darn" about that point again.
My point is that there is no such thing as a "physical" device, there are only block device drivers and instances thereof. And if I want to take a snapshot of
Have you got your LWN subscription yet?
But it sucks less.
Of course that is also the argument for windows. Xorg is much better than xf86 and of course computers with gigs of ram and dual core processors are nice too.
I loved aqua for macosx but I guess X is here. The fonts are now getting usable for those used to them on windows or macosx though you need David Turners patches for real clear type rendering.
http://saveie6.com/
Ah, but then you trot out the old, "Democracy is two wolves and a sheep voting on what to have for dinner." Which is the definition of the tyranny of the majority, in terms even the sheep can understand.
It doesn't do the sheep any good to "get out the vote" and convince more wolves to join in the voting.
Can you be Even More Awesome?!
It's good to see that somebody is still finding a use for those old foxfire catalogs.
Right-- there is no technical reason this *cannot* be done. But there are technical reasons this *should* not be done. Perhaps we are talking past eachother on this minor point.
LedgerSMB: Open source Accounting/ERP
Yup, it amazes me how incredibly patient Windows users are. They will put up with *anything* and come back for more. My guess is that they all have a sado-masochism fetish. If Windows gives me shit at work, I pop in a Linux bootable CD and carry on working while the rest of the people around me all head for the water cooler. Windows is the greatest *unproductivity* tool ever invented.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
Hmm, that post is just so uninformed that I guess the Honorable Coward hasn't ever actually used Linux.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
No, something like this could be best handled in the bottom half of the filesystem (inode layer). However, I do not think you need knowledge of the stack to do it. If what you want is a fully transactional write, you write to a new set of blocks and then mark the old ones as free. You would need to keep some state handling data in the inode, of course to make this work, but it would be completely feasible to do. It would even allow you to do transactional writes even when not using RAID.
The problem with implementing this in the RAID layer is that this layer has no knowledge of the filesystem data structures. You *could* do this with a RAID array encapsulated behind an interface like OSD, but but behind just a block interface.
LedgerSMB: Open source Accounting/ERP
ZFS is layered.
ZFS may be more layered than a Baumkuchen inside, but that's totally irrelevant. From the point of view of the rest of the world, ZFS is just an amorphous blob of code that talks to the disk at one end and presents a file abstraction at the other, and that's what matters. Why does it matter? Because ZFS duplicates a lot of functionality that already exists elsewhere in the kernel and because its internal abstractions aren't available (or likely even suitable) for building other things on top of. All the shiny, pretty diagrams Sun draws about its internal structure don't change that.
You can even, y'know, read the source before you start making ignorant claims. I guess this wouldn't be Slashdot if people did that, though.
You can even, y'know, stop being a total fanboi before you start making ignorant claims. I guess this wouldn't be Slashdot if people did that, though.
Well, another solution is to use ZFS.
is responsible for all of the current failings of Linux -- desktop adoption, user-friendliness, consumer software, and gaming
Well, and a Solaris/ZFS dev says that to a Linux dev ? Talking about thinking holistically...
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
Right-- there is no technical reason this *cannot* be done. But there are technical reasons this *should* not be done.
There is not a single one that I know of, and in particular, you have not presented a single technical argument against. "Used in fundamentally different ways" is not a technical argument. Neither is "could result in severe data corruption". If I want severe data corruption I can produce it in any number of fine and otherwise useful Unixy ways. You need to take off those blinkers and think about "could be useful" and "could make life nicer for users" and "could be implemented nicely".
Have you got your LWN subscription yet?
It's not like Solaris with ZFS is any different in this regard. To use snapshotting or any other zfs feature on a physical disk, you have to add the raw block device to the zfs pool. Just like you have to add a physical disk to LVM before you can use LVM snapshots on Linux. You can't just snapshot arbitrary block devices on either OS, holistic layering violations or not.
The reason? It is not "fast enough" for IP. Though Solaris IP, using (heavily modified) stream modules is an excellent performer.
Unfortunately Linux is more guilty of NIH than many others.
"Any design problem can be solved by adding an additional level of indirection, except for too many levels of indirection." This is known as Cargill's quandry. But - does anyone know who Cargill was?
Pick one. Use it. Done.
You only found OSX easy to pick because there was only one. If OSX doesn't do something quite the way you'd like it to, you ignore it because either
a) you have no choice in the matter
b) you don't know it's a possibility
So just pick one. From the US, Red Hat for Gnome (OSX like) or SuSE for KDE (windows like). That was no more difficult than "Do I pick Windows or OSX?".
You claim this requires the storage model to be part of the filesystem, but it doesn't.
All it requires is sharing some minimal extra information, such as letting the block layer have a concept of free/used data blocks, and allow the filesystem to give hints about the strip width, for example. Sufficient information to deduce those values might even already be there with the existing API's (I don't know - I haven't looked much at that part of the Linux kernel).
If done right, other filesystems would be able to take advantage of it with little effort, instead of having to duplicate this functionality in every filesystem that wants it.
As for snapshotting, that also at most needs knowledge of which blocks are free and which are not - that's sufficient to expose a virtual block level device to the upper levels while doing copy-on-write transparently on the physical disk as long as there's enough space. You can even do it without understanding free space, but it means you need to preallocate some space for block copies which you can only reclaim if a snapshot gets deleted but that's usually ok for a lot of scenarios (such as using snapshots for hot backups).
dude, If you run Ubuntu 7.04 now and drop Beryl on that you will notice that your world gets rocked. Linux is just as good as windows(and with Beryl its beautiful). I showed Ubuntu to my mom and she was like "put that on my computer" and i was like its easy just pop this disc in and press install! It really is that easy. LiveCD makes life easy for Linux. The only problem with Linux is 1) too many distros. If everyone working on Linux focused on to maybe 5 major distros a lot more work would get done. and 2) marketing. MS goes out in the world and tells people why they need windows and how wonderful their life will be with the power of windows or some shit like that. The only good press about Linux is on the internets. I wanna see tv ads showing me a better life with the magical powers of Linux! and 3) games. This will change when game developers support the platform which means more people need to HAVE the platform. When everyone that uses Linux is sad that game developers don't really support their platform they need to realize that its not because developers don't wanna support Linux. Money talks to them as they are businesses. More user base = more support. Simple as that. So once Linux gets the user base the games will come and we can fly on magical unicorns and shit into the sunset. Support the new OLPC - One Linux for every PC. chakow!
Balderdash!
nt.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
Linux was built on "oh no, I need a Unix and I don't have any money and AT&T is suing BSD so I'd better not use that".
For me, that's not much of a premise.
I'd rather they went for the "lets stop working on Linux" option.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
I agree. It does fail, mostly. However, if you look at the success of wireless, you can kind of see what the potential is (luckily, pretty much all devers are interested in wireless). Perhaps there should be some kind of body that can put together initiatives consisting of a stacks of projects from drivers up to the GUI. I think that would be pretty sweet. And maybe the coding is even being done already, the packages could at least be strung together or grouped somehow.
Two Examples:
Handheld Syncing
Printing/Scanning
Cheers.
-- A cat is no trade for integrity!
Linux could have done the same thing, since all of the layers of ZFS are implemented inside the kernel. It didn't, and you can draw whatever conclusions from that you wish.
I am TheRaven on Soylent News
I know next to nothing of the particulars of this topic technically, but at a high level it seems to me like we're having a debate between sometimes complex and annoying configuration to do what should be simple, and a monolithic approach that is easy, but does not fit all circumstances.
So, aren't there other high level options here? Either (simplest) use the monolithic tool when it does what you want, configure by and when it doesn't, or... find an existing open source tool, or write your own (script/gui tool, whatever you prefer) that follows a convention to save yourself the time when the convention does what you want, but allow access to the underlying configuration you need to tweak (exposes those options in the script/gui tool).
Someone tell me why these high level options aren't available, or I just plan on ignoring all arguments/complaints as being based on an either/or fallacy.
If I understand you correctly, you're proposing to hide the disk driver behind your "LVM" driver. /dev/hda provides a number of ATA/ATAPI specific system calls. The same is true for devices representing SATA, SCSI or whatever drives. This means your "LVM" needs to know about those system calls.
/dev/hda means your "LVM" driver?
:)
This is a problem because
And the Linux kernel may have a problem with the entire ideia of "hiding" drivers: how can your "LVM" reference the ATA driver while the remaing kernel thinks
Another aproach would be adding snapshotting capabilities to your disk driver.
Then again, you're making all this mess just to hide stuff from users.
I have had MSDN for many years, and recently TechNet Plus also. Burning an ISO and installing the software is a no brainer. Within several days though, Microsoft operating systems start to fall apart as the default options start banging into each other. So you loaded Vista and Office successfully. Big Deal. The question is whether you can reliably use them to get real work done on a schedule day after day. Try looking at the system logs for information about services that fail to launch because of cascading problems in the operating system and application installs. You say you loaded up Visual Studio on Vista with no problem. I doubt that.
In the KDE version of Ubuntu, adding second head monitor involves: System Settings > Display > Second Monitor (click on checkbox). How is that different from what you wanted? (Someone can tell us what the equivalent is in GNOME.) And then you can choose "clone of primary monitor" or "dual monitors", and if you choose the latter, you can set what the position of the monitors is relative to each other.
By the way, I'm still using the third-newest version of Kubuntu, and in fact KDE has had these features even before I switched over to Ubuntu --but since I didn't have two monitors before, I never tried to use it.
Next example of lack of GUI for configuration, please.
404555974007725459910684486621289147856453481154 in hex is "You sank my Battleship?"
[GPG key in journal]
The free radeon driver is a bit better than nv. Except it won't play 1080i with xv, a bug that hasn't been fixed for years. But that didn't quite work correctly with the last nvidia card + nv driver I tested, either.
About the binary drivers, I don't know - I'd never use these. They're a bitch to install and require certain kernel versions, they break things, they simply don't belong on a stable system. Anyone using these would be better off running Windows.
The one family of graphics chips that really works is the onboard Intel GMA e.g. on 965G boards. Everything just works, although you might have to use the new modesetting branch for non-standard resolutions like 1920x1200. No problems with xv or the TMDS PLL like with ATI, no stuttering video and general weirdness (or complete lack of support) like with nv.
Intel GMA doesn't really work for games, but for these, you need that large game loader called Windows anyway.
I might know a guy who knows somebody that knows.
Only three things are certain; death, taxes, and apocryphal quotations - Ben Franklin.
Of course it's easier to create innovative features by writing spaghetti code that circumvents well-defined (and limited) interfaces. But it makes the codebase harder to maintain! The spaghetti code will break more likely than code that goes through the interfaces and adheres to the contract. Layer violations are a strong indicator that the current interfaces are not flexible enough or that they are too high-level. But the longterm solution is cerainly not spaghetti code.
Well duh. What do you think an LVM virtual device is? Right, an instance of a block driver. Now what do you think a "physical" block device is? Right again, an instance of a block driver. Spot the difference? No, neither can I. Get my point yet? Ah, darn.
A "physical" block device represents a piece of hardware (usually a hard disk) with some fixed capacity. A LVM block device represents a chunk of some LVM volume group. It makes no sense to have LVM-style snapshotting on block devices that aren't LVM logical volumes - snapshots are implemented using copy-on-write, which requires extra storage space for the modified sectors, and there's nowhere sane to put that with a normal block device.
Never going to happen; even if it did, why should someone writing code to scratch a personal itch care?
Now think: TdR
Would he do what the kernel guys say, or rewrite from scratch?
qmail was abandoned long ago, it now consists of software + patches patched with patches to workaround the software's limitations. These patches, of course, add functionality written by 3rd parties that introduces their own vulnerabilities. Since there is no development "tree" per se, many patches are completely incompatible with each other.
Anyone still using it is desperately clutching the "it's so secure" line, because that's what djb said. They forgot to check osvdb, though, where it's readily evident that it's not so secure.
Kernel Team
No! Layers! Onions have layers. Linux
have layers! Onions have layers. You
get it? They both have layers. (he heaves
a sigh and then walks off)
End User
(trailing after the Kernel Team) Oh, they both
have layers. Oh. {Sniffs} You know,
not everybody likes Linux. Games! Everybody
loves Games! Games have layers.
Kernel Team
I don't care. what everyone likes.
Linux are not like games.
End User
You know what else everybody likes?
Fancy GUIs. Have you ever met a person,
you say, "Let's get some fancy GUIs," they
say, "Hell no, I don't like fancy GUIs"?
Fancy GUIs are delicious.
Kernel Team.
No! You dense, irritating, miniature
beast of burden! Linux OSes are like onions!
And of story. Bye-bye. See ya later.
- these are not the droids you are looking for -
Golly, you've got a great argument sir. Of course, your argument is directed at *non-games*. As the poster was trying to point out, games sell in the order of a 6 month time span and old games are a crap shoot to get working properly on modern OSs. Or is it not fair that I point out that NTVDM is a shitty game system for DOS games, and I'm left trying to finagle with DOSBox or some other emulator on both Windows and Linux if I'm really interested in "archaic" games? But, yea, preach on brother about how you can use all your old non-games in Windows. That was entirely the subject at hand to discuss and the exact lacking that was being pointed out in Linux.
Eurohacker European paranoia, gun rights, and h
Do you have a copy of StarOffice from the mid-to-late 90's? Try running that in Linux now. Do you have a copy of MetroX from say, 1998? Try running that in Linux now. Are you still using the original Linux binaries for any games released in the late 90's?
Running old software (>10 years old) in Linux is actually pretty straightforward. All you need are the older versions of the libraries that it was designed for. That's it. Most distributions still maintain "compatibility" packages for older versions of GCC, X-related libraries, etc. I believe little has changed in the kernel such that backward computability was ever broken.
In Linux, you can run "ldd" on a binary and it will tell you what libraries it needs and what it found on your system.
If you actually remember how Windows software worked ~10-15 years ago, you probably remember games complaining that different versionf of the Microsoft runtime libraries such as VBRUN300.DLL were not found.
If I understand you correctly, you're proposing to hide the disk driver behind your "LVM" driver.
/dev/hda provides a number of ATA/ATAPI specific system calls. The same is true for devices representing SATA, SCSI or whatever drives. This means your "LVM" needs to know about those system calls.
/dev/mapper, md, floppy, whatever) the upstream interface is the standard block device api. Only the downstream side of a "physical" driver knows about sata, scsi etc.
/dev/hda means your "LVM" driver?
/dev/hda, hmm?
/dev/hda, we could get rid of the device mapper layer entirely.
You might say that, though this is nothing new because disk drivers are already hidden behind LVM drivers.
This is a problem because
No that is not a problem because in all cases (hda,
And the Linux kernel may have a problem with the entire ideia of "hiding" drivers: how can your "LVM" reference the ATA driver while the remaing kernel thinks
Device mapper already knows how to "hide" the actual driver[1] and the remaining kernel knows how to sort that out. How do you think it is possible to create a device mapper device as a linear mapping (say) and later even while the device is in use stack a mirror or snapshot on top of it? So how come I can't do that with
Another aproach would be adding snapshotting capabilities to your disk driver.
And do it separately for each different kind of disk? Bleah.
Then again, you're making all this mess just to hide stuff from users.
You don't know that. Perhaps this is more simple and powerful than the mess we currently have, which is a great example of bogus layering with no good plan for abstracting across it.
[1] How does device mapper stack one block driver on top of another while the device is in use? Easy, it allocates a new device number internally, moves the existing block driver from the old device number to the new device number, then initializes the new block driver to do its downstream IO to the new device number, and finally stuffs the new block driver into the old device number. There is no reason why this stacking trick could not be learned by all block devices (by lifting it into the generic block layer) then not only would we be able to stack a mirror or snapshot on top of
Have you got your LWN subscription yet?
Sorry, but I can't agree with your reasoning. To explain, let met set out a few realities of software development, as I've personally come to see them after some years as a developer:
Now, these are obviously blanket statements, and no doubt there are valid exceptions to each rule. Indeed, the rules themselves seem almost contradictory, though I think that is just an illusion: what they really tell us is that balance is required, both in the degree of concentration of effort vs. parallel development, and in the degree of future planning and generality vs. the efficiency of developing what will actually be used in the end.
Now, take a look again at the argument you gave in the parent post.
If you have 100 different ideas for ways to solve a problem, then go ahead and develop 100 different solutions to see which work best. But in reality, do you really, or are there only really a relatively small number of solutions, but many implementations of each with few real differences?
OK, let's assume there really were 100 unique approaches, and that in practice 5 of them turn out to work best. Did the other 95 learn something developing their failed alternatives? Perhaps, but they might have learned something helping to develop the successful alternatives as well. After all, if so few of the ideas really worked out, would it not be better to study those and the ideas and techniques of the people behind them?
You also suggested that the 95 would have given something back to the community. But would what they gave back have much value? Again, it seems unlikely that if 5 different solutions were all viable, the other 95 would really offer many further ideas in terms of high level design or general approach. And again, if they did, perhaps those same good ideas could have made the better solutions even better still if the development teams had compared notes earlier, giving a best-of-both result. The 95 might also have some neat implementation tricks, but as I suggested above, in reality it seems very rare for one project to borrow code from another in this way unless the code in question was specifically developed as a self-contained library, in which case it's not really specific to one of the 95 failed attempts anyway.
It's not hard to see that a small number of ideas, perhaps 2, perhaps 5, might offer genuine pros and cons, and with a concentration of development effort they might be able to learn from each other's experience and systematically share code for common functionality so that all benefit. But really, I don't see this happening at a micro level. It has to be mac
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
A "physical" block device represents a piece of hardware (usually a hard disk) with some fixed capacity. A LVM block device represents a chunk of some LVM volume group. It makes no sense to have LVM-style snapshotting on block devices that aren't LVM logical volumes - snapshots are implemented using copy-on-write, which requires extra storage space for the modified sectors, and there's nowhere sane to put that with a normal block device.
You have confused "specific driver" with "block device interface". Sigh. Hint: in a C++ context it would be the difference between an abstract interface and an instance of an object of a class derived from the abstract interface. In other words, you are a couple of levels of abstraction away from enlightenment. I suggest starting by learning about the kernel block device interface.
Have you got your LWN subscription yet?
First, if you are so convinced that you are right and the world is wrong, nothing is stopping you (aside from your own technical understanding) from maintianing a patch to the Linux source which does exactly as you describe. If it is really that useful, I am sure other people will want it to be included in the source tree. Even if it is only useful in certain niche cases, I am sure you will have a lot of help maintaining it. But I personally don't think it would be very useful at all, and making the software which is everything to everyone is a recipe for disaster. Always.
However, my argument is simply that the LVM layer is a *better* place for this because it can be guaranteed to be reasonably safe and meaningful there. In fact, honestly I think that the bottom half of the filesystem is the best place for this functionality simply because the driver has enough knowledge to do this safely under all circumstances. Perhaps the *best* way to handle it would be to put hooks into the LVM and md layers to make it handle this through calls from the filesystem driver.
LedgerSMB: Open source Accounting/ERP
Why would you want to do this? It does not buy you anything, as you can already set your system up to create LVM volumes of all your block devices and forget the non-LVM volumes exist. The existence of "raw" block devices that don't allow you access to the LVM features doesn't take away any functionality from you.
This is an old religious debate coming back to haunt us once more... When the ARPAnet was up, running and mature, the ISO proposed a reference model for networking which lad seven layers, unlike the ARPAnet's three (network, host-to-host and application).
Some non-practitioners believed far too much in the reference model, and went so far as to claim that layering should never be adjusted for actual cases.
The people recommending the ISO Reference Model (ISORM) were referred to as the "ISORMists" by the ARPAnauts, and were good, critical colleagues: they started some intelligent discussions. The people claiming that other layerings were improper and/or infeasible were less than useful, and eventually were named the ISORmites, pronounced eye-sore-mites.
Mike Padlipsky has a good description of the religious wars in his booklet "The Elements of Networking Style" Englewood Cliffs, NJ (Prentice-Hall) 1985. I recommend it if you can get your hands on a copy.
--dave
davecb@spamcop.net
Why would you want to do this? It does not buy you anything, as you can already set your system up to create LVM volumes of all your block devices and forget the non-LVM volumes exist. The existence of "raw" block devices that don't allow you access to the LVM features doesn't take away any functionality from you.
Did you bother reading the article? Did you see the part about "the Linux community's inability to design, plan, and act in a holistic manner"? You are an excellent example of that inability.
So much for the rhetoric, now please pay attention if you wish to stop being part of the problem. What this buys is removal of a whole bogus layer that has no technical justification whatsoever for existing. In the process, life becomes simpler for the system admin and we open the door to fixing some of the stupid limitations that we have been saddled with for years because of the idiotic device mapper misfactoring. Got your attention? Think I'm wrong? Well here is some advice: you should not only have read the article and understood it before posting, you should have read the code in question and you obviously have not.
Have you got your LWN subscription yet?
Design by committee produces chaos and frustration - those involved secretly desire monarchy and some control.
Design by fiat is often effective, but very narrow - those involved secretly desire freedom and a voice.
I suspect most respondents prevaricating rather than addressing the original subject are quite well aware of both sides of this.
Cheers, thanks a lot.
Nice post, but you're casting pearls before swine by posting it on this site. There's too much emotional investment here for any real discussion.
Anyway, the discussion is by no means limited to snapshotting. What about mirroring, higher level raid, encryption, multipathing, etc? Do you want to roll all those features into your one, perfect, all singing all bloating filesystem to the exclusion of all other filesystems? Perhaps the *best* way to handle it would be to put hooks into the LVM and md layers to make it handle this through calls from the filesystem driver. Wow! You are so close. What if I were to tell you that those hooks already exist with minor correctable deficiencies, and that they do not need to be incorporated into the lowest level of the LVM as you wish, because that whole layer is a bogus fabrication of some muddled mind and needs to die?
Sorry about ridiculing your post. To avoid such ridicule in the future, please leave out the empty rhetoric and stick to informed, supportable arguments. If you are unable to do this, then please ask a friend who is informed and thoughtful to post in your place.
Have you got your LWN subscription yet?
Have you got your LWN subscription yet?
True, and what kind of sense does that make? It is purely an artifact of the incumbent low level LVM model. Please go back and read the original post and notice how you misinterpreted it. And you can't zfs snapshot anything that isn't a zfs filesystem. Is that really insightful? If you create a ufs filesystem you can't magically make it have all the attributes of a zfs filesystem, and if you create a linux filesystem on a raw device rather than an lvm you can't magically make it have all the attributes of an lvm. Sun lets you do the former, linux lets you do the latter--because that's what the users want. Why is it ok for you in solaris, but a horrible problem in linux?
from: http://blogs.sun.com/bonwick/entry/rampant_layerin g_violation
...
...
...
...
...
text follows:
Rampant Layering Violation?
Andrew Morton has famously called ZFS a "rampant layering violation" because it combines the functionality of a filesystem, volume manager, and RAID controller. I suppose it depends what the meaning of the word violate is. While designing ZFS we observed that the standard layering of the storage stack induces a surprising amount of unnecessary complexity and duplicated logic. We found that by refactoring the problem a bit -- that is, changing where the boundaries are between layers -- we could make the whole thing much simpler.
An example from mathematics (my actual background) provides a useful prologue.
Suppose you had to compute the sum, from n=1 to infinity, of 1/n(n+1).
Expanding that out term by term, we have:
1/(1*2) + 1/(2*3) + 1/(3*4) + 1/(4*5) +
That is,
1/2 + 1/6 + 1/12 + 1/20 +
What does that infinite series add up to? It may seem like a hard problem, but that's only because we're not looking at it right. If you're clever, you might notice that there's a different way to express each term:
1/n(n+1) = 1/n - 1/(n+1)
For example,
1/(1*2) = 1/1 - 1/2
1/(2*3) = 1/2 - 1/3
1/(3*4) = 1/3 - 1/4
Thus, our sum can be expressed as:
(1/1 - 1/2) + (1/2 - 1/3) + (1/3 - 1/4) + (1/4 - 1/5) +
Now, notice the pattern: each term that we subtract, we add back. Only in Congress does that count as work. So if we just rearrange the parentheses -- that is, if we rampantly violate the layering of the original problem by using associativity to refactor the arithmetic across adjacent terms of the series -- we get this:
1/1 + (-1/2 + 1/2) + (-1/3 + 1/3) + (-1/4 + 1/4) +
or
1/1 + 0 + 0 + 0 +
In others words,
1.
Isn't that cool?
Mathematicians have a term for this. When you rearrange the terms of a series so that they cancel out, it's called telescoping -- by analogy with a collapsable hand-held telescope. In a nutshell, that's what ZFS does: it telescopes the storage stack. That's what allows us to have a filesystem, volume manager, single- and double-parity RAID, compression, snapshots, clones, and a ton of other useful stuff in just 80,000 lines of code.
A storage system is more complex than this simple analogy, but at a high level the same idea really does apply. You can think of any storage stack as a series of translations from one naming scheme to another -- ultimately translating a filename to a disk LBA (logical block address). Typically it looks like this:
filesystem(upper): filename to object (inode)
filesystem(lower): object to volume LBA
volume manager: volume LBA to array LBA
RAID controller: array LBA to disk LBA
This is the stack we're about to refactor.
First, note that the traditional filesystem layer is too monolithic. It would be better to separate the filename-to-object part (the upper half) from the object-to-volume-LBA part (the lower half) so that
Why do people insist on making up weird plurals of the word "virus"?
Wow, the article (orig) is absolutely right. It's the first intelligent Linux big-picture criticism I have ever read, and it's spot on. I absolutely agree. The nature of a disperse collection of programmers can't easily, and has not yet created a coherent, consistent, and integrated user experience that makes Windows and Mac OS even moderately successful despite all their shortcomings.
And yet, it's hilarious to hear all the Slashdotters rail on and on about the individual parts, and give excuses about how well it works, and blah blah blah-- ZFS's "layering" isn't the problem, he's talking about the bigger picture, and until all the programmers working on the integrated Open Source OS platform GET IT, Linux will never be successful. No one gets the big picture, and if you're arguing with that, then you're not either. Move along...
"!"
I'm sure there's a way for the traditional linux FS stack and ZFS to co-exist. Basically, identify the layers that are replaced by ZFS and disallow those layers to run on the same resources(volumes, drives?) that ZFS is using. Then the user has the choice of using ZFS or the normal stack.
However, I think that ZFS IS NOT the end all and be all of file systems. It's an impressive system but it alone is not worth throwing away all of the manhours put into the linux FS stack. It might warrent a fresh look at the stack though to see if there are places for refactoring available..
Cheers
Ben
it is already ported to FreeBSD
Oh, so if you misconfigure qmail it's insecure? I'll bet I can misconfigure anything and cause it to be insecure. Let's start with any PHP program and turn on register_globals. Let's run Apache as root.
Don't piss off The Angry Economist
Yeah - right - misconfigure.
1998 called, they want their arcane limitations back. 8gb of ram? Have you noticed than you and I can go to best buy and purchase a machine capable of of running 128GB (amd64/windowsxp64)?
Famous quote of 2007: "8GB Should be enough for anyone"
It's not like Solaris with ZFS is any different in this regard. To use snapshotting or any other zfs feature on a physical disk, you have to add the raw block device to the zfs pool. Just like you have to add a physical disk to LVM before you can use LVM snapshots on Linux. You can't just snapshot arbitrary block devices on either OS, holistic layering violations or not.
Exactly my point. It is a stupid, unnecessary restriction whether the stupidity is implemented on Linux or Solaris. Hey, think about this: before some clever ancient person invented and carefully explained to everyone the concept of zero, the human race was unable to count below the number one. Needless to say, negative numbers were beyond the comprehension of the average person. Similarly, you are unable to perceive the benefit of removing a bogus interface layer while at the same time simplifying the user interface and making it easier to add new, sophisticated storage features. Well that is not my problem, it is yours.
Have you got your LWN subscription yet?
I'm with you, Dan. I'm really looking forward to cleaning this up sometime. :-)
(And if you need anyone to review your
posts to remove excessive attitude, I'm available
- that other Dan
As a systems administrator I appreciate two interfaces to my data - one through a filesystem (and I really dont mind which one as long as it follows the usual semantics), and one that relates to the raw data as written to the spinning magnetic media.
When I copy all my raw devices I am confident that I have captured the full state of my machine, and I know exactly how much space and time that copy will take.
Whilst the ideas proposed may work in an academic way, I worry that the side effects will damage reliability. We're used to thinking about raw devices and filesystems, and to extend this to pondering filesystems as seen at certain points in time complicates disk usage because you can now no longer look at the size of data that you are storing and know how large a physical volume is needed to store it.
If you can write to a snapshot, then you will use up your physical storage in a manner that is invisible to the standard view of the filesystem.
We already have a way to implement this as seen using the layered filesystem in Knoppix, where the basic copying on writing goes on in a separate file. Maybe allocating storage sizes for changes when the snapshot is turned into a writable area could provide the same protection.
I'm not convinced that making this integrated any closer with block devices will bring more benefits (probably speed) than it may cause confusion (darn, I've got lots fo disk space, why can't I write this file). Computers dont just run the operating system - they run applications. Indeed, their very purpose is to run applications since all the operating system does is to provide the interface between the application and hardware. If changes in the operating system trigger application failures then you're heading in the wrong direction, and if you are optimistic enough to suppose all application developers test for all error codes and handle them sanely and correctly I've got some lottery numbers for you.
-- Don't believe everything you read, hear or think