My understanding always was, Linus wasn't too proud of having driven a wedge through the middle of the kernel community, that's what the "Git" was about. Now that he went on to change the entire world of revision control, I would say... the pain was worth it. Probably.
RSDL wasn't included in the mainline kernel because Linus considered Con to be unreliable, and wanted to have a scheduler with a developer he could count on to maintain the code and fix bugs.
What a fine way to drive away a talented developer and long time contributer. Linus does not get it right all the time.
It's not just designing the rich API, it would also require porting everything to it and dealing with all the corruption bugs that would no doubt result from such an effort -- *nothing* is in a state where it could use the proposed volume manager efficiently.
You mean, use the new functionality of the volume manager efficiently. Because it will of course support the standard block device API, so anything that supports a block device will be able to use it. But only a filesystem aware of the new API would be able to do things like define variable RAID geometry, as ZFS does, or set a region of the block device a "free space" or "all zeroers". We extend in-kernel APIs like that all the time.
If you really want to reuse BTRFS as a volume manager for other filesystems, make yourself a filesystem image file on top of BTRFS.
I understand the preference for a layered approach, but I just don't think it's competitive here.
But you did not say why, other than "designing a rich API is hard". It is also hard to get the last corruption bug out of a program that is more complex than necessary, as a combined LVM+filesystem surely must be. So given two hard tasks, only one of which yields a component that can be shared, I favor the hard API design task.
Your post is bang on. Kernel politics have gotten firmly in the way of making significant advances in integration between RAID/LVM and filesystems, creating a perfect setting for the hype from Sun about how we absolutely have to suck the whole LVM into the filesystem a la ZFS, which a lot of otherwise intelligent people have bought into. When you actually strip away the rhetorical fluff, what you find is that there are certain advantages to storing RAID metadata on the filesystem, among them to avoid wasting time syncing up data that isn't even in use. Thus nugget of truth was exaggerated to the hyperbole that the entire LVM has to be implemented inside the filesystem. This is nonsense as we shall demonstrate in due course.
Meanwhile, Sun deserves a chance to capitalize on the hype, don't you think? They've had little but bad luck lately. And it should be a good lesson for we penguins, that if politics obstructs progress enough, somebody is going to catch up and pass even the mighty Linux machine. Yes, it's true Virginia. Sun has gotten out ahead with ZFS and we are playing catchup. We have only ourselves to blame, complacency and all that.
And it doesn't really matter. Personally I don't care if Sun takes a healthy bite out of Linux share. It's all open source, and progress marches on. Sun merely lit the proverbial fire under Penguin bottoms and usually that has good results.
rying different kernels and different X servers would at least show an attempt at understanding what's going on.
And if you had bothered to read the article, you would have noticed they carefully specified the kernel used in each version, the X server and the compiler.
The EEE had me hopefull we would see really affordable laptops. But then it was a big hit. Prices went up specs went up.
The eee 1000 with 40 GB SSD is less than $500 now. I call that a real bargain. I'm just waiting a respectable interval before I pick one up to round out my eee collection, currently consisting of a 9" model. I justify this by being able to hack more comfortably on the road. Admittedly I'm stretching but the only reason I contemplate this extravagance is the low price. Two of them and barely half the price of the Thinkpad I just got rid of. Admittedly the Thinkpad is much more powerful, but in my experience, not a lot more useful, and I certainly would not carry it around in the flap of my camera bag like I do the eee.
It would be trivial to have a pseudo directory tree (Linux already uses them for/proc and/sys) which allows you to see any regular filesystem via whatever view you felt like having.
I'm glad you think so. As far as I know, serious problems still remain with FiST, the only serious attempt at an in-kernel stacking filesystem solution for Linux that I know of. Things like getting changes in the underlying filesystem to appear on the stacked filesystem to replace old data the stacked filesystem was caching.
The Linux VFS just wasn't designed for filesystem stacking. You'd probably have more luck with a library preload scheme in userspace, but then it wouldn't be just like proc.
One of the things they noticed was that the file system grows very slowly over time, such that you would normally get a bigger hard drive anyway by the time you fill up a drive with Venti.
This claim does not square with the more or less permanent state of disk fullness that many users experience. Me for example. Which comes about because I load the disk up without ever bothering to clean up, until it's full, then I start spending time deleting things, until that finally gets too painful or impossible and then I break down and buy a new disk. Or as I typically end up doing, a new computer with a disk several times the size of the old one.
In a Venti world, those hordes of users would all be forced to buy new disks instead of having the option of deleting files. A fine deal for disk drive makers I guess.
I really like that it doesn't commit everything unless you explicitly tell it to - I've been bitten by that far too many times with Subversion.
The crazy thing about Git is that it doesn't commit changed files that are under version control unless you provide the secret code. Now, if that's the behavior you really want you're in the minority and that behavior should require the option. Instead, Git breaks the default, expected behavior, and violates the principle of least surprise.
Indeed, that's my number one complaint about Mercurial. But it's doable, just not in a pleasant way. On the other hand, try just about anything with git. Like commit. Oh wait, commit -a or it just doesn't do what the naive person would expect.
most of the items that git 'claims' to be better on is something IDE plugins fix
You have no idea what you are talking about. Subversion doesn't even have real branches for crying out loud, and is pathetically slow compared to Git. I have used Subversion extensively and it often makes me feel like slitting my wrists.
By the way, take a look at Hammer if you want to see something closer to the truth than ZFS. It has a cluster replication model too, considerably in advance of ZFS replication.
I don't know about that either. There are consistent reports that ZFS is slower than Ext3 on many common workloads. Also reports of instability.
While I do respect some of the engineering achievements in ZFS, I do not consider it to be the last word in filesystem design, or even the best filesystem for many applications. I also have doubts about the wisdom of some of the design decisions, such as inhaling the LVM into the filesystem, using 128 byte block pointers, and making a distinction between filesystem snapshots and clones.
When you have the LVM and RAID inside the file-system, you can easily do things like do a RAID rebuild of *ONLY* the data the file-system is using.
Surely you can imagine a relatively trivial mechanism whereby LVM raid could be instructed to resync only data which is currently allocated by the filesystem, a "you can treat these ranges as empty until written" advisory.
A humble proposal: We need another layer, between the block layer and the filesystem layer -- call it an extent layer -- which is simply concerned with allocating some amount of space, and (perhaps) assigning it a unique ID. Filesystems could sit above this layer and implement whatever crazy optimizations or semantics they want -- linear vs btree vs whatever for directories, POSIX vs SQL, whatever.
Sun is perfectly able to GPL whatever code of theirs they wish, including ZFS.
Personally, it does not bother me a bit that Sun has not put ZFS under GPL, that is Sun's loss not ours. ZFS is hardly the last word in filesystem design.
You should quietly smile at whatever they come up with and fail to participate in it. Patents, and particularly software patents, are a huge drain on tech industry and a net drain on society. Be part of the solution, not the problem.
Slashdot Mirror
My understanding always was, Linus wasn't too proud of having driven a wedge through the middle of the kernel community, that's what the "Git" was about. Now that he went on to change the entire world of revision control, I would say... the pain was worth it. Probably.
Do users really need an open source desktop suite when they can meet their needs using a server based suite?
I can't meet my needs with a server based office suite, so I need an open source desktop suite.
Lesson learned: Geeks now require social skills, at least on a professional level, or their work might not be taken seriously.
How about the lesson: Geeks leading projects now require social skills or talented developers will leave their projects?
RSDL wasn't included in the mainline kernel because Linus considered Con to be unreliable, and wanted to have a scheduler with a developer he could count on to maintain the code and fix bugs.
What a fine way to drive away a talented developer and long time contributer. Linus does not get it right all the time.
It's not just designing the rich API, it would also require porting everything to it and dealing with all the corruption bugs that would no doubt result from such an effort -- *nothing* is in a state where it could use the proposed volume manager efficiently.
You mean, use the new functionality of the volume manager efficiently. Because it will of course support the standard block device API, so anything that supports a block device will be able to use it. But only a filesystem aware of the new API would be able to do things like define variable RAID geometry, as ZFS does, or set a region of the block device a "free space" or "all zeroers". We extend in-kernel APIs like that all the time.
If you really want to reuse BTRFS as a volume manager for other filesystems, make yourself a filesystem image file on top of BTRFS.
You'd joking, I hope :-)
I understand the preference for a layered approach, but I just don't think it's competitive here.
But you did not say why, other than "designing a rich API is hard". It is also hard to get the last corruption bug out of a program that is more complex than necessary, as a combined LVM+filesystem surely must be. So given two hard tasks, only one of which yields a component that can be shared, I favor the hard API design task.
Your post is bang on. Kernel politics have gotten firmly in the way of making significant advances in integration between RAID/LVM and filesystems, creating a perfect setting for the hype from Sun about how we absolutely have to suck the whole LVM into the filesystem a la ZFS, which a lot of otherwise intelligent people have bought into. When you actually strip away the rhetorical fluff, what you find is that there are certain advantages to storing RAID metadata on the filesystem, among them to avoid wasting time syncing up data that isn't even in use. Thus nugget of truth was exaggerated to the hyperbole that the entire LVM has to be implemented inside the filesystem. This is nonsense as we shall demonstrate in due course.
Meanwhile, Sun deserves a chance to capitalize on the hype, don't you think? They've had little but bad luck lately. And it should be a good lesson for we penguins, that if politics obstructs progress enough, somebody is going to catch up and pass even the mighty Linux machine. Yes, it's true Virginia. Sun has gotten out ahead with ZFS and we are playing catchup. We have only ourselves to blame, complacency and all that.
And it doesn't really matter. Personally I don't care if Sun takes a healthy bite out of Linux share. It's all open source, and progress marches on. Sun merely lit the proverbial fire under Penguin bottoms and usually that has good results.
rying different kernels and different X servers would at least show an attempt at understanding what's going on.
And if you had bothered to read the article, you would have noticed they carefully specified the kernel used in each version, the X server and the compiler.
I just read it and found it pretty devoid of information. It is one of those mindless performance reviews in as many pages as possible.
That is nonsense, this is Phoronix, a consistently reliable source.
The EEE had me hopefull we would see really affordable laptops. But then it was a big hit. Prices went up specs went up.
The eee 1000 with 40 GB SSD is less than $500 now. I call that a real bargain. I'm just waiting a respectable interval before I pick one up to round out my eee collection, currently consisting of a 9" model. I justify this by being able to hack more comfortably on the road. Admittedly I'm stretching but the only reason I contemplate this extravagance is the low price. Two of them and barely half the price of the Thinkpad I just got rid of. Admittedly the Thinkpad is much more powerful, but in my experience, not a lot more useful, and I certainly would not carry it around in the flap of my camera bag like I do the eee.
It would be trivial to have a pseudo directory tree (Linux already uses them for /proc and /sys) which allows you to see any regular filesystem via whatever view you felt like having.
I'm glad you think so. As far as I know, serious problems still remain with FiST, the only serious attempt at an in-kernel stacking filesystem solution for Linux that I know of. Things like getting changes in the underlying filesystem to appear on the stacked filesystem to replace old data the stacked filesystem was caching.
The Linux VFS just wasn't designed for filesystem stacking. You'd probably have more luck with a library preload scheme in userspace, but then it wouldn't be just like proc.
One of the things they noticed was that the file system grows very slowly over time, such that you would normally get a bigger hard drive anyway by the time you fill up a drive with Venti.
This claim does not square with the more or less permanent state of disk fullness that many users experience. Me for example. Which comes about because I load the disk up without ever bothering to clean up, until it's full, then I start spending time deleting things, until that finally gets too painful or impossible and then I break down and buy a new disk. Or as I typically end up doing, a new computer with a disk several times the size of the old one.
In a Venti world, those hordes of users would all be forced to buy new disks instead of having the option of deleting files. A fine deal for disk drive makers I guess.
I really like that it doesn't commit everything unless you explicitly tell it to - I've been bitten by that far too many times with Subversion.
The crazy thing about Git is that it doesn't commit changed files that are under version control unless you provide the secret code. Now, if that's the behavior you really want you're in the minority and that behavior should require the option. Instead, Git breaks the default, expected behavior, and violates the principle of least surprise.
Try deleting a branch some time.
Indeed, that's my number one complaint about Mercurial. But it's doable, just not in a pleasant way. On the other hand, try just about anything with git. Like commit. Oh wait, commit -a or it just doesn't do what the naive person would expect.
The problem with Git is the utter lack of visualization, plugins, and integration with other tools.
Have you tried gitk? Or with Mercurial, "hg view". Awesome, light and tight.
most of the items that git 'claims' to be better on is something IDE plugins fix
You have no idea what you are talking about. Subversion doesn't even have real branches for crying out loud, and is pathetically slow compared to Git. I have used Subversion extensively and it often makes me feel like slitting my wrists.
Mercurial is slower than GIT on Linux
Sometimes slower, sometimes faster, usually about a tie in my experience. Measurements on kernel tree import and initial commit showed roughly a tie.
But Mercurial is _way_ more obvious and pleasant to use than Git. I use both, but any time I have the option I choose Mercurial.
Well, actually, no. Not unless you're suggesting that LVM is appropriate at the file level -- as in, creating a new device per-file.
I do not see what in your description above would require a device per file.
By the way, take a look at Hammer if you want to see something closer to the truth than ZFS. It has a cluster replication model too, considerably in advance of ZFS replication.
I don't know about that either. There are consistent reports that ZFS is slower than Ext3 on many common workloads. Also reports of instability.
While I do respect some of the engineering achievements in ZFS, I do not consider it to be the last word in filesystem design, or even the best filesystem for many applications. I also have doubts about the wisdom of some of the design decisions, such as inhaling the LVM into the filesystem, using 128 byte block pointers, and making a distinction between filesystem snapshots and clones.
When you have the LVM and RAID inside the file-system, you can easily do things like do a RAID rebuild of *ONLY* the data the file-system is using.
Surely you can imagine a relatively trivial mechanism whereby LVM raid could be instructed to resync only data which is currently allocated by the filesystem, a "you can treat these ranges as empty until written" advisory.
A humble proposal: We need another layer, between the block layer and the filesystem layer -- call it an extent layer -- which is simply concerned with allocating some amount of space, and (perhaps) assigning it a unique ID. Filesystems could sit above this layer and implement whatever crazy optimizations or semantics they want -- linear vs btree vs whatever for directories, POSIX vs SQL, whatever.
Woohoo, you've invented an LVM.
It wouldn't be possible to duplicate RAID-Z with LVM.
I often hear that claim but never see any support of that claim.
Sun is perfectly able to GPL whatever code of theirs they wish, including ZFS.
Personally, it does not bother me a bit that Sun has not put ZFS under GPL, that is Sun's loss not ours. ZFS is hardly the last word in filesystem design.
You should quietly smile at whatever they come up with and fail to participate in it. Patents, and particularly software patents, are a huge drain on tech industry and a net drain on society. Be part of the solution, not the problem.
and beware lawyers with mod points.