Slashdot Mirror
IBM Promises Logical Volume Management For Linux
An Anonymous Coward writes: "I found the following message posted to several Linux mailing lists:
************************* Begin Message **************Well, the message sounded interesting so I checked out their website. I found the specified white paper in the "Documents / White Papers" section. After giving it a quick read, I must say that this isn't like any other LVM I've ever heard of! The use of "plug-in modules" to control what devices the LVM sees, how it partitions those devices, and how it uses those partitions to create a volume is incredibly flexible. I also like the part about the LVM working with the filesystem on a volume so that a volume can be resized without any data loss (and without all of the manual steps that must currently be done on Linux). The paper discusses this LVM as an architecture, and it avoids discussing user interfaces and implementation issues. I wonder if this is some pie in the sky vision or if they have any code to back this up?"Hello! Since IBM has begun to publicly support Linux, many of our customers have started showing an interest in Linux. We have received many requests from our customers asking us to enhance certain areas of Linux (logical volume management in particular) in order to make Linux a more acceptable platform for their IT operations. Furthermore, we have been asked to provide a migration path from existing platforms (both IBM and non-IBM) to Linux. IBM has been moving to satisfy these requests by contributing developers and technology to the Linux Community. This is what drove IBM's decision to release JFS to the Linux Community, and it is driving the decision to release logical volume management technology to the Linux Community.
IBM is releasing one of its most advanced architectures for a Logical Volume Management System. This architecture is quite interesting as it completely integrates all disk and volume management into a single, highly extensible, easy to use entity. We hope that the release of this technology will lead to a world class logical volume management system for Linux, one which satisfies the requirements of our customers as well as those of the Linux Community.
The first of several white papers describing the LVMS architecture can be found at the IBM Linux Technology Center website:
Since we would like to have an honest, open discussion about this, I would suggest that all interested parties post their comments to the LVM mailing list (unless someone has a better suggestion!). All comments are welcome!
Thanks!
Ben Rafanello
IBM Linux Technology Center
PS - Information about the LVM mailing list can be found at:
***************** End Message *************************
Your objections don't even make sense, and it's clear you've never worked with enterprise environments:
LVs complicate system management; in particular, they make it more difficult to figure out what physical devices a file system actually depends on, and they make it much more likely that you make a mistake when setting up disks.
Not really, the LVM is always happy to tell you what maps to what, and this is not problem unless you create a logical root volume in which case you deserve what you get. Disks are too cheap these days for that sort of foolishness, and if you need LVM you can afford it.
LVs break the correspondence between block numbers and head positions. With simple physical mappings, small differences in block numbers usually correspond to small head movements, something file system designs tend to rely on, but with LVs, all bets are off.
What planet have you been living on, Mr. Jetson? The geometry the OS sees has had little or no relation to the physical geometry of the disk since disk controllers became more intelligent than the first PCs several years ago. Where things will actually land physically on modern disks has been totally out of your control for years, which is the reason Sun dropped the ability to twiddle cylinder groups way back in the move from SunOS to Solaris.
If you take advantage of LVs spanning multiple disks, you just multiplied your risk of data loss, because if any one of those disks goes, so does the whole file system.
That's why LVMs and RAID travel in packs - they form a symbiotic realtionship. With disks as cheap as they are, mnay people just use 0+1 (mirroring and striping), which eliminates this concern. If you're cheap and don't mind abysmal performance in degraded mode, you can go with 5.
If you need file systems bigger than a single disk, use RAID.
Have you ever done this? RAID tools that do this are inherently *doing* LVM, even if they're not calling it that and it's not a separate tool. RAID per se typically handles Mirroring, Striping and Parity, the volume manager handles concatenation. These are often merged in a single tool, but concatenation of disks is not strictly speaking a RAID function.
GNU Parted and PartitionMagic already provide you with the ability to resize partitions without a full backup and restore; you don't need LVMs for that.
While decent, these are hardly enterprise-class tools. I also sincerely doubt that either is capable of resizing volumes that span physical disks, since that *does* imply LVM functionality, unless there's an LVM hiding under the covers and lying to these tools. There are people that have a real need for single files that are tens or hundreds of gigabytes, so *not* spanning disks isn't really an option.
One of these days, Linux may get concatenated mounts, which would give you another, very reliable and simple way of having file systems span multiple disks. Adding concatenated mounts would probably not be any harder than hacking in an LVM.
I'll confess to being ignorant of concatenated mounts (although I asume this would allow simply allow concatenation at mount time, but LVMs have significant benefits, and the people who need them have been using them quite successfully for years. There's certainly noting that's going to prevent the need for them in the near term. LVMs make "big computing" possible on small computers. There's just no effective way around that.
LVMs may have thier shortcomings, and they aren't appropriate everywhere. Most people won't need them, and the ones that do can handle the minimal incremental complexity. Further, what are often called LVMs are usually today also RAID and JFS tools. This sort of capabilty is not easily duplicated in other ways or we'd have been doing it already for years.
It seems you really are living in Jetson-land as your ID indicates if you think things like "object based disk storage systems" are going to have large-scale real-world impacts anytime soon. Ah, the naivete of youth...
"The future's good and the present is nothing to sneeze at." - Roblimo's last
because those illogical volumes are really a bitch to manage. Like herding cats....
-Denor
A good LVM for Linux is here.
Look at what they're doing with Sun and Java. After working with Sun for 4 years, they refuse to ratify Sun's version of EJB, in effect leaving Sun out to dry. IBM was given access to all of Sun's java IP and this is how they repay them?
I'm just a bit leery (sp ?) of IBM cozying up with Linux and Apache. They may think everyone has forgotten how predatory they are but some of us haven't. I think most people see them as a MS foil and as a result consider them to be one of the good guys. Just remember, folks. 15 years ago, IBM was the oppressor of the tech industry. The only reason they aren't as powerful today is because they got too big and unresponsive, which led them to make several bad decisions.
-- You see, there would be these conclusions that you could jump to
Okay, on behalf of all the slashdotters who realize how much IBM is contributing to Open Source I would like to apologize to all IBMers who read the above post.
Besides Linux pure-plays like RedHat and VA Linux no company gets it more than IBM. Their contributions to Apache are impressive, they are releasing laptops with Linux, implementing a better JDK than Sun's for Linux plus they are behind the AlphaWorks site.But instead of thanking them or being grateful for their contributions certain people feel that they should bitch and moan about how IBM will turn on Open Source (how? By stealing GPLed code???). The tiff with Sun you describe is simply that IBM licenses some Java technologies but refused to pay Sun's exorbitant licensing fees for the J2EE brand. Wow, that is so evil, they implement a better version of Java than Sun then refuse to pay Sun for the permission to call it Enterprise Java.
Ingratitude is sad to witness.
IBM would do well to work with the existing project, or at least to establish a good-faith basis for communucation with the developer community already committed to solutions under public license.
The same could be said of Linus and FreeBSD. I think we're all better off with competition, because Open Source competition benefits both projects.
--
The real question is 'will they actually develop it and make it work?'. The reason it's a question is that IBM has been releasing code to the community for a little while now and targetting it at Linux, but they have not put sufficient developer time into making it work.
Their JFS (journalling file system) is a good example of this. released quite some time ago, no effort to port it to current versions on the kernel and no apparent discussion about that is going on.
The LVM could be similar. *IF* they had released the LVM last year and *IF* they had supported it with a reasonable amount of programming resources we might care (and I say this as a person who has used and *loved* IBM's LVM on AIX). On the other hand, if they just release the code but don't support the transition and don't make an active effort to participate in community development of it, who cares?
Contrast this to SGI who, for all of their failings as a company, are releasing code *and* supporting its inclusion into the kernel through active participation in the linux-kernel list, web sites and other mailing lists. I'd like to see IBM get on board in the same way.
First the easy ones:
I wonder if this is some pie in the sky vision or if they have any code to back this up?"
Yes, we have code. IBM has a prototype of this LVMS on at least one of its platforms that I know of.
As long as it's released under a liscense like the GPL, we don't need to trust them
We are currently releasing the technology to the Linux Community. The current white paper provides a high level view of the basic architecture. As the architecture is discussed with the Linux Community, additional white papers will be produced to explain, in greater detail, various aspects of the architecture. Should things progress to the point where we are actually releasing code for some or all of it, then any code we release will be under the GPL.
Heinz Maulshagen and team have already written an LVM for Linux. This LVM is already in the 2.3.x series of kernels. What is IBM's reasoning for this duplication of effort? Might it not be more effective to assist in the current LVM implementation instead of bringing additional complexity? Or are there advantages in IBM's approach to logical volume management?
Heinz and his team have done an excellent job with their LVM. However, we feel that our design has some advantages, and that we are looking at the problem from a different perspective. Let me explain.
IBM's initial concept of a logical volume management system was not all that different from what we find in the Linux Community today. However, over the years, IBM's concept has evolved based upon input from our customers, as well as usability studies performed on our customer base. Our current concept of a logical volume management system integrates disk partitioning and management, bad block relocation, software RAID, encryption, filesystems, logical volume creation, deletion, and management, etc. into a single, coherent, open ended architecture. All of the components of this architecture communicate with the Logical Volume Manager, who co-ordinates their activities and handles all interactions with the user. The Logical Volume Manager is a single program with three interfaces: a command line interface, a text mode interface, and a graphical user interface. Thus, there is a single place for the user to turn to for any aspect of logical volume management. This makes logical volume management easier for the user to learn and use, and it reduces the chances for user error and data loss.
As an example of IBM's concept, lets examine the case where an encrypted volume is to be shrunk. We will assume that the user has already been through any authentication process which may be required to access the encrypted volume and has been granted full access. The user would start the LVM. The LVM would allow the user to select a volume. Once selected, the user would indicate that the volume should be shrunk. The LVM would examine the volume, find which features are in use on the volume (encryption, in this case), find what filesystem is in use on the volume, and then ask each feature that would be affected by the shrink if that feature can handle having the volume shrunk without data loss. If a feature indicates that it can NOT handle having the volume shrunk, then the LVM informs the user that the volume can NOT be shrunk without data loss. If all of the features indicate that they can handle having the volume shrunk, then the LVM would ask the filesystem on the volume how much the filesystem can be shrunk without data loss. If the filesystem indicates that it can't be shrunk without losing data, the LVM will inform the user appropriately. If the filesystem can be shrunk, it will indicate how much it can be shrunk before data loss begins. The LVM will then present this value to the user and allow the user to specify how much to shrink the volume by, limiting the user's choice to only those values that prevent data loss. Once the user has specified how much the volume is to be shrunk by, the LVM will then notify the filesystem to shrink itself. Once the filesystem is done shrinking itself, the LVM will notify the features on the volume that the volume is about to be shrunk. After this, the LVM will actually shrink the volume. Once the volume has been shrunk, the LVM will notify the features on the volume that the shrink has been completed. The filesystem will also be notified that the shrink has been completed. Once all of this has been completed, then the user will be notified that the volume has been successfully shrunk.
The above example is quick and dirty, and it leaves out quite a bit, but I hope it gives you an idea of what we are trying to accomplish. Basically, we are trying to bring together all of the various aspects of logical volume management into a single, cohesive, seamless entity. Furthermore, we are trying to do this in a way which is as flexible and expandable as possible. Consider the following:
The LVMS architecture uses logical disks. Logical disks are created and controlled by plug-in modules called "Device Managers". Thus, anything that can be made to appear as a logical disk through the use of a plug-in "Device Manager" module can be used by the LVMS. As an example, lets say we have a "Device Manager" plug-in which can access a Storage Area Network (SAN). By just adding this plug-in to an existing system, the LVMS would be able to allocate storage on a SAN, make it appear as a logical disk, and then use any of the LVM capabilities that could be used on a local disk. No code in the LVMS, existing plug-in modules, or LVM utilities needs to be modified or changed.
The LVMS Architecture uses logical partitions. Logical partitions are controlled by plug-in modules called "Partition Managers". If you wanted to access a drive that was partitioned for use with a Mac, you would just need a "Partition Manager" plug-in module that understands the partitioning scheme used by the Mac. No LVMS, existing plug-in module, or LVM utility needs to have its code changed. All that needs to be done is have the Mac "Partition Manager" plug-in added to the system.
The LVMS Architecture employs something we call "Feature Plug-ins". Feature Plug-ins control how logical partitions are combined into logical volumes, such as through drive linking, the various forms of mirroring, or software RAID. They can also be used to filter I/O to a volume, as would be the case for encryption. They can also be used to redirect I/O, as in the case of bad block relocation. Furthermore, Feature Plug-ins can be stacked to produce a volume, so that multiple Feature Plug-ins can be used on a single volume (encryption combined with software RAID and bad block relocation, for example). Every volume in the system can employ different Feature Plug-ins, and volumes which do use the same Feature Plug-ins can have them stacked differently. Thus, the user has an enourmous amount of flexibility. One final point to make here is that, since all of the plug-in modules communicate with the LVM, the LVM can coordinate their activities to ensure that the user doesn't do something which will result in the loss of data (at least without adequate warning).
Given all of the above, I hope everyone understands why we think the LVMS Architecture we are releasing has some advantages over other approaches.
Well, this post has gotten excessively long, and I apologize for that. I will try to address additional questions/comments/concerns later, hopefully with smaller posts!
Ben Rafanello
IBM Linux Technology Center
Many IBMers expressed relief (on the IBM internal forums) that Linux was beyond the control of IBM, so the company couldn't screw it up the way they did with OS/2. Many of them are still bitter about that. Many of the old OS/2 guard have moved to Linux, happy to have an OS that lets them work the way they want to work.
I think it's cool that they're adding neat features to the OS. Having them contributing cool features why not trying to manage or market the OS seems to be the right way of doing things. Perhaps they should look at modifying their other business units to work in this sort of paradigm, with IBMers only contributing to projects managed by other business units...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
http://linux.msede.com/lvm/
Check Freshmeat for updates.
IBM would do well to work with the existing project, or at least to establish a good-faith basis for communucation with the developer community already committed to solutions under public license.
Jeremiah
"Flyin' in just a sweet place,
Never been known to fail..."
~# mount -t ext2 /dev/sda2 /home
Error: fuzzy dice in foo buffer found.
~# mount -t ext2 /dev/sda2 /home -o fuzzy_dice
Error: fuzzy dice requires car.o module
~# insmod car.o
Unresolved symbols: car_need_oil, car_need_gas
~# dd if=/dev/cash of=/dev/bank bs=1 count=300
0+300 records in
0+300 records out
~# dd if=/dev/bank of=/dev/wallet bs=1 count=50
0+50 records in
0+50 records out
~# insmod car.o io=/dev/wallet
~# mount -t ext2 /dev/sda2 /home -o fuzzy_dice
Succeeded.
~# cd /home
~# ls
KERNEL PANIC!
000:000 FF 00 CC G0 BB LE DE G0 0K
Goddamn illogical drives...
IBM promised to bring more and more stuff to Linux..And they are doing it
In this article at ZDNET, they talk about how IBM is releasing their Small Business Pack for Linux. They are trying to port more and more over to Linux. A lot of people don't seem to give them credit. I support this outright...and hope they release more stuff under a more flexible license like the GPL. I think that every corporation can see the light eventually. Here's to hoping.
Think about it. IBM sells a *wide* variety of products, from laptops to mainframes and from software to service. It complicates matters for them if they have to be "multilingual" with their platform systems; Linux gives them a way around this by, as suggested in the Halloween documents, treating the OS as basically a generic commodity.
The more popular Linux gets, and the better IBM's equipment works with it, the better their bottom line gets. If Linux gives them a platform to standardize around, then they can sell more hardware and support services and not have to worry as much about certain problems (compatibility, stability, etc) that live at the OS level. This is very much in their interest, but it's also in the interest of consumers as well, if Linux becomes a reliable system component along the lines of, say, a hard drive or a word processor. The consumer gets the benefit or a robust & rewarding system platform that works on most any equipment available; IBM gets the benefit of having their entire product range appeal to customers because it all speaks the lingua franca of the computer world that they are positively contributing to.
Consider EBCDIC. Wasn't that the standard text format on IBM machines back in the day? Is it in use on any modern system? [I'm asking -- not rhetorical...] It seems to me that they had the wisdom to (eventually) switch to the open ASCII standard before, and are promoting the new Linux standard now. I see no reason to be suspicious of this behavior -- everyone stands to benefit, or at least everyone that isn't trying to shill a proproetary in-house OS... :)
DO NOT LEAVE IT IS NOT REAL
There are better solutions to the problem LVs are supposed to address:
IBM's LVM was one of the reasons I hated using AIX (they did similarly oddball and nonstandard stuff in some other areas). I consider it a poorly designed facility. While we can't keep people from porting stuff to Linux, I hope Linux distributions will not incorporate that kind of nonsense; Linux configuration and system management needs to get simpler, not more complex.
The drive for systems like LVM is understandable because UNIX and Linux file systems and large scale data management are not perfect. For example, a big ISP that runs out of disk space on some important partition and needs more space quickly has a legitimate problem. But rather than rushing to a half-baked solution like LVMs, let's identify what the problems are we really want to solve and come up with good solutions to them. With upcoming technologies like object-based disk storage systems, there seem to be much more straightforward and reliable answers than LVM.
Allrighty, everybody. I think that anyone who starts to think "why would we need another LVM?" should /definitely/ go read the LVMS whitepaper. It's not the same thing as the previous LVM or LVM's that may have existed. Go. Read.
/definitely/ something that the Linux community is going to want to jump at.
/has/ changed alot. They're actually pretty damned cool! I would highly recommend anyone looking for a cool Linux-related job to consider IBM. Hey, look at me! I'm getting paid to write Open-Source Software!
Let me tell you one of the neato things about working at IBM in Austin - I got to talk to Ben (yes, Ben Rafanello, the guy who posted the stuff to the linux mailing lists) all about this last night, and he told me he was planning on posting that stuff. So anyway, I read the LVMS whitepaper last night, and it is
For those of you who are wondering about what license it will be released under - well, I can't say for sure. Just remember that IBM GPL'd their JFS, and we continue to work on that here as well. (I'm actually working on the JFS here in Austin.)
And yes, IBM
-Will the Chill
Creator of RPerl, Scouter, Juggler, Mormon, Perl Monger, Serial Entrepreneur, Aspiring Astrophysicist, Community Organiz