IBM Promises Logical Volume Management For Linux

Several folks wrote in with news that IBM has announced via mailing list its intention to release a Logical Volume Management (LVM) architecture for Linux. Here is the LVM white paper referred to below. Neither the note below nor the IBM Web site seem to declare what sort of license it will be released under, but the expressed hope "that the release of this technology will lead to a world class logical volume management system for Linux" would fit logically with the GPL, wouldn't it? ;) (More)

An Anonymous Coward writes: "I found the following message posted to several Linux mailing lists:

************************* Begin Message **************

Hello! Since IBM has begun to publicly support Linux, many of our customers have started showing an interest in Linux. We have received many requests from our customers asking us to enhance certain areas of Linux (logical volume management in particular) in order to make Linux a more acceptable platform for their IT operations. Furthermore, we have been asked to provide a migration path from existing platforms (both IBM and non-IBM) to Linux. IBM has been moving to satisfy these requests by contributing developers and technology to the Linux Community. This is what drove IBM's decision to release JFS to the Linux Community, and it is driving the decision to release logical volume management technology to the Linux Community.

IBM is releasing one of its most advanced architectures for a Logical Volume Management System. This architecture is quite interesting as it completely integrates all disk and volume management into a single, highly extensible, easy to use entity. We hope that the release of this technology will lead to a world class logical volume management system for Linux, one which satisfies the requirements of our customers as well as those of the Linux Community.

The first of several white papers describing the LVMS architecture can be found at the IBM Linux Technology Center website:

link

Since we would like to have an honest, open discussion about this, I would suggest that all interested parties post their comments to the LVM mailing list (unless someone has a better suggestion!). All comments are welcome!

Thanks!

Ben Rafanello

IBM Linux Technology Center

PS - Information about the LVM mailing list can be found at:

link

***************** End Message *************************

Well, the message sounded interesting so I checked out their website. I found the specified white paper in the "Documents / White Papers" section. After giving it a quick read, I must say that this isn't like any other LVM I've ever heard of! The use of "plug-in modules" to control what devices the LVM sees, how it partitions those devices, and how it uses those partitions to create a volume is incredibly flexible. I also like the part about the LVM working with the filesystem on a volume so that a volume can be resized without any data loss (and without all of the manual steps that must currently be done on Linux). The paper discusses this LVM as an architecture, and it avoids discussing user interfaces and implementation issues. I wonder if this is some pie in the sky vision or if they have any code to back this up?"

Comments

Linux LVM

[Anonymous Coward]

A good LVM for Linux is here.

Re:Can We Trust IBM?

[_Swank]

One of the main reasons that IBM left "Sun out to dry" as you put it was due to Sun's backtracking on putting Java in the hands of an open standards body and instead to keep it under their own control. You seem to be upset with IBM for standing up for the "open standards are good" that you (and purt near everyone on /.) believe in. And despite this, IBM remains one of the biggest backers (if not the) of Java in the industry. Hardly "leaving Sun out to dry."

The only reason they aren't as powerful today is because they got too big and unresponsive, which led them to make several bad decisions.

That you feel IBM is less powerful today than the late 80's surprises me. By almost every indication financially the company is doing better and the general public (specifically the IT industry) has a more positive view of the company and not (I believe) for the reason you site. So IBM may not have a stranglehold on the tech industry lilke they used to, but that is "power" that they don't want. Look where it got them, and look where it is getting Microsoft.

As for bad decisions hurting IBM, they made their share of them. Their major mistake being their structural divisions of the company into different operating units in preparation for the governments forced breakup that never came. The ways in which they did this killed IBM in more ways than anyone not in the company at the time can even imagine. So even if Microsoft doesn't get divided, it could be very interesting anyway.

And I think that IBM's strategic movements towards Linux and other open source projects are much too broad and pervasive to be an IBM version of Microsoft's "embrace, extend, extinguish" that you believe it is.

Biting the Hand that Feeds You...

[Carnage4Life]

Okay, on behalf of all the slashdotters who realize how much IBM is contributing to Open Source I would like to apologize to all IBMers who read the above post.
Besides Linux pure-plays like RedHat and VA Linux no company gets it more than IBM. Their contributions to Apache are impressive, they are releasing laptops with Linux, implementing a better JDK than Sun's for Linux plus they are behind the AlphaWorks site.But instead of thanking them or being grateful for their contributions certain people feel that they should bitch and moan about how IBM will turn on Open Source (how? By stealing GPLed code???). The tiff with Sun you describe is simply that IBM licenses some Java technologies but refused to pay Sun's exorbitant licensing fees for the J2EE brand. Wow, that is so evil, they implement a better version of Java than Sun then refuse to pay Sun for the permission to call it Enterprise Java.

Ingratitude is sad to witness.

Ben Rafanello responds

[BenR]

I see that IBM's announcement has generated some confusion as well as many questions. I will try to answer all of the questions I can.

First the easy ones:

I wonder if this is some pie in the sky vision or if they have any code to back this up?"

Yes, we have code. IBM has a prototype of this LVMS on at least one of its platforms that I know of.

As long as it's released under a liscense like the GPL, we don't need to trust them

We are currently releasing the technology to the Linux Community. The current white paper provides a high level view of the basic architecture. As the architecture is discussed with the Linux Community, additional white papers will be produced to explain, in greater detail, various aspects of the architecture. Should things progress to the point where we are actually releasing code for some or all of it, then any code we release will be under the GPL.

Heinz Maulshagen and team have already written an LVM for Linux. This LVM is already in the 2.3.x series of kernels. What is IBM's reasoning for this duplication of effort? Might it not be more effective to assist in the current LVM implementation instead of bringing additional complexity? Or are there advantages in IBM's approach to logical volume management?

Heinz and his team have done an excellent job with their LVM. However, we feel that our design has some advantages, and that we are looking at the problem from a different perspective. Let me explain.

IBM's initial concept of a logical volume management system was not all that different from what we find in the Linux Community today. However, over the years, IBM's concept has evolved based upon input from our customers, as well as usability studies performed on our customer base. Our current concept of a logical volume management system integrates disk partitioning and management, bad block relocation, software RAID, encryption, filesystems, logical volume creation, deletion, and management, etc. into a single, coherent, open ended architecture. All of the components of this architecture communicate with the Logical Volume Manager, who co-ordinates their activities and handles all interactions with the user. The Logical Volume Manager is a single program with three interfaces: a command line interface, a text mode interface, and a graphical user interface. Thus, there is a single place for the user to turn to for any aspect of logical volume management. This makes logical volume management easier for the user to learn and use, and it reduces the chances for user error and data loss.

As an example of IBM's concept, lets examine the case where an encrypted volume is to be shrunk. We will assume that the user has already been through any authentication process which may be required to access the encrypted volume and has been granted full access. The user would start the LVM. The LVM would allow the user to select a volume. Once selected, the user would indicate that the volume should be shrunk. The LVM would examine the volume, find which features are in use on the volume (encryption, in this case), find what filesystem is in use on the volume, and then ask each feature that would be affected by the shrink if that feature can handle having the volume shrunk without data loss. If a feature indicates that it can NOT handle having the volume shrunk, then the LVM informs the user that the volume can NOT be shrunk without data loss. If all of the features indicate that they can handle having the volume shrunk, then the LVM would ask the filesystem on the volume how much the filesystem can be shrunk without data loss. If the filesystem indicates that it can't be shrunk without losing data, the LVM will inform the user appropriately. If the filesystem can be shrunk, it will indicate how much it can be shrunk before data loss begins. The LVM will then present this value to the user and allow the user to specify how much to shrink the volume by, limiting the user's choice to only those values that prevent data loss. Once the user has specified how much the volume is to be shrunk by, the LVM will then notify the filesystem to shrink itself. Once the filesystem is done shrinking itself, the LVM will notify the features on the volume that the volume is about to be shrunk. After this, the LVM will actually shrink the volume. Once the volume has been shrunk, the LVM will notify the features on the volume that the shrink has been completed. The filesystem will also be notified that the shrink has been completed. Once all of this has been completed, then the user will be notified that the volume has been successfully shrunk.

The above example is quick and dirty, and it leaves out quite a bit, but I hope it gives you an idea of what we are trying to accomplish. Basically, we are trying to bring together all of the various aspects of logical volume management into a single, cohesive, seamless entity. Furthermore, we are trying to do this in a way which is as flexible and expandable as possible. Consider the following:

The LVMS architecture uses logical disks. Logical disks are created and controlled by plug-in modules called "Device Managers". Thus, anything that can be made to appear as a logical disk through the use of a plug-in "Device Manager" module can be used by the LVMS. As an example, lets say we have a "Device Manager" plug-in which can access a Storage Area Network (SAN). By just adding this plug-in to an existing system, the LVMS would be able to allocate storage on a SAN, make it appear as a logical disk, and then use any of the LVM capabilities that could be used on a local disk. No code in the LVMS, existing plug-in modules, or LVM utilities needs to be modified or changed.

The LVMS Architecture uses logical partitions. Logical partitions are controlled by plug-in modules called "Partition Managers". If you wanted to access a drive that was partitioned for use with a Mac, you would just need a "Partition Manager" plug-in module that understands the partitioning scheme used by the Mac. No LVMS, existing plug-in module, or LVM utility needs to have its code changed. All that needs to be done is have the Mac "Partition Manager" plug-in added to the system.

The LVMS Architecture employs something we call "Feature Plug-ins". Feature Plug-ins control how logical partitions are combined into logical volumes, such as through drive linking, the various forms of mirroring, or software RAID. They can also be used to filter I/O to a volume, as would be the case for encryption. They can also be used to redirect I/O, as in the case of bad block relocation. Furthermore, Feature Plug-ins can be stacked to produce a volume, so that multiple Feature Plug-ins can be used on a single volume (encryption combined with software RAID and bad block relocation, for example). Every volume in the system can employ different Feature Plug-ins, and volumes which do use the same Feature Plug-ins can have them stacked differently. Thus, the user has an enourmous amount of flexibility. One final point to make here is that, since all of the plug-in modules communicate with the LVM, the LVM can coordinate their activities to ensure that the user doesn't do something which will result in the loss of data (at least without adequate warning).

Given all of the above, I hope everyone understands why we think the LVMS Architecture we are releasing has some advantages over other approaches.

Well, this post has gotten excessively long, and I apologize for that. I will try to address additional questions/comments/concerns later, hopefully with smaller posts!

Ben Rafanello

IBM Linux Technology Center

IBM and Linux

[Greyfox]

Linux gives IBM the ability to make good on the IBM dream of having a single OS running across all its hardware platforms. It was originally to be OS/2 but they screwed that up pretty well. They tried to port AIX to the PC archetecture. AIX for Microchannel was probably the shortest-lived creation in the history of information services. Most of their mainframe OSes kind of look like one another but are still pretty different from anything on the PC.

Many IBMers expressed relief (on the IBM internal forums) that Linux was beyond the control of IBM, so the company couldn't screw it up the way they did with OS/2. Many of them are still bitter about that. Many of the old OS/2 guard have moved to Linux, happy to have an OS that lets them work the way they want to work.

I think it's cool that they're adding neat features to the OS. Having them contributing cool features why not trying to manage or market the OS seems to be the right way of doing things. Perhaps they should look at modifying their other business units to work in this sort of paradigm, with IBMers only contributing to projects managed by other business units...

Logical?

[Signal 11]

As opposed to .. what.. illogical volumes? I can just see it now...

~# mount -t ext2 /dev/sda2 /home
Error: fuzzy dice in foo buffer found.

~# mount -t ext2 /dev/sda2 /home -o fuzzy_dice

Error: fuzzy dice requires car.o module

~# insmod car.o
Unresolved symbols: car_need_oil, car_need_gas

~# dd if=/dev/cash of=/dev/bank bs=1 count=300
0+300 records in
0+300 records out

~# dd if=/dev/bank of=/dev/wallet bs=1 count=50
0+50 records in
0+50 records out

~# insmod car.o io=/dev/wallet

~# mount -t ext2 /dev/sda2 /home -o fuzzy_dice
Succeeded.

~# cd /home
~# ls

KERNEL PANIC!

000:000 FF 00 CC G0 BB LE DE G0 0K

Goddamn illogical drives...

Not surprising

[babbage]

All the posts about IBM's relationship with Linux seem a bit odd to me; I can't see why it's such a big surprise or matter of debate.

Think about it. IBM sells a *wide* variety of products, from laptops to mainframes and from software to service. It complicates matters for them if they have to be "multilingual" with their platform systems; Linux gives them a way around this by, as suggested in the Halloween documents, treating the OS as basically a generic commodity.

The more popular Linux gets, and the better IBM's equipment works with it, the better their bottom line gets. If Linux gives them a platform to standardize around, then they can sell more hardware and support services and not have to worry as much about certain problems (compatibility, stability, etc) that live at the OS level. This is very much in their interest, but it's also in the interest of consumers as well, if Linux becomes a reliable system component along the lines of, say, a hard drive or a word processor. The consumer gets the benefit or a robust & rewarding system platform that works on most any equipment available; IBM gets the benefit of having their entire product range appeal to customers because it all speaks the lingua franca of the computer world that they are positively contributing to.

Consider EBCDIC. Wasn't that the standard text format on IBM machines back in the day? Is it in use on any modern system? [I'm asking -- not rhetorical...] It seems to me that they had the wisdom to (eventually) switch to the open ASCII standard before, and are promoting the new Linux standard now. I see no reason to be suspicious of this behavior -- everyone stands to benefit, or at least everyone that isn't trying to shill a proproetary in-house OS... :)

Just say "No" to "logical volumes"...

[jetson123]

I have used LVM for a number of years, including IBM's LVM. As far as I am concerned, logical volumes are a bad idea. Why?

• LVs complicate system management; in particular, they make it more difficult to figure out what physical devices a file system actually depends on, and they make it much more likely that you make a mistake when setting up disks.
• LVs break the correspondence between block numbers and head positions. With simple physical mappings, small differences in block numbers usually correspond to small head movements, something file system designs tend to rely on, but with LVs, all bets are off.
• If you take advantage of LVs spanning multiple disks, you just multiplied your risk of data loss, because if any one of those disks goes, so does the whole file system.

There are better solutions to the problem LVs are supposed to address:

• If you need file systems bigger than a single disk, use RAID.
• GNU Parted and PartitionMagic already provide you with the ability to resize partitions without a full backup and restore; you don't need LVMs for that.
• If you use several file systems to limit how much space particular applications can eat up, consider using quotas instead.
• One of these days, Linux may get concatenated mounts, which would give you another, very reliable and simple way of having file systems span multiple disks. Adding concatenated mounts would probably not be any harder than hacking in an LVM.

IBM's LVM was one of the reasons I hated using AIX (they did similarly oddball and nonstandard stuff in some other areas). I consider it a poorly designed facility. While we can't keep people from porting stuff to Linux, I hope Linux distributions will not incorporate that kind of nonsense; Linux configuration and system management needs to get simpler, not more complex.

The drive for systems like LVM is understandable because UNIX and Linux file systems and large scale data management are not perfect. For example, a big ISP that runs out of disk space on some important partition and needs more space quickly has a legitimate problem. But rather than rushing to a half-baked solution like LVMs, let's identify what the problems are we really want to solve and come up with good solutions to them. With upcoming technologies like object-based disk storage systems, there seem to be much more straightforward and reliable answers than LVM.

LVMS is /not/ LVM

[Will the Chill]

Allrighty, everybody. I think that anyone who starts to think "why would we need another LVM?" should /definitely/ go read the LVMS whitepaper. It's not the same thing as the previous LVM or LVM's that may have existed. Go. Read.

Let me tell you one of the neato things about working at IBM in Austin - I got to talk to Ben (yes, Ben Rafanello, the guy who posted the stuff to the linux mailing lists) all about this last night, and he told me he was planning on posting that stuff. So anyway, I read the LVMS whitepaper last night, and it is /definitely/ something that the Linux community is going to want to jump at.

For those of you who are wondering about what license it will be released under - well, I can't say for sure. Just remember that IBM GPL'd their JFS, and we continue to work on that here as well. (I'm actually working on the JFS here in Austin.)

And yes, IBM /has/ changed alot. They're actually pretty damned cool! I would highly recommend anyone looking for a cool Linux-related job to consider IBM. Hey, look at me! I'm getting paid to write Open-Source Software!

-Will the Chill