Slashdot Mirror

LVM existed 4 years ago. I used it. Remember the IBM/SUSE relationship in 2001??? Well IBM ported
LVM to Linux and it was included in SUSE since version 6.3! That's over 4 years ago. Technically
SUSE was more advanced than RedHat. The only thing Redhat was good generating money from the blood and sweat of open source developers. SUSE innovated.

The Logical Volume Manager (LVM) - Part 1

Michael Hasenstein

This document describes the LVM in SUSE LINUX. It is freely distributable as long as it remains unchanged. The original version of this document (PDF) can be obtained at http://www.suse.com/oracle/.

The Logical Volume Manager (LVM) - Part 1

Michael Hasenstein

This document describes the LVM in SUSE LINUX. It is freely distributable as long as it remains unchanged. The original version of this document (PDF) can be obtained at http://www.suse.com/oracle/.

SUSE has included a Logical Volume Manager since SUSE LINUX 6.3. The LVM in SUSE LINUX is Heinz Mauelshagen's implementation, the homepage is now available at http://www.sistina.com/lvm/.

SUSE Inc. 2001
Michael Hasenstein

Like I said, I've been active in the Linux Community since 1992.

But it can be helpful in the future to dedicate, say 10% of your drive to an LVM snapshot space....

I haven't done this yet (I'm lucky! I have a real tape drive to backup my stuff.....) but I plan to make my system take a snapshot every hour and every day (total of two) so that at most I lose an hour's worth of work.

Also, I've always wondered if it was possible to make an operating system that would take as long to destroy something as it did to create it. For example, your term paper took ten days to write, so the rm termpaper.tex command would take ten days to run :)

You'll be wanting a distributed cluster filesystem. There are several available, with their pros and cons. They are also all aimed at enterprise / HPTC installations. For home use you'll be better off with a set of RAID disks.


GPFS from IBM. This is free for academic use, but you pay for commercial use. Linux or AIX only.


GFS from sistina. Commercial offering. Linux only.


Lustre. This is beta quality code, but is freely available. It might work wonderfully, or it might eat your files.


(open)AFS. Works, but has limitations. It does not support large files and clients aren't available for all OSes.

In short, it doesn't. That has to be done by the filesystem layer or application layer itself. A Fibre Channel-based SAN setup such as is common in enterprise deployments doesn't normally provide for this either (Fibre Channel zoning is an exception, but that's a feature not normally required by most setups). If you really want to do concurrent filesystem access by multiple machines, you need a distributed lock manager of some kind, similar to what you have in Oracle RAC's cluster filesystem, Sistina's Global Filesystem, or OpenGFS.

Linux LVM already does something like this.

Nope. LVM was done by Sistina, I've been running it on my lightly loaded home server for ~3 years with no major problems. Minor problem with my mp3 collection which is on Reiserfs over LVM which has been extended a couple of times (as you do). Some of the files are corrupted, starts off as Thin Lizzy then jumps to a Cure track and then Dire Straits. I don't know if it was the resize_reiserfs that did it or the lvextend (or my PC objecting to Dire Straits). I have extended another logical volume which is ext3 over LVM without any problems but I don't think you can draw any conclusions from that. Just my .5 Euro worth.

LVM has snapshot capabilities built in, independent of the filesystem (as it should be!).

I haven't used it in a production environment yet, but I've experimented with it and it seemed to work well.

I chose ReiserFS over ext3 because it allows online resizing of my filesystems (in combination with the excellent LVM of course).

I heard Linux LVM does snapshots too. So FreeBSD is actually behind Linux in this.

This is a prime example of what's wrong with slashdot today. Whoever modded this up is a fucking idiot. Let's break it down:

You heard linux LVM does snapshots. So basically, you have no idea. Shot in the dark. I don't know why you used your karma bonus on such an obvious gem of wisdom. But hey, you were right at least.

"So FreeBSD is actually behind Linux in this." I like how you made an assumption, and then stated a fact based on that. Karmaworthy indeed. The thing is, YOU'RE WRONG. But nevermind that, I knew you were an idiot already.

What really bothers me is, I know I'm not the only who followed this exact same train of thought, but here two days later I'm the only one replying to smack this guy down. So instead of the majority of people who read this then remembering the real facts, they're going to remember what this guy said and probably go around quoting it in a smug manner thinking they're oh-so-leet.

Until they meet me, and I kill them.

Linux is about the only... major OS that doesn't have some kind of filesystem-snapshot support.

You do realise that dump doesn't give you a filesystem snapshot? Even on Solaris - the most venerable of modern UNIX - the manpage for ufsdump clearly states:

When running ufsdump, the file system must be inactive; otherwise, the output of ufsdump may be inconsistent and restoring files correctly may be impossible.

There's a good reason why nobody seriously uses dump anymore.

And Linux does support filesystem snapshots. The Linux LVM explicitly lists it as a feature.

Moderators, this person was not informative, they were simply wrong.

Yes there is. It's called LVM. I've used its snapshot capabilities before on my Linux server, it's very nice.

Try out LVM from Sistina. It's part of 2.4.x, why not use it? ;-) In fact, you can "apt-get install lvm10" on Debian Woody. If you want to compile it yourself, it's pretty easy. Sistina did a good job wrapping up the software and kernel patches with a well documented installation procedure. I couldn't live w/o my LVM.

Try out LVM from Sistina. It's part of 2.4.x, why not use it? ;-) In fact, you can "apt-get install lvm10" on Debian Woody. If you want to compile it yourself, it's pretty easy. Sistina did a good job wrapping up the software and kernel patches with a well documented installation procedure. I couldn't live w/o my LVM.

Personally, I've always found LVM to be a bit disconcerting -- but I'm an admitted Sun bigot who's simply used to Veritas. Everyone's mileage may vary.

AFAIK, LVM is a completely independent development effort by Sistina, though obviously based on IBM's LVM from AIX (or earlier?).

If you're going to share a disk/fs between multiple machines, you will need a filesystem capable of performing proper file locking in order to avoid data corruption and race conditions.

Global File System (aka GFS) can do this. I believe that it was originally developed under a OSS license, but eventually went commercial. There's rumors of a GNU/GPL GFS (called OpenGFS) but I don't have many details as to the maturity of the project, or any experience with it at all.

I found GFS's learning curve to be pretty steep, but if I was able to set it up, I'm sure that you can work through it.

Lastly, I have only used GFS with a SAN cluster, connecting multiple machines via fabric fibre channel (you might want to consider into using a third box as a RAID host). I know that you are using a very different solution than I did, on a different budget -- so YMMV.

I hope that this is helpful to you.

--Turkey

The Linux LVM (Logical Volume Manager) supports this, take a look at the LVM page

Use LVM, filesystem snapshots, and tar (or cpio).

Ahem. I assume you mean VxVM (which is what does the partitioning). And since you ask, there are three things that do that on Linux: LVM, EVMS, and yes, genuine Veritas VxVM (and also VxFS thrown in, to boot). Solaris may be better at some things than Linux, but the number of things that fall in that category is shrinking rapidly. Volume management no longer qualifies.

Anyone know how well Gentoo works with LVM, Linux's logical volumn manager? I run Mandrake 8.1 at work and probably won't switch but at home, I also run Mandrake and would likely switch if the dist properly supported lvm. Particularly if that dist supported KDE 3 which I see Gentoo does.

Shoot The Other Machine In The Head. An Explanation. I'm serious.

Then again, if it's just for checkpointing (create snapshot), installing experimental stuff and then committing (delete snapshot) or rollback (restore from snapshot, delete snapshot), it should do the trick wonderfully.

If I had my choice on what to implement for the ultimate network distributed filesystem, I would concentrate on LDAP, GFS, and Kerberose. LDAP, by its very nature was designed to be a distributed, redundant resource locator and data respository. It can be back-ended by any number of engines, including your more popular RDBMS's. It may seem a bit overwhelming, but well worth the investment in time and energy. Check out the OpenLDAP site for more information.

The second issue you're trying to address is data redundancy and failover. You want a high-availability solution. Look into using the Linux Global Filesystem (GFS). In a nutshell, it's a clustered journaling filesystem whose participants are equally responsible for the data on disc. If one of the servers in the cluster goes down, the first server to see it plays back the unfinished journal of the downed server, and the whole cluster continues on its merry old way.

So, it would be one GFS+LDAP cluster with multiple 1U, fiberchannel servers attached to a fiberchannel disc array. Tack on a gigabit ethernet backbone, and you've got a winner.

Three little words: logical volume manager

Just take a look at the 2.4.16 changelog. There really weren't that many changes to the kernel, and this bug is a fairly troublesome one. I would only sit on 2.4.15 if I had a UPS and I touched the /forcefsck file in root (you should do that now, anyway).

There really is no reason NOT to install the new kernel. You probably haven't racked up much uptime anyway, and not that uptime on 2.4.15 is really worth bragging rights anyway.

Personally, I upgraded when 2.4.16-pre1 came out. I also converted many of my partitions to ext3 (finally). I've been waiting for ext3 to be merged in with stable for a very long time!

Another improvement that wasn't detailed because of the famous "...merge with Alan..." messages in the ChangeLog was that most of LVM is up to date in the stable kernel now. LVM has been at the 1.0.1rc4 release for some time now, and not having to patch my kernel is pretty nice (although, the LVM crew made creating patches quite simple). If you haven't checked out LVM yet, do so. It's quite sweet!

GFS is now commercial-only (fairly "cheap" from a business point of view - 1000 US$/node before rebates and special offers, but completely unreasonable for home hackers who'd like to try it for the heck of it - might be fun trying to combine it with iSCSI).

OpenGFS seems to have taken off in the free software side of the camp.

Cluster-wide locking requires applications understanding it, so it's not easy. I'm not sure, for instance, what would MySQL (a popular app that might benefit from this) if two processes tried to access the same storage read-write concurrently - even if the locking semantics were perfectly implemented by the filesystem.

Where I work I'm trying to set up a mixed active/standby+active/active configuration (shared Fibre Channel-connected storage, applications that can run independently do so, and those who cannot run in hot-standby). I'm almost ready to go live (glee).

This limit is for a SINGLE IDE disk. Now, if you use Logical Volume Management (which is in the standard 2.4 kernel, no patches required) you can combine multiple disks into one.

Since my machine has 2 IDE controllers, with 2 buses each, and 2 drives per bus, you could make a system with 8 144 pB drives, put an XFS partition on it, and have 1152.92 pB of storage.

And for meaningless statistics sake: I make my MP3s (from CDs that I own, thankyouverymuch) at an average of 160 kb/sec. At that rate, the specified drive array would store 1826693 YEARS of MP3s. None of which would be Brittany Spears.

I don't get it. I'm laying in bed, on my laptop with wireless lan. i'm very drunk, and linus torvalds is laying next to me. He gets fed pouches (of food) rather than patches from alan. alan is another cat but ive lost his picture hehe...
linus just seems to want to sleep! How is he supposed to hack VM being just a cat?
On a more sane note - I think Alan has done a wonderful job. I use -ac kernels almost exclusively, apart from my main machine where I use LVM which doesn't apply very well to the -ac kernels.
I read alan's diary regularly, if only for the british humour that american's just don't get :)
I personally would love a job where I could rise after mid-day and hack on kernels all night and eat lots of fish fingers.
I am however quite glad that Alan will be able to concentrate on other things - from what I've seen of his diary, he seems more suited to hacking around, trying new things for the hell of it, rather than just keeping something working.
Best of luck to Marcelo, I am sure you have Interesting Times ahead!

There is a possibility of using MOSIX together with GFS (which gives true device sharing) so that you don't need to use something like NFS. This way, a migrated process will be able to access the device directly, without needing to go through its home node.

AFAIK, this option is still not production-level, though.

You forgot to mention the Global File System (GFS). Not only is it a journalling file system, but it also allows speedy access to network drives. Its NFS on 'roids.

--
I hope we shall crush in its birth the aristocracy of our monied corporations ...

-dB

Now you guys can all test GFS (GLobal Filesystem) it's yet another journaled file system for linux. However this one is different. It's cluster aware. With this FS you can really use all those disks and FC cards you are buying from E-Bay. You can actually mount the same fs as though it's local on all the machines that can see the drives on your FC-AL or Fabric.

Connect the machines to a storage area network and store the data on GFS!!!

I'm in the same situation, but my options are wide open (I haven't decided on any products yet). I have been researching Global File System (GFS) http://www.globalfilesystem.com.

At that web site they have a section where you can see the current status of all the Fiber Channel controllers and drivers for Linux, including other people's experience with different FiberChannel cards.
Namely, you can find it here: http://www.sistina.com/gfs/howtos/fibrechannel_how to/node6.html.
Hope that helps.

NAS is nice since there are a lot of simple off-the-shelf solutions that allow you to put a bunch of disks up with a server that can be accessed by many computers at the same time for both reading and writing. NFS is simple old technology with support in any $500 Linux box with a $20 ethernet card. The disadvantage is that it is slow ... as much as 100x slower than local hard drives due to all of the networking overhead.

True SAN gets rid of the TCP/IP and NFS and just directly attaches disks to computers using something like a Fibre network (SCSI-3). This is blazingly fast (approximately local HD speeds), but requires more complex networking. Since each computer is basically mounting SCSI devices, you also don't have any easy way to have multiple computers that can read and write from the same SAN storage. Shared-SAN software is in the pipeline from Tivoli and Veritas, but you might want to take a look at Global File System, which allows you to have multiple Linux boxes on a Fibre SAN (or SCSI bus!) read and write from the same disks.

With Oracle coming out with Oracle Parallel server Linux *will* have a commercial clustering product.

I'm hoping that GFS (http://www.sistina.com/gfs/) will help with further clustering developments. The nice thing with GFS will be that two servers can work together or failover for a particular task without having to worry about keeping data in sync. Check it out!

-dB

SGI previously released their FailSafe application monitoring and restart service. Having the Phoenix stuff underneath it and available for the GFS file system, and using the existing linux-ha bits would pretty much be a complete cluster solution. That would be good.

-dB