Unionfs for Linux?

Lukey Boy asks: "A machine in my network is currently a large fileserver, and holds many hard disks full of media (namely my music and what not). Each drive is running a standard EXT3 filesystem with the same layout (/media, /media/mp3, and so on). My problem and question is how do I join these drives to look like a single hierarchy? I would like to, when I check /all/mp3, view the contents of each drive in this combined directory. FreeBSD has a unionfs filesystem type which supports the unioning of two drives - but only two is a fairly bad limitation, especially when I add a new drive. It appears that Al Viro is working on a unionfs for Linux 2.5, with again only two mount points supported. I was also considering using the Linux Volume Manager system, or possibly a software RAID striping arrangement; does anyone have any experiences doing anything similiar? Is there any decent inheritable filesystem (IFS) available for Unix machines?"

Comments

MOSIX

[MBCook]

I belive that this is what the Mosix file system does. You can find out at the Mosix website (sorry I don't know the URL off the top of my head).

Of course, you could always port unionfs ;)

Use your own suggestion -- LVM

[0x0d0a]

This is *exactly* what LVM is designed to do. Multiple physical volumes comprising one logical volume. Follow up on your own suggestion. :-)

Re:Use your own suggestion -- LVM

[|_uke]

So does the previous poster. LVM is great because it virtualizes the hardware side of things. A volume (virtual partition) can be spread out over any number of drives. I was using LVM a while ago, it works quite well.

The only real problem is volume resizeing. Although LVM has great support for resizeing the actual volumes, there is always a little trickery involved with resizeing a filesystem ON those volumes =)

Anyways, I believe LVM is now part of EVMS.

I would give it a try, LVM is quite cool to use. I was using it when I had to switch all my drives around on a regular basis. Its nice to have a single device for my root partition and not have to make large changes to fstab and whatnot just to get my system to boot =)

(Now with a stable drive layout, I am back to using normal partitions... but thats just because I was too lazy to setup LVM again =)

BTW, nice nickname Lukey Boy :P

Re:Use your own suggestion -- LVM

[Polo]

I believe resizing with LVM is quite easy now. The command is e2fsadm and it knows about both ext2/ext3 and lvm.

Although LVM doesn't join two filesystems at the directory level, what it will do is allow you to have separate disk partitions and allocate and deallocate space from them from a pool made of of all your physical disks.

You can still have your /usr and /var (or /music or /video), but you can also add a new disk and give 20% of it to /music and 60% to /video and leave the rest unallocated for later. Or maybe shrink /music by 10gig and give it to /video.

I'm using redhat 7.3 and lvm is included. The only problem was that I didn't know how to load the lvm module at boot time. In the end, I mucked with /etc/rc.d/rc.sysinit to change this line from:

if [ -e /proc/lvm -a -x /sbin/vgchange -a -f /etc/lvmtab ]; then

to:

if [ -x /sbin/vgchange -a -f /etc/lvmtab ]; then

and everything worked fine.

Re:Use your own suggestion -- LVM

[jaffray]

I use LVM with reiserfs on all my boxes - resizing both the volume and the filesystem is easy. For instance, if I want to grow my MP3 volume by 10G: "umount /riaa; lvextend -L +10G /dev/quadcuda/riaa; resize_reiserfs /dev/quadcuda/riaa; mount /dev/quadcuda/riaa /riaa". Done. Shrinking is just as easy, but in that case the filesystem resize happens before the volume resize instead of vice versa.

Looks like you could do the same with ext2 and resize2fs if you wanted. If I recall correctly, when I set up my first box with LVM, resize2fs had lots of "beware, this is somewhat experimental code" warnings and I was reluctant to use it, which was one reason why I used reiserfs instead. Maybe a couple more years of testing have increased confidence in the utility, since those warnings aren't there now. I'd rather use reiserfs anyway.

Re:Use your own suggestion -- LVM

[red_dragon]

You can resize ReiserFS volumes online!

lvextend -L +10G /dev/tardis/stash
resize_reiserfs /dev/tardis/stash

There's no need to unmount a volume before growing it. You still have to unmount it before shrinking it, though; resize_reiserfs will refuse to work if you don't do so.

Re:Use your own suggestion -- LVM

[Hast]

The bad thing about LVM is something I've experienced first hand. Harddrive failiure. I lost 200G of data because the first disk on a LVM failed and took the others with it. Since then I've stayed away from it.

Otherwise it's a nice system though. But I'd say use RAID5 to create redundant storage, then use LVM to put these together into one logical unit.

quick solution

[austad]

You could always go the ghetto booty route and just make symbolic links under the /all directory.

cd /media/mp3
cp -Rl * /all/mp3

Do this for each drive. Or write a cronjob to do it once a day. This may or may not work depending on your directory structure.

I do a similar thing to make my music more "browsable". /media/audio has a subdirectory for each artist, but what if I want to browse by genre? I have /media/audio/genre/[classical,rap,jazz,rock,etc], and each subdirectory has symbolic links back to the artist directories in /media/audio.

Of course, once iTunes has Vorbis support, it won't really matter how the stuff is organized on disk.

why does it hurt when i do this...

[kevin lyda]

there's an old joke about a guy going to a doctor who complains that it hurts when he holds both his arms above his head and yodels. (or something like that) the punch line?

don't do that.

unionfs with two fs's really isn't all that great idea. and no offence, but unionfs with more then two fs's is monumentally stupid. where do new files get created? how do you deal with conflicts? what happens with files that you edit? what if you remove a file that exists on more then one fs?

there are a slew of different answers to these questions so either the kernel inflicts (and yes, i mean inflicts) policy or you have to provide a way to configure all those options. should mounting fs's get as complicated as firewall rules?

there are lots of solutions to your problem. pick one of them rather then a bigger problem.

Re:why does it hurt when i do this...

[Paul Jakma]

what he's saying is that unionfs is not the way to do it here.

what he wants can easily be done with regular old mounts, eg he could sort his mp3s into 'classical', 'jazz' etc.. and have each one in a seperate dir and on a seperate disk. or he could symlink into the various disks. finally the most transparent way to merge the disks into one is LVM.

--paulj

Re:why does it hurt when i do this...

[adolf]

Witty. Constructive. +4.

I like it. Not.

If "there are lots of solutions" to the problem, showing at least one of them would perhaps turn outright discouragement into something useful.

The problems are obvious. And they have been solved for a long time - witness TVFS under OS/2. Does something similar exist for a free OS? LVM might be close.

Mounting filesystems should be no more nor less complicated than is needed to achieve the desired goal. If this means that it's "as complicated as firewall rules," so be it.

Seems that this is already the case, anyway. I've got three lines of firewall rules. The fstab on the same machine trounces that handily, without doing anything creative or silly - just mounting various partitions to various points.

(and before anyone asks how it can possibly be secure, I'll say this: It's -STABLE, and that's good enough for me.)

unionfs 2-fs limitation shouldn't be a big deal

[polymath69]

if they can be chained...

For example, maybe you have

/dev/hda1:/media/mp3 as /tmp/a/mp3
/dev/hdb1:/media/mp3 as /tmp/b/mp3
/dev/hdc1:/media/mp3 as /tmp/c/mp3
/dev/hdd1:/media/mp3 as /tmp/d/mp3
...
/tmp/join/a unions /tmp/a/mp3 and /tmp/b/mp3
/tmp/join/b unions /tmp/c/mp3 and /tmp/d/mp3
/all/mp3 unions /tmp/join/a and /tmp/join/b

Mind you, I haven't looked at whether any given unionfs implementation would support this configuration, but loopback-type support typically isn't hard.

As as aside... wherever I type "space slash", slashdot eats the space, making the above a little harder to read than it should be. Anyone know why that is, and/or a workaround?

Soudns like TVFS

[LordNimon]

The Toronto Virtual File System for OS/2 does exactly this. It lets you combine multiple directories into one mount point. You can even designate individual directories as read/write or read-only. It's used by OS/2 developers a lot. You can overlay a read-write empty directory onto a read-only directory that has the SDK in it. Whenever you change a file, the changed file gets written to the read-write directory, hiding the read-only version.