Unionfs for Linux?

Lukey Boy asks: "A machine in my network is currently a large fileserver, and holds many hard disks full of media (namely my music and what not). Each drive is running a standard EXT3 filesystem with the same layout (/media, /media/mp3, and so on). My problem and question is how do I join these drives to look like a single hierarchy? I would like to, when I check /all/mp3, view the contents of each drive in this combined directory. FreeBSD has a unionfs filesystem type which supports the unioning of two drives - but only two is a fairly bad limitation, especially when I add a new drive. It appears that Al Viro is working on a unionfs for Linux 2.5, with again only two mount points supported. I was also considering using the Linux Volume Manager system, or possibly a software RAID striping arrangement; does anyone have any experiences doing anything similiar? Is there any decent inheritable filesystem (IFS) available for Unix machines?"

Comments

Use your own suggestion -- LVM

[0x0d0a]

This is *exactly* what LVM is designed to do. Multiple physical volumes comprising one logical volume. Follow up on your own suggestion. :-)

Re:Use your own suggestion -- LVM

[|_uke]

So does the previous poster. LVM is great because it virtualizes the hardware side of things. A volume (virtual partition) can be spread out over any number of drives. I was using LVM a while ago, it works quite well.

The only real problem is volume resizeing. Although LVM has great support for resizeing the actual volumes, there is always a little trickery involved with resizeing a filesystem ON those volumes =)

Anyways, I believe LVM is now part of EVMS.

I would give it a try, LVM is quite cool to use. I was using it when I had to switch all my drives around on a regular basis. Its nice to have a single device for my root partition and not have to make large changes to fstab and whatnot just to get my system to boot =)

(Now with a stable drive layout, I am back to using normal partitions... but thats just because I was too lazy to setup LVM again =)

BTW, nice nickname Lukey Boy :P

Re:Use your own suggestion -- LVM

[Polo]

I believe resizing with LVM is quite easy now. The command is e2fsadm and it knows about both ext2/ext3 and lvm.

Although LVM doesn't join two filesystems at the directory level, what it will do is allow you to have separate disk partitions and allocate and deallocate space from them from a pool made of of all your physical disks.

You can still have your /usr and /var (or /music or /video), but you can also add a new disk and give 20% of it to /music and 60% to /video and leave the rest unallocated for later. Or maybe shrink /music by 10gig and give it to /video.

I'm using redhat 7.3 and lvm is included. The only problem was that I didn't know how to load the lvm module at boot time. In the end, I mucked with /etc/rc.d/rc.sysinit to change this line from:

if [ -e /proc/lvm -a -x /sbin/vgchange -a -f /etc/lvmtab ]; then

to:

if [ -x /sbin/vgchange -a -f /etc/lvmtab ]; then

and everything worked fine.

Re:Use your own suggestion -- LVM

[red_dragon]

You can resize ReiserFS volumes online!

lvextend -L +10G /dev/tardis/stash
resize_reiserfs /dev/tardis/stash

There's no need to unmount a volume before growing it. You still have to unmount it before shrinking it, though; resize_reiserfs will refuse to work if you don't do so.

symlinks?

[Erebus]

rm /all/mp3/*
for f in /media/*
for g in /media/$f/*
ln -s /media/$f/$g /all/mp3/$g
done
done

quick solution

[austad]

You could always go the ghetto booty route and just make symbolic links under the /all directory.

cd /media/mp3
cp -Rl * /all/mp3

Do this for each drive. Or write a cronjob to do it once a day. This may or may not work depending on your directory structure.

I do a similar thing to make my music more "browsable". /media/audio has a subdirectory for each artist, but what if I want to browse by genre? I have /media/audio/genre/[classical,rap,jazz,rock,etc], and each subdirectory has symbolic links back to the artist directories in /media/audio.

Of course, once iTunes has Vorbis support, it won't really matter how the stuff is organized on disk.

Re:why does it hurt when i do this...

[Paul Jakma]

what he's saying is that unionfs is not the way to do it here.

what he wants can easily be done with regular old mounts, eg he could sort his mp3s into 'classical', 'jazz' etc.. and have each one in a seperate dir and on a seperate disk. or he could symlink into the various disks. finally the most transparent way to merge the disks into one is LVM.

--paulj