Cross Platform, Low Powered Home Servers w/ RAID?

Milo_Mindbender asks: "At home I've collected too much data to easily backup, so I've been thinking about RAID5 for a little extra data security. I multiboot my computers for both Linux and Windows so I really need a RAID solution that will make the data at least readable by both OS's. I don't think this can be done on a single machine (can it?) so I'm looking to put together a Linux home server with RAID5 serving both SAMBA and NFS. Aside from the usual questions (software/hardware RAID, types of disk to use...etc) because I live by myself in an apartment I have a few tricky requirements I hope the Slashdot crowd can help me with." How would you set up a RAID5 server to perform Samba/NFS sharing duties without it wasting a lot of wattage, while it idles? "I hate to waste electricity, so how can a Linux RAID5 server be setup to automatically spin down to the lowest possible standby power use, then spin back up when a computer accesses it? I don't have a basement, garage or other remote place to put the thing, so it needs to be quiet or at least not die a thermal death if I lock it in a closet. What's the sweet spot for choosing CPU type/speed, hardware/software RAID controller, motherboard and memory to make a home server? Since this is only going to be serving a few machines (and maybe doing router/gateway duty), I'm sure there's a point where adding more CPU horsepower doesn't improve performance much. Any suggestions on motherboards, cases or even complete systems that work particularly well for this kind of small headless home server?"

Comments

Yep ..

[douggmc]

I use an AOpen i855 motherboard w/ a Pentium M proc. There is a newer one from Aopen called i915 also. I use it as a gateway/firewall/server (use a distro called clarkconnect .. http://www.clarkconnect.org/ works wonderfully). Low power, quiet, cool ... ** But why NFS? Just use Samba.

Don't worry about CPU

[MerlynEmrys67]

Pick an old slow CPU, it really doesn't matter at 100Mbs speeds, if you were going multi gig - well then maybe. However don't skimp on RAM - put as many 1 GB sticks in it as you have memory slots (2 GB sticks are too expensive right now, but maybe)

Go with slower hard drives, ie 7200 RPM drives, maybe slower - and you won't have the heat problems. However you might want to look into RAID 15, so if you can get a system that will hold 6, even better.

Now remember, to drop back CPU power, and raw disk speed for the thermal/power savings

Search..

[Daxster]

Have you even searched Slashdot, let alone google?!
http://ask.slashdot.org/article.pl?sid=05/09/25/02 59222&tid=198&tid=4
http://ask.slashdot.org/article.pl?sid=05/10/07/23 1223&tid=198&tid=230&tid=98&tid=4
http://ask.slashdot.org/article.pl?sid=05/11/09/02 0242&tid=232&tid=198&tid=126&tid=4
http://ask.slashdot.org/article.pl?sid=05/04/27/15 20246&tid=222&tid=198&tid=230&tid=4

*cough*

I'm using something like that...

[TheWanderingHermit]

First, I have to say I'm truly awed that you have that much data. You must really love collecting pr0n -- er, have a lot of sound and video files.

I recently had to set up two new servers. One is for business, and one is for personal data. For both, I used RAID 5. They run NFS and Samba, with different directories shared as needed to other systems. RAID 5 is EXTREMELY simple to set up (it's a one line command, once you install mdadm, which, on Debian, installed like a dream), and I'd just suggest Googling for mdadm and tutorial. You'll get several tutorials. There's really no need to pay for hardware RAID cards on Linux (unless you're using an old, slow system). Besides, until you get into the range of something like $300, the RAID cards all do the work through drivers anyway, so you might as well just get a cheap ($10-$20) PCI IDE Controller card to add to your existing IDE channels. Just make sure it works on Linux and is NOT Adaptec (they fsck with the drive order).

On both my systems, all the drives are the same size and model number. I figure you can't always tell if a 160GB drive is 160GB or 140GB, and I didn't want to mess with that. RAID 5 takes 3 drives, but with mdadm, you can add a spare for failover (and the monitoring daemon will e-mail an account on that system in case of failure, so you have a warning to replace the bad drive). My only concern about using the same model for all drives is that there may be a flaw in that model. I found drives that were given a large number of good reviews at NewEgg.

You can also add more spares and more devices with mdadm, or replace faulty devices (not hot swappable, unless you have special hardware, and I don't even know if Linux supports that).

One last note on mdadm: when you first set up a RAID 5 array with it, you'll get an immediate warning of something like a degraded event. This is normal. I think (can't remember details) mdadm and the kernel (mdadm is by the person who wrote the RAID code for the Linux kernel) don't do an exact version of RAID 5 and, instead, use something that lets it rebuild on a new drive faster than it would otherwise.

Re:I'm using something like that...

[swillden]

You can also add more spares and more devices with mdadm, or replace faulty devices (not hot swappable, unless you have special hardware, and I don't even know if Linux supports that).

Here's another tip: If you're using Linux software RAID, carve your drives into multiple partitions, build RAID arrays over those, then use LVM to weld them into a larger pool of storage. It may seem silly to break the drives up into paritions, just to put them back together again, but it buys you a great deal of flexibility down the road.

Suppose, for example, that you had three 500GB drives in a RAID-5 configuration, no hot spare. That gives you 1TB of usable storage. Now suppose you're just about out of space, and you want to add another drive. How do you do it? In order to construct a new, four-disk array, you have to destroy the current array. That means you need to back up your data so that you can restore it to the new array. If there were a cheap and convenient backup solution for storing nearly a terabyte, this topic wouldn't even come up.

If, instead, you had cut each 500GB drive into ten 50GB partitions, created ten RAID-5 arrays (each of three 50GB partitions) and then used LVM to place them all into a single volume group, when it comes time to upgrade, you will have another option. As long as you have free space at least equal in size to on of the individual RAID arrays, you can use 'pvmove' to instruct LVM to migrate all of the data off of one array, then take that array down, rebuild it with a fourth partition from the new disk, then add it back into the volume group. Do that for each array in turn and at the end of the process you'll have 1.5TB, and not only will all of your data be safely intact, your storage will have been fully available for reading and writing the whole time!

Note that this process isn't particularly fast. I did it when I added a fifth 200GB disk to my file server, and it took nearly a week to complete. A backup and restore would have been faster (assuing I had something to back up to!). But it only took about 30 minutes of my time to write the script that performed the process and then I just let it run, checking on it occasionally. And my kids could watch movies the whole time.

For anyone who's interested in trying it, the basic steps to reconstruct an array are as follows. This example will assume we're rebuilding /dev/md3, which is composed of /dev/hda3, /dev/hdc3 and /dev/hde3 and will be augmented with /dev/hdg3

• pvmove /dev/md3 # Move all data off of /dev/md3
• vgreduce vg /dev/md3 # Remove /dev/md3 from the volume group
• pvremove /dev/md3 # Remove the LVM signature from /dev/md3
• mdadm --stop /dev/md3 # Stop the array
• mdadm --zero-superblock /dev/md3 # Remove the md signature from the disk
• mdadm --create /dev/md3 --level=5 --raid-devices=4 /dev/hda3 /dev/hdc3 /dev/hde3 /dev/hdg3 # Create the new array
• pvcreate /dev/md3 # Prepare /dev/md3 for LVM use
• vgextend vg /dev/md3 # Add /dev/md3 into the array

In order to make this easy, you want to make sure that you have at least one array's worth of space not only unused, but unassigned to any logical volumes. I find it's a good idea to keep about about 1.5 times that much unallocated. Then, when I run out of room in some volume, I just add the 0.5 to the logical volume, and then set about getting more storage to add in.

Re:I'm using something like that...

[swillden]

I'll follow your tip, but will probably add boot and swap partitions to every drive. Not because it's needed on every drive, just to keep the setup consistent over all drives.

I did that too. I also had a couple of drives which were actually slightly bigger than 200GB, so I used the extra space for /root partitions (mirrored).

This will work with those newfangled extended partitions, right? Didn't use those since the days we dual booted DOS and OS/2.

Yep. In fact, to keep the numbering clean, it's a good idea to make all of your RAID partitions extended. That means they start at, eg., hda5 (1-4 are primary partitions). It's also a good idea to use RAID device numbers that correspond to the numbers of the drives they contain. So, put hda5 in md5.

If a drive dies, essentially the 10 raids will go on degraded and once I pop in a new drive, I can go and rebuild them with the LVM rather uninterested in the whole process.

Right. LVM has no idea anything has occurred unless the RAID goes beyond degraded mode and actually fails.

However, what happens if the new drive is bigger than the old one?

If you have a RAID-5 made of, say 50GB partitions, you lose one of them (because the drive died) and then you add a new partition back into it, that new partition must be at least 50GB, and if it's larger than 50GB the extra space will not be used.

However, just because you added a larger drive doesn't mean you need to make the partitions on the new drive larger. You could just make more of them. I recently added a 250G drive (the fifth in my box) and I used 200GB of it as hot-spare partitions for my RAID arrays and the other 50GB I just mounted for use as scratch space for stuff I don't care if I lose.

Can I extend the raids while rebuilding them, or do I have to rebuild and then extend one after the other? How does that work?

You can extend them while rebuilding, but it would make me (perhaps irrationally) nervous. To extend, the first thing you have to do is to get your data moved off of the array you want to extend. The array will have to continue running in degraded mode during that time, and it will take significantly longer to perform the pvmove operation than it would to resynchronize a new partition into the array. For my partitions, which are 20GB, it takes about 20 minutes to get an array back to full operational mode from degraded mode. It takes two to three hours to migrate that data to another partition, during which time losing another disk would be fatal to my data. It's unlikely that anything's going to happen in that time, but I prefer to get the RAID fixed first, then think about relocating data and rebuilding individual arrays.

In practice, I use RAID-5 over four disks plus a hot spare. So if I lose an active disk, chances are good that the RAID will already be rebuilt onto the spare before I even notice the e-mail warning me of the issue.

A raid can span ATA as well as SATA drives, right?

It's much better than that. The Linux software RAID system can use any block device. RAID devices (/dev/md*) are block devices. LVM also works on any block device, and LVM volumes are block devices. You could, if you wanted to, build an absolutely insane structure of RAIDs of RAIDs of LVM volumes of RAIDs... I like to say that the true measure of the power of a tool is how large a mess you can make if you misuse it, and by that definition Linux software RAID and LVM are powerful tools indeed. I've never come up with any reason to do anything other than LVM over RAID, but there may be one.

Another thing Linux can do is network block devices. Because my server is basically full -- I can't add disks, I have to replace them -- I've toyed with the idea of putting another big disk in my desktop machine and exporting it as a network block device, then adding that into my RAID arrays on the file server. The performance hit shouldn't be too ba

Re:I'm using something like that...

[MacJedi]

I hope you excuse me for intruding in the thread, but I find this to be a quite interesting optimization problem. You ultimately want to optimize, T, the total size of the LVM of arrays, where:

T = Sum_{j=1}^{m}[(N_{j}-1) * S_{j}]

where Sum_{j=1}^{m} is the sum over m arrays, N_{j} is the number of partitions in the jth array and S_{j} is the size of a partition in the jth array. One thing that this equation shows us, is that each array in the LVM does not need to be the same size! The other constraint in this problem are:

N_{j} >= 3

which simply means that each array needs at least 3 partitions to be valid.

Now, there is another piece to this problem that I have not found a good way to express, which describes how to split a physical disk into partitions and then hand how to map each partition onto an array, j, since each disk does not have to contribute a partition to each array. This is needed to prove that a solution is optimal, but we should have enough intuition now to get a good solution.

For your specific example with 4 disks of 120, 160, 200 and 250 GB. I suggest an LVM made up of 3 arrays:

1. 4 partitions of 60 GB coming from each disk for an 180 GB array. Free space is now 60, 100, 140 & 190 GB from disks 1 through 4 respectively.
2. 4 partitions of 60 GB coming from each disk for an 180 GB array. 0, 40, 80 and 130 GB free.
3. 3 partitions of 40 GB coming from disks 2, 3 and 4 for an 80 GB array. 0, 0, 40 and 90 GB left-over.
4. each array made up of 3 partitions. The first, second and third arrays have S_{j} = 50 GB and comes from disks 2, 3 and 4. This yields 100 GB in each array for a total of 300 GB. At this point there is 120 GB free in disk 1, 10 GB free in disk 2, 50 GB free in disk 3, and 100 GB free in disk 4. Now you can make another array with 3, 50 GB partitions coming from disks 1, 3 and 4, for a total of 400 GB in the LVM. Finally, you can take 10 GB from disks 1, 2, and 4 to add an additional 20 GB to the raid for a total of 420 GB. This leaves 60 GB free in disk 1, 0 GB free in disk 2, 0 GB free in disk 3 and 40 GB free in disk 4.

hmm

[GigsVT]

I'm surprised no one has mentioned that RAID5 is no replacement for backups.

I guess if it's just porn you got for free or whatever it doesn't matter, but if the data is important you still need some sort of backups.

RAID protects against:
Disk Failure

Backups protect against:
Disk Failure
Accidental Deletion
Malicious users
Malicious programs
Filesystem corruption
Errant program causing file corruption

RAID won't protect you from any of those other things one bit.

Re:hmm

[Homology]

I'm surprised no one has mentioned that RAID5 is no replacement for backups.

Indeed. In RAID options for OpenBSD you see the following warning:

While a full discussion of the benefits and risks of RAID are outside the scope of this article, there are a couple points that are important to make here:

* RAID has nothing to do with backup.
* By itself, RAID will not eliminate down-time.

If this is new information to you, this is not a good starting point for your exploration of RAID.

Check LinkSys NSLU2

[Frodo420024]

Given that low power and backup are the main purposes, I suggest you take a close look at the LinkSys NSLU2. It takes two external USB2 drives and provides Samba shares for them. It'll automatically pull backups from your network, then back up one of the drives to the other. If you use laptop (2½") drives with the unit, it can supply enough power to them, and the whole setup will use less than 10 watt total. Takes less space, too.

Having the backup done by normal file copying rather than RAID is not a problem in my view - after all, backup is the purpose, and that's done by the firmware. RAID ain't always ideal: A friend of mine had a nice RAID5 setup in his computer. Then the primary drive got corrupted - and that was immediately mirrored to the second drive! He lost all his data...

No mention of the NSLU2 is complete without noting that it's eminently hackable. :)

Re:Hardware RAID

[tzanger]

For our Linux boxes, we just run the following bash script every hour in cron:

Why in the hell do you do that? Go look up mdadm's --monitor --scan and -ft modes, and then configure smartd to also email you out warnings. Beats the shit out of some manual process that relies on the /proc format not changing over time!

Synthesis from /. + new advice

[advid.net]

See this journal for information about this topic, gathered on slashdot.

I've set up a fileserver in my garage, Linux mandriva 2005, serving NFS and SaMBa shares. Running since 3-4 months
I use EVMS as professional LVM. Raid 0 or 1 available, and bad blocs relocation too. Also SMART monitoring is running as daemon.

Your main problem for spinning down drives is the filesystem:
With journaled FS (recommended) disks will spin up every 10mn or so, after some tuning. For me too it's still too much and I'd like to stop them for hours if I don't use the shares...
I plan to study this: remount read only and then turn read-write access automatically on serving files. As nothing happen on RO journaled FS, discs will remain down when not in use - I hope.

I'll will gather all good advice on this thread later to update the fileserver entry.