Ask Slashdot: Best Kit For a Home Media Server?

First time accepted submitter parkejr writes "I started off building a media library a few years ago with an old PC running Ubuntu. Folders for photos, ogg vorbis music from my CD collection, and x264 encoded mkv movies. I have a high spec machine for encoding, but over the years I've moved the server to a bigger case, with 8 TB of disk capacity, and reverted back to Debian, but still running with the same AMD Sempron processor and 2GB RAM. It's working well, it's also the family mail server, and the kids are starting to use it for network storage, and it runs both link and twonkyserver, but my disks are almost full, and there are no more internal slots. The obvious option to me is to add in a couple of SATA PCI cards, to give me 4 more drives, and buy an externally powered enclosure, but that doesn't feel very elegant. I'm a bit of an amateur, so I'd like some advice. Should I start looking at a rack system? Something that can accommodate, say, 10 3.5" drives (I'm thinking long term, and some redundancy)? Also, what about location — I could run some cat6 to the garage and move it out of the house, in case noise is an issue. Finally, what about file format, file system, and OS/software? I'm currently running with ext3 and Debian Squeeze. Happy with my audio encoding choice, but not sure about x264 and mkv. I'd also consider different media server software, too. Any comments appreciated."

Comments

Larger disks

[Anonymous Coward]

Larger disks. 4 TB should be available very soon (maybe now?).
http://www.extremetech.com/computing/108665-hitachi-ships-worlds-first-4tb-hard-drive-sticks-it-to-thor

HP Microserver

[quenda]

For a slightly more sane solution than rackmounting at home, consider the HP microserver.
Very low power (12W CPU), small, quiet, cheap, server grade, no Windows tax, holds four pluggable 3.5" drives plus optical (which some people swap for a 5th HDD for RAID5.)

http://blog.thestateofme.com/2011/05/14/review-hp-microserver/
http://www.silentpcreview.com/HP_Proliant_MicroServer
http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-4237916-4237918-4237917-4248009.html
http://forums.overclockers.com.au/showthread.php?t=905262

If 8TB is full, you need to stop the obsessive collection of warez/pr0n/torrentz you are never likely to watch again.

Hard drive math and hobbyist heuristics

[Wrath0fb0b]

Ballpark figures, this isn't exact, redo it with your preferred constants, I'm just trying to explain my reasoning against huge enclosures with > 10 drives,

Standard drive idle usage (W) ~ 10W [1]
Low-power (green) drive idle usage (W) ~ 5W [1]
Cost of power ~0.20 $/KWH
Cost of an older drive per year = $17
Cost of a green drive per year = $8.50
Replacing 6x500GB older drives with one 3TB green power savings = $95/yr

So think about that for a sec. At $150[2] for a 3TB drive, you cover the price in power savings in 18 months. That's assuming that there is zero fixed-cost per drive. At the point where you are talking about adding SATA controllers or fancy multi-bay enclosures or, worse, external enclosures with their own PSUs (and fans!), the turnaround-point for older drives is far sooner.

I'm a hobbyist, I understand that it's really cool to make do with older hardware and feel like you aren't letting anything to go waste but sometimes using old hardware instead of buying new is penny-wise and pound-foolish. Spending money on increasing how many hard drives you can accommodate instead of just buying newer high-capacity lower-wattage drives is absolutely batty; especially when you get into the price for anything remotely good in the RAID dept.

My advice, move everything to the largest capacity drives that are reasonably priced (after the flood damage is sorted). Replace the drives when you can do between 4:1 and 6:1 replacement -- should be every 3-4 years. Live happily, quietly and simpler. Save money on power transparently.

[1] http://hothardware.com/Reviews/Western-Digital-2TB-Caviar-Green-Power-Hard-Drive/
[2] I bought some Hitachi 3TBs before the Thailand floods at $130 on Newegg. Of course you would be silly as heck to buy hard drives now for your hobby storage project before they at least fall back to pre-flood level.
[3] http://www.newegg.com/Product/Product.aspx?Item=N82E16817182221
[4] Older drives need not go to waste, they can become offline storage with a simple USB dock[3] -- make a backup, throw it in an anti-static bag, leave it at your relative's house when you visit!
[5] http://en.wikipedia.org/wiki/File:Hard_drive_capacity_over_time.svg

Re:No reason to change from H.264

[Ltap]

Actually, Matroska has a number of good advantages:
1: More open than MP4 -- it has none of the ugly MPEG-LA overtones.
2: More codec support than MP4.
3: More consistency -- there aren't PS3-oriented versions or AppleTV versions of Matroska. MP4's device support might be wide, but when every player and device seems to have its own version with its own ridiculous, restrictive standards, it doesn't really qualify as supported at all.
4: Content management: Matroska is the easiest to mux and makes it great to work with alternate audio and video streams, subtitles, etc. You get the widest range of options of any container out there.

While Matroska might be used by some because it is perceived as more "elite", there are many very valid reasons for using it, mainly revolving around flexibility and openness. As someone who has worked with both containers, I can say that Matroska far more easily delivers what I want and the tools for working with it are generally free (in both senses), more usable, and more powerful.

Re:Mac mini or apple Tv

[BLKMGK]

Ummmm, why two nic? The HDD and it's bus are the bottleneck - I cannot max out a gig-E nic now on my server but can play 1080P via x.264 with surround sound DTS encoded on a 100meg connection. No stuttering, no issues, no muss no fuss. It's wired for gig-E but sadly this nic refuses to synch at it.

I WOULD run some sort of redundant storage. However I wouldn't go traditional RAID. Striping data across disks along with parity makes for lots of speed, also means the disks never shut down. No thanks! I happen to use unRAID from Lime Technology. Parity isn't striped, it's held on a single disk. Each disk uses a standard format - ResierFS (ick). But's journaled and standard enough that recovery is easier. On top of that I can pull a disk and get data from it on another machine - one disk removed I still see all my data. Last but not least - if I lose multiple disks at once I only lose the data on THOSE disks and not the entire thing. Losing one disk I lose nothing. In 6+ years or so I've never lost more than one disk. Since parity isn't striped and neither is data one of my servers has ALL drives spun down and quiet, the other has just 2. Between them I have a bit over 22TB worth of disk BTW and not all are 2TB disks but as I fill smaller ones I swap in 2TB disks, the system migrates the data fine. Other programs can be run on this system, it's Linux based, but it's trickier than a full OS install - which has been done by some too.

Honestly to me it sounds like his current setup is working save for disk space. Why not just upgrade to 2TB disks? Surely he has more than 4 ports? If add-on cards are needed they are plentiful. I would also suggest using 4n1 cages for easy swapping. A low speed CPU is fine, underclock it maybe too - HDD bus is the bottleneck!

Re:No reason to change from H.264

[Osty]

Others have touched on your other points, so I wanted to address this:

Switch to a standard .mp4 container. Much better supported on hardware or software. Some day you will want to be able to stream from your server to a thin set-top box or load a file on your kid's phone. On that day .mkv will make you cry.

Plenty of thin set-top box clients play mkvs already. Devices from Western Digital (WD TV Live), Netgear, Seagate, Roku, Popcorn Hour, Boxee, and many others all support mkv out of the box, with header compression support, subtitles, chapters, multiple audio and video streams, and some even support 3D (not the new mk3d format yet, but SBS works) and will play subtitles correctly. Most mkv files contain MPEG2, h264, or VC-1 video and AC3 or DTS (or the newer Dolby TrueHD, DTS-HD, etc) and all of these players handle those just fine. Don't blame the container for being flexible enough to allow any random codec. Blame whatever crap source you stole your videos from for using a random codec. Those of us archiving our DVDs and Blu-Rays will either encode in H264 or remux the original MPEG2/H264/VC-1 streams untouched and have no problems.

If you want to move videos to a phone, that's easy enough to do. The beauty of being a completely open container format means that it's trivial to demux Matroska containers into their component streams, which than then be remuxed into mp4 for devices that suck. Since you'll probably want to down-res the videos anyway for handheld formats (even on tablets you won't want higher than 720p), there's no reason to keep the originals in mp4. Keep your original, untouched videos in mkv and re-encode at lower resolution and bitrates into mp4 using Handrake for mobile devices.

Divx (yeah, whatever, they're still relevant) has adopted MKV as their HD container format, and the proliferation of "networked media tank" devices plus Matroska's openness makes it not only relevant but desirable for long-term video storage. Using 5+ year old devices like Xbox 360 and PS3 as your benchmark for what containers to use would be a bad idea (Xbox still doesn't even support 6-channel AAC in mp4, never mind supporting AC3).