Slashdot Mirror

Here's a case I've seen suggested.
http://www.servercase.com/Merchant2/merchant.mvc?Screen=PROD&Product_Code=CK4020&Category_Code=4UBKBLN
Norco has a similar one; I'm not sure about the exact differences.
http://www.newegg.com/Product/Product.aspx?Item=N82E16811219033

You can pick up SAS controllers at the usual sorts of places, newegg, supermicro, whatever. Watch out for the newer couple series of DELL PERC controllers since I hear they've started restricting you to only use them with dell branded drives.

I don't yet use the above types of enclosures, but may pick some up for a work project soon.

As for OS -- just look at FreeBSD 8.0/8.x with ZFS or maybe OpenSolaris with ZFS if they ever release a public B138 / 2010.03 release or better (which are both now long past expected due).
Both should be thoroughly checked under load / production simulation conditions for stability and FS integrity before trusting data to them. Since you deal with large (video) files and not millions of small ones you probably won't be using anything too new / fancy like dedup so you should be safer without the more esoteric / bleeding edge FS features in use.

I'd suggest getting three or more chassis and populating each with about 4-6 1TB HDDs in RAIDZ or RAID-1 mirrored pairs to start with, and set up a redundant cluster of NAS shareed volumes from each of these drive pools from each of the servers such that any one of the server machines itself could totally fail and the other 2+ servers would redundantly have your data preserved via use of a second level RAIDZ/mirror off of the server pool exports. There's discussion about this on the zfs-discuss mailing list archives lately as I recall as well as on the freebsd zfs list IIRC.

Use dedicated 1Gbit LAN links between the servers of course.

If you really don't want to sysadmin an OS look at FreeNAS or NexentaStor's community edition or OpenFiler and see if any of those suit you, but given your space/performance needs, I suspect just running an OS on the servers will work.

As for the PC motherboard, you don't need anything especially high end, a mini-ITX like the D510MO or one of the Zotac ATOM boards from newegg will get you a CPU and motherboard for $80-$200, though usually getting in on one of the fry's weekend bundle sales of a CPU+microatx motherboard will often get you a CPU+Motherboard in the $30-$100 range. All that matters is having enough of the right kind of slots for a SAS controller or whatever you want to use (SATA for lower end), and of course a slot for a gigabit NIC if the motherboard doesn't already have enough of those for you.
Supermicro / tyan have some serverish motherboards if you're interested in those, but usually those are closer to $250-$350 and proabably don't do you much good. Just put 4GB or 8GB RAM into the things and ZFS will be happy running under a 64 bit OS.

Look at putting your cache/ZIL on the main "head" server onto a SSD if you want better performance.

The backblaze design is interesting, though I don't personally see that it is needed at the scales you're talking about which can be readily enough handled with an off the shelf case with a few drives per case and multiple servers for redundancy / backup.

Tahoe-LAFS may be of interest to you if you want to check that out, though it sounds like ZFS itself should just work for you.

Anyway read the freebsd-zfs and zfs-discuss related lists, they're full of good into on DIY SAN/NAS solutions.

While I agree with many of the previous post regarding Newegg, I would also have to add in that http://www.servercase.com/ has a much better selection of cases, especially for things like disk arrays and specialized accessories associated with rackmount hardware.

If you're looking for low volume (2 - 10 units), you're going to end up paying close to retail. That's just how it works.

For medium volume (25+ units) talk to the guys at http://www.servercase.com/ ... They've been really helpful to me in the past.

For larger volumes (100+) talk directly with VIA, they have always been extremely helpful in working with the little guys (in my experience), and if you're actually building a product, or developing something nifty with their boards you can generally get direct access with their devs and help in marketting the products. Hell, they even payed for us to go to COMDEX and CES to show off our EPIA based products, not exactly the typical response from a big company ya' know?

I own a large collection of DVD, and decided recently to price some large-volume storage so I could have a digital backup of my discs. What I figured out was that a 250 GB hard drive currently costs about $200. This works out to about $0.80/GB. Your average DVD contains about 7 GB of data, so you can figure a per-disc storage cost of $5.60 per disc. Based on these numbers, you can store about 35 movies per drive, so if you happen to have a couple hundred discs, you'll need at least seven drives for a RAID5 solution. Go ahead and throw in $500 for a SATA RAID card, another $450 for a case with 7 hot swap bays. And then you have to build the rest of the machine. If you spent $300 to do so, your total cost is $2650. Divide this by the total storage capacity (233 discs), and your net storage cost is $11.30 per disc. Most movies can be acquired on eBay in perfect condition for this amount or less, and you don't have the ongoing expense of also replacing drives when the die.

Obviously, these numbers are quite variable variable, and you could certainly use cheaper parts, but there is an absolute minimum cost for everything here. My conclusion was that until there is a fundamental change in the world of mass storage, in either techology or cost, this is just going to have to wait.

I bought a case from http://www.servercase.com/, a 3Ware RAID Controller and 8 200GB IDE drives. I've got 1400GB of usable space in RAID5. It runs Linux with Samba and NFS. I also use it for a MythTV Backend.

Unfortunatly, once you have all this space, you WILL find a way to use it all and need more. I put this system together about 10 months ago, and it's at 85% capacity now. I'm preparing to build a new server with 12 250GB drives, to have just over 4TB between the 2 systems.

Instead of making stupid comments regarding VMWare or laptops, here is something that will actually help you out. I just completed my rack a few months ago. Building a home rack is not that expensive but you will need to look around because most of the rackmount products are business oriented and they seem to be more expensive. Here are the links to the stuff I used. Some of the prices may be slightly off (previous slashdot post) but you should be able to track down most of the stuff. I do agree with the posts regarding the 1U servers it will be much easier for you to stick with 2U and up servers, the extra space makes it much easier and cheaper to work with. Some of the links are not quite working (l-com and bestbuy) but I think it is from problems on the vendors website, hopefully they will work tomorrow or I will post new links.

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
60.00 - Base(Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
1200.00 - LCD Monitor (21")
41.49 - LCD Swivel [cyberguys.com]
119.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables [techdepot.com]

Rack Mount UPS
249.99 - UPS 1500VA CyberPower 1500AVR(free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Instead of making stupid comments regarding VMWare or laptops, here is something that will actually help you out. I just completed my rack a few months ago. Building a home rack is not that expensive but you will need to look around because most of the rackmount products are business oriented and they seem to be more expensive. Here are the links to the stuff I used. Some of the prices may be slightly off (previous slashdot post) but you should be able to track down most of the stuff. I do agree with the posts regarding the 1U servers it will be much easier for you to stick with 2U and up servers, the extra space makes it much easier and cheaper to work with. Some of the links are not quite working (l-com and bestbuy) but I think it is from problems on the vendors website, hopefully they will work tomorrow or I will post new links.

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
60.00 - Base(Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
1200.00 - LCD Monitor (21")
41.49 - LCD Swivel [cyberguys.com]
119.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables [techdepot.com]

Rack Mount UPS
249.99 - UPS 1500VA CyberPower 1500AVR(free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Instead of making stupid comments regarding VMWare or laptops, here is something that will actually help you out. I just completed my rack a few months ago. Building a home rack is not that expensive but you will need to look around because most of the rackmount products are business oriented and they seem to be more expensive. Here are the links to the stuff I used. Some of the prices may be slightly off (previous slashdot post) but you should be able to track down most of the stuff. I do agree with the posts regarding the 1U servers it will be much easier for you to stick with 2U and up servers, the extra space makes it much easier and cheaper to work with. Some of the links are not quite working (l-com and bestbuy) but I think it is from problems on the vendors website, hopefully they will work tomorrow or I will post new links.

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
60.00 - Base(Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
1200.00 - LCD Monitor (21")
41.49 - LCD Swivel [cyberguys.com]
119.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables [techdepot.com]

Rack Mount UPS
249.99 - UPS 1500VA CyberPower 1500AVR(free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

You mean something like this?

Well I happen to be doing the exact same thing. I decided to use an open relay rack with casters for space/ease of use purposes. Unfortunately this has not been assembled yet (physical presence issue, out of country, should have it assembled/pics by the 1st of Jan).

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
111.00 - Base x2 (Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)
15.00 - Center Mount Brackets

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
999.00 - LCD Monitor (20")
41.49 - LCD Swivel
109.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables

Rack Mount UPS
249.99 - UPS 1500VA (free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Well I happen to be doing the exact same thing. I decided to use an open relay rack with casters for space/ease of use purposes. Unfortunately this has not been assembled yet (physical presence issue, out of country, should have it assembled/pics by the 1st of Jan).

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
111.00 - Base x2 (Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)
15.00 - Center Mount Brackets

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
999.00 - LCD Monitor (20")
41.49 - LCD Swivel
109.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables

Rack Mount UPS
249.99 - UPS 1500VA (free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Well I happen to be doing the exact same thing. I decided to use an open relay rack with casters for space/ease of use purposes. Unfortunately this has not been assembled yet (physical presence issue, out of country, should have it assembled/pics by the 1st of Jan).

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
111.00 - Base x2 (Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)
15.00 - Center Mount Brackets

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
999.00 - LCD Monitor (20")
41.49 - LCD Swivel
109.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables

Rack Mount UPS
249.99 - UPS 1500VA (free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Well I happen to be doing the exact same thing. I decided to use an open relay rack with casters for space/ease of use purposes. Unfortunately this has not been assembled yet (physical presence issue, out of country, should have it assembled/pics by the 1st of Jan).

Relay rack 38U steel, with base (should fit under a doorway with casters)
129.00 - Rack
111.00 - Base x2 (Freestanding Application)
40.00 - Casters

Rackmount Computer Case
130.00 - Computer Case
30.00 - Power Supply (300 Watt)
15.00 - Center Mount Brackets

LCD / KVM Switch / Mounting (need to make custom rack mount for LCD Swivel)
999.00 - LCD Monitor (20")
41.49 - LCD Swivel
109.95 - Rack Mount KVM (8 Port)
18.99 - KVM Cables

Rack Mount UPS
249.99 - UPS 1500VA (free shipping from BestBuy, easy to return if problems, not cheapest price)

Hope that helps,
Jad

Bus 0, device 12, function 0: PCI bridge: Digital Equipment Corporation DECchip 21152 (rev 3). Master Capable. Latency=64. Min Gnt=4.

But you can't use this to connect a rack of computers. For one thing the max cable length for connecting two busses would be just a few inches. For putting PCI cards in 1.75" high 1U rackmount cases, there are PCI risers with a short ribbon cable that connects to the PCI slot. Even these short cables often cause timing problems. For instance, with the riser, cards may only work in the first one or two slots that will otherwise work in all the slots.

But even if you could cable all the computers together on one giant PCI bus, it would still be a bad idea. A good 24 port gigabit ethernet switch (~$2000) has a 480MB/sec switching fabric, to support full speed full duplex on each port. 32 bit 33Mhz PCI is only about 132 MB/sec, not nearly as fast. You'd need a 64 bit 66 Mhz PCI bus to keep up. And there are more expensive gbit switches with more ports that have 100 Gbit/sec fabric. And this is just gbit ethernet, the slowest and cheapest of the high speed interconnects used in modern Beowulf clusters.

There are faster ways to connect computers than gigabit ethernet. The EE times article is very untechnical, but this one has some more information. LLNL has used a very fast and very expensive interface called quadrics. This is probably the fastest way to connect computers in a Beowulf. People like Cray/SGI and IBM have faster things still, but they cost real big bucks. Other ways to connect a Beowulf are the above mentioned gigabit ethernet (~$100-$250 a node for up to 24 nodes), myrinet (~$1400-$2000 /node up to 128 nodes), and SCIhardware and software (~$1400-$2100 /node). Myrinet uses a switch like gigabet ethernet and the largest switch they have is 128 ports. SCI is switchless, each card has multiple cables (1-3), and is connected in into a ring, 2D or 3D torus.

My current DNS, mail, and web server is 11.5" x 2.5" x 10.75" and draws about 20 watts. It's based on the VIA C3 EPIA motherboard. The only downside for your use is that there's only room for one hard drive inside this tiny case. But it's cheap (less than $200) and as the power input is 12V, I use two paralleled gel-cells for a UPS. (That way I can swap out one battery for maintenance without interrupting anything.) My DSL router also runs off 12V. Linux installed very easily.

There's a similar VIA-CPU based low power motherboard for a little less money that draws so little power that there's no CPU fan. For reliability this may be a good choice as it reduces the number of moving parts.

If you insist on room for two hard drives, see these cases.

Since PV cells output DC I suppose it would be more efficient to use a DC->DC ATX power supply. If you really wanted a desktop machine instead of a laptop that is.