Slashdot Mirror

Since you're such a low UID I'll bother answering your question.

Thank you, this is one of the few valid answers to my primary question which is of actual experience with clustered file systems.

I had already thrown out OCFS2 and GFS2 as possible candidates, but that was irrelevant to my reply. Also currently I am unaware of any non-proprietary hardware or software RAID (mdadm in particular) that supports active/active or active/passive on a shared backplane at any RAID level other than 1 or 0 (i.e. DRBD) and rather expensive and not yet released Areca external RAID controllers. Also I'm looking for whitebox OSS solutions.

One: The title is a borderline lie. Yes, you can buy 12x 1TB drives for about a grand. But if I'm going to build an array and bench mark it and constantly compare it to buying a Core i7-975 Extreme, the drives alone don't do me any good! (And I love how you continually reiterate with statements like "The Idea: Massive Hard Drive Storage Within a $1,000 Budget")

Two: Said controller does not exist. They listed the controller as ARC-1680ix-20. Areca makes no such controller. They make an 8, 12, 16, 24 but no 20 unless they've got some advanced product unlisted anywhere.

Three: Said controller is going to easily run you another grand. And I'm certain most controllers that accomplish what you're asking are pretty damned expensive and they will have a bigger impact than the drives on your results.

Four: You don't compare this hardware setup with any other setup. Build the "Uber RAID Array" you claim. Uber compared to what, precisely? How does a cheap Adaptac compare? Are you sure there's not a better controller for less money?

All you showed was that we increase our throughput and reduce our access times with RAID 0 & 5 compared to a single drive. So? Isn't that what's supposed to happen? Oh, and you split it across seven pages like Tom's Hardware loves to do. And I can't click print to read the article uninterrupted anymore without logging in. And those Kontera ads that pop up whenever I accidentally cross them with my mouse to click your next page links, god I love those with all my heart.

So feel free to correct me but we are left with a marketing advertisement for an Areca product that doesn't even exist and a notice that storage just keeps getting cheaper. Did I miss anything?

Add to that, if I my memory doesn't fail that they published the RAID 5 wasn't safe to run with 1TB disks... crap.

For the ads... I see no ads. The old "Proxomitron" takes care of everything

Two: Said controller does not exist. They listed the controller as ARC-1680ix-20. Areca makes no such controller. They make an 8, 12, 16, 24 but no 20 unless they've got some advanced product unlisted anywhere.

They screwed up the model number. They clearly state that they used the model with 16 internal ports and 4 external ports - which is the ARC-1680ix-16. If anything the WTF here is that Areca call their 20 port controller a ...-16.

One: The title is a borderline lie. Yes, you can buy 12x 1TB drives for about a grand. But if I'm going to build an array and bench mark it and constantly compare it to buying a Core i7-975 Extreme, the drives alone don't do me any good! (And I love how you continually reiterate with statements like "The Idea: Massive Hard Drive Storage Within a $1,000 Budget")

Two: Said controller does not exist. They listed the controller as ARC-1680ix-20. Areca makes no such controller. They make an 8, 12, 16, 24 but no 20 unless they've got some advanced product unlisted anywhere.

Three: Said controller is going to easily run you another grand. And I'm certain most controllers that accomplish what you're asking are pretty damned expensive and they will have a bigger impact than the drives on your results.

Four: You don't compare this hardware setup with any other setup. Build the "Uber RAID Array" you claim. Uber compared to what, precisely? How does a cheap Adaptac compare? Are you sure there's not a better controller for less money?

All you showed was that we increase our throughput and reduce our access times with RAID 0 & 5 compared to a single drive. So? Isn't that what's supposed to happen? Oh, and you split it across seven pages like Tom's Hardware loves to do. And I can't click print to read the article uninterrupted anymore without logging in. And those Kontera ads that pop up whenever I accidentally cross them with my mouse to click your next page links, god I love those with all my heart.

So feel free to correct me but we are left with a marketing advertisement for an Areca product that doesn't even exist and a notice that storage just keeps getting cheaper. Did I miss anything?

maxtorman, thank you very much for explaining the technical details. We just ordered two 20-drive cases of the 1TB ES.2 model ST31000340NS drives from Provantage on December 31, 2008 for a new server build. After this news hit TechReport.com, TomsHardware.com, Slashdot, and started to spread through the news aggregators like Reddit and Digg this week, we went to check if this model was affected by our choice of SAS controller, and indeed it was. We freaked out after going through the Seagate KB and support discussion forums, and this morning we were starting to consider returning all the drives unopened to Provantage and ordering Western Digital RE3 units instead, as the server build hadn't started yet.

Your patient description of the underlying technical issues and responses to others' questions in Slashdot gave us enough confidence to hold off until next week Friday before deciding whether or not to return the drives, based upon what we see happening in the 1.5TB drive firmware issue. Possibly longer, if Provantage will let us exchange for Western Digitals after a month of holding onto the case of unopened Seagates, should that prove necessary. At this point, we can't tell if the SAS controller issue we have identified is related to the 320th log entry issue you described, but they sound related, so one possible question for you is if you are aware of any relation between the issues. We plan a tremendous lot of slack into our server build projects, so we would rather have Seagate get it right with this next firmware release than try to push a release out just for appearance's sake. The data on these drives would be configured for software-based 3x-RAID1, and redundantly backed up to LTO4 tape as well, so a loss of a drive's data is not a concern for us. However, time spent rectifying an issue is a big deal, and while we are patient with issues before the server build, but once it is built and in production, issues like this have a very adverse operational impact upon us.

Yes, software RAID is great, especially if you like your writes being really slow. :-/
If you're going to dump hundreds of $$ into hard drives, cough up a bit more for a HW raid controller!

Says the person who's never done any real benchmarking of these things...

Unless you buy the right raid card, you'll likely get worse performance from it than you would from software raid. I'm talking the name brands too - LSI, Adaptec, 3Ware. They all suck. Of the 3, 3ware is the best. On a LSI SAS raid controller I recently tested, I only got a 30% I/O speedup going from a single drive to a 6 drive raid 5. That's pathetic! Software raid at least gave me 140% improvement.

If you really want good numbers, get an Areca controller. They perform very well and have drivers right in the linux kernel (2.6.19+).

The older RocketRaid cards (Highpoint) performed fairly well, but were not really hardware raid - they were "hardware assisted" raid. Most of the work was really software raid in the driver. As long as you had a fairly fast cpu, you got great numbers. I believe the newer ones are true hardware raid now, but I haven't benchmarked them yet as they only had up to 8 port controllers in PCIe last I checked.

You can get a sata-II to FC adapter from areca, these are pretty expensive, but the nice thing is that you don't need a motherboard in your case. Combine it with a chenbro 3U 16 bay case and you have a relatively affordable setup.

http://www.areca.com.tw/products/html/fibre-sata.h tm

I've been putting together the specs for such a beast. I decided to go with SATA for cheap drives and "SATA-II" (or whatever you want to call it, since there isn't a standard name for NCQ and 3.0Gbps support) for future-proofing.

1) The natural first choice was 3ware. 12 port SATA-II controller (9550SX-12), for about $800. 3ware products are very well supported on Linux. The only downside is that it's a PCI-X device (this is NOT "PCI Express"!), and PCI-X busses are generally only found on very high end motherboards for servers and workstations. Any athlon motherboard or single-processor opteron board claiming to have PCI-X is lying, they really mean PCI express (AMD chipsets did not support PCI-X at all until around the time dual opteron motherboards were being created)

So since I didn't want to spend $500 on a motherboard that had built in scsi raid, support for 16GB of ram and dual opteron processors just to use that $800 card, I looked around some more...

2) And found a serious contender, the 12 port Areca 8x PCIe ARC-1230 (also about $800). While most low end motherboards don't provide an 8x PCI Express slot, they DO provide a 16x slot which will work just fine for this card (after all, this will be the fileserver, so a motherboard with crappy built in video will do, we're not playing Doom 3 here). Linux drivers are provided as source, even including a kernel tree patch which will build the driver into the kernel rather than as a module, making booting directly from the RAID controller easy.

Slap the Areca into Tom's Hardware's 37 watt computer (motherboard has built in GigE, but pentium-Ms are 32 bit processors, making giant files/filesystems a pain. An Athlon 64+cheap mini-ATX can be had cheaper, but uses more power), add in a stack of 10 watt 400GB WD Caviar Raid Edition 2 drives, and you're set for a very low power fileserver with a lot of storage.

Now, my turn to "ask slashdot":

Where do I get a 250-300 watt powersupply with 12 SATA power connectors?

Alternatively, do the SATA drive cages (like 3ware's RDC-400-SATA (pdf) have their own SATA power connectors built in and use standard molex connectors on the outside? Do I need special cages to support 3Gbps drives (ok, not a serious problem for now, but futureproofing)? 3ware's website says it'll work, their product PDF doesn't.

3ware's SATA implementation is ugly; it's effectively a bridge from their PATA one, so it doesn't support NCQ.

Personally I use Areca cards - a 16-way card that can run RAID '6' (RAID 5 but with two parity discs) and a hot-spare, has its own Ethernet port for remote access to the firmware is rather good. Oh, and it has (unofficial) kernel sources suitable for 2.3 - 2.6.

Very good.

A few months ago, I was looking for a hardware RAID 5 solution using IDE disks, with hotswap and the whole shebang, and OS independant. I found:

this pretty spiffy unit .

A bit pricy, but does a great job. It is totally independant from the OS, since the motherboard just sees it as a normal disk, and does pretty much everything you may want from a hardware RAID 5 solution (hotswap, transparent rebuilds, array roaming, capacity expansion...).

Note: my first unit was faulty and I had to get it fixed a few times, until finally I got it exchanged. Since then, no more problems with it. But note that it did a great job even through its own hardware failures: all I had to do was put all the disks in the new unit, and my array was back up.