Slashdot Mirror

We did the same thing. Another data point.

As far as future expansion, two options, get SATA 12 port 3ware cards and the 3ware Parallel to Serial converter that 3ware sells for $30, or go with something like external ATA-SCSI from acnc that can be chained up to infinity.

These cases from CalPC could house something like 30 of these, for a massive beowulf cluster. :) They cost about $500 without power supply.

Oops, the link should have been this

From the story:

These drives will also carry a three-year warranty.

huge capacities up to 320 GB for corporate archiving and media recording; and unique manufacturing and quality for 24/7 operations with mean time to failure (MTTF) rates exceeding one million hours.

Guess all you SCSI zealots are going to have to eat your hat.

All of our large archive arrays at work are already ATA. Not everyone needs high speed and large capacity, a lot of company's data needs just require a lot of space, and speed isn't too critical. ATA is stealing this market away from SCSI and tape very quickly. Maxtor is just filling this niche that already existed.

As a side note, 3ware already has a serial ATA RAID card out with 10 ports per card, and great linux support. 2.5 TB for $4500 in a single full tower case. Nice.

Not as many days of MP3s as this.

Since I built it, I did the math on the MP3s first thing. What else does one do when presented with an incomprehensible amount of space.

Lets assume 2MB per minute, a little high for 128kbps, but lets assume you have 160s in there too.

Server 1: 1.6TiB
Server 2: 1.8TiB (I think the 1.9 on the page is a typo)

That's 3.4 TiB, 3481GB, 3.5Mln MB rounded.

divide by 2, 2mb per minute.
divide by 60 minutes in an hour
12 hours in a day
365 days in a year.

6.7 Years of Mp3s without a single repeat. For less than $17,000.

If you assume 1MB/minute of MP3, which is closer to reality, it's over 13 years of MP3s.

Read speed is 128MBytes/sec, write speed is 42MBytes/sec.

You don't need a cray, you can build this at home, scaled down.

Well, we did this.

Inspired by Slashdot's earlier story that was nearly identical, and with the help of Peter Ashford from ACCS, we built two servers, both with capacities well over a TB, for around $8000 each. They have the capacity to expand to 3TB if need be.

Story here

As far as performance:
(from my memory)
EXT3: About 16MB/Sec block write, 45MB/sec block read
ReiserFS: About 20MB/sec block write, 130MB/Sec block read (that's no typo).
XFS: About 30MB/sec block write, 85MB/sec block read.

It seems that file system plays a large role in performance. The arrays are three RAID5 in hardware using Linux software RAID0 on top of the RAID5 arrays to tie them together.

IDE RAID controllers are 3ware Escalade 7810. Write performance can be greatly increased by using 7850 cards that have more cache.

We stuck with XFS, Reiserfs had a bigfile bug, files created over 2GB would lock up the computer basically. XFS in general seemed much more mature, reiserfs seems more like someone's college thesis project, that they never cleaned up to be production grade.

We experimented with different RAID0 stripe sizes, the hardware RAID5 stripe size is fixed at 64k, there are 7 active disks in each array and one hot spare. Stripe size tweaking seemed to mostly trade off read for write speed, within a certain range of values, with a taper off in performance at either extreme, (down around 8k stripes, or over 1024k stripes)

We eventually went with 1024k stripes. That is what the benchmarks above reflect. The variance in file system performance could very well be due to interactions with stripe size, but there seemed to be common themes (reiser always read fastest no matter what stripe, XFS was always better at writes)

I have been in so many arguments with SCSI zealots on here over this RAID... I wish people would understand what price/performance ratio means. IDE isn't a superior technology, but every now and then, it is the right tool for the job, when price is a goal too.

Yeah, Right

Reads at >80MB/Sec writes ~25MB/sec. Cost 1/3 of what a SCSI equivalent cost.

I can't believe you guys modded a troll up to 3...

I guess this sucks?

Or This?

This?

This?

These?

This stuff?

IDE is here to stay in the high end market, and it's going to kick SCSI's ass. Why pay 3X more per drive for the same HDA with a different interface board?

This is from the server in the first link above. Note that most of the write bottleneck is caused not by the drives but by the hardware RAID5 controller.

Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
bedford 1G 24436 11 22834 13 83890 43 361.2 2