Posted by
CmdrTaco
on from the now-thats-a-lotta-pr0n dept.
Delta-9 writes "Here is a writeup on how to combine 6 200GB IDE drives into a small tower and hack together some firewire controllers to give you one giant 1.2TB firewire drive." Very cool project, both technically and aesthetically.
Wow... that's some big access logs hes being hit with then;)
Re:Right about now....
by
veritron
·
· Score: 1, Informative
Actually, the way they get all those sick specs is by overclocking the system like crazy and installing a really nasty cooling system. Considering the insane prices they're charging for the thing it's so not worth it.
But yeah, if you do order one, it'll probably live up to the claims - it'll just be a huge waste of money.
You'd think if you have the mad skilz to hack up a 1.2 TB server, you'd be smart enough to get yourself better than a 256 kbps adsl connection to the net...
When I wired up 5 60MB SCSI-25 drives back in the day to get a whooping capacity of 5x60MB...
And still that amount of data is almost half of one of today's most popular RO mediums.. =)
But none the less, nice article and with the disk prices these day's it's getting closer within rage for many of the people that spend that much on electronics... I sure do =P
Unfortunately, you could buy a full system for the cost of the four-bay enclosure, gaining higher performance at the same price.
Does anyone know where to get a firewire-ide adaptor or a multi-drive firewire enclosure for a fair price?
Re:Uhm..
by
Anonymous Coward
·
· Score: 1, Informative
you don't need to buy a full box - you can buy a cheap old scsi box, remove the centronics ports from the back and replace them with firewire boards to connect drives to - check out the boards at firewire bridgeboards
Those boards are more than twice as expensive as the cheapest full enclosure (with case and power supply) you can find! This reinforces what I said. Why would the board be so much more expensive? Does anyone sell just the board for a reasonable price? A reasonable price would be cheaper than buying the same board as part of an enclosure. (I thought this would be obvious but I guess not)
Re:Slashdotted again...
by
green+pizza
·
· Score: 4, Funny
So you've got a terrabyte of data, but can it handle Slashdot?
1.2 TB RAID -- $900 386-based Web Server -- $0.25 The satisfaction of a slashdotting -- priceless
Re:Slashdotted again...
by
Tackhead
·
· Score: 2, Funny
> > So you've got a terrabyte of data, but can it handle Slashdot? > > Increasing the amount of data without increasing the bandwidth is not the way to avoid slashdoting.
Hell, the guy could have had an OC-48, we're talking about a Slashdotting, even Firewire's gonna be a bottleneck. (Those poor, poor, poor drives. He should give them to me.)
Ha ha you just have a T3! I got duel OC3's + a T3!
Even with an unlimited bandwidth though, theres also the bandwidth in the computer itself. With the drives mentioned in the article I get 34.7 MB/s read speed (Benchmarked that exact type of drive just yesterday) which translates to about 278 Mb/s before overhead. have a 3MB page and get 12 hits per second or more and its going to overload the server no matter what kind of speed your internet connection is.
-- All misspellings and grammatical errors in the above post are intentional and part of my artistic expression.
Hmmm.. you'd need some aluminium bar-stock, a small die-casting setup, watchmakers lathe and an ultra-clean glovebox for assembly. And of course sacrificial drives to grab the platters and heads out of. You'd need to redesign the driver board and remember to feed extra power to the motor to counter all the extra mass. And while you're at it, why not cap the whole thing off with a perspex window and internal LEDs.
Hmmm.. you'd need some aluminium bar-stock, a small die-casting setup, watchmakers lathe and an ultra-clean glovebox for assembly. And of course sacrificial drives to grab the platters and heads out of. You'd need to redesign the driver board and remember to feed extra power to the motor to counter all the extra mass.
And while you're at it, why not cap the whole thing off with a perspex window and internal LEDs.
But what a wonderfully uber-kewl project it would be:)
As the other posters are saying, your price was too high. In 1988 I bought an 80 MB drive for $650, and I was very pleased that I was able to break the $10/MB price point.
he used fire wire and not usb or scsi - or well anything else. this will save us from the hundred or so "Why not firewire?" posts every time somebody discusses some other method of moving data around.
.
-- It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
Probably but it won't save us from the "Noone needs that much harddrive space"-trolls and the counter attack "yeah you do if you want to do raw video editing"
Absolutely correct, for streaming transfers those 6 disks have an aggregate transfer rate on the order of 160MB/s, putting them behind a single firewire interface is strangling them.
So when I put 6 individual disks on my 3ware controller in a 2x 64-bit slot inside my computer case why don't I get to be on the front page of slashdot? It runs way faster than this guy's setup and it was probably cheaper to put together.
did he connect all those drives to a single port or controller on his mac? I'd agree with you, unless he's just wanting a large filesystem and isn't particularly interested in the overall throughput.
as to being on the front page, do some case mods, add some CCFLs, and some blinking LEDs for each drive, and you should be all set!
Actually, I do run Gentoo on my boxes. Not because it's fast or source based, but simply because it's easy as hell to update. I'm lazy =)
-- - It's not the Macs I hate. It's Digg users. -
/.'ed
by
Anonymous Coward
·
· Score: 5, Informative
A few years ago while browsing the Halted Anniversary Sale, I came across a 4 bay 5.25" SCSI drive case for something really cheap (I think $35). I can't pass up a deal like this so I snagged it. Well it sat around for a long time, past the point of me giving up on SCSI. I had replaced my 3 36GB SCSI drives with 2 80GB IDE drives and never wanted to go back. So it sat. While performing the SparcStation ITX hack, I discovered that firewire to IDE bridgeboards could be had in the $50 to $80 range. After using one there I started a little thread in the back of my mind about what other nefarious uses I could find for these little gems. Then one day I saw an ad for a full height 5.25" box that held 3 3.5" hard drives. Suddenly inspiration struck me like a bolt of lightning, and in true Dr. Bob fashion, I took it to an extreme.
The largest drive available at the time I started this hack was the Maxtor 200GB.
What do you think?
Here's how I did it:
1. Start with the empty case.
2. The original case fans were very noisy. In addition to that, the fan grilles cause lots of turbulence noise. So I cut them all out and replaced them with PanaFlow fluid bearing fans and wire grilles. I had to make custom power cable harnesses for these fans as well
3. As long as I was replacing noisy fans, I replaced the fans in the drive carriers with think PanaFlow FDB fans. I threw their grilles out altogether as they operate with their doors closed and the grille is, well, pointless.
4. Next I downloaded the art work for the firewire logo from Apple's web site. I printed out one that would fit and glued it to the boring beige top case. Black indicated material to be removed. First I drilled pilot holes to get the tool bits in. Then I started cutting to remove the big chunks, then I cut closer to the edges with my dremel tool, and finally filed it smooth with my half round bastard (not shown here). Those that know the joke are now snickering.
5. After this the whole case was sanded and painted with Krylon Fusion Burgundy Red. This paint takes 7 days to fully polymerize so I set it aside and focused on the electronics. I also bought a hunk of clear acrylic from TAP plastics and a 30mm round for the center of the logo.
6. OK I've got a firewire hub that mounts in the same hole as the old Centronics connector did (firewire depot), and 3 dual drive FireWire to IDE controller cards. Plus I need to supply power and route the cables for data and the LED's. I decided to mount them on the empty panels between the back of the drives and the back panel. First I had to measure the card for the stand off. Never leave home without your trusty calipers.
7. Now the cards can be mounted on my 3/4" standoffs and 4/40 screws. This project would be impossible without round IDE cables. The powered hub is visible in the lower left of the 1st picture.
8. This might look like a chaotic mess to you, but it's actually a carefully choreographed symphony of cable. The truth is, it's the only way it would all fit.
9. This is glue. Strong stuff.
10. When the front was dry, I hit it with some 3M Imperial Hand Glaze. That made it nice and shiny. Mmmmmm Shiny. (droooool)
11. Now it's time to get silly. I installed 2 6" and 2 12" tri-color cold cathode lamps. These will really spice up my life. After messing with EL wire, I have decided that it's not bright enough to be worthwhile for almost any use. CCFL lamps however are bright enough to be seen in any lighting conditions including camera flash. EL wire is only visible in low light. CCFL lamps also last longer.
12. Like EL wire, cold cathodes require a high voltage inverter.
13. Finally I mounted some LED's in the front connected to the busy signal outputs on the firewire controller cards. I may at a later date remove this metal grill to improve the lighting and airflow.
All done. Here are some beauty shots:
Please visit my archive of art work photos for this project. Click on any picture for a very high resolution photo. Some of these really move me.
Tech Specs:
Firewire 400 (sustained transfer rate of 35MB/s, max for firewire 400)
Oxford 911 chipsets
6 Maxtor 200 gig ATA 133 hard drives
4 cold cathode lamps with a combined output of 12 watts
16 LED's
Powered firewire hub
... and yes I caught the Blues Brothers reference in #9.
-- The higher the technology, the sharper that two-edged sword.
isn't there a question out there on this??
by
peragrin
·
· Score: 2, Interesting
hey guys isn't this an article already here. or maybe he should of waited an hour????
http://ask.slashdot.org/article.pl?sid=03/09/23/20 41246&mode=thread&tid=137&tid=188&tid= 198
-- i thought once I was found, but it was only a dream.
What good will that do you?
by
psoriac
·
· Score: 5, Funny
Tell me, Mr. Anderson, what good is a 1.2TB firewire drive if you can't serve any content from it?
-- I browse Slashdot at +3, Funny
Re:What good will that do you?
by
ItalianScallion
·
· Score: 1
yes, maybe if you had used scsi or usb...
Re:What good will that do you?
by
McAddress
·
· Score: 3, Funny
Lets see, at 700 MB each, that works out to about 1700 movies. Plus about 3,000 songs. Sounds good enough for me.
Re:What good will that do you?
by
stienman
·
· Score: 1
Lets see, at 700 MB each, that works out to about 1700 movies. Plus about 3,000 songs. Sounds good enough for me.
What, is DVD too good for you? 150 DVD movies, with extras.
-Adam
Re:What good will that do you?
by
McAddress
·
· Score: 1
huh? thats like saying "is wav is too good for you. 400 wav songs, with cover art."
Re:What good will that do you?
by
dododge
·
· Score: 1
Lets see, at 700 MB each, that works out to about 1700 movies.
Or about 60 compressed HDTV movies.
Or maybe one uncompressed HDTV movie. But it'll have to be a short one:-)
Re:What good will that do you?
by
Josh+Booth
·
· Score: 1
Oooh, 1.2 TB on USB 2.0. That would be scary. Mwahahahaha! What, USB can do 127 devices per bus, right? Minus two for mouse and keyboard (unless you want to go oldschool), and buy 125 of these and you have yourself 15 TB of HD space all LVM'ed into various partitions. To make it efficient, you'd have to fill your ~6 PCI slots with some of these USB 2.0 cards and stripe the drives, although you would still have only ~30 USB ports. Oh, duh, it wouldn't matter because they aren't all on the same bus... Anyhoo, we need a baseline anyway, so that's about 5 drives per port, which is still going to suck, but not if you stripe them properly. Actually, all thoes drives would cost about $26500 before shipping, and would draw an ungodly amount of power. Is anyone willing to see whether it will work?
I'm assuming the RIAA hates the idea of anyone having a homebuilt 1.2TB drive hooked to IRC or Kazaa??
Does that warrant a supoena for trying to build such a thing?
/.'ed but i can imagine...
by
stratjakt
·
· Score: 1
You get an old mobo, stick a bunch of drives in it, install linux, perhaps on a CF card, install a firewire card, and let software munge the whole thing into a big firewire drive.
I guess if I could see it I might be impressed case-mod wise, but technically it sounds like any other fileserver, just using firewire instead of network.
Am i missing something?
--
I don't need no instructions to know how to rock!!!!
Re:/.'ed but i can imagine...
by
stratjakt
·
· Score: 1
Ok, just RTFA, so before my fanclub jumps up and down eager to flame me, I'll correct myself.
He crazy glued a firewire hub and a bunch of idefirewire converters into some snazzy looking box. So essentially the PC sees 6 200gig firewire drives, not one big one. So case-mod wise he gets points, but nothing new tech-wise.
A linux patch to make all its drives look like one big firewire drive via the firewire port would be cool. I think "mods" should address both hardware and software.
I like my idea better. What's the partition size limit, anyways? Does firewire do LBA48 or what?
--
I don't need no instructions to know how to rock!!!!
Re:/.'ed but i can imagine...
by
Raffaello
·
· Score: 2, Informative
If you actually read the article, you'll see that he's running Mac OS X, and that the OS sees his box as a single, 1.1 TB, Mac OS Extended volume called "BigHonkingDrive." So, no, the "PC" doesn't see it as 6 200 gig firewire drives.
Re:/.'ed but i can imagine...
by
GigsVT
·
· Score: 1
I approached a linux kernel SCSI developer once about "drive aggregation" using ATA drives (you could use SATA these days) in a full computer, and the SCSI port as the output, i.e. you'd plug the whole linux box into another computer and it'd look like one big SCSI drive (like what ACNC does with their hardware).
Anyway, he basically told me to fuck off, and spouted a bunch of highly technical stuff, in a very sarcastic way. (It doesn't surprise me that the linux SCSI code is so bad after talking with him)
I still think it would be a cool idea. In theory there should be no issue making a SCSI card connect to another SCSI card, they are both SCSI devices. You'd just have to change the SCSI ID of one of them. I know someone's used SCSI as a network card once as an experiment, so it's not unprecedented to make odd things appear on the SCSI bus from software.
-- I've had enough abrasive sigs. Kittens are cute and fuzzy.
Re:/.'ed but i can imagine...
by
piznut
·
· Score: 1
"How many drives does Apple support inside a case"
If you buy a G5, the answer is 2.
Re:/.'ed but i can imagine...
by
Chaset
·
· Score: 1
Along those lines, older Apple laptops (back when they all used to have SCSI ports rather than firewire) can be placed in a "scsi disk mode".
This allowd you to hook the laptop up to any Mac desktop and use it as a SCSI disk. This was possible even on Powerbooks with internal IDE drives.
I guess what I'm saying is that yeah, something like that should be possible, although the Apple solution probably required some support in firmware (BIOS).
-- --
"This world is a comedy to those who think, a tragedy to those who feel."
Re:/.'ed but i can imagine...
by
wolrahnaes
·
· Score: 1
They still can do this over firewire.
-- I used to get high on life, but I developed a tolerance. Now I need something stronger.
Forgive my hardware ignorance but...
by
Kethinov
·
· Score: 4, Interesting
if you use firewire controllers to give you one giant 1.2TB firewire drive, doesn't that essentially make 6 hard drives pretend to be one? (AKA the OS doesn't realize it's many) And if just ONE of those drives failed, aren't you shit out of luck with your data?
Again, forgive my hardware ignorance if I'm way off.
-- You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Re:Forgive my hardware ignorance but...
by
kasperd
·
· Score: 5, Informative
Doesn't sound like ignorance to me. If the six 200 GB drives make up a 1.2TB logical drive, there cannot be any redundancy. Six IDE drives, and no redundancy - I don't hope he have any important data there. Had he at least used RAID-4 or RAID-5 giving him a 1TB logical drive and one redundant disk, he would have a fair chance of keeping his data (assuming the broken disk gets replaced before the next fails).
Re:Forgive my hardware ignorance but...
by
fpu
·
· Score: 1
Actually, I was wondering whether the fact that these six disks are connected to the same Firewire hub makes the OS believe they are a single entity -- doesn't that require anything more intelligent than some IDE-to-IEE1394 adaptors? Or is this kind of thing handled by MacOS X (and never mentioned in the text)?
-- /usr/games/fortune: command not found
Re:Forgive my hardware ignorance but...
by
Kethinov
·
· Score: 1
Yeah I suspected that but I didn't want to go making grandiose claims without being sure. Personally, I'd leave them as separate drives and use them for different purposes and/or operating systems. That way if one fails, everything else is independent and you only have to worry about recovering THAT drive's data because it's been modularized.
Personally I'm not a RAID fan. I operate with three hard drives. One 40gb drive, one 20gb drive, and one 8gb drive. Yeah/. crowd I know that's pretty old school, but I don't need a lot of storage. What I do need is physical separation of data. One drive is my Wintendo (8gb), another is Gentoo (40gb), and another is my all purpose storage bitch (20gb). By separating the purpose of each drive you ensure that it if one goes down, the rest remain stable. Maybe this guy should have tried that? But then again, he may have been in it for bragging rights;)
-- You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Re:Forgive my hardware ignorance but...
by
Brummund
·
· Score: 1
Uh, where's the redundancy in that? What happens is your "all purpose storage bitch" goes down? I really hope you backup that one.
I use RAID1 to mirror my/home etc., and it has saved my ass a couple of times. Installing an OS is easy, but having to redo all my projects would be a PITA.
Re:Forgive my hardware ignorance but...
by
Kethinov
·
· Score: 1
Uh, where's the redundancy in that? What happens is your "all purpose storage bitch" goes down?
CDs =P
Keep in mind that my all purpose storage bitch is only 20gb of storage and the hard drive isnt even close to being full. As for the 40 gigger, all it is is open source stuff and my personal projects, which are all backed up on the webserver. The 8 gig Wintendo is also backed up on CD.
If I had more data I'd probably be RAIDing but at this point it'd be a frivolous use of my limited money.
In either case I still believe in physically separating drives for the sheer fact that I run two operating systems and I don't feel like playing around with separate partitions and filesystems on a single RAIDed or non-RAIDed drive.
-- You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Re:Forgive my hardware ignorance but...
by
Brian+Blessed
·
· Score: 1
Raid-5 would give him 1200*2/3 = 800GB, but as the OS will see the 6 drives the best bet is to use mirroring to get 600GB because the performance of Raid-1 is so good (in Linux).
- Brian.
Re:Forgive my hardware ignorance but...
by
TCM
·
· Score: 1
Are you sure about the *2/3 part? IIRC RAID5 is N-1 usable disks, giving you 1TB storage (5x200G) with 6 disks.
-- Of course it runs NetBSD.
BTC: 1NT7QvbetmANwaMzhpVL6
Re:Forgive my hardware ignorance but...
by
Raven42rac
·
· Score: 1
Seems to me that it would more than likely write 1/6th of the data to each drive at once. So if one drive failed you would only be 1/6th shit out of luck.
-- I hate sigs.
Re:Forgive my hardware ignorance but...
by
JimRay
·
· Score: 4, Informative
It's a little confusing, especially if you're not on an OS X box, but this guy has built a software RAID setup. Essentially, all six disks are acting as one because he's used the OS X Disk Utility to set them up as one.
The problem with this is that OS X's Disk Utility doesn't support RAID 5 in software, at least not out of the box. So, you either have to stripe the six disks (lots of space, no redundancy) or mirror them (as much space as your smallest drive, full redundancy) . It looks like he went for the striping option, which is how he got over a terabyte. However, as it's been pointed out several times already, this is a bad idea because if one of those disks fails, his data is lost. And I seriously doubt he's backing this "disk" up...
What he should do (and quite possibly is doing for all I know, it's not detailed) is use something like Raid Toolkit to create a RAID 5 setup. Since RAID 5 uses both data striping and parity, his data is protected even if a disk gets hosed.
However, software-based RAID 5, at least in my understanding, isn't exactly a performance champ, so if he's doing a lot of reading and writing to that drive, he's probably better off getting a real RAID controller. However, this would make a killer media backup box.
The linux based software RAID HOW-TO is actually pretty informative for a general understanding of software RAID.
Cheers
-- My other computer is your Windows box
Re:Forgive my hardware ignorance but...
by
Polo
·
· Score: 1
I believe they show up as 6 individual drives.
Re:Forgive my hardware ignorance but...
by
Scott+Laird
·
· Score: 1
Done properly, software RAID 5 can be as fast (or even faster) then hardware RAID 5. After all, in most cases, "hardware" RAID is really just software RAID on an embedded CPU. I've dealt with way too many hardware RAID cards that slow to an utter crawl when doing RAID 5. On the other hand, Linux's software RAID is pretty zippy; I've seen 200MB/sec on the right hardware.
There can be performance issues if you're short of CPU on your system, or if you're low on RAM, or if you're nearing the limits of your system bus, but none of those are usually the case on modern storage servers--you don't usually need to run them that close to the edge.
The one thing that hardware RAID can buy for you is a non-volatile cache; that lets you get away with caching writes, and just generally buys you better performance and consistency. One of these days, someone will add journalled RAID-5 to Linux, though, and then you'll be able to accomplish the same thing with a PCI-based battery-backed RAM card for cheap.
Re:Forgive my hardware ignorance but...
by
Fred+Ferrigno
·
· Score: 1
In RAID systems, the data is broken up and distributed evenly across the drives. So rather than simply missing 1/6th of your files, you're missing 1/6th of EVERY file, which is a lot harder to recover from, if not impossible.
Re:Forgive my hardware ignorance but...
by
afidel
·
· Score: 4, Informative
Wrong calculations, that's for a 3 drive RAID-5, for a 6 drive array it's 5/6ths or 1TB in this case. In general it is (N-1)*capacity.
-- There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Re:Forgive my hardware ignorance but...
by
Darth+Hubris
·
· Score: 2, Insightful
Shit. The things I absolutely can't lose, that can't be downloaded again(OSes, apps, pix, etc)amount to 16 Mb. Everything else is expendable.
-- The party's over... the drink... and the luck... ran out
Re:Forgive my hardware ignorance but...
by
ScrewMaster
·
· Score: 1
Redundancy, reschmundancy. This is just very large and very cool, and will likely be used to mass quantities of MP3s and ripped DVDs. I like it.
-- The higher the technology, the sharper that two-edged sword.
Re:Forgive my hardware ignorance but...
by
i.r.id10t
·
· Score: 1
Yeah, but there is no parity bit or other way of recovering the missing info from the other 5 drives.
-- Don't blame me, I voted for Kodos
Re:Forgive my hardware ignorance but...
by
Thomas+A.+Anderson
·
· Score: 1
I've always been amazed how we only loses 1 disks worth of space in raid 5, regardess of number of disks (and yeah, I know it's not one physical drive that is unavailable). How does this work? Seems like magic to me.
-- Personally its not God I dislike, its his fan club I cant stand (bash.org)
Re:Forgive my hardware ignorance but...
by
Raven42rac
·
· Score: 1
That is what I was saying, only rephrased.
-- I hate sigs.
Re:Forgive my hardware ignorance but...
by
Raven42rac
·
· Score: 1
That is what I was saying, only during a vain attempt at humour. Yes you are screwed if one of the drives died, totally screwed. There is no "evil" bit either. =P
-- I hate sigs.
Re:Forgive my hardware ignorance but...
by
afidel
·
· Score: 1
The parity calculations are computed as if there was one parity disk and then they are distributed across all the disks. The parity blocks are the same size as the data blocks and as long as only one disks dies the other disks will contain all but one of the data blocks plus the the parity block (on average, any particular file may be missing a data or parity block). There is also another newer form of RAID that some vendors called RAID-6 which writes two disk failures per array but at the price of losing two disks worth of capacity to parity data, in real world experience I would say that is overkill because most situations that would lead to more than one HDD per array dying will kill most/all of them. This page's illustration might make it a bit more clear for you.
-- There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Re:Forgive my hardware ignorance but...
by
Thomas+A.+Anderson
·
· Score: 1
Thanks - that did help!
-- Personally its not God I dislike, its his fan club I cant stand (bash.org)
Re:Forgive my hardware ignorance but...
by
216pi
·
· Score: 1
Build a hancy-fancy uber-raid? Allways remember:
There are two types of hard disks: those that have crashed and those that will.
Re:Forgive my hardware ignorance but...
by
Fred+Ferrigno
·
· Score: 1
My mistake. It seemed to me that you were trying to say that it wasn't such a problem. Perhaps you were being sarcastic.
Re:Forgive my hardware ignorance but...
by
afidel
·
· Score: 1
Baydel call's it that, and I think I remember at least one other vendor doing the same.
-- There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Re:Forgive my hardware ignorance but...
by
Raven42rac
·
· Score: 1
I was, I should have said as much. No offence taken, none meant.
Wouldn't it make more sense to build a SATA RAID array? Using the 3Ware 8 channel SATA controller and a bunch of big ass Maxtor SATA drives you can get more storage for probably less cost and complication.
I'd like to put about ten drive in my current case and a new motherboard with 64bit / 66mhz PCI slots to support that fast 3Ware RAID controller. Eight drives in RAID 5 on the 3Ware, and two drives mirrored on another card (to be named) for the system. If the drives in the RAID 5 configuration were 300GB SATA drives I'd have enough space to last me, my family, and my friends about 10 years. Woohoo! Unfortunately that would run me about $8,000 by the end of it because I'd be building from the ground up with the exception of the case, which I would probably buy knew anyway.
It would be cool if the OS saw it as 1 huge 1.2GB drive, but that would be a hell of a hack to make it display as unified piece of media.
I would imagine he has the drives striped in software to appear as one large drive. This is pretty easy to do with Windows, OS X, and pretty much any semi-modern un*x.
The memory stick RAID is even worse. With a lot of random access he's going to kill memory sticks pretty fast.
But at least he doesn't take himself too seriously, and has something of a style all to himself
If you want to know if it is RAID 0 or RAID 4 or RAID 5, you are asking the wrong guy, I build floppy disk drive RAIDs none of that fancy shmancy stuff. All I know is that my FDD raid rules! Well, anyway, you drag each volume over to the RAID making thing and then click CREATE. After a very entertaining display of USB FDD flashing lights, whirling drives, and an interesting rhythm of spinning technology sounds, my RAID was complete. At first I thought it was screwed up, but it just took a while for the various units to meld themsleves into a single super duper kalimazooper floppy drive.
Incidentally he got a whopping 115.2KB/s out of his raid array. I can dl off the Internet faster than that via my ADSL connection - I get peaks as high as 200KB/s -> Free DSLAM modems rule (French ISP).
Only one thing wrong...
by
TheSHAD0W
·
· Score: 3, Insightful
Firewire is SLOW. You're taking drives capable of bursting 100 or 133 megabytes per second and plugging them into a bus that maxes out at 50, with a practical limit of half that. Also, aren't those little bridges expensive? You might be better off getting a RAID controller and boosting your throughput to 1/2 gigabyte per second or better.
Of course, Firewire is a lot more convenient. But if you want convenience, why not just buy single-drive externals and stack them? I suppose you may have an old case lying around, but I'd personally find a bunch of drives that were easily separable more useful. If I needed to take the data on one with me, I could just unhook it and bring it along.
Re:Only one thing wrong...
by
EinarH
·
· Score: 2, Informative
1/2 gigabyte per second?
I'm not aware of any IDE RAID controller that can do that, but I could be wrong.
The 3ware Escalade 7500 series is some of the best IDE RAID controllers out there and they do burst at max 190MB/s streaming (RAID 5) in read and max 70 MB/s sustained.
--
Melius mori in libertate quam vivere in servitute.
Re:Only one thing wrong...
by
vadim_t
·
· Score: 1
Well, PCI-X can certainly handle half a gigabyte per second, so I suppose that there's got some really expensive card that can do it too.
Re:Only one thing wrong...
by
toddestan
·
· Score: 1
Besides, he's using the drive to store media files. You don't need speed for that at all. Sure, it might be a little tedious copying 1TB of data over to the drive, but once that data is there he'll be fine. Heck, my USB 2.0 external can serve up MPEG2's and DiVX movies just fine when hooked to a USB 1.1 port.
Re:Only one thing wrong...
by
toddestan
·
· Score: 1
It is very painfully slow. But the 12mbps (about 1.5 mb/s) is fast enough to serve up MPEG2 and DiVX.
For example, I have 3:40 MPEG2 music video that's 132MB. 3:40 is 220 seconds, so to serve it up I need atleast 600k/s, which is less than half the speed of USB 1.1. So it's good enough in this case.
Re:Only one thing wrong...
by
bill_mcgonigle
·
· Score: 1
-- My God, it's Full of Source! OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
6 drives, no redundancy.. Stupid.
by
Ferrule
·
· Score: 5, Insightful
Ok, so the guy goes to great lengths to build a 6 drive 1.2TB external storage device.
Doesn't menton how the drives become one.. It's not raid-5 as that would be 5X200MB + 1 parity drive. So it's either striped, or the large volume properties were faked.
IMO buying 6 drives and not running RAID 5 is really dumb.
or it is the perpetually restarting Micro$haft defrag utility that always ends when the drive is still fragmented and says it is finished. Sort of like a premature ejaculation.
--
From excellent karma to terible karma with a single +5 funny post...
What is this tear-uh-bite that you speak of? I've never heard of such a thing. Please tell us how big this drive is in units of Libraries of Congress or in terms of how high a stack of floppies it would take.
-- "People that quote themselves in their signatures bother me" - athakur999
The stack of floppies would be 2.52 Kilometers (or 1.57 Miles) high. This assumes one double-sided 1.4MB floppy disk is two millimeters think. If we calculate 1.2TB to mean 1,200,000 MB - then the height of our floppy disk stack becomes 2.4 Kilometers or 1.49 Miles high.
As a Canadian, I am unfamiliar with this "Congress" of which you speak, though im sure she has a very big libraries.
If a floppy disk is 1/8 of an inch (I'm guessing here) and holds 1.44MB And there are 1024 MB in a GB, and 1024 GB in a TB, you would need: 873,813.33 floppy disks to hold 1.2 TB, which would measure 4551.11 feet high
Or better, how many Libraries of Congress could those 3.5" floppies fill? Well, lets see.... they would fill a room 40ft x 22ft x 10ft. What that's all? And they could only pave half a mile of road. Quite lame.
However you could stitch together some fine floppy-disk jump-suits for 10 blue whales! Much more impressive.
Calculating with values of 3.3mm for thickness of floppy disk, TB=1024^4 and Floppy Disk capacity = 1.44MB, then we get a simple total of 7.8x(Distance from Earth to Moon) for a stack of floppies.
According to this google post and this one, the ISO spec of a floppy disk says it's 3mm thick. One floppy holds 1.44MB of data in decimal format. Divide 1.2TB by that and you get 833,333 and 1/3. Multiply that by 3mm and convert to a larger unit understood by Americans and you get: 1.55 mile-high stack of floppies.
The Library of Congress holds 10TB of data. Convert the 1.2 decimal terabytes to hexadecimal and you get ~1.12TB. Divide the exact number into 10 and you get: 11.18% of the Library of Congress
funny thing about assumptions... outrageous assumptions are as legitimate as reasonable ones. as long as the assumption is clearly stated, there is no foul. and assuming that the grandparent is not a moron, he makes perfect sense:)
With this much space, I can backup all my vhs head-cleaner tapes to dv format!
I built an even bigger array last week...
by
dewpac
·
· Score: 1
(8) 200GB WD w/ 8MB cache (1) 3ware 7506 (1) Linux box that already was serving files with nfs + samba on a much smaller drive..
So what? Now I've got 1.4TB (at least I don't lose all/part of my data if the drive fails, unlike this guy). I've only used maybe 10% of that. I've moved all the Videos, mp3s, ISOs, and any other random garbages off all the other pc's in the house to it.
I'd call it a waste, but its too damn cool.
Re:I built an even bigger array last week...
by
ericdano
·
· Score: 1
But how fast are the file transfers compared to firewire? I too am looking for massive storage solution that is cheap, and something like what this guy did sounds great, mainly for the access speeds......
--
It's either on the beat or off the beat, it's that easy.
I moderate therefore I rule!
--
Re:Another simple problem with an elegant solution
by
kvandivo
·
· Score: 2, Informative
http://www-cs-faculty.stanford.edu/~knuth/news.htm l
He's probably thinking that he should have used the Ultra160 SCSI solution after all.
must be using Software RAID...
by
madcoder47
·
· Score: 1
According to ther site, the guy uses three firewire-to-ide interface cards and a firewire hub. Obviously, from the screenshot, he uses Mac OS X, so I assume has has to be using software RAID... I dont see any mention of a RAID controller or some sort of combinatorial device other than the firewire hub, and that cetainly will not combine drives.
FYI, Mac OS X includes software RAID by default, it's accessible in Disk Utility, located in/Applications/Utilities
Re:One 1.2TB drive to the OS, or a bunch of 200GBs
by
mixmasta
·
· Score: 1
huh? It does look like one disk. It's called software raid.
--
#6495ED - cornflower blue
Re:One 1.2TB drive to the OS, or a bunch of 200GBs
by
Drakin
·
· Score: 1
Uh... it looks like it -does- see it as one single drive, going by the screen shot.
-- The higher the technology, the sharper that two-edged sword.
guess the firewire aint fast enuff to prevent /..
by
Anonymous Coward
·
· Score: 1, Informative
to prevent the/. effect.
the firewire terabyte drive has just maxed out
----------------- Tech Specs:
Firewire 400 (sustained transfer rate of 35MB/s, max for firewire 400)....
--------------- thats not enuff for you being able to escape the wrath of slashdot....
OS X software RAID / LVM
by
djtack
·
· Score: 2, Informative
While it isn't mentioned in the writeup, the firewire bridges will not make the drives appear as one. The OS will still see 6 different drives. OS X pretty easily supports software RAID and LVM, so he's almost certainly using one of those methods.
And yes, if any one of those drives dies, he's SOL, although as somebody else mentioned a RAID 5 would help this situation.
Is where he got the chipset? I have wanted to do something like this but everything I find is either a complete ready to go unit, or a case with the chipset in it. I want just the chipset, no case or drives or anything.
Shesh, some things are sooo hard to find now-aday's.
i think the project is cool irregardless of just what was used. my statements have to do with/. more than the technology. whenever there is a post on/. about someone using usb or something else- tons of posts pop up saying 'why not firewire?' or something to that effect. drives me nuts- those people get on my nerves- not people acutally using firewire. sorry if that was unclear.
.
-- It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
So he took off the shelf parts and used them in the way they where intended (gasp) what a 1337 hardware hacker.
Wow that's the best impression of the sarcastic geeky guy who owns the comic book store in the Simpsons.. oh wait, nevermind.
Re:RAID Fun (I just did the math)
by
Stephen+Samuel
·
· Score: 1
6x200MB drives giving 1.2Terabytes... OMB: This means that the guy is using RAID-0. In other words, when (not if) one of those drives goes south, he's gonna have 1.0Terabytes of allmost usable filesystem.
If FSCK manages to make any sort of sense out of what's left, it's gonna take a week of babysitting to get it to the point where it'll do much of anything useful with that data.
-- Free Software: Like love, it grows best when given away.
Well, that was a bit difficult to determine, considering his web page is thoroughly slashdotted.
Sure, pay for it
by
Anonymous Coward
·
· Score: 1, Insightful
I'd be happy to build a much simpler SATA raid with hardware RAID5. If you'd be so kind to send me the money for it, I'd love to build it.
I perform hardware reliability testing for a living, so for all those nay sayers griping about the reduction in reliability I say: I'm well aware of exactly what the impact of MTBF is, and since this array is ONLY intended to serve as a backup for my own systems, and me digital media throughout the house, it's worth the risk. I am fully aware of the consequences of a failure of any single part of the system.
buying drives for an array
by
MarcoAtWork
·
· Score: 4, Insightful
I would hope that this person (and anybody else that is thinking about creating an array) is not going to buy all 6-8 drives at the same time from the same supplier.
More often than not drives built in the same batch tend to fail fairly close to each other, and if more than one fail at the same time you can kiss goodbye to your RAID-5 array (and you were backing up your 1+TB of data, weren't you? after all it takes 'only' about 250 DVDs to do it, doesn't it?)
I think that ideally you'd want to buy your drives over a 6-8 months period from different suppliers for every drive, while it's definitely messier in terms of warranty etc. the additional protection from 3 drives failing at the same time should be worth the hassle...
just my 2c
-- --
the cake is a lie
Re:buying drives for an array
by
_avs_007
·
· Score: 1
If I used it to store pr0n, I would still want redundancy:p
Re:buying drives for an array
by
seibed
·
· Score: 1
better that they should buy the same drive, but late in its product cycle, when most of the bugs have been worked out (better code, consistent media, reliability return info folded back into the product... those kinds of things.)
Re:buying drives for an array
by
WuphonsReach
·
· Score: 1
I've seen the cluster of failures as well. When one drive goes in the array, a second one is often sure to follow.
Which means... not only should you be running RAID5, but you should also have either a hot-spare drive in the array, or a ready-to-go drive boxed and ready for installation. (You did opt to install hot-swap bays, such as the ones Promise makes?)
The bigger risk for this yokel, is that he's trying to put (3) 3.5" HDs into (2) 5.25" bays. I actually have a bay cooler that lets me do it (from dirtcheapdrives). However, even with the fan running, I long ago decided that it was a very bad idea to pack those drives that close together. So now I just install (2) drives in that device.
He's almost guaranteed to lose the entire array within (6) months if he's running 7200rpm drives that are packed into a case like sardines. Better luck if he uses 5400rpm drives that run cooler. (5400rpm drives make great USB enclosure drives because they run cooler and air flow in a USB drive enclosure is usually sub-par.)
-- Wolde you bothe eate your cake, and have your cake?
Re:buying drives for an array
by
pmz
·
· Score: 3, Insightful
I think that ideally you'd want to buy your drives over a 6-8 months period from different suppliers for every drive...
I respectfully disagree. For a high-availability array, which would you rather have:
- a set of six matched drives, with the same firmware revision and protocol implementation nuances providing thousands fewer variables when troubleshooting a failing system.
- six randomly purchased drives with who-knows-what and who-knows-how-they-will-interact providing only the possibility of trial-and-error chance resolutions of problems.
I think there's a reason why Sun manages the firmware revision of their harddrives as part of their complete software configuration. Sun even provides patch sets to upgrade drives to fix anomolies that come up.
Yes, there is more than just brand-name behind Sun's high OEM prices (and Sun knows it too...that'll be $600, please).
In other news the RIAA demands 45 trillion $
by
NoSuchGuy
·
· Score: 1, Funny
In other news the RIAA accused of theft of 300,000,000 (three hundred million) MP3s (4MB each MP3). The lawyers of the RIAA demand a compensation of 150,000 USD per stolen song. This summs up to 45,000,000,000,000,000 USD or 45 trillion USD. That is 9 times the GNP of the USA in 1998. The shares of the RIAA members are going balistic.
Further more these lawyers argue there is no legimate use of 1200GB of disk space for a private person.
-- Grundgesetz * 23. Mai 1949 - 30. November 2007 - http://www.vorratsdatenspeicherung.de/
I realize this is somewhat redundant, but how well would this work with RAID5? Since the IDE/firewire converters are three separate physical units, all parity information would have to be processed by the host (PC, or mac in this case). Firewire should be fast enough to handle the extra data, but I'm not sure the added overhead (calculating parity data, sending it over firewire) would fare well for performance. Perhaps a solution which handles RAID5 on the target end would be better?
Re:One 1.2TB drive to the OS, or a bunch of 200GBs
by
jo_ham
·
· Score: 2, Informative
He does in fact, have it striped in OS X.
He had a screenshot of the Finder's 'get info' window for the drive. He named it, aptly enough, "BigHonkingDrive".
This person better get their ass in gear. The properties dialog said he had only used 130 megs of the drive and that's just not right.
I wanna see what happens when you log onto Kazaa with a TB of mp3s.
c'mon you were all thinking it...
by
Anonymous Coward
·
· Score: 1, Funny
"Gee, that's a lot of pron"
This is/. right?
Re:6 drives, no redundancy.. Stupid.
by
Ferrule
·
· Score: 1
Poor coward.. I'll rescue you from the 'beneath my threshold pergatory'
Put them all in a box with a cheap mobo with raid controller, have it run linux, and do some haxoring to expose it as a ginormous HDD to the firewire port.
I think what you suggest is excellent. I imagine there is a IP over fireware module for Linux, so you could use Samba to get at the big drive you suggest.
What would be cooler is if you could create a great big raided partition on a linux box which could be mounted as a USB or firewire drive on another. The huge partition wouldn't me mounted locally at all, just ready to be served to hot-plug clients.
-- If all this should have a reason, we would be the last to know.
Hardly newsworthy
by
Anonymous Coward
·
· Score: 2, Interesting
Fore everyones information, this 'project' is nothing new or special. I will not metion its name, but connected to the better servers of a 'top' P2P application are dozens if not hunreds of people with shares (ie. storage solutions) of comparable and rarely even greater capacity. On some servers, they won't even let you in if you have less than a hundred gigabytes of shared 'infomation'...
But why is this thing restricted to Windows and Mac? Is it using some weird driver? I would think it would appear as a firewire storage device, and thus be OS-independent. A bit of searching shows that people are using these with Linux, but I don't know if they're really a good idea yet.
Re:Firewire enclosure
by
Jeff+DeMaagd
·
· Score: 1
I don't know but I really doubt they need special drivers unless you call OHCI special, which it is not.
I tried this last year, but the devices on the same firewire chain got assigned somewhat randomly by the Linux driver so it was very difficult to tell which device corresponded to which physical drive. If you had trouble with one device it was difficult to tell which drive it was.
I tried this on Linux with some problems.
by
Rolman
·
· Score: 4, Interesting
I tried this on Linux and got terrible performance at the first try, I got a 23MB/s RAID-0 when each HDD is capable of 26MB/s by itself (everything according to Bonnie++ and hdparm). I didn't know what to blame, the bus, the cables, the Linux SCSI layer, or the whole IEEE1394 support on Linux. Windows was noticeably faster with up to 28MB/s.
Then I made some more research and it turned out the problem was caused by the sbp2 kernel module. This module had some good fine-tuning parameters (sbp2_max_sectors, sbp2_max_outstanding_cmds and spb2_max_cmds_per_lun) up to 2.4.20, but these got ditched in 2.4.21 in the name of a "better way of handling these parameters". I understand the logic behind this move, but the tweakable granularity should have been kept.
Using 2.4.20, I managed to get better performance by tweaking these parameters, then modified sbp2.c on 2.4.22 to reflect the changes. However, I haven't been able to get the 35MB/s this guy got so easily on MacOS X, I'm currently stuck at 29.22MB/s maximum and it's painfully slow to test all combinations of those variable parameters on the sbp2 module.
I just wish there was some document which could explain more about the relationship between these parameters for people not actually involved on the linux1394 project. The comments on sbp2.c are not helpful beyond this point.
By the way, I'm using two Oxford-based bridges to connect two 8MB cache Matrox HDDs, and I'm using Bonnie++ and hdparm for testing. YMMV but the least I can say is Linux RAID support on Firewire still has a long way to go.
-- - Otaku no naka no otaku, otaking da!!!
The object of the game ...
by
ScrewMaster
·
· Score: 1
It's not so much that it can be done in a complicated way, it's just that figuring out how to build something like that, and then actually doing it, is really most of the fun. Actually using it is somewhat anticlimactic in comparison. (sigh) unfortunately for me my own RAID array is getting rather full, and this article has me thinking. Dammit all to hell.
-- The higher the technology, the sharper that two-edged sword.
appearantely nothing
by
Anonymous Coward
·
· Score: 1, Interesting
People put those in PC's all the time which are probably much more sensitive to noise than my simple system. Keep in mind that the drives themselves are fully enclosed in grounded metal. So the only exposure risk is to the FW controllers. They don't seem to care.
To verify integrity I wrote a 1TB file of pseudorandom data using PRBS23 and then read it back and verified the data integrity with no errors. As a side note, it takes about a day to do this over firewire, although some delays were because my powerbook's cpu is too slow to decode the data at full bus speed.
MO buying 6 drives and not running RAID 5 is really dumb.
Or a RAID 0+1 arrangement with three drives per volume. In this case, we would still be talking two ~500 gb volumes, which is quite impressive of its own volition. Honestly, how many of you could even get to 75% on a 1 TB volume anyway?
Or, if you just want to act silly you could build two and mirror them...
Re:What about a USB 2.0 Drive tower?
by
ultrapenguin
·
· Score: 1
At this place you can buy what looks like a USB2 to 4 IDE controller adapter. What you don't see at the manufacturer's site is that this board fits into a standard ATX power case, takes standard ATX power supply, so you can easily make a cool 4 200 or 300 or whatever HDD external case. As you can see Here, that board being mounted into a case, where you can hook up the hdds, using the case's power supply, etc.
Too bad its USB2, and too bad each IDE controller only supports one device, otherwise this would make for a killer 8-device controller...
Any comments about warez, should meet similar fates, although a home server done this way would likely result in n^666 years prison time, if RIAA has their way!!
Hey, that can't be that bad...You'll still get out in polynomial-time....666^n would be much worse...
-- If religous zealots don't believe in Evolution, then why are they so worried about bird flu?
Be sure to use a journalled fs and LVM
by
Tracy+Reed
·
· Score: 3, Informative
Others have mentioned the necessity of RAID 5 in a setup like this but let me point out that you don't want to fsck 1T of disk. I have had to watch the fsck of 500G of disk back before we had journalled fs and it was terrible. When we started attaching many terabytes of disk to Linux boxes we needed a better solution. So you will want to use a journalled fs. Reiserfs is my favorite. Then you will not want to have to backup/restore when you decide your current partitioning layout was a bad idea or just generally want to shuffle things around so be sure to use LVM also. I use LVM on all of my machines, even desktops, and it has really made life easier. Often you will need more room on/home but notice that/var has a couple gig unused and with LVM you just shrink/var and expand/home all without reboot and you are good to go.
Re:Be sure to use a journalled fs and LVM
by
Jah-Wren+Ryel
·
· Score: 1
Fsck speed is almost completely dependent on the number of inodes, not necessarily the size of the drive. So, if you really wanted to use a non-journalling FS and were storing primarily large files, like say 700MB DiVX rips, then by creating the filesystem with a signficantly reduced number of inodes (like 100th of the original inode density) your could FSK a terabyte or so in a reasonable amount of time.
Why would you want to do this instead of using one of the many journaling filesystems available for linux? I dunno, but you could.
-- When information is power, privacy is freedom.
Re:Be sure to use a journalled fs and LVM
by
pe1chl
·
· Score: 1
Last time I built a new system with two 120GB disks I thought "well let's consider to use LVM", after having seen it on IBM AIX.
So, I started experimenting with SuSE Linux installation on LVM. Of course I wanted RAID-1. Well, that did not seem to be considered. You could make two partitions, RAID-1 them and run LVM on top of that, but no RAID-1 as part of the LVM (as in AIX).
Next, it turned out that you cannot have root on LVM. And no swap either.
So, I would need to make 3 partitions on both drives, make 3 RAID-1 sets, use one for root one for swap and the remainder could then finally be used for LVM. At that time, I decided that it was not worth it (for my home machine)
When you have LVM, it should be a total solution, not something you can only use for/var and/home.
(after some time I found out that my root partition is too large and the remainder-of-the-disk partition that I mounted as/local and have put things like/home and/usr/local in is too small. LVM would have helped if it was good, but it would not have helped me because of the restrictions I mentioned)
Conclusion: LVM is a good idea but the implementation in Linux needs more work.
Re:Be sure to use a journalled fs and LVM
by
Tracy+Reed
·
· Score: 1
I'm not sure what happened with your LVM install but I have been able to do many of the things you could not. You use MD to do RAID-1. LVM and MD work together. You can have root on LVM but it is a bit of a pain so I don't usually bother. I am running swap on LVM right now. The Linux LVM implementation is quite nice and mature.
Re:Be sure to use a journalled fs and LVM
by
pe1chl
·
· Score: 1
It looked good at first but the Yast2 installer (SuSE) disallowed both root on LVM and swap on LVM... Hopefully it will be improved in the future.
Re:Be sure to use a journalled fs and LVM
by
bill_mcgonigle
·
· Score: 1
Can you recommend a good, current reference for LVM? I've found some spotty ones and some old ones via Google, but you obviously found the right ones.:)
-- My God, it's Full of Source! OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Re:Be sure to use a journalled fs and LVM
by
Tracy+Reed
·
· Score: 1
Hmm...I've just read the LVM howto, followed the linux-lvm mailing list, and used it for a year or so. No great secrets to reveal.
Re:One 1.2TB drive to the OS, or a bunch of 200GBs
by
R2.0
·
· Score: 1
You do realize that if you realize your quest in your sig, then you won't be able to read your own posts?
-- "As God is my witness, I thought turkeys could fly." A. Carlson
Jeez, if you're going to build a funky storage project that uses a firewire hub, why wouldn't you build it around Hubzilla?
Personally, I'm putting together PCs at home so fast that I've already got over 100Gig just in boot volumes, with a total of about 200Gig of storage across all the PCs in one room. I've also still got two or three free IDE ports, one free SATA port and about eight free firewire ports. Maybe I'll need a funky storage solution somwhere around 2012.
I'm not advocating it be used as-is, but artistically it's a much more interesting start than a simple board. It could have been a Godzilla-themed case with a little fighting scene visible through a window. Or something.
I should look into this to use for storing video files. I need to be able to use huge video files to edit and composite on both a Windows/Linux box and my Mac, and obviously I can't afford a fancy RAID server thingie, so something like this would be perfect - just move a cable around.
First I drilled pilot holes to get the tool bits in. Then I started cutting to remove the big chunks, then I cut closer to the edges with my dremel tool, and finally filed it smooth with my half round bastard (not shown here). Those that know the joke are now snickering.
Well, I tried googling for "half round bastard file joke" and got nothing. Clue me in, someone?
Like goimg yp McDonald's and buying a Billion hamburgers just to watch them change the signs?
the joke (naughty words)
by
Anonymous Coward
·
· Score: 2, Informative
Moderator - I'm not a user and I'm not familiar with your policy on naughty words so feel free to edit this post for content.
A religious woman gets a job at the local hardware store. The manager shows her where everything is so she can help customers. Some time later a guy comes in looking for a file. So she takes him to the file section. While there, she picked up a rat tail file and said "How about this nice round file here?" and the customer replies "No I'll take this flat bastard instead." The woman is shocked and runs to the manager. "That man called that file a Bastard!" THe manager explained that the term bastard refers to the type of cut on the file and that it's a perfectly normal thing to say.
A few days later another guy comes in looking for a file. Again she leads him over to the file section, but this time eager to impress, she picks up another file and says "How about this nice half round bastard?" and the guy replies, "No I'll take this little bitty motherf*cker over here."
ba dum bum
Bobby "Never though I'd have to explain that line to 4 million nerds" Kinstle
Re:6 drives, no redundancy.. Stupid.
by
dusty123
·
· Score: 1
If you have gone that far it would be a lot simpler and a lot handier to make a NAS (Network Attached Storage) out of it.
Gigabit Ethernet is not that expensive any more and ~80MB/s should be fast enough. And Linux can speak nearly any network protocol one can think of.
Anyway, note that you need a lot of processing power for this, forget CPU's < 1Ghz for such a project.
Nope - iSCSI or Fibrechannel
by
haraldm
·
· Score: 1
That would have been cool, and more data center like.
-- open (SIG, "</dev/zero");
$sig = <SIG>;
close SIG;
This kind of array may start becoming more common in the corporate world not as cheap online storage, but as cheap backup. The cost of a single IDE/SATA drive roughly compares to the cost of enterprise backup media such as DLT tapes etc. This kind of array would make for a quick and inexpensive way to backup a large server storage volume, such as a SCSI array.
These small cheap arrays could be bought in volume and simply stored offline in existing off site storage facilities, negating the long term reliability issues of IDE quality disks. Costs for backup may also fall with this solution as no expensive hardware such as tape libraries are required.
Steve
Re:What about a USB 2.0 Drive tower?
by
Curtman
·
· Score: 2, Informative
Get your own installation of the ensembl genome browser and related apps. Why? Just because it's cool to have half a dozen genomes in your computer to play with:)
Promise also makes a (6) wire SATA RAID card that supports RAID0/1/5 as well as hot spares. I have the ATA/100 version of the card with 128Mb cache and I get 6-10 Mbytes/sec of throughput. (Real world usage according to PerfMon with an interval of 120sec.)
Anytime you're doing large arrays like this, you need to have a hot spare and you also need to have some sort of generational backups. (For when the O/S decides to randomly encrypt around 1/4 of the files on your 275Gb file system.)
Promise also makes an (8) drive, external RAID case that holds (8) IDE drives and outputs to a SCSI connector. That lets you get around 1.3-1.5Tb for around $4500 (total cost).
-- Wolde you bothe eate your cake, and have your cake?
"Blueprint"... am I right?
by
makkverk
·
· Score: 1
The guy was not very specific. Am I correct in assuming that he did something like this?
Where HD1...HD6 are the 6 harddrives, FWBB1...FWBB3 are the dual FireWire BridgeBoards, FWHUB is the FireWire hub, and FWOUT is the FireWire cable leading from the hub to the main computer.
That's really cool that it's working for you. However, 6 to 10 MB/s is rather slow. I get 20MB/s with my Adaptec 1200a with two mirrored drives and that's considered poor performance.
The theoretical speed for 33mhz/32bit PCI is 132MB/s so an ATA/133 disk should be getting nearly full speed (depending on read/write method - some file systems are obviously faster than others).
The 3Ware 8 and 12 channel SATA RAID controllers are 66mhz/64bit PCI which yields speeds of 528MB/s. If you were to RAID 5 a set of eight ATA133 disks you should be able to get speeds in the hundreds of megabytes per second.
Looking at the graph on this page you'll see that the lowest reported speed is still twice as fast as yours. And the high end speeds with no redundancy are nearly 80MB/s.
-- No sig for you. YOU GET NO SIG!
Re:Thank God..but it's still IDE drives
by
temojen
·
· Score: 1
Maybe they didn't fit in his case, or wouldn't fit with adequate cooling & power supply.
But FireWire is NEW!!! This is totally different!
by
FatSean
·
· Score: 1
Well it kinda is, since devices other than computers can talk FireWire...but essentially this is a useless exersize. The typical fire-wire user wouldn't be able to plug in the thing if there wasn't an arrow...and someone technical enough to need this would likely use an IDE RAID card.
Oh well...hip-hip-hooray for intellectual masturbation!
6-10Mb/sec is pretty standard for long term average use (such as decrypting a block of files) on the Promise RAID5 card (ATA/100 drives). I've seen it peak as high as 13-15Mb/sec (doing pure reads). All depends on how sequential/random your data access pattern is (and whether you're weighted more towards reads or writes). The amount of memory in the box also makes a big difference.
My video capture box can push 72Mb/sec doing sequential reads, but real work performance is usually more around 15-20Mb/sec. The video capture box uses the PCI Promise ATA/100 RAID1 card connected to a pair of 7200rpm IBM ATA/100 drives. (Comparable to your Adaptec 1200a in RAID1 mode).
Anyway, the Promise card is a lot better the then old craptastic Adaptec IDE RAID card which allowed you to connect (4) 33Mhz IDE drives. Performance on that (even with max cache memory) was only around 500Kb/sec. Since then, I've been very skeptical of the Adaptec IDE RAID cards.
And to put the numbers in perspective, on our dedicated server down at the server farm, I get 40-55Mb/sec raw sequential reads, with real-world performance down around 20Mb/sec. I'm assuming those drives are 10,000rpm and the drives are setup as RAID1.
All of these values were taken from a 5 minute average (sustained throughput) once the benchmark program had settled down and the cache/buffers were filled.
-- Wolde you bothe eate your cake, and have your cake?
Re:One 1.2TB drive to the OS, or a bunch of 200GBs
by
mixmasta
·
· Score: 1
Well, I don't spend a lot of time reading my own posts, thankfully =).
Wow, that's genius. Buy multiple drives and a firewire hub, use multiple firewire to ide converters, and then use the OS on a connected Mac to software RAID the things, and get 35Mb/sec max transfer to a single client.
This guy's a freaking rocket scientist.
Or, wait... do what I did... buy a 4U aluminum case and toss in a recycled motherboard/CPU to match, bolt a system drive to a PCI card blank and mount it inside along with a Gig-E network card, get an actual RAID controller like a 3ware 7500-8, toss in eight drives (120GB in my case, bought in the middle of last year) set up 7 drives on RAID 5 and the eighth as a hot spare, toss Linux and SAMBA (I'm agnostic, use whatever OS you like) on the thing, and voila, 800-something gigs of sustained transfer rates of around 140Mb/sec (60Mb/sec writes). Accessible to every machine in the house via SAMBA or web (tossed on Apache, too) and if I chose to open it up (which no, I don't), the net... without having to host them off my desktop machine.
If you don't think this is a big deal, do you have any idea how much RAM it takes for *any* OS to cache information about a terabyte of disk?
All in all, with all of the fireware converters and hokum, he spent less than I did, by a couple hundred bucks (figuring the hard drive cost and size differences in; I built mine in the summer of 2002, and drive prices were a bit higher). Figure in the existing sunk cost of the mobo and RAM and the system drive and the processor, and yup, I tossed about $400 more into my case. But...
Doctor Bob doesn't actually give specs on his real transfer rates either; he says 35Mb/sec... which is the theoretical max for Firewire.
Yeah, I want to live in Theory too.. everything works there. Realistically, figure in the software RAID over the firewire, I'd be surprised if that thing ever makes it past 20Mb/sec. That's about... one-fifth the speed of an ATA/133 IDE drive.
Welcome to 1994.
Man, this guy gets smarter every second I think about it. Not to knock him for building a big externally housed drive system. Projects are fun, and this sounds like he had fun which, if that was the whole point, is great. If he wanted to sink that much money into something reasonable and sane, well, he failed.
Doctor Bob is *not* a brain surgeon.
And is it just me, or does that thing look like an explosion at an LED factory? I find this type of case mod ugly as hell. Yeah yeah, I know people will disagree with me... it's just my opinion. The firewire logo on the top was a nice touch, but you might as well put a Yugo hood ornament on top of it too, while you're advertising.
Why some people think having eleven randomly-picked colors of blinky lights on a case is uber-k3wl is beyond me. I modded one case once, before it was the "in" thing, and while I took pains to not make it l337-flashy but instead just interesting, I still regretted it once it became as common as it did. It's sitting in a basement now, scavenged of parts.
Ah yes, reminds me of some coworkers at a past job who couldn't figure out why the dual processor uber-dekstop they built was frying itself, even with high-speed fans going.
Until I pointed out that all of the case fans were blowing out (exhaust).
Slashdot Mirror
This guy is wishing he hadn't posted 3 megs worth of pictures.
When I wired up 5 60MB SCSI-25 drives back in the day to get a whooping capacity of 5x60MB...
And still that amount of data is almost half of one of today's most popular RO mediums.. =)
But none the less, nice article and with the disk prices these day's it's getting closer within rage for many of the people that spend that much on electronics... I sure do =P
I've been thinking about something like this but using a standard PC case to house the drives. Still, it wouldn't look as nice as this does.
..actually there are quite cheap external boxes for usual hard drives including a firewire plughole. Cheap, finished - it works out of the box.
Ok, why get it easy if it can be done complicated as well..
What brave karma-worthy soul will have the courage to post a mirror?
So you've got a terrabyte of data, but can it handle Slashdot?
If yer gunna homebrew your drive you should put all the platters in the same casing! -anyone can have a bunch of drives
See the Pictures of the Flood of '08
Wow that went down fast. Sounds like a very cool project. To bad I can read aobut it :)
'Fraid it's only a mozilla grab... clicky. Images are still uploading, but I got them all...
Just dug up an old receipt just recently for a Seagate 20MB hard drive, 1992.
Total: $495.24
he used fire wire and not usb or scsi - or well anything else. this will save us from the hundred or so "Why not firewire?" posts every time somebody discusses some other method of moving data around.
.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
The largest drive available at the time I started this hack was the Maxtor 200GB.
What do you think?
Here's how I did it:
1. Start with the empty case.
2. The original case fans were very noisy. In addition to that, the fan grilles cause lots of turbulence noise. So I cut them all out and replaced them with PanaFlow fluid bearing fans and wire grilles. I had to make custom power cable harnesses for these fans as well
3. As long as I was replacing noisy fans, I replaced the fans in the drive carriers with think PanaFlow FDB fans. I threw their grilles out altogether as they operate with their doors closed and the grille is, well, pointless.
4. Next I downloaded the art work for the firewire logo from Apple's web site. I printed out one that would fit and glued it to the boring beige top case. Black indicated material to be removed. First I drilled pilot holes to get the tool bits in. Then I started cutting to remove the big chunks, then I cut closer to the edges with my dremel tool, and finally filed it smooth with my half round bastard (not shown here). Those that know the joke are now snickering.
5. After this the whole case was sanded and painted with Krylon Fusion Burgundy Red. This paint takes 7 days to fully polymerize so I set it aside and focused on the electronics. I also bought a hunk of clear acrylic from TAP plastics and a 30mm round for the center of the logo.
6. OK I've got a firewire hub that mounts in the same hole as the old Centronics connector did (firewire depot), and 3 dual drive FireWire to IDE controller cards. Plus I need to supply power and route the cables for data and the LED's. I decided to mount them on the empty panels between the back of the drives and the back panel. First I had to measure the card for the stand off. Never leave home without your trusty calipers.
7. Now the cards can be mounted on my 3/4" standoffs and 4/40 screws. This project would be impossible without round IDE cables. The powered hub is visible in the lower left of the 1st picture.
8. This might look like a chaotic mess to you, but it's actually a carefully choreographed symphony of cable. The truth is, it's the only way it would all fit.
9. This is glue. Strong stuff.
10. When the front was dry, I hit it with some 3M Imperial Hand Glaze. That made it nice and shiny. Mmmmmm Shiny. (droooool)
11. Now it's time to get silly. I installed 2 6" and 2 12" tri-color cold cathode lamps. These will really spice up my life. After messing with EL wire, I have decided that it's not bright enough to be worthwhile for almost any use. CCFL lamps however are bright enough to be seen in any lighting conditions including camera flash. EL wire is only visible in low light. CCFL lamps also last longer.
12. Like EL wire, cold cathodes require a high voltage inverter.
13. Finally I mounted some LED's in the front connected to the busy signal outputs on the firewire controller cards. I may at a later date remove this metal grill to improve the lighting and airflow.
All done. Here are some beauty shots:
Please visit my archive of art work photos for this project. Click on any picture for a very high resolution photo. Some of these really move me.
Tech Specs:
Firewire 400 (sustained transfer rate of 35MB/s, max for firewire 400)
Oxford 911 chipsets
6 Maxtor 200 gig ATA 133 hard drives
4 cold cathode lamps with a combined output of 12 watts
16 LED's
Powered firewire hub
hey guys isn't this an article already here. or maybe he should of waited an hour???? http://ask.slashdot.org/article.pl?sid=03/09/23/20 41246&mode=thread&tid=137&tid=188&tid= 198
i thought once I was found, but it was only a dream.
Tell me, Mr. Anderson, what good is a 1.2TB firewire drive if you can't serve any content from it?
I browse Slashdot at +3, Funny
I'm assuming the RIAA hates the idea of anyone having a homebuilt 1.2TB drive hooked to IRC or Kazaa??
Does that warrant a supoena for trying to build such a thing?
You get an old mobo, stick a bunch of drives in it, install linux, perhaps on a CF card, install a firewire card, and let software munge the whole thing into a big firewire drive.
I guess if I could see it I might be impressed case-mod wise, but technically it sounds like any other fileserver, just using firewire instead of network.
Am i missing something?
I don't need no instructions to know how to rock!!!!
if you use firewire controllers to give you one giant 1.2TB firewire drive, doesn't that essentially make 6 hard drives pretend to be one? (AKA the OS doesn't realize it's many) And if just ONE of those drives failed, aren't you shit out of luck with your data?
Again, forgive my hardware ignorance if I'm way off.
You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Wouldn't it make more sense to build a SATA RAID array? Using the 3Ware 8 channel SATA controller and a bunch of big ass Maxtor SATA drives you can get more storage for probably less cost and complication.
No sig for you. YOU GET NO SIG!
... 1.2x10^12 bytes or 1.2x1024^4 bytes?
Should have been http://vader.inowknowwhataslashdottingfeelslike.co m.
Can someone explain to me where this style of response came from? Is it based on some article or email that Knuth wrote? I'm serious.
I browse Slashdot at +3, Funny
It would be cool if the OS saw it as 1 huge 1.2GB drive, but that would be a hell of a hack to make it display as unified piece of media.
I would imagine he has the drives striped in software to appear as one large drive. This is pretty easy to do with Windows, OS X, and pretty much any semi-modern un*x.
Here's a guy who striped 5 floppy drives to make a floppy RAID... he's my hero:
http://ohlssonvox.8k.com/fdd_raid.htm
Firewire is SLOW. You're taking drives capable of bursting 100 or 133 megabytes per second and plugging them into a bus that maxes out at 50, with a practical limit of half that. Also, aren't those little bridges expensive? You might be better off getting a RAID controller and boosting your throughput to 1/2 gigabyte per second or better.
Of course, Firewire is a lot more convenient. But if you want convenience, why not just buy single-drive externals and stack them? I suppose you may have an old case lying around, but I'd personally find a bunch of drives that were easily separable more useful. If I needed to take the data on one with me, I could just unhook it and bring it along.
Ok, so the guy goes to great lengths to build a 6 drive 1.2TB external storage device.
Doesn't menton how the drives become one.. It's not raid-5 as that would be 5X200MB + 1 parity drive. So it's either striped, or the large volume properties were faked.
IMO buying 6 drives and not running RAID 5 is really dumb.
Sure is a purty case though.
No no no - it's not slashdotted. He's just running Norton Disk Doctor. Check back in November.
__ Someday, but not this morning, I'll finally learn to use the preview button.
What is this tear-uh-bite that you speak of? I've never heard of such a thing. Please tell us how big this drive is in units of Libraries of Congress or in terms of how high a stack of floppies it would take.
"People that quote themselves in their signatures bother me" - athakur999
Less than 10 replies and the site is slashdotted.
If only it was this easy to disrupt Verisign's "services"...
# wrote sig.txt, 23 lines, 31337 chars
With this much space, I can backup all my vhs head-cleaner tapes to dv format!
(8) 200GB WD w/ 8MB cache
(1) 3ware 7506
(1) Linux box that already was serving files with nfs + samba on a much smaller drive..
So what? Now I've got 1.4TB (at least I don't lose all/part of my data if the drive fails, unlike this guy). I've only used maybe 10% of that. I've moved all the Videos, mp3s, ISOs, and any other random garbages off all the other pc's in the house to it.
I'd call it a waste, but its too damn cool.
http://www-cs-faculty.stanford.edu/~knuth/news.htm l
http://www.WinWithRealEstate.com/
He's probably thinking that he should have used the Ultra160 SCSI solution after all.
According to ther site, the guy uses three firewire-to-ide interface cards and a firewire hub. Obviously, from the screenshot, he uses Mac OS X, so I assume has has to be using software RAID... I dont see any mention of a RAID controller or some sort of combinatorial device other than the firewire hub, and that cetainly will not combine drives.
/Applications/Utilities
FYI, Mac OS X includes software RAID by default, it's accessible in Disk Utility, located in
huh? It does look like one disk. It's called software raid.
#6495ED - cornflower blue
Uh... it looks like it -does- see it as one single drive, going by the screen shot.
Do you think he might be compensating for something, eh?
</Shrek voice>
to prevent the /. effect.
the firewire terabyte drive has just maxed out
-----------------
Tech Specs:
Firewire 400 (sustained transfer rate of 35MB/s, max for firewire 400)....
---------------
thats not enuff for you being able to escape the wrath of slashdot....
While it isn't mentioned in the writeup, the firewire bridges will not make the drives appear as one. The OS will still see 6 different drives. OS X pretty easily supports software RAID and LVM, so he's almost certainly using one of those methods.
And yes, if any one of those drives dies, he's SOL, although as somebody else mentioned a RAID 5 would help this situation.
What is this tear-uh-bite that you speak of? I've never heard of such a thing.
European or African... I mean, base 2 or base 10 TB?
Kjella
Live today, because you never know what tomorrow brings
I would probably consolidate my whole music collection and video collection.
Combined with on of the many Home Theater PC projects it would make for one slick setup.
Video on demand from your own private collection.
EA David Gardner -"... but the consumers have proven that actually what they want is fun."
And now, just how can I hack my TiVo into using this?
Newsflash: The OS does see it as one great big 1.2TB drive. Look at the screenshots. (Hes' using OSX - not sure if that is why it is seen that way).
Sidenote: With windows dynamic disks, you could see the drive as one unified peice of media. Not sure how hard it would be with linux offhand though.
.
Is where he got the chipset? I have wanted to do something like this but everything I find is either a complete ready to go unit, or a case with the chipset in it. I want just the chipset, no case or drives or anything.
Shesh, some things are sooo hard to find now-aday's.
snowulf.com
i think the project is cool irregardless of just what was used. my statements have to do with /. more than the technology. whenever there is a post on /. about someone using usb or something else- tons of posts pop up saying 'why not firewire?' or something to that effect. drives me nuts- those people get on my nerves- not people acutally using firewire. sorry if that was unclear.
.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
He's now on the list of companies being sued for claiming higher than actual drive capacities.
Step 2 : Buy firewire bridgeboards
Step 3 : Plug it all in
Step 4 : Post on slashdot and crash web server
So he took off the shelf parts and used them in the way they where intended (gasp) what a 1337 hardware hacker.
"Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
If FSCK manages to make any sort of sense out of what's left, it's gonna take a week of babysitting to get it to the point where it'll do much of anything useful with that data.
Free Software: Like love, it grows best when given away.
Well, that was a bit difficult to determine, considering his web page is thoroughly slashdotted.
I'd be happy to build a much simpler SATA raid with hardware RAID5. If you'd be so kind to send me the money for it, I'd love to build it.
I perform hardware reliability testing for a living, so for all those nay sayers griping about the reduction in reliability I say: I'm well aware of exactly what the impact of MTBF is, and since this array is ONLY intended to serve as a backup for my own systems, and me digital media throughout the house, it's worth the risk. I am fully aware of the consequences of a failure of any single part of the system.
Bobby Kinstle
The Mirror
I would hope that this person (and anybody else that is thinking about creating an array) is not going to buy all 6-8 drives at the same time from the same supplier.
More often than not drives built in the same batch tend to fail fairly close to each other, and if more than one fail at the same time you can kiss goodbye to your RAID-5 array (and you were backing up your 1+TB of data, weren't you? after all it takes 'only' about 250 DVDs to do it, doesn't it?)
I think that ideally you'd want to buy your drives over a 6-8 months period from different suppliers for every drive, while it's definitely messier in terms of warranty etc. the additional protection from 3 drives failing at the same time should be worth the hassle...
just my 2c
-- the cake is a lie
...what those high-voltage inverters for the cold cathode flourescents do to the delicate machines they're in.
tasks(723) drafts(105) languages(484) examples(29106)
Comment removed based on user account deletion
In other news the RIAA accused of theft of 300,000,000 (three hundred million) MP3s (4MB each MP3). The lawyers of the RIAA demand a compensation of 150,000 USD per stolen song. This summs up to 45,000,000,000,000,000 USD or 45 trillion USD. That is 9 times the GNP of the USA in 1998. The shares of the RIAA members are going balistic.
Further more these lawyers argue there is no legimate use of 1200GB of disk space for a private person.
Grundgesetz * 23. Mai 1949 - 30. November 2007 - http://www.vorratsdatenspeicherung.de/
bad habit - thanks.
.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
I realize this is somewhat redundant, but how well would this work with RAID5? Since the IDE/firewire converters are three separate physical units, all parity information would have to be processed by the host (PC, or mac in this case).
Firewire should be fast enough to handle the extra data, but I'm not sure the added overhead (calculating parity data, sending it over firewire) would fare well for performance. Perhaps a solution which handles RAID5 on the target end would be better?
He does in fact, have it striped in OS X.
He had a screenshot of the Finder's 'get info' window for the drive. He named it, aptly enough, "BigHonkingDrive".
This person better get their ass in gear. The properties dialog said he had only used 130 megs of the drive and that's just not right.
I wanna see what happens when you log onto Kazaa with a TB of mp3s.
"Gee, that's a lot of pron"
/. right?
This is
Poor coward.. I'll rescue you from the 'beneath my threshold pergatory'
Put them all in a box with a cheap mobo with raid controller, have it run linux, and do some haxoring to expose it as a ginormous HDD to the firewire port.
I think what you suggest is excellent. I imagine there is a IP over fireware module for Linux, so you could use Samba to get at the big drive you suggest.
What would be cooler is if you could create a great big raided partition on a linux box which could be mounted as a USB or firewire drive on another. The huge partition wouldn't me mounted locally at all, just ready to be served to hot-plug clients.
Ian
His was 1Q92, yours was 4Q92.
If all this should have a reason, we would be the last to know.
Fore everyones information, this 'project' is nothing new or special. I will not metion its name, but connected to the better servers of a 'top' P2P application are dozens if not hunreds of people with shares (ie. storage solutions) of comparable and rarely even greater capacity.
On some servers, they won't even let you in if you have less than a hundred gigabytes of shared 'infomation'...
2c
But why is this thing restricted to Windows and Mac? Is it using some weird driver? I would think it would appear as a firewire storage device, and thus be OS-independent. A bit of searching shows that people are using these with Linux, but I don't know if they're really a good idea yet.
I tried this last year, but the devices on the same firewire chain got assigned somewhat randomly by the Linux driver so it was very difficult to tell which device corresponded to which physical drive. If you had trouble with one device it was difficult to tell which drive it was.
I tried this on Linux and got terrible performance at the first try, I got a 23MB/s RAID-0 when each HDD is capable of 26MB/s by itself (everything according to Bonnie++ and hdparm). I didn't know what to blame, the bus, the cables, the Linux SCSI layer, or the whole IEEE1394 support on Linux. Windows was noticeably faster with up to 28MB/s.
Then I made some more research and it turned out the problem was caused by the sbp2 kernel module. This module had some good fine-tuning parameters (sbp2_max_sectors, sbp2_max_outstanding_cmds and spb2_max_cmds_per_lun) up to 2.4.20, but these got ditched in 2.4.21 in the name of a "better way of handling these parameters". I understand the logic behind this move, but the tweakable granularity should have been kept.
Using 2.4.20, I managed to get better performance by tweaking these parameters, then modified sbp2.c on 2.4.22 to reflect the changes. However, I haven't been able to get the 35MB/s this guy got so easily on MacOS X, I'm currently stuck at 29.22MB/s maximum and it's painfully slow to test all combinations of those variable parameters on the sbp2 module.
I just wish there was some document which could explain more about the relationship between these parameters for people not actually involved on the linux1394 project. The comments on sbp2.c are not helpful beyond this point.
By the way, I'm using two Oxford-based bridges to connect two 8MB cache Matrox HDDs, and I'm using Bonnie++ and hdparm for testing. YMMV but the least I can say is Linux RAID support on Firewire still has a long way to go.
- Otaku no naka no otaku, otaking da!!!
It's not so much that it can be done in a complicated way, it's just that figuring out how to build something like that, and then actually doing it, is really most of the fun. Actually using it is somewhat anticlimactic in comparison. (sigh) unfortunately for me my own RAID array is getting rather full, and this article has me thinking. Dammit all to hell.
The higher the technology, the sharper that two-edged sword.
People put those in PC's all the time which are probably much more sensitive to noise than my simple system. Keep in mind that the drives themselves are fully enclosed in grounded metal. So the only exposure risk is to the FW controllers. They don't seem to care.
To verify integrity I wrote a 1TB file of pseudorandom data using PRBS23 and then read it back and verified the data integrity with no errors. As a side note, it takes about a day to do this over firewire, although some delays were because my powerbook's cpu is too slow to decode the data at full bus speed.
Bobby Kinstle
Or a RAID 0+1 arrangement with three drives per volume. In this case, we would still be talking two ~500 gb volumes, which is quite impressive of its own volition. Honestly, how many of you could even get to 75% on a 1 TB volume anyway?
Or, if you just want to act silly you could build two and mirror them...
Who did what now?
At this place you can buy what looks like a USB2 to 4 IDE controller adapter. What you don't see at the manufacturer's site is that this board fits into a standard ATX power case, takes standard ATX power supply, so you can easily make a cool 4 200 or 300 or whatever HDD external case. As you can see Here, that board being mounted into a case, where you can hook up the hdds, using the case's power supply, etc. Too bad its USB2, and too bad each IDE controller only supports one device, otherwise this would make for a killer 8-device controller...
Hey, that can't be that bad...You'll still get out in polynomial-time....666^n would be much worse...
If religous zealots don't believe in Evolution, then why are they so worried about bird flu?
Others have mentioned the necessity of RAID 5 in a setup like this but let me point out that you don't want to fsck 1T of disk. I have had to watch the fsck of 500G of disk back before we had journalled fs and it was terrible. When we started attaching many terabytes of disk to Linux boxes we needed a better solution. So you will want to use a journalled fs. Reiserfs is my favorite. Then you will not want to have to backup/restore when you decide your current partitioning layout was a bad idea or just generally want to shuffle things around so be sure to use LVM also. I use LVM on all of my machines, even desktops, and it has really made life easier. Often you will need more room on /home but notice that /var has a couple gig unused and with LVM you just shrink /var and expand /home all without reboot and you are good to go.
You do realize that if you realize your quest in your sig, then you won't be able to read your own posts?
"As God is my witness, I thought turkeys could fly." A. Carlson
Personally, I'm putting together PCs at home so fast that I've already got over 100Gig just in boot volumes, with a total of about 200Gig of storage across all the PCs in one room. I've also still got two or three free IDE ports, one free SATA port and about eight free firewire ports. Maybe I'll need a funky storage solution somwhere around 2012.
I should look into this to use for storing video files. I need to be able to use huge video files to edit and composite on both a Windows/Linux box and my Mac, and obviously I can't afford a fancy RAID server thingie, so something like this would be perfect - just move a cable around.
And yeah, I think Firewire would be fast enough.
I belong to the ______ generation.
Well, I tried googling for "half round bastard file joke" and got nothing. Clue me in, someone?
Like goimg yp McDonald's and buying a Billion hamburgers just to watch them change the signs?
Moderator - I'm not a user and I'm not familiar with your policy on naughty words so feel free to edit this post for content.
A religious woman gets a job at the local hardware store. The manager shows her where everything is so she can help customers. Some time later a guy comes in looking for a file. So she takes him to the file section. While there, she picked up a rat tail file and said "How about this nice round file here?" and the customer replies "No I'll take this flat bastard instead." The woman is shocked and runs to the manager. "That man called that file a Bastard!" THe manager explained that the term bastard refers to the type of cut on the file and that it's a perfectly normal thing to say.
A few days later another guy comes in looking for a file. Again she leads him over to the file section, but this time eager to impress, she picks up another file and says "How about this nice half round bastard?" and the guy replies, "No I'll take this little bitty motherf*cker over here."
ba dum bum
Bobby "Never though I'd have to explain that line to 4 million nerds" Kinstle
Gigabit Ethernet is not that expensive any more and ~80MB/s should be fast enough. And Linux can speak nearly any network protocol one can think of.
Anyway, note that you need a lot of processing power for this, forget CPU's < 1Ghz for such a project.
That would have been cool, and more data center like.
open (SIG, "</dev/zero"); $sig = <SIG>; close SIG;
This kind of array may start becoming more common in the corporate world not as cheap online storage, but as cheap backup. The cost of a single IDE/SATA drive roughly compares to the cost of enterprise backup media such as DLT tapes etc. This kind of array would make for a quick and inexpensive way to backup a large server storage volume, such as a SCSI array.
These small cheap arrays could be bought in volume and simply stored offline in existing off site storage facilities, negating the long term reliability issues of IDE quality disks. Costs for backup may also fall with this solution as no expensive hardware such as tape libraries are required.
Steve
Yep
No, really I have:i d=01187
http://www.coolcasegallery.net/ccg/viewcase.php?c
That's a whole lotta pr0n..
Get your own installation of the ensembl genome browser and related apps. Why? Just because it's cool to have half a dozen genomes in your computer to play with :)
---- Take the Space Quiz!
Software striping isn't the same as HW striping. The OS sees it natively as JBOD, and presents it as a big disk.
Promise also makes a (6) wire SATA RAID card that supports RAID0/1/5 as well as hot spares. I have the ATA/100 version of the card with 128Mb cache and I get 6-10 Mbytes/sec of throughput. (Real world usage according to PerfMon with an interval of 120sec.)
Anytime you're doing large arrays like this, you need to have a hot spare and you also need to have some sort of generational backups. (For when the O/S decides to randomly encrypt around 1/4 of the files on your 275Gb file system.)
Promise also makes an (8) drive, external RAID case that holds (8) IDE drives and outputs to a SCSI connector. That lets you get around 1.3-1.5Tb for around $4500 (total cost).
Wolde you bothe eate your cake, and have your cake?
Where HD1...HD6 are the 6 harddrives, FWBB1...FWBB3 are the dual FireWire BridgeBoards, FWHUB is the FireWire hub, and FWOUT is the FireWire cable leading from the hub to the main computer.
That's really cool that it's working for you. However, 6 to 10 MB/s is rather slow. I get 20MB/s with my Adaptec 1200a with two mirrored drives and that's considered poor performance.
The theoretical speed for 33mhz/32bit PCI is 132MB/s so an ATA/133 disk should be getting nearly full speed (depending on read/write method - some file systems are obviously faster than others).
The 3Ware 8 and 12 channel SATA RAID controllers are 66mhz/64bit PCI which yields speeds of 528MB/s. If you were to RAID 5 a set of eight ATA133 disks you should be able to get speeds in the hundreds of megabytes per second.
Looking at the graph on this page you'll see that the lowest reported speed is still twice as fast as yours. And the high end speeds with no redundancy are nearly 80MB/s.
No sig for you. YOU GET NO SIG!
Maybe they didn't fit in his case, or wouldn't fit with adequate cooling & power supply.
Well it kinda is, since devices other than computers can talk FireWire...but essentially this is a useless exersize. The typical fire-wire user wouldn't be able to plug in the thing if there wasn't an arrow...and someone technical enough to need this would likely use an IDE RAID card.
Oh well...hip-hip-hooray for intellectual masturbation!
Blar.
6-10Mb/sec is pretty standard for long term average use (such as decrypting a block of files) on the Promise RAID5 card (ATA/100 drives). I've seen it peak as high as 13-15Mb/sec (doing pure reads). All depends on how sequential/random your data access pattern is (and whether you're weighted more towards reads or writes). The amount of memory in the box also makes a big difference.
My video capture box can push 72Mb/sec doing sequential reads, but real work performance is usually more around 15-20Mb/sec. The video capture box uses the PCI Promise ATA/100 RAID1 card connected to a pair of 7200rpm IBM ATA/100 drives. (Comparable to your Adaptec 1200a in RAID1 mode).
Anyway, the Promise card is a lot better the then old craptastic Adaptec IDE RAID card which allowed you to connect (4) 33Mhz IDE drives. Performance on that (even with max cache memory) was only around 500Kb/sec. Since then, I've been very skeptical of the Adaptec IDE RAID cards.
And to put the numbers in perspective, on our dedicated server down at the server farm, I get 40-55Mb/sec raw sequential reads, with real-world performance down around 20Mb/sec. I'm assuming those drives are 10,000rpm and the drives are setup as RAID1.
All of these values were taken from a 5 minute average (sustained throughput) once the benchmark program had settled down and the cache/buffers were filled.
Wolde you bothe eate your cake, and have your cake?
Well, I don't spend a lot of time reading my own posts, thankfully =).
#6495ED - cornflower blue
Wow, that's genius. Buy multiple drives and a firewire hub, use multiple firewire to ide converters, and then use the OS on a connected Mac to software RAID the things, and get 35Mb/sec max transfer to a single client.
This guy's a freaking rocket scientist.
Or, wait... do what I did... buy a 4U aluminum case and toss in a recycled motherboard/CPU to match, bolt a system drive to a PCI card blank and mount it inside along with a Gig-E network card, get an actual RAID controller like a 3ware 7500-8, toss in eight drives (120GB in my case, bought in the middle of last year) set up 7 drives on RAID 5 and the eighth as a hot spare, toss Linux and SAMBA (I'm agnostic, use whatever OS you like) on the thing, and voila, 800-something gigs of sustained transfer rates of around 140Mb/sec (60Mb/sec writes). Accessible to every machine in the house via SAMBA or web (tossed on Apache, too) and if I chose to open it up (which no, I don't), the net... without having to host them off my desktop machine.
If you don't think this is a big deal, do you have any idea how much RAM it takes for *any* OS to cache information about a terabyte of disk?
All in all, with all of the fireware converters and hokum, he spent less than I did, by a couple hundred bucks (figuring the hard drive cost and size differences in; I built mine in the summer of 2002, and drive prices were a bit higher). Figure in the existing sunk cost of the mobo and RAM and the system drive and the processor, and yup, I tossed about $400 more into my case. But...
Doctor Bob doesn't actually give specs on his real transfer rates either; he says 35Mb/sec... which is the theoretical max for Firewire.
Yeah, I want to live in Theory too.. everything works there. Realistically, figure in the software RAID over the firewire, I'd be surprised if that thing ever makes it past 20Mb/sec. That's about... one-fifth the speed of an ATA/133 IDE drive.
Welcome to 1994.
Man, this guy gets smarter every second I think about it. Not to knock him for building a big externally housed drive system. Projects are fun, and this sounds like he had fun which, if that was the whole point, is great. If he wanted to sink that much money into something reasonable and sane, well, he failed.
Doctor Bob is *not* a brain surgeon.
And is it just me, or does that thing look like an explosion at an LED factory? I find this type of case mod ugly as hell. Yeah yeah, I know people will disagree with me... it's just my opinion. The firewire logo on the top was a nice touch, but you might as well put a Yugo hood ornament on top of it too, while you're advertising.
Why some people think having eleven randomly-picked colors of blinky lights on a case is uber-k3wl is beyond me. I modded one case once, before it was the "in" thing, and while I took pains to not make it l337-flashy but instead just interesting, I still regretted it once it became as common as it did. It's sitting in a basement now, scavenged of parts.
Rock is dead. Long live scissors and paper!
Ah yes, reminds me of some coworkers at a past job who couldn't figure out why the dual processor uber-dekstop they built was frying itself, even with high-speed fans going.
Until I pointed out that all of the case fans were blowing out (exhaust).
I'm surprised the damn case didn't pucker in.
Rock is dead. Long live scissors and paper!