Slashdot Mirror
Do-It-Yourself Fibre Channel Array
skarphedin writes "There's an interesting story here on a do-it-yourself fibre channel array. These guys make one for under $250 and it can perform up there with 15k SCSI in some cases." You know you want one.
1) Buy a lot of fibre
2) Fly to Folkestone, England
3) Rent a sailboat
4) Sail to Calais, France, laying an array of fibre behind you
5) Congratulation on your Do-It-Yourself Fibre Channel Array.
It seems the links to the store selling the goodies is already slashdotted. The $40 a piece FC hostbus adapter page now shows $800 adapters, or a 100 pack for just over $60000. Beowulf anyone?
But seriously, I bet if I wired this in my dorm room I could get some mean negative pings in UT '03. Kinda like a 'spider sense' for the pc.
suck my ping!
I have used FC tech for several years. It's amazing technology, but performance is not it's main advantage. It's advantage is the possibility of stacking up incredible amounts of storage, with rendundant paths, at up to 100 m from the attachment point (one of the servers). This kind of environment is also very mindful of quality, and a self-made solution is not acceptable. Would you stack dozens of these self-made boxes and bet your career that they'll not fail. I know I wouldn't.
On the other hand, if I just want performance, I will do better with SCSI, and even save some money.
In this respect, I don't quite see what kind of niche would the solution in the article cover.
Sigged!
Fibre chanels I thought were used because .
1) Huge expansivity
2) Faster speeds, esp. over LAN (Storage area networks)
Why would one want to use it in a home setup?
You probably are not going to buy more than 3 or 4 Harddisks. I say if you want speed use more RAM(*though you wont get much for $250 * results might vary). If you want expansivity(not too much) and relatively fast (depends on a lot of stuff) access speeds and standards based setup, may I suggest iSCSI
.ACMD setaloiv siht gnidaeR
How long do you think till the mac-heads credit apple with bring down the price of FC by including it in the X-raid? Just like they credited apple with bring down the price on SCSI, USB, etc.
I know I'm going to hell, I'm just trying to get good seats.
We had great plans for building an FC array up until a while ago. For those who think FC is too expensive, take a look at this:
180GB ATA drive: $200
Qlogic FC host adapter: $40
10 18GB 10k drives (eBay): $99
10 T-cards: $50
UTP-cable: $20
--------------
Total: $209
Of course, there's the cost of running the array as well, which is the reason we never finished our project (We did get the hostadapter and built a couple of T-cards though). We calculated that our FC array would cost us an additional $2-300 in electricity every year. After getting hit with a $500 surprise electricity bill for our current equipment, we simply decided it wasn't worth it and got another IDE drive instead. Still, an interresting project. =)
Yeah this is cool and such like but what if you want to mount it across two machine using two FCA's? You need software that allows file locking (such as SGI's CXFS) and that costs. Mind you if you only wanted it on one machine why not just buy a load of disks because in honesty when are you going to need such high amounts of bandwidth?
Rus
Cheap UK and US VPS
The drive mounting and enclosure was a bit of a kludge. Are there any reasonably priced boxes that you can install the drives in, with the correct mounting hardware and backplane?
Mea navis aericumbens anguillis abundat
Is anyone as annoyed as I am about aticles spreat across 4376245 pages for no fucking reason? Page 2 for example (which is only 2 paragraphs of actual text) uses less than 50% of the page space! You have to scroll WAY past the end of the artlcle to even see all the ad's on the left column.
I guess the fact that there is no normal "print version" link like MOST sites have is the most annoying. There is however a link to a PDF version on the Very Last Page which helps, but html is much prefered.
And what's with the wood blocks that looked like they were cut with a chain saw or hacked apart with an exacto blade? Ever hear of sheet metal? Hell, at LEAST pick up a used DRIVE case instead of a TAPE case. Even NEW they are pretty cheap.
The CONCEPT of the project is interesting, but the implementation leaves MUCH to be desired.
...porn storage at the speed of light department
6) Profit!
Why should this be surprising? FC drives are in every single case SCSI drives with a different, more expensive, interface. Although they tend to be cheaper on the surplus market, which I think is the *real* point.
Then he goes and mounts them in the case with wood!! Why? Its an insulator!!! Ok, maybe he didn't have the necessary metal skills (or equipment even) to make a custom bracket, but using wood to mount a drive just seems a bit dangerous to me.
- 18.4GB Fibre Channel 3.5LP 10K RPM 25.4MM
(Hitachi) - (DK32CJ18FC)
$119.00
- QLA2100/66 64bit PCI FC Host Adpt COPP w/Cab
(Qlogic) - (QLA210066)
$858.20
So where did he get those goodies? Ohh! he was just good buddies with a ex- Enterprise storage Admin?Robert
Optical fiber is not required for shorter distances, however, because Fibre Channel also works using coaxial cable and ordinary telephone twisted pair. Fibre Channel offers point-to-point, switched, and loop interfaces.
Too bad whoever wrote that was completely full of it. Fibre Channel does not exist over coax or twisted pair.
Coax really isn't used much for data cabling any more.
Twisted Pair? Who knows. I expect the error rates are too high for the extremely tight specs required by Fibre Channel.
SirWired
From an english langauge pov:
http://dictionary.reference.com/search?q=fibre
From a google pov:
"fibre channel array" hit count = 2600
"fiber channel array" hit count = 213
Why put all of 2 disks into an ugly assed old compaq case with those shady adapters when you can go buy a used 11,14 or 22 disk fibre-channel array with redundant power and dual loops made by a certain manufacturer (hey I ain't givin up all my secrets) for well less than $500 empty and around $650 with ~180GB 10K disks in it?! And yes of course they do make FC cards with internal adapters on them too. Here's a hint: SENA. As for FC not having performance, all I can say is 'HUH?!' I'll take a single 1gbps loop over scsi320 parallel or whatever they're calling it any day. Beyond my own benchmarking FC devices, if SCSI were better/faster don't you think people like EMC, HDS and Compaq (believe it or not Compaq makes some pretty kickin arrays) would still use SCSI back ends or even front ends on their storage products? For years EMC has been slammed about using SCSI back ends in their arrays and finally have FC throughout the machines. FC is saweeet and it runs SCSI above the FC layer as well. It can also run other protocols like IP but I've yet to see that implemented well.
Where I used to work, we had a few Sun servers with FC disk arrays.
Here's the Sun engineer's explanation of why FC is so interesting for servers:
1) The FC protocol has a 100MByte/s dedicated bandwidth to data. The communication between disks etc. will not interfer with this bandwidth.
2) Modern SCSI has two modes: one for data (burst mode) and one communication mode. The communication mode is a lot slower (first scsi standard) in order to remain compatible with older disks. This means that scsi is a lot more advantageous to users reading large files than small files.
This is where FC becomes interesting: If you have a striped disk array, you will read many small segments from different disks instead of large segments from single disks. In this special case, FC is faster than SCSI, even though it is "slower" by looking at the burst rates in the specs.
Has anyone looked at the prices from the story? I could not find a $25 FC drive anywhere. Cheapest was $120. So, yah, $240 for 2 drives, that kinda blows the under $250 out of the water.
Anyway, being a 'mac head' and a 'linux head' and a 'computer head' in general, I don't necessairly get the orig posters issue. Apple does a lot of cool things. And some things just happen to hit the Mac market before the PC market. Big deal. Why not bitch about Billy G. then?
My work has been setting up a FCA for the past three weeks using Linux and there have been some major problems. They have a fat array with 32 15k rpm U320 drives hooked up to a IBM x440 via 4 HBAs. The interesting thing is that no distro they've tried can transfer faster than Windoze due to Linux kernel and driver issues. I was a little shoked. The x440 has 8 Xeons w/hyperthreading. The more cpus that are enabled, the more the performance degrades. The sysadmin says he thinks it has something to do with single-threaded io calls in all Linux kernels - the more cpus try to access io, the more threads that get blocked. Me and the other sysadmin - Gentoo 'heads' - start scratching our heads wondering what all the Linux 'Enterprise' stuff is that everyone is talking about. And yes, these components were all given the 'good to go' stamp by all their manufacturers. Since the prob is with the kernel itself, this is kinda major.
So, these things are cool, but there are definitely reasons for and against having them. I doubt you could build anything useful for under $400-$500 and they're really only going to shine in a server or workstation environment whith sustained io. It's nice to have these things as proof of concept at home (hell, I have all kinds of weird servers and drives and such), but when you start talking about a production environment in the enterprise, it takes on a whole new ball of wax.
I still think the XServe is interesting, however, I'm not too thrilled about ATA drives. Guess that was for cost tho. Go mac heads!
Anyone seen ide to fibre channel convertors ? Before you flame - They DO exist this product has them in it see here :-
http://www.axus.com.tw/br1200fc.htm
Anyone seen single drive versions of this ?
I made a Fibre Channel array like this last year. The only difference being that I used a Mylex eXtremeRAID 3000 (eBay for $200), a 256MB Crucial DIMM for cache, and four Seagate ST39102FC 9GB 10,000 RPM disks.
My whole point to the project was EXTREMELY fast disk access (up to ~160MB/sec sustained transfers, see here) that I could locate at the far end of a REALLY long cable. I've got my machine in my office and the hard drives on the other end of a 30m cable, nestled nicely down in the basement where I cannot hear it.
There are a few basic pictures of the external assembly available here. Works really, really well. It's amazing what hugely fast disk IO does for the rest of a machine.
Having built my own fiber channel backplanes based of a previously slashdot mentioned cinonic backplane, I agree, there are difficulties in setting up a fibre channel array. I got a db9-db9 cable and a hssdc-hssdc cable orgiinally, and ended up trying to solder them together. I'd never tried soldering shields together, and presumed it would take a while, but I kinda found out its pretty much impossible. So I just pulled on over the other and wire wrapped the hell out of it. I suspect its because of the cabling, but my fibre setup is fairly intollerant to electrical fields. I have to place the drives and cabling as far away from everything; cat 5, computer systems, et al. Kinda not pretty. Before proper positioning, you'd get occasional sometimes fatal SCSI errors. Kernel panics were driving me nuts. I realized the drive was between a switch and a computer, so I spent a while and figured out the path of least interference. Now I can run bonnie++ on my 5 disc soft-raid 5 JBOD for a week straight. And fry eggs as the same time! And no more kernel panics. Thats always a plus. Its a pain, but in the proper linux motto, once you can get it running, you can bet its not gonna stop.