Slashdot Mirror
Building a Massive Single Volume Storage Solution?
An anonymous reader asks: "I've been asked to build a massive storage solution to scale from an initial threshold of 25TB to 1PB, primarily on commodity hardware and software. Based on my past experience and research, the commercial offerings for such a solution becomes cost prohibitive, and the budget for the solution is fairly small. Some the technologies that I've been scoping out are iSCSI, AoE and plain clustered/grid computers with JBOD (just a bunch of disks). Personally I'm more inclined on a grid cluster with 1GB interface where each node will have about 1-2TB of disk space and each node is based on a 'low' power consumption architecture. Next issue to tackle is finding a file system that could span across all the nodes and yet appear as a single volume to the application servers. At this point data redundancy is not a priority, however it will have to be addressed. My research has not yielded any viable open source alternative (unless Google releases GoogleFS) and I've researched into Lustre, xFS and PVFS. There some interesting commercial products such as the File Director from NeoPath Networks and a few others; however the cost is astronomical.
I would like to know if any Slashdot readers have any experience in build out such a solution? Any help/idea(s) would be greatly appreciated!"
Considering that there are billion dollar companies whose only job it is to provide secure and redunant storage of the type that you describe, what makes you believe that someone on slash-dot would give you a solution for free?
The kind of thing you are talking about is non-trivial. If people have ideas concerning these matters you should pay them for them.
What a lot of gall!
Also, if you are being paid to do this by someone, then they obviously hired the wrong person to do the work.
Well, in the event of scarling between a few TB and >= 1 PB, you'd be talking about a rather large price range.
Combine that with the fact that the prices are probably changing all the time with market factors, and the likelihood of someone selling a petabyte storage system with anything meaningfully called a retail price is absurd -- these are big, custom pieces not something you have the clerk run to the warehouse and see if there are any in stock.
This isn't like an Nvidia card where the manufacturer says "reccomended retail" is x and you go to the price comparison sites.
Lost at C:>. Found at C.
A certain very large brick and mortar bookstore had a multi-day mainframe outage as a result of EMC bungling. The outage shut down loading docks across the country, etc.
They were running EMC storage on their IBM in a raid 1 config. Yes, EMC prefers that you to run the system drives on their storage. They tried to get me to move my system drives onto their storage years ago (crashola when you can't swap due to EMC performance issues/cache full).
After upgrading one of the redundant nodes ("source load", whatever that is), it wouldn't come up. No problem, they thought.. They shutdown and replicated the contents of the other node's system volume. Then, neither would come up.
It was very, very bad. Don't always believe EMC hype. They were clueless on this and the damage was in the many millions (just due to the docks being shutdown).
I could tell you stories about performance on EMC vs. their hype.. But you'd die of boredom. Instead, get your boss to kick down the cash for the 'real thing' and go eat free food with them, see major sporting events and concerts. They have suites in all major baseball, football, hockey and basketball arenas. Your boss sounds cheap, party with EMC.
Have you ever seen a single drive failure cause a 30TB SAN to grind to a halt? I have, thanks to EMC.
Needless to say, we are keeping our future storage options open.
Give IBM a call, the DS4000 series sounds like just the thing you are looking for. IBM doesn't screw you over in software license renewals either (unlike some other companies).
But hey, EMC did buy us lunch after our 4 hour outage.