Slashdot Mirror
Building a Massive Single Volume Storage Solution?
An anonymous reader asks: "I've been asked to build a massive storage solution to scale from an initial threshold of 25TB to 1PB, primarily on commodity hardware and software. Based on my past experience and research, the commercial offerings for such a solution becomes cost prohibitive, and the budget for the solution is fairly small. Some the technologies that I've been scoping out are iSCSI, AoE and plain clustered/grid computers with JBOD (just a bunch of disks). Personally I'm more inclined on a grid cluster with 1GB interface where each node will have about 1-2TB of disk space and each node is based on a 'low' power consumption architecture. Next issue to tackle is finding a file system that could span across all the nodes and yet appear as a single volume to the application servers. At this point data redundancy is not a priority, however it will have to be addressed. My research has not yielded any viable open source alternative (unless Google releases GoogleFS) and I've researched into Lustre, xFS and PVFS. There some interesting commercial products such as the File Director from NeoPath Networks and a few others; however the cost is astronomical.
I would like to know if any Slashdot readers have any experience in build out such a solution? Any help/idea(s) would be greatly appreciated!"
register a few thousand gmail accounts and write the interface that will make writing of data to gmail inboxes invisible to the app.
I know a certain recent Zombie network that was discovered which collectively had quite a few Pbs of storage... Of course I wouldn't recommend going down that road as it leads to you know ... jail.
News Reporters Make Tasty Polar Bear Treats!
I never thought I'd see the day when sites were boasting a petabyte of porn. .avis -- if you sat down and watched them end-to-end, you'd have 348 years of "backdoor sliders", "dribblers to short", "pop flies", and "long balls". We live in an enlightened age.
That's over 3 million hours of
With a project this large, they may be able to do it in-house and still take advantage of economies of scale. They can buy HDDs, motherboards, rackmount cases, etc. by the pallet or container load and temporarily up-hire some of their part-timers to do the assembly.
With a network bootable bios, the nodes could just be plugged in and install an image off a server, then customize it based on their MAC.
Just don't call it PetaFile.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
"I can't imagine having that much data I didn't care about."
Hollywood script archive.
The global economy is a great thing until you feel it locally.
Dear Slashdot,
I have been tasked with (insert very difficult, very important job). This is very important to my company. I have (insert number much lower than it should be) dollars to do this. I do not want to use (insert company name specializing in this exact thing) because management thinks they are too expensive. I think I can do this (insert better/faster/cheaper/...) than said company, even though they have vastly more experience and have invested much more time and research than I have. My continued and future employment probably rests on this project. Please advise.
Maybe not
And you'll probably get at least one nice lunch out of the sales deal. I recommend saving your lunch money and asking for sales visits from all of the major players.
Why not store the data randomly in a dilithium matrix with asynchronous data transfer and AJAX? Maybe some RUBY on RAILS too - I hear that's hot right now. Of course, you'd have to make use of a couple of Heisenberg compensators configured in parallel to keep track account for any memory addressing issues, but no need to state the obvious there.
"Nokia is not a country, it's the capital of Finland!" -Moderated "Informative". Yeesh.
Maybe he needs it for a swap file? I heard Microsoft upped the memory requirements in the next version of windows.
what on Earth kind of data do they anticipate will take a petabyte of contiguous storage?
I know. They don't know I know, but I do. It's data gathered by the black helicopters, by Echelon, by Carnivore, by our very own printers, by RFID, about every movement of every single one of us... *They* do it. They.
Stupidity is an equal opportunity striker.
Fellow slashdotter Bill Dog