Slashdot Mirror
IBM Introduces Petabyte-Capacity 'Storage Tank'
statikuz writes "Wired is reporting that IBM's new data storage system, codenamed "Storage Tank", uses software to link servers in multiple locations over an IP network, creating a sort of mega-server capable of connecting thousands of computers and processing multiple petabytes of data. 'Storage Tank has the potential to become to an organization's data what the Dewey Decimal system is to a library,' said Dan Colby, general manager of storage systems at IBM. 'It reinvents the way information is filed, managed, shared and accessed within an organization.' CERN is currently using a beta version of the system to store data from the Large Hadron Collider particle accelerator, which is being used to recreate the first moments of the Big Bang. IBM expects Storage Tank eventually will be able to handle 10 to 20 terabytes of CERN data. Get your own 'starter configuration' for only $90,000!"
For my house, of course.
Quote:
"Storage Tank has the potential to become to an organization's data what the Dewey Decimal system is to a library"
Strange that he compares it to a system that few libraries use anymore. Yes, it revolutionized cataloguing. Right before it became obsolete (because it cost too much).
Not too long ago Slashdot reported on the owners of the Dewey Decimal system suing a hotel in New York for using it as the theme for their room numbering. How long until IBM starts suing everyone with a storage tank?
When we finally manage to federate the sum of human knowledge, we'll have some where to put it.
Intentional Mispelling
Dude, what the fuck does this have to do with DRM?
Quote:
"IBM expects Storage Tank eventually will be able to handle 10 to 20 terabytes of CERN data. By 2007, when the proton smashing is scheduled to commence in earnest, CERN will be generating data at a minimum rate of 5 to 8 petabytes a year."
Wow! This monster storage tank will be able to handle 20 terabytes of data! In four years?! That's just amazing!! A whole 1/1000th of the required yearly storage!
Imagine a Beowulf cluster of these!
*snicker*
hadron colliders, that is.
Wait 'til the NeoCons get a hold of this term.
This space for rent.
Stop talking about petabytes, go to PETA and oyu might get laid by a goat. Or buy rohypnol, I use that to get laid.
Whats the difference between this and a SAN (storage area network) which has been around for years?
'Storage Tank has the potential to become to an organization's data what the Dewey Decimal system is to a library,'
I'd be careful about making that comparison, unless you want a lawsuit from the Online Computer Library Center.
funny how pr0n still seems to drive technology..
xao
xao
http://TheHillforum.hopto.org
Had a hard drive crash the other day without backups.
Are there any easy solutions that can write data out to two HDs redundantly, perhaps to two SCSI or USB external drives?
Just like everyone's gunna say:
WOW, that can hold A LOT OF PORN!
bossesjoe is a known troll who's been plaguing /. for months with useless gibberish. Let's get rid of him once and for all.
That's 90,000 dollars and comes preinstalled with one 36Gb drive. Additional drives can be purchased at the low-low price of 4,000 dollars apiece.
Assuming you can put 6 HDDs in 1U enclosure and a
standard 44U rack and those 300 Gig monster HDDs,
that's still 12 racks worth of HDDs. Holy...
Now I'll sit back and wait for the obligatory "... bah! Tank-shmank! Gimme a few of these Maxtor monsters, and I'll roll my own "storage tank" using a spare full-tower chassis, a PIC controller and some duct tape..."
This has been done. I think they call it KaZaa though.
-----
considering it isn't too difficult or expensive for an average Joe to assemble a terabyte's-worth of storage from off-the-shelf parts; a petabyte isn't really that much.
"Get your own 'starter configuration' for only $90,000!"
:P
Can I get $10,000 if I include some Cracker Jack box tops? If not, I'm ordering a petabyte-capacity storage tank for sea monkeys.
My blog
Anonymous Coward is a known troll who's been plaguing /. for months with useless gibberish. Let's get rid of him once and for all.
Storage Tank comes extremely late - it was first promised to come out in early 2001.
According to this article at The Register, IBM failed to provide such features of Storage Tank as, "link servers and storage systems from all vendors, making it possible to view and access a file from any system. ". Instead, it will only support AIX and Windows platforms starting this November. Support for other Unix versions, including Linux, is expected not earlier than mid-2004.
...who read that one link as 'Large Hardon Collider' ...yeesh, I think I need to get out more.
Imagine a Beowulf cluster of storage tanks!
SAILING MISHAP
<thud>
"The best argument against democracy is a five minute chat with the average voter."
--Winston Churchill
I always thought a good idea was multiple RAID storage across the entire network. So all the files are spread throughout the network. With multiple copies so if two or three computers go down, that data is not lost...kind of a cross between SAN and RAID.
So what's a file called in a petabyte-capacity storage tank?
A petafile!
Ha, I crack myself up.
open source solution that already stores 100s of terabytes that is called LUSTRE... LUSTRE is already deployed in a few live aplications run by the NCSE (hope I remembered that right)....
At the symposium this year, the fellow mentionned they were working on scaling to petabyte storage for next year.
Sincerely,
Mentally Challenged Parents Association
(What's a Petafile, Walter?)
--
Power to the Peaceful
Look at the score breakdowns on some of these stories. There are almost no Score:3 posts -- meaning that no moderation is occuring. Did CmdrTaco break something?
Storage Tank has the potential to become to an organization's data what the Dewey Decimal system is to a library.
Shh!!! Don't mention Dewey Decimal or you might get sued!!
So would anyone take a shot at actually specifying the hardware and cost for a 1 Petabyte system? Include HDs, systems, # of racks, (don't forget the switches for the network). Assume no RAID.
it's a hardon collision
Hope they have lots of backup. Of course, how do you backup a system like this?
"Open the pod by doors, Hal" > "I'm afraid I can't do that, Dave" sudo "Open the pod bay doors, Hal" > alright
I've been considering an idea like this for years. I mean, what's the problem in splitting a file system up into lots of smaller chunks and storing them on many different computers? My idea was to introduce redundancy so that even if not all of the nodes are active or reachable at any given time, the information could be located or constructed from other information. By doing so, a distributed storage system could be placed on millions of computers worldwide, in a sort of SETI@home-like setup, and users could donate a tiny chunk of their hard drive to help scientific research or student projects or whatever, where the people using the storage can't afford to pay for it. What's 50 megs, or 100, or 200, in today's hard drives anyway? People could easily "donate" the unused space on their drives and never feel a difference.
# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 999T 48G 998T 0%
# wget --theinternet
Muahahahaha!
The Register claims that contents may have settled during shipping.
... I have nothing profound to say but hey, as my nick is a petabyte I figured I should chime in.
..
Then again, I'm only a petabyte here, usually I'm in a larger configuration.
Ah, good times good times
Isn't this just one way to implement a P2P network? By selling it for enterprise use, IBM is supporting the argument that P2P networks have legitimate use and should not be outlawed as the RIAA has attempted.
I have not used either, but Storage Tank seems to deliver similar functionality as Waste, though on a larger scale and with a different UI paradigm. Perhaps if Nullsoft had released Waste as a way for small and medium sized businesses to share files, AOL would have acted differently.
That's a lot of pron.
Hell, who needs pron anyway? My disk is getting hard just thinking about having so many bits to fill.
I think I'll hold off until it's the size of a quarter and costs $50.
That is the hottest thing I've seen all day!(My girlfriend is out of town)
While the Dewey Decimal system was revolutionary for its day, it's long fallen into disuse in any serious library. A lot of school libraries still use it and some local libraries use it, but I can't think of one university or college library I've been to that didn't use the library of congress system. It's a lot more useful as most people who have used both would say.
Also, it's interesting to note that the library at Amherst College, where the Dewey decimal system was created (by Dewey!) no longer uses the Dewey decimal system.
Don't IEEE numbers use the Dewey Decimal system as the theme for their specs? 802.11a, 802.3, etc.
cool, now I can keep all my pr0n, illegally downloaded movies, and music in one place. it would take big pipes to fill this tank, if you know what i'm saying
Investing forum
It's still not enough.
grammar-lesson free since 1999. (rescinded - 2005)
The Mango system was only produced for versions of Windows up to 95, with spotty NT support. The premise was pretty cool: each user of the system allocated part of their hard drive to a single network share. All of this space was added up and appeared as a single shared mapping to the network. Each file was copied to two users for safety. If a user accessed a file, they would get a copy and someone else would lose it. If someone's computer became unavailable, the rest of the system would reconfigure. It was sort of a large RAID-1 system.
Coda information available here: http://www.coda.cs.cmu.edu/
Hey, if you make a file that is 100GB you would have a file that is a petabye or a petafile...get it hehe :) flam me if u want
Man, imagine a Red Sox world championship...
GO SOX!!! FUCK THE YANKEES!
Me email iz skyewalkerluke at microsoft's free email service.
check OIDs in SNMP, looks precisely as a decimal-position addressing system.
Less is more !
Maybe I'm just old, but doesn't it strike anyone that a system built to handle petabytes of data should cost more than $90,000? That's not a whole lot of money for enterprize level hardware.
Hell, I remeber seeing an IBM System 38 with 16 gigabytes of storage, bloody thing took up a room and cost a couple million bucks. All they did with it was keep a driver's licence database on it and run print batches.
$90,000? CHEAP!
a porn solution that works!
Oh my god, that's $630,000 dog dollars!
'Thats they exact same thing a banana wrench monkey.'
oops, watch out for those hadrons!
It seems that IBMs system is just a specialized P2P file sharing/serving network, not really anything new and "revolutionary."
...you will discover that 1 petabyte is enough
room for more Divx encoded porn than a man could
watch in a lifetime with no sleep or bathroom
breaks. Think about that for a second.
Oh man, imagine hacking Tivo to work with this!
It would be cool if the sox won, but you know they have no chance. They only have one good piture and he's done for the series...
startime - 10-01-03
endtime - 10-01-13
Distributed storage, that is.
Or is it a 1 pibibytes?
... how long before I fill it worth porn.
http://saveie6.com/
that "10-20" terabytes line has to be a typo.
I spoke w/ some people from CERN regarding their CASTOR HSM, and a few years ago they were up in the petabyte range already. By now, they're probably sitting at at least a few hundred TB online, and probably 5 PB offline, as a conservative guess.
IBM's been doing GPFS filesystems in the > 50 TB size, w/ > 1 GB/sec. throughput for years. That, and even's IBM's mid-tier FAStT products can confortably carry 12 TB on one dual-controller storage head.
Still, further abstracting the issue of locality is very exciting stuff. I'd be interested to see exactly how they go about doing it, and if it's anything that you can't get w/ Lustre when it's ready.
PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
These large units confuse me. Tera, Tebi, Peta, Pebi? Could someone give me the equivalent in LoCs?
"Sic Semper Tyrannosaurus Rex."
the Andrew File System (AFS), and to some degree it sucessor DFS.
Do you know why programmers always confuse halloween and christmas?
hhahahahahahahahaha
The RIAA sues Big Blue for creating a "haven for filthy music pirates"..
Kazaa file numbers shoot up
SCO sues IBM (again) because this "Storage tank" is just like the one they got to hold the shit that comes out of the SCO office toilet before it is tossed at Linux users
ok so it's not THAT funny
Suchetha
learn from yesterday, plan for tomorrow, party tonight
or one out of three ain't bad
i am still alive because of the hope of vr pr0n. your not aiming high enough :(
OK. So, SCO is pissy with IBM over it's Linux/UNIX usage, right? So, now IBM has a system called the Storage Tank which could hold a whole buttload of data.
How long will it take before the RIAA jumps on it because it has the potential to house and share an immense amout of media? If some schmo with the CD collection from Hades and some extra green- or t-backs (t for technicolor as in the new 20) to toss around gets ahold of one of these and has the free time to fill it up with tunes...
And was I the only person who glanced at the "Large Hadron Collider" text and think Hardon off the bat? Oh well, such is the curse of being a young American male with a perverse sense of humor...
Blog Prophyts - Right On, Man
Get your own 'starter configuration' for only $90,000!
Hey, it beats the 300gb Maxtor
I have over 70 freaks, do you?
...a mouth full of vegatables to me!
Google to avoid joke flybys.
---
DRM is like antifreeze, to the MPAA/RIAA it's sweet, to the consumers it's poison.
New genetically engineered Seamonkeys!
Imagine a storage tank of those!
The Storage Tank may have finally arrived, but according to this article it falls rather short of what IBM claimed it was going to be.
No bathroom breaks? That would defeat the purpose of pr0n!
I had the chance to see the actual installation at CERN. The tank filesystem and principles are quite neat. I'm looking forward to get this working well for commercial stuff also.
Some of the basic principles:
Maybe not everything is in place yet, but the strategy is good !
Despite what IBM might claim, SGI's CXFS filesystem is clearly the first (and the most) heterogenous, clustered filesystem available today.
Only one word for it:
owned!
Woah
Shouldn't that read Pitybyte or something like that? ;-P
Okay okay, it's PebiByte, but hey...one can try =)
...who are braindead, one Petabyte is approximately 10 to the 15th power in bytes. If you're a harddrive manufacturer.
It's also estimated that a Petabyte is all the storage space required to hold the memories of a hundred year old human being.
Edward@Tomato - /home/Edward/ man woman
man: no entry for woman in the manual.
"Qua!?"
10 to 20 Terabytes of data is what the LHC collisioner is going to generate each second while it is running. CERN is expecting to generate at least 5 petabytes of data per year.
It should also be noted that CERN is a large user of lower cost large storage arrays based on 3ware cards, but those won't scale to what the LHC will require.
You don't. If all your data is lost, you just re-launch the experiment with that single neutron aimed for that single quartz and BANG there is your data again (albeit different)... ;-)
More related to OpenAFS than Coda.... Looked at both extensively recently. OpenAFS scales like a champ, Coda does not. Mostly due to the way Coda stores metadata. You have to run a seperate server instance of Coda for every (approx) 23 gig of file space you serve out. There are other limitations...
OpenAFS seems to be much more open ended at this point, though it does not support Disconnected operation or Read/Write replicas. Though that isn't necessarily a bad thing. Neither likes files >2gig right now.... OpenAFS is still in active developement, but Coda seems stalled at this point.
"Variety's the spice of life. I like a wide selection. Sometimes I'm in the mood for nasty close-ups, sometimes I like them arty and air-brushed. Sometimes it's a spread brown-eye kind of night, sometimes it's girl-on-girl time. Sometimes a steamy letter will do it, sometimes -- not often, but sometimes -- I like the idea of a chick with a horse. "
Hopefully they aren't using IBM hard drives in it. We've seen a 63% failure rate on them.
Me (Insightfill) again - AC as I'm getting OT.
I hadn't really looked into OpenAFS, as it didn't come up in many other references for me, but I'm looking forward to trying it out. I tried running Coda in Windows using Cygwin (as the documentation says you can), but ran into a bug that was traced down to an update to Cygwin. The Coda devs. aren't planning to address it any time soon. Yup: stalled.
I was looking for Disconnected operation or read/write replicas, so this might be my thing. Could've really used the >2gig support. Oh well. I'm going to give OpenAFS a whirl - thanks!
It sounds to me like IBM is just struggling to keep up with the Jonses here... in this case the Jonses being Spinnaker Networks. They already have a system in production that can scale to 12 petabytes today (no need to wait until next year!). It's also not limited to Windows and AIX; they support Linux and a whole host of other unix flavors too.
Nothing IBM is doing here is new technology (other than to IBM). As examples, SGI has been doing this for this years with CXFS and ADIC has had the StorNext File System (used to br called CVFS) for a while as well.
It's kind of sad that no one on the board here seems to understand what this product actually is though. It essentially is a metadata controller that will allow read/write access to the same LUN from multiple servers on a SAN (i.e. straight block level access to a disk(s) via SCSI over FC). Even though this isn't a new concept, it's good to see a large storage player adopt this idea.
only one good pitcher? Why is the series 2-2 now then? Wakefield and Low aren't that bad, you know. Its the relief pitchers who suck.
that read "CERN is currently using a beta version of the system to store data from the Large Hardon Collider" and thought "Well, it would be great for all of my pr0n, but I don't think I'd want any gay pr0n, let alone 1PB of it!"
Other than that, show us some benchmarks! Is it fast enough to capture uncompressed video to?
Imagine the uber-l33t PVR you could build with this! a hundred TV tuner cards, capture ALL your favorite shows -- and your friends' favorite shows -- and all the HBO and skinemax flicks! I mean... uncompressed.. of course! why waste valuable CPU cycles on compression? then, when you've catalogued every movie, every show, every commercial, every thing that has been on TV since you got the shit set up, well... you notice that you've only used about a quarter of your storage! now you're pissed! you wasted all that money on space that's gonna go unused?!?! so you start compressing everything into DVD compatible MPEG-2, DivX, VCD compliant MPEG-1, VCD2.0 compliant MPEG-2, Real, WMV, Hasbro VideoNOW, and whatever other formats you can think of -- now if a friend wants a show or movie in a specific format, no problem! Okay, you have 1/4 of a PB left. Good enough -- now to watch all the pr0n you captured from the playboy channel!
Error 666 - SCO source has been found in your Linux kernel. Please remove it.
Formerly kdsolutions
If you're curious about how Storage Tank works, check out the paper from the IBM Ssstems Journal.
Now can we please hear someone saying
/.
"But I can build it cheaper myself, why would anyone wannt to BUY this for $90,000, when you can build it using several RAIDS and cheap IDEs for a quarter of the price"
yeah..
Assuming we are going with high quality, 23.9 fps DiVX. A 120 minute porno should be about 730MB.
Petabyte: 1,125,899,906,842,624 bytes
730MB: 1,048,576*730 = 765,460,480 bytes
That's 1,470,879 and 120 minute porno films.
This comes to a total of 176,505,480 minutes.
Hours worth of porn: 2,941,758
In a non-leap year, there are 8760 hours per year.
That's right.. It would take 335 years to watch all that porn with no breaks.
My suggestion would to be to purchase one of those monitor arrays featured on slashdot the other day and run a separate video on each. With 10 displays, that'll get you down to around 33 1/2 years or porn.
** your mileage may vary as I am going by 1024 bytes per KB in my calculations. Leap years not calculated.
Yes but is it in PiB or PB?
I certainly want my PB drive to be PiB's not PB's =)
I don't really care for the sox, but I want the Yankees to lose miserably, so GO SOX! :)
200GB ought to be enough for everybody.
please emphasize the "suck" on sauerbeck. what a mistake...
Me email iz skyewalkerluke at microsoft's free email service.
It's a good thing the Be File System created for the BeOS supports 32 petabytes!