Petabyte Storage Array

knight13 writes "Engadet is reporting that EMC is rolling out a petabyte RAID array. From the article, "And if you're ready for that level of storage, there's now someplace to get it: EMC has launched its first petabyte array, a version of the company's flagship Symmetrix DMX-3 system that includes nine room-filling cabinets of drives." The price? A mere $4 million."

Comments

1 Peta?? How many

[IdleTime]

JPEGS would that be?

Re:1 Peta?? How many

[rnpg1014]

If every JPEG was 500 KB, 4,708,523,520 of them. This doesn't account for the operating system, if there is one. Still, when would you ever need to store nearly 5 trillion JPEGs, unless you're Google Caching?

Re:1 Peta?? How many

[gatzke]

Is that one billion copies of tubgirl and goatse?

Re:1 Peta?? How many

[Gryle]

jpegs
I think you misspelled porn.

Re:1 Peta?? How many

[mikequad]

Enough for you to appreciably contribute to global warming through your mechanical expenditure.

Re:1 Peta?? How many

[Anonymous Coward]

> If every JPEG was 500 KB, 4,708,523,520 of them.

If the size was reduced somewhat, you'd be able to have a jpeg for every member of the world population.

Re:1 Peta?? How many

[Anonymous Coward]

Still, when would you ever need to store nearly 5 trillion JPEGs

That's 5 billion.

Re:1 Peta?? How many

[HUADPE]

The obvious use of the internet...porn.

Re:1 Peta?? How many

[thefirelane]

How many JPEGs would that be?

1 Really big one

Re:1 Peta?? How many

[Anonymous Coward]

How many JPEGs would that be?

All of them.

Re:1 Peta?? How many

[stfvon007]

What if someone wanted to make a photo album with 1 picture for each person on the planet? They Would run out of space! They would need to buy 2 in order to hold all the pictures! And what if they wanted bitmaps? Even more would be needed!

Re:1 Peta?? How many

[mobby_6kl]

>What if someone wanted to make a photo album with 1 picture for each person on the planet?

How about this, limit the sample to women aged 18-35, but take several photos? Maybe even more space could be saved by taking off the extra clothes...

Re:1 Peta?? How many

[ltbarcly]

Account for the operating system?

ROFL

Re:1 Peta?? How many

[LifesABeach]

Well, 2 Peta Bytes would be the equivalent storage capacity of one person. You figure that for a safe system to store one person's life could be done with about 6 Peta Bytes. In the next 20 twenty years that will be easily realistic for Joe Average Six Pack.

The next step is to create the method of downloading a persons history.

Re:1 Peta?? How many

[Hegh]

If every JPEG was 500 KB, 4,708,523,520 of them. This doesn't account for the operating system, if there is one. Still, when would you ever need to store nearly 5 trillion JPEGs, unless you're Google Caching?

In case nobody realized...4,708,523,520 is in the billions, not trillions as mentioned above.

Re:1 Peta?? How many

[IdleTime]

Call me when I can get an ipod sized device with 1 petabyte disk :)

Re:1 Peta?? How many

[batboy78]

There is an Operating System on a SAN. It is typically stored on the first 3-5 drives of the DPE. These disks are referred to as the "Vault" disks. These disks can be used for storage but as much as 20 percent of the disk is reserved for the OS.

Re:1 Peta?? How many

[HTH NE1]

1 Peta[byte]?? How many JPEGS would that be?

JPEGs are too small for the margin of error in the reporting of this 1 PB array. From the article, it appears to be a 1.2 PB array (2400 * 500 GB), presumably to push it over the 1 PiB capacity(*). And that's assuming the whole capacity is available and none of it used for redundancy. (That's a lot of data to lose over one drive failure.)

(*) The difference between 1 PB and 1 PiB is nearly 126 TB! That's a lot of space to miss over not knowing what units you're talking about. Byte @

Re:1 Peta?? How many

[Poltras]

We could do the 1 trillion $ web page, and pay for the petabyte array... business case here, gentlemen.

Re:1 Peta?? How many

[Carnildo]

4,708,523,520...5 trillion
What scale are you using? Under short scale, that number would be called "five billion", while under long scale it would be "five thousand million" or "five milliard".

Re:1 Peta?? How many

[rnpg1014]

Sorry for the trillions != billions thing, happens to me sometimes. For example once I was thinking to say "blazer" and "jacket" and I said "jazer"...

Kinda Interesting

[synthparadox]

This is pretty interesting in that it's yet another item that we all wish we had just for overkill purposes.

However, I doubt they'll sell many of these. The only places I can think of that would benefit from this are supercomputing institutes, but they often build their own redudant RAID systems and/or NAS systems.

It's nice and all, but seriously people, who's the audience?

Re:Kinda Interesting

[TinyManCan]

You're mistaken.

If this was slightly less high-end disk (DMX's are EMC's top of the line) it would be perfect for disk-to-disk backups. We send approx 50 TB a day of data to tape to send offsite. I would *love* to have the last 50 days data on disk, onsite for instant restores.

Re:Kinda Interesting

[synthparadox]

But again, I refer to my question. Is it worth $4 million? Are you willing to pay $4 million for something like this? Usually the people who are interested in this have their own ability & money to build a NAS or backup systems and support them.

Re:Kinda Interesting

[AKAImBatman]

If the man has 50 Terabytes of critical data that he needs to backup and ship every day, I'd say he probably has a budget that could accomidate one of these things. While multi-terabyte arrays are more common than they once were, anyone carrying around that much data still needs to spend millions just to keep their infrastructure intact.

Now all he's got to do is get his boss to sign the check. :-P

Re:Kinda Interesting

[Anonymous Coward]

"It's nice and all, but seriously people, who's the audience?"

Pretty much anyone who bought into the ERP/CRM hype. Oracle does an amazing job of filling up space, and for a very large database, this amount of space isn't really that ridiculous, particularly when you consider what kind of space you'll need to grow into ...

Re:Kinda Interesting

[Anonymous Coward]

Wow, just wondering, what type of work are you doing that you generate 50 TB a day of data?

Re:Kinda Interesting

[TinyManCan]

WHile not wanting to say too much, that cost is definitely reasonable considering the benefits.

Imagine just once, you had to wait 4 hours for some tapes to come back onsite. Now that is four hours times approx 40,000 people (number of employees unable to work). That one outage just cost you 160,000 hours of downtime, where you could not serve your customers. Assuming you pay on average $25/per employee/per hour you've paid for the system in one go.

This is not including the amount of money lost because product has not been sold, or customers who got bad treatment and left.

Also, while the company certainly has people with the skills to assemble a large array, I can say that we would never do it. It is much easier to offload the support and performance worries to a vendor.

Re:Kinda Interesting

[Neil Blender]

Just a guess but...

He probably works for a company that provides backups to other companies in a data center. He'll still need to move the data offsite and the only practical way to move that amount of data is by tape. Having a $4 million dollar array would probably mean he wouldn't have to drive to the offsite storage facility three times a day to fetch tapes to do recoveries and could spend more time reading slashdot. The money this would cost to buy and maintain could easily cover several FTEs for many years.

Re:Kinda Interesting

[Anonymous Coward]

Actually this solution is pretty cheap. If you look at a system with a 3ware card (12 port) you can get the cost per TB to about $1700. That way you would need 500 of them. (Each system has 12 400G drives. RAID 5 loose 2 per system to stay under 2TB.) Cost would be roughly $1.7M. This does not include networking or racks, etc.

Thus 2x that cost is pretty good and you get a better package.

I agree, there would be a lot of people who would still build there own, but this is getting very close.

Oh industries who would like this. Medical (these guys create 10x the data that nuclear physics creates), oil and gas, rendering (data duplication to reduce hot spots), disk backups (as mentioned before), call centers with large db (almost anyone now days), financial industry. Really with a solution like this, you can also branch out with other filesystem and access patterns to your computers (lustre and gpfs).

Re:Kinda Interesting

[Anonymous Coward]

-1 Redundant Acronym Is Dumb

Re:Kinda Interesting

[DerekLyons]

However, I doubt they'll sell many of these. The only places I can think of that would benefit from this are supercomputing institutes, but they often build their own redudant RAID systems and/or NAS systems.

I suspect the marketing strategy is to sell the smaller versions of the system - the petabyte version is just an assembly of modular components.

It's nice and all, but seriously people, who's the audience?

For the full meal deal? Probably nobody - but it makes a hell of an advertisement for the smaller systems in the same product line.

Re:Kinda Interesting

[TinyManCan]

For the full meal deal? Probably nobody - but it makes a hell of an advertisement for the smaller systems in the same product line.

I'd bet you that you are wrong on this. EMC is going to sell a lot of these systems.

Previously you could get a 230 TB (? might be off, going from memory?) DMX3000 array. EMC has a lot of customers with several (many in some instances) of these installed. A good percentage of these customers would probably consolidate into a single array. Some customers like the advantages of many smaller arrays.

But certainly EMC is going to sell more than 0 of these. This year.

Re:Kinda Interesting

[Anonymous Coward]

The problem is that he'd need somewhere offsite to backup the petabyte array.

Re:Kinda Interesting

[MikeFM]

It'd be awesome if a major hosting provider bought some of these and let you buy space in whatever size chunks you needed.

Re:Kinda Interesting

[Theatetus]

Imagine just once, you had to wait 4 hours for some tapes to come back onsite. Now that is four hours times approx 40,000 people (number of employees unable to work). That one outage just cost you 160,000 hours of downtime, where you could not serve your customers. Assuming you pay on average $25/per employee/per hour you've paid for the system in one go.

Only if you use Enron math. You have to pay $25 per employee per hour either way. The only thing that matters is what you mentioned as a side note, revenue from customers lost during the outage. If whatever system relies on this backup is generating you $1,000,000 per hour, then an array like this would pay for itself in one four-hour outage. But, that doesn't take into account opportunity cost: you could still be better off if you put that $4 million to use generating revenue; if it made back more than the outage costs you you're still on top.

Re:Kinda Interesting

[dilvish_the_damned]

The only places I can think of that would benefit from this are supercomputing institutes
We have been in the design phase for a storage solution that would provide scalability up to an exabyte (on paper, in theory, with some dark magic and light kludges).
The only thing that has amazed me to this point is not the fact we can do it, its the number of applications that require (and the others that logically should require if they were not stuck in the 90's) a serious storage solution.

SHAMELESS PLUG BELOW:
However the main problem is not how to store and retrieve the data in a seamless fasion, its power and the the fact that, for most applications, you rarely need the drives spinning... So we took that and moved it a step further and power the whole damned thing of, one unit and layer at a time. Add in some deep thought in the power management and you have what we are striving for. Oh, that and 'dial in your own reliability' this last is my phrase.

Re:Kinda Interesting

[TinyManCan]

I'm interested in this approach, but have always had concerns on the effect of constant power cycling on the drives. Have you seen a reduction in lifespan on the drives from this on-off-on-off behavior? Does this increased failure rate, and the increase in the number of drives to 'dial in the reliability' end up costing more in the long run?

Re:Kinda Interesting

[Nutria]

The problem is that he'd need somewhere offsite to backup the petabyte array.

Think of the petabyte array as near-line storage for the main array.

The 50TB of tapes per day going out to Iron Mountain will still be going out every day. "They" will just be doing a whole lot fewer tape recalls.

Re:Kinda Interesting

[Faltargan2006]

Hmmmm.....
(http://ask.slashdot.org/article.pl?sid=06/01/30/2 258225)

Maybe?

Re:Kinda Interesting

[jurt1235]

I design disk backup systems, and use low end disks in it, with sufficient redundancy. I also device ways in it so you do not need so much disk space to go back in time for very prolonged periods (deltas which to the user look like full backups). Mail me, it is always fun to design a new system.

Re:Kinda Interesting

[Mikeytsi]

You don't have much involvement with Intenet companies, do you?

The customers for this are Google, MSN (and it's related projects), iTunes,... Basically any company that has a presence that requires a large amount of data storage with fast retrieval. Most of these companies probably already use SANs from EMC, I know MSN does.

Re:Kinda Interesting

[rjstanford]

If whatever system relies on this backup is generating you $1,000,000 per hour, then an array like this would pay for itself in one four-hour outage. But, that doesn't take into account opportunity cost: you could still be better off if you put that $4 million to use generating revenue; if it made back more than the outage costs you you're still on top.

Well, yes and no. Let's say that the system is generating $1mm per hour in revenue. What do you think the potential backlash would be from a four hour downtime in terms of customer satisfaction, media coverage, et cetera? I'd guess well over that $1mm of lost revenue, and it can last for a long, long time.

Re:Kinda Interesting

[Krid(O'Caign)]

You're missing the reputation stain that results from a 4-hour outage. The company's PR damage from the downtime would likely cost them significant revinue, and thus in this case it's probably worthwhile.

Car with airbags, car without airbags. Most airbags are never used, but people don't buy them because they expect to use them. They buy them because the waste cost of having and not needing is in the $100 range, but the cost of needing and not having is anywhere between $40,000 and death.
The question is if the company is large enough for that kind of down time to do enough damage to justify it's cost.

Re:Kinda Interesting

[DocSavage64109]

I also device ways in it so you do not need so much disk space to go back in time for very prolonged periods (deltas which to the user look like full backups).
Rsnapshot does much the same thing in linux. It uses rsync to optimize transfer of data across the network and and hard links to allow multiple backups to share the same files. You can easily have snapshots that are months old alongside snapshots from every day of the week.

Re:Kinda Interesting

[Anonymous Coward]

We send approx 50 TB a day of data to tape to send offsite. I would *love* to have the last 50 days data on disk, onsite for instant restores.

50 TB for full backup I suppose, right? You don't really generate 50 TB of new data every day, don't you? Then you'll need less than (50 TB/day * 50 days) to keep onsite for instant restores.

Re:Kinda Interesting

[ls -la]

redudant RAID

how ironic...

Re:Kinda Interesting

[parc]

There are several organizations that are required by law to keep records of data for several years. An example:

When I was in the telecom field, we were required to keep a log of every call made for 6 years. Now imagine you are a tiny mom and pop organization -- let's just call it Sam and Bob's Communications. You buy another small mom-and-pop sop, Art, Trent, & Trevor. You've now got a combined customer base of, say 500 million people. Half of them also have a cell phone, and 20% have two phone lines. That's a lot of data.

And, of course, there's the ever-present need for Homland Security to log every breath that every person in america takes.

Re:Kinda Interesting

[Hans Lehmann]

However, I doubt they'll sell many of these. The only places I can think of that would benefit from this are supercomputing institutes, but they often build their own redudant RAID systems and/or NAS systems.

That's funny, I was saying almost the same thing about Terabyte storage not many years ago. Now I wish I had a full terabyte of space at home to store music, pictures, videos, etc.

Re:Kinda Interesting

[Jokerz17]

how ironic...

How redundant.

Re:Kinda Interesting

[at_18]

$1mm

What's that? A small form factor $1 banknote with a 1mm side?

Re:Kinda Interesting

[Tweekster]

You are somewhat correct... But depending on the job, that four hours has to be made up somehow, aka, they have to be paid to actually do the work now.

Re:Kinda Interesting

[Anonymous Coward]

Hrd disks are usualy rated for a certian number of spin-up cycles. Nowadays its usually in the milions.

Re:Kinda Interesting

[hesiod]

> Oh industries who would like this. Medical (these guys create 10x the data that nuclear physics creates)

Exactly my thoughts. Of course, I work in a hospital, so that's obvious to me... Diagnostic Imaging devices in a small hospital can easily put out a terabyte a year -- that's just images, no medical records, etc. If you move up to a larger hospital, a petabyte of storage for a PACS (picture archiving system) would be EXTREMELY valuable.

Re:Kinda Interesting

[Anonymous Coward]

Well, the responses to this article illustrate perfectly the kind of useless dubmasses who join me in reading this site every day.

aaaaand this is why you shouldn't read slashdot if you know what's good for you :)

lots of people here who have knowledge a mile wide, but an inch deep,

and are too arrogant to admit that: YES INDEED i do not know anything outside mainaining a single user system;
  see the clown who thinks that the OSS shit will provide an acceptable replacement to an enterprise backup solution.

Yessir folks! sometimes you have to pay money to get things done properly!
You can't just flinch at every up front cost. hold it all together with duct tape and perl, and expect to be taken seriously! *

but hey, we're a bunch of sensitive nerds who hate non-programmers who have control over us/i.e. management

riiiiiiightttttt?

* just today, we're gonna use ASP.NET, I hear it:
But PHP is FREE!!!! Why you gotta pay all that money?!!
Answer: because SOMETIMES people need to get work done!

Holy Truman, Batman!

[MarkRose]

Interesting calculation: If you live 80 years, that's 435.5 KB per second -- enough for a TV-quality video of your entire life.

Re:Holy Truman, Batman!

[genkers]

Is there enough tv for all that space?

Re:Holy Truman, Batman!

[pintomp3]

yes, it can document my entire life. that sure is a lot of porn.

Re:Holy Truman, Batman!

[Anonymous Coward]

But the question remains: How much of a market is there for super high resolution videos of some guy jacking off?

Re:Holy Truman, Batman!

[cgenman]

Interesting calculation: If you live 80 years, that's 435.5 KB per second -- enough for a TV-quality video of your entire life.

If you live for 80 years, that's 75 years longer than an average hard drive will last. That's 6.9 Megs of data breaking every second.

Re:Holy Truman, Batman!

[ImaLamer]

Does that mean I can fast forward through the commercials?

Re:Holy Truman, Batman!

[McWilde]

The phrase "TV-quality" is always amusing to me. Digital television is just starting to roll now in Holland (or maybe I'm just now starting to notice it) and the one feature that is supposed to sell it is the great quality of the picture. But that's not really the problem with television. Everyone I know has cable tv, which is good enough for me. The problem is there's never anything on.
So saying that you could make a TV-quality video of your life is like saying your life sucks.

Re:Holy Truman, Batman!

[Marxist Hacker 42]

Yes- and for the purpose of documenting my life, I don't need no stinkin' TV quality either. Given this, what I want is a digital camera that can handle sound and picture, recorded constantly to a Type III CF slot, at 240x360 pixels and 16 bit stereo sound, in MPEG-2 format. I then would want the Petabyte array at home- and Hitachi's new 6GB microdrive on the road. I'd then be able to record all the interesting (read waking) hours of my life, and have them indexed by day and hour for recall, pretty easily.

Re:Holy Truman, Batman!

[mvdw]

Why anybody would think that their life is interesting enough to warrant 24/7 recording of it is beyond me.

Re:Holy Truman, Batman!

[Marxist Hacker 42]

It isn't- most of those gigabytes will be wasted. What this is useful for is as an external memory periphereal to your brain. Human brains are very efficient at storing data, but notoriously inefficient at data retrieval- by the time you're 30 the pathways for retrieval are begining to degrade in reverse- you'll be still able to remember what your locker combination was in high school, but what you had for breakfast this morning may be lost forever.

Such a "life recording system", especially if incorporated into a device small enough to carry around with you that could say, keep local the recordings of the past 168 hours in a low-res format, backed up weekly on your home petabyte data store, would be incredibly useful. What did the boss tell you to do on Monday just before going off on that trip? Rewind to Monday and watch it!

NOBODY except maybe a voyeur on 8x fast forward is going to want to watch your whole life- but the porn opportunities for somebody of the opposite sex or homosexuals are amazing so such cameras would have to be banned from communal locker rooms and the like.

Re:Holy Truman, Batman!

[cgenman]

have them indexed by day and hour for recall, pretty easily.

I have a friend who has photographic memory. She can take pictures of things with her mind, and look back at them later. If she wants, she can snap an entire textbook and read it later.

The problem is, though, that whenever she wants data she still has to read it. If she doesn't study for tests, then she has to flip through textbooks in her mind to try and find the data, which is a lot more tedious than you would think. If you had a recording of your life and wanted to know your boss's exact statement about your project 6 months ago, you will need to spend hours and hours and hours flipping through footage looking for it.

A 24-7 documentation of your life would take 24-7 to watch.

Re:Holy Truman, Batman!

[Marxist Hacker 42]

If you had a recording of your life and wanted to know your boss's exact statement about your project 6 months ago, you will need to spend hours and hours and hours flipping through footage looking for it.

That's where visual search tools come into use- I'm not saying we have the technology RIGHT NOW to find stuff in a 6 month archive of video- I'm saying that the storage is coming close (perpendicular recording Hitachi Microdrives are coming out in 2007, at which point you'll be able to carry 60GB around pretty easily), and the software too thanks to the War on Terror (the same software algorithims that allow a Predator to target a single individual with a Hellfire missile or allows the TSA to pick out a face in the crowd for extra scrutiny at the airport, will one day allow your child to automatically sift through a petabyte data store of video to find a first meeting of a first love). With a Petabyte of storage, you can also afford to devote some space to diary indexing- entering a note on your PDA that syncs to the camera to mark a date-time-stamp on the video as something of interest, with notes as to what was interesting about it. With it being a RAID array, it can be hot swapable and continuous back up, so that a single drive dying doesn't mean you lost 3 months of your life.

We're talking convergence in the near future, not the present. But there's at least 4 currently separate technologies that are coming together quite quickly to allow you to do all of this...and it will cause a change in the culture, because such PDA/camera systems are sure to become quite ubiquitous.

Re:Holy Truman, Batman!

[smoker2]

I have a friend who has photographic memory. She can take pictures of things with her mind, and look back at them later. If she wants, she can snap an entire textbook and read it later.

I don't think I have a photographic memory, but back when I was in school, I used to cram for exams by scanning (fast reading - not electronically) relevant books or texts. Then when I was in the exam room and read a question, I had to remember which page of the book the answer was on (or the subject was dealt with) and then I could "read" it from the mental image of the page.
Seemed to work quite well. Of course, you had to know the subject quite well anyway to judge relevance, but to have the original material available gave a better starting point for your answers. I still use the same system now, and I don't think it differs much from the "Incredible Memory Man" techniques.

Heh

[Anonymous Crowhead]

I was just thinking about how 4 years ago you could build a terabyte array for about $5-10,000 down from many millions 8 years ago. Today, you can get a terabyte for less than $500. In a few years, a petabyte is only going to cost $5,000. If you just buying space for future growth, it seems like a total waste of money.

Re:Heh

[Anonymous Coward]

Um...8 years ago they made drives larger than 500 megabytes (being that 500GB is the largest consumer drive available today), so I doubt that array was millions 8 years ago. Also, disk space is really starting to level off, and won't grow much until we have some major tech changes.

Re:Heh

[Anonymous Crowhead]

In 1997 the cost per gigabyte was roughly $100. So, yes it would have been under a million for the disks. Still, only 12 to 18 months earlier, a gig was running nearly $1000 so if I said 10 years, it would have cost millions. In 2001, a gigabyte cost about $6. Now it's about $0.50. For the entire history of computing I have heard "X will level off at Y unless some techical advances are made." Technical advances have always been made.

Re:Heh

[Anonymous Coward]

And back in _MY_ day, a mere 10M hard drive cost $1000!!! Ah you young whipper-snappers have it SO easy these days!!!

Re:Heh

[alex_guy_CA]

"Today, you can get a terabyte for less than $500"

Where? I've been looking for that! (no I'm not kidding or trolling)

Re:Heh

[Just Some Guy]

Where? I've been looking for that! (no I'm not kidding or trolling)

Put four of these in a spare box. Price before tax+shipping: $480 for over a terrabyte of storage.

Re:Heh

[alex_guy_CA]

Anything that comes in a box? (no spares here)

Re:Heh

[Tesla Tank]

I bought a computer in 98 with 8 gig of HD. You're telling me it would've costed around $800? The entire PC was $1000 and I'm sure the HD costed only around $100.

Re:Heh

[Anonymous Crowhead]

For almost every year since the early 90s the cost per megabyte dropped by about half. So the cost of a drive at the beginning of 1997 could cost 4 times as much (per meg) as a drive bought at the end of 1998. In those two years the price per meg would have fallen by half two times. The reason you were able to buy a sub $1000 pc, was in part due to the steep drops in the price of drives.

Bragging rights

[utlemming]

I love the last line -- if you just want bragging rights -- these days that'll last 3 days.

On a related note...

[mr_stinky_britches]

When will we have support for hardware raid in the 2.6 kernel?
(Then I can just make..my...own...*)

*Scale: 1:1000
--

http://wi-fizzle.com Ruby on Rails and stuff

Re:On a related note...

[Anonymous Coward]

There are, in fact, when it does work, you may just see one drive, though you'll probably need to use a setup disk/utility if it doesn't default to something. Numerous people get compaq/hp proliants with scsi raid cards to work.

You might like this, it doesn't just apply to SATA.

http://linux.yyz.us/sata/faq-sata-raid.html

http://developer.skolelinux.no/info/prosjektet/del prosjekt/hw-raid-info.html

Been there done that

[hackstraw]

http://www.archive.org/web/petabox.php

By those who truly care about the human tradition, and spreading the music of the Grateful Dead and other freely available media.

Is this another slashvertisement?

Re:Been there done that

[PornMaster]

The I/O performance of a petabox is just *slightly* lower than a DMX-3.

This box, and the software used to manage it, make it considerably more useful than a petabox.

Re:Been there done that

[hackstraw]

The I/O performance of a petabox is just *slightly* lower than a DMX-3.

Are you being facetious here? Any details?

Keep in mind that these are two very different beasts here. The petabox is one rack, the EMC "box" is 9 racks. More drives always gives you better performance. I'd be happy with a petabox for my music and porn. That would serve me fine for a couple of years.

My point was that this is being _advertised_ as something new. Being that most slashdotters are still in their mother's basements trying to get Linux to run their PIIs and PIIIs "fast", I doubt that this box is anything that they would know what to do with over a petabox.

Re:Been there done that

[name773]

and i don't think the petabox is redundant at the hard disk level

Re:Been there done that

[r_weaver]

It's only a "Petabox" in name, not capacity. (unless by "box" they mean a 20'x8'x8' shipping container)

From the linked site:

* High density-- 100 Terabytes per rack
* Colocation friendly-- requires our own rack to get 100TB/rack, or 50TB in a standard rack

So even with the special Internet Archive racks, you'd need 10 of the racks to get a Petabyte.

Though it seems that capricorn-tech has improved on the capacity since the Internet Archive page was written, advertising up to 80 TB per standard rack, so you could get by with just 6.25 racks for a full Petabyte of storage if you had those new higher capacity nodes in the special racks.

Other interesting specs: 900 lbs per standard 19" x 80" high (42U) rack, 80W/node, 3.2 KW/rack, 40KW for an entire Petabyte.

Re:Been there done that

[Animats]

The Archive is working, as I understand it, to move the Greatful Dead fan recordings off to somewhere else. Some of that content is streaming-only, and the fans keep playing the same streams over and over again. This is using up too much of the Archive's bandwidth.

Re:Been there done that

[rikkards]

The petabox is one rack, the EMC "box" is 9 racks. More drives always gives you better performance
No, the petabox is 10 racks to get a peta. You only (only?) get 100 terabytes in a rack. See the second bullet in the Overview.

Re:A petabyte of pr0n

[Compuser]

Funny, the poster just before you said that this was
"enough for a TV-quality video of your entire life".
Conversely, this is enough for twice the pr0n you could
possibly watch (assuming you sleep and eat).

Failure rate

[joNDoty]

Let's assume for a moment that the average lifetime of one hard disk in this petabyte array is 6.5 years. Since there are 2,400 hard drives, that means that once this thing has been running for a while, you will be replacing, on average, one broken hard drive per day, for the entire lifetime of the array. That's about $350 per day in replacement parts alone!

Re:Failure rate

[Anonymous Coward]

No, it doesn't work like that. When you buy an array like (esp. from EMC) you buy a _service plan_ to go with it.

You pay $xxx,xxx.xx up front for a years service. The EMC arrays call home when a drive is getting ready to die (i.e. well before there is _any_ data loss) and EMC sends a tech onsite. The drive is swapped out and you as the customer notice absolutely nothing, except a line in the security log where the tech showed up at the datacenter.

Re:Failure rate

[imoou]

I guess if you the money to spend $4 million on storage, you wouldn't have to worry about $350 per day anymore.

Re:Failure rate

[muletool]

That probably small potatoes compared to the electric bill.

Re:Failure rate

[hackstraw]

Let's assume for a moment that the average lifetime of one hard disk in this petabyte array is 6.5 years. Since there are 2,400 hard drives, that means that once this thing has been running for a while, you will be replacing, on average, one broken hard drive per day, for the entire lifetime of the array. That's about $350 per day in replacement parts alone!

Good math!

However, for a cool $4mil, hopefully this would include some kind of drive replacement program. With those raw numbers, a drive a day would cost on average of $127,750/year. Not including the power and the goon to replace the drives.

Who knows? I just retired my NetApp because $50k for a nonexpandable storage system that is less than 1/2 TB is stupid for me to justify. I would be more than happy with a petabox.

Re:Failure rate

[Faltargan2006]

Shhhh!......(job security!)....shh....

Re:Failure rate

[improbable]

I saw a couple of reps from Google talk a few weeks ago, and they said in response to a question that they get 2 hardware failures per day at some of their data centers.

Of course, they're Google.

Re:Failure rate

[Anonymous Coward]

If you assume MTBF of 500,000 hours, you get an average of one failure every 500,000 hours / 2400 drives / 24 hours/day = ~8.7 days. Keep in mind that you're probably paying $800-$1600/drive (EMC drives ain't cheap), so it's closer to $100-200/day for maintenance. Now, that may be included in the price, it may be included in a service contract, or it may be paid out-of-pocket.

Close to 50 years ago, you would pay $35,000/year for a 5MB disk drive (the first hard drive ever, IBM's 5MB 305 RAMAC). 20 years ago you were lucky to even be able to pick up a 1GB drive.

dom

Re:Failure rate

[TopSpin]

Large data centers often have far more than 2400 operational disks. Under these conditions, at any given moment, some fraction of all storage has faulted and repair activity is continuous. This is one reason SCSI hardware is preferred: the disks are more uniform (capacity, electrical interface, etc.,) and replacements remain available over longer intervals.

This isn't the slightest bit unusual. At any moment some fraction of the power transmission and distribution system has faulted. Some percentage of all aircraft are grounded. Various segments of all wide area communications systems are down. Repairs never cease.

$350 equates to a few minutes of aggregate labor costs spent financing, provisioning, securing and monitoring a petabyte of storage. Other large ongoing costs include power and cooling. $350/day is lost in the noise.

EMC's new offering will reduce many of these costs for a given amount of storage. The thing to do then is build data centers to host these machines by the dozen.

Re:Failure rate

[TallMatthew]

I worked for a search engine in the Bay Area (not Google) which employed about 200-300 database servers, each stocked with 3 hard drives (one root, one mirrored set for the database files), the mirrored disks pegged with I/O.

We replaced 4-5 a week, on average.

Re:Failure rate

[nigelc]

And having worked at a company which had one of the smaller EMC storage arrays, this pretty much happened weekly. Got to the point that the technician was stopping by our office every couple of days!

T'was claimed that they'd had a bad batch of drives from IBM :-)

Worst part was, as the company was winding down, we had some kind of problem with the "phone-home" logic on the storage array. So every ten minutes, you'd hear the very loud and anachronistic sound of a modem dialing out and trying to warble a connection.

Re:A petabyte of pr0n

[Anonymous Coward]

"enough for a TV-quality video of your entire life".

I watch my pr0n in HD ;)

Re:fp!

[Mahou]

redundant array of independent disks array! redundant indeed. can you use this if you have a nic card? oh, and don't forget to keep your pin number and ssn number safe!

Re:A petabyte of pr0n

[joNDoty]

this is enough for twice the pr0n you could possibly watch (assuming you sleep and eat)

Never heard of TV dinners?

Re:A petabyte of pr0n

[Leroy_Brown242]

1 Petabyte is enough pr0n?

For who?

You're new here, aren't you? :)

Look at the Engadget link

[mentaldrano]

I know some people need a lot of storage, but c'mon:

http://www.engadget.com/2006/01/30/emc-rolls-out-4 -million-petabyte-array/

Four million petabytes?

Anyway, anyone else remember Arthur C. Clarke's "3001 The Final Odyssey"? By the time you start talking about petabytes, we could think about storing not just great works of art, but also the artist who created them. Let's see, some math is in order:

Fat artist: 75kg = 75000 g. Since we're 70% water, we'll say the average molar mass is 20g, so that's 3750 moles of molecules. Multiply by Avogadro's number and we get 2.2575 x 10^27 molecules. One petabyte is 10^15 bytes, so we need another 12 orders of magnitude before we can even devote 1 byte per molecule.

Even millions of petabytes leaves us short. Yawn, wake me up when we have 4 million zettabyte drives. Of course, by then the drive manufacturers will have redefined the prefixes again...

Re:Look at the Engadget link

[mentaldrano]

Sorry for replying to my own comment, but it seems that the wikipedia article for yottabyte beat me to the punch.

http://en.wikipedia.org/wiki/Yottabyte

Re:Look at the Engadget link

[MobileTatsu-NJG]

"Four million petabytes?"

No. $Four million for a pedabyte. Read the title, not the URL. Don't be ignorant of the bling, foo.

Re:Look at the Engadget link

[Professr3]

I say, rip meh to DIVX!!!

Re:Look at the Engadget link

[mfukar]

I believe we can all see now, the importance of DNA. And brains.

Re:A petabyte of pr0n

[Compuser]

I pity you.

true

[xusr]

It's amazing how quickly storage increases and prices go down. On the other hand, it's interesting to keep in mind that as amazing as an iPod nano would be in 1985, the invention of paper was the single biggest leap in storage density we've ever seen.

Re:true

[mrchaotica]

...the invention of paper was the single biggest leap in storage density we've ever seen.

Oh, come on -- dividing by zero doesn't count!

(d/dx density = new medium/old medium;
paper/nothing = n/0)

Re:true

[name773]

your mind's storage density is greater than paper

Re:true

[Anonymous Coward]

You're implying that our mind was "invented", which I don't mind, cause I think we're just part of god's game of pool involving reality.

Re:true

[pizza_milkshake]

What, you've never heard of stone tablets? They last FOREVER, but the write latency is a killer.

Re:true

[rikkards]

your mind's storage density is greater than paper

Yeah but the transfer from reality to memory is inconsistent from one person to another. Need some ECC on it.

You're posting on Slashdot...

[mrchaotica]

...my entire life. that sure is a lot of porn.

...so no, it's not. : P

Re:You're posting on Slashdot...

[stuffman64]

No, it's how much porn he's watching, not making.

Storage Limits

[Doc Ruby]

Each of your eyes has about 8Kx6K retinal detector cells, signalling at 40Hz. Nyquist sampling means we'd need 16Kx12K 24bit pixels to fill them at 40fps, or 368Gbps. Two ears at even 1Mbps falls into the rounding error, but 10:1 compression is a reasonable minimum expectation: 3.7Gbps. 100 years is 1450PB . The average American earns about $40K:y for 40 years, or $1600K. That means when we get $1:PB, we'll be able to afford to store everything we see and hear in our entire lives.

25 years ago, 10MB cost $5K to store, or $1:2KB. Today $1 stores 1GB, a 500K-fold increase. The 1500-fold increase to $1:PB is relatively around the corner.

Re:Storage Limits

[Anonymous Coward]

Each of your eyes has about 8Kx6K retinal detector cells, signalling at 40Hz.

Our eyes are 48 megapixels?! There are only about a million nerve fibers running from the eye to the brain. Near the center of the eye, there is a 1:1 matching between nerve fibers and photoreceptors but towards the peripheries each nerve fiber connects to many photoreceptors [indirectly via a great deal of retinal circuitry]. Optical blurring means that the resolution of the eye is not limited by the photoreceptor density, that is probably so high to increase sensitivity and perhaps to improve colour vision.

Nyquist sampling means we'd need 16Kx12K 24bit pixels to fill them at 40fps,

I'm not sure whether your 24 bits are to account for colour or dynamic range. The visual system adapts so that you can detect a few photons in the dark or see clearly in bright sunlight but at any given time you probably only distinguish between 1024-2048 levels (10-12 bits).

The 40fps is a bit hazy too. Two pathways run from the eye to the brain - the 'magnocellular pathway' is made of large nerves that transmit a lot of information - primarily motion information. The 'parvocellular' pathway transmits colour and fine form information relatively slowly - probably only a few fps. Chromatic information is pretty heavily compressed - you can't see fine detail that is defined by colour alone.

or 368Gbps. Two ears at even 1Mbps falls into the rounding error, but 10:1 compression is a reasonable minimum expectation: 3.7Gbps. 100 years is 1450PB . The average American earns about $40K:y for 40 years, or $1600K. That means when we get $1:PB, we'll be able to afford to store everything we see and hear in our entire lives.

25 years ago, 10MB cost $5K to store, or $1:2KB. Today $1 stores 1GB, a 500K-fold increase. The 1500-fold increase to $1:PB is relatively around the corner.

In summary, biology!=computer science. Our senses don't take in more than a few megabytes per second, so you're out by a factor of a few hundred thousand. You also forgot to take into account the fact that we sleep about 1/3 of the time.

Re:Storage Limits

[Doc Ruby]

All of my estimates are overestimates, as I'm looking for the worst case of affordable storage. But there are over 130M retinal receptors in each eye. There is less than 1 fiber per 100 receptors from the eye to the brain, but I'm not talking about recording the optic nerve - I'm talking about recording the images received by the eye. The 40Hz rate is derived from the optic nerve, but it's an oversimplification of the async signalling of images by the retina. And your point about sleeping might contribute to my 10x compression becoming 30x, though I'd like to record what I would see if I weren't sleeping.

Computer science and biology have much different analytical techniques. But I started working in imaging engineering over a decade and a half ago. Engineers developing devices to augment or replace biology can look at upper limits to necessary performance, and then optimize within those limits. The upper limit I proposed makes "total recall" look feasible soon. Your further reductions make it look even more feasible. The Wikipedia page indicates that both eyes generate 1Mbps, or under 400TB raw. That means the storage is already "affordable", though we'd need more than retinal projectors - we'd need a nerve interface. But any way we look at it, we've nearly reached the amount of storage that is "enough for anybody".

Re:Storage Limits

[Anonymous Coward]

Each of your eyes has about 8Kx6K retinal detector cells, signalling at 40Hz. Nyquist sampling means we'd need 16Kx12K 24bit pixels to fill them at 40fps, or 368Gbps.

Just a little pitnicking here: the human eye has a far greater dynamic range than eight bits, and the CiE color gamut representing the human eye's spectral reponse exceeds RGB by quite a bit, most notably the blue-green "hole".

So, add three more primaries for the full CiE gamut, and 64 bits per channel of floating point data for dynamic range, and we get a number that is 2x 8x = 16x your number, which is 5.888 Tbps.

Re:Storage Limits

[Doc Ruby]

"Pickback": The 24bit pixels I mentioned are sufficient to cover the eye's dynamic range. They're not RGB by any means - the eye's gamut is mostly green, and only slightly blue->violet. But I haven't seen any evidence for 64bits of range in color info - even our 40bits tests on CiE (and other colorspaces) demonstrated substantial overkill.

Re:Storage Limits

[Anonymous Coward]

The unique thing about the optical nerve compared to other tracts in the brain, is that it's one-way. There are no axons that transmit information from the brain back to the retina. This means that if you could process the retinal image to the format in which signals are sent down the optic nerve, this would be entirely sufficient to evoke an identical response in the brain. So there'd be very little point in recording all 130 million cell responses - most of that data gets thrown away very early on - even before the first synapse. Many of those rod and cone cells are actually syncytia - cells that are connected electrically and are therefore functionally a single unit.

When you read a book that you're holding, you compensate for the shaking and it seems steady. When you read a book that someone else is holding, it often seems to be shaking all the time.

In the same way if you could simply record the signal from your retina then replay it to your brain later, it probably wouldn't be a very comfortable or illuminating experience. 'Seeing' is not a photographic process, but involves directing your eye movements towards whatever part of the world currently has your attention. Without being able to recreate your eye movements exactly and in synchrony with the retinal video, you would only perceive a jerky nonsensical series of snapshots of the world.

Rather than asking 'how can I record something equivalent to the retinal image' the question should be 'how can I record an image sufficient to allow me to scrutinise the environment I was in to the same level of detail as before.' If you took a high-resolution video through a fisheye lens then the files would be simply huge - far bigger even than your estimates. If you linked a camera to an eye tracker with an image stabiliser, you might get something a bit more reasonable.

By the way, each photoreceptor records only one channel (RGB/LMS in biospeak) so you would not need to multiply your 130e6 by three. You would only need 10-12 bits to record the full dynamic range of the photoreceptor, provided you could also account for global adaptation + to a lesser extent local adaptation on the retina.

It would be perfectly possible to make a camera with the same gamut as the retina - in fact I have done so. The retina records three channels and so long as the spectral sensitivities of your camera elements are identical to those of the LM and S photoreceptors in the eye, the chromatic information you record will be identical. The problem is one of display gamut. You wouldn't need any extra information to drive a display with a larger gamut and it would be quite easy to make one (an Israeli company has already done this).

Re:Storage Limits

[Anonymous Coward]

It would be perfectly possible to make a camera with the same gamut as the retina - in fact I have done so. The retina records three channels and so long as the spectral sensitivities of your camera elements are identical to those of the LM and S photoreceptors in the eye, the chromatic information you record will be identical. If your receptor sensitivities are different, e.g. RGB, you will record an image that has the same amount of information, but this information will be slightly different.

The problem is one of display gamut. You wouldn't need any extra information to drive a display with a larger gamut and it would be quite easy to make one (an Israeli company has already done this).

Re:Storage Limits

[Doc Ruby]

Of course, digital images can be jittered into a targeted retina. With realtime feedback from a retina position sensor. But there are bigger problems.

When we were making digital cameras for publishing (and I was mastering colorspaces, even working with the JPEGroup), we worked with a RG/YB when compensating for resolution (4Kx3K) with gamut dynamic range (40bits). We also subsampled, stepping the 512x512 video sensor 8x6 times across half-pixel for a single scan.

We found the most obvious evidence of the process, vs photography or even drumscanners, was in the "grids". The sensor grid of photodiodes, and its stepping pattern, was too regular, and its rectangular grid was perceptible even in film prints from rastered analog CRTs. The retina is more circular, more stochastic, and detects the grid. Moreover, video frames are detectable per se by the retina firing without a lockstep clock.

I think replicating the retinal image's total light modulation characteristics will require clockless sensors, distributed perhaps stochastically, fractally, or somehow more organically than a grid. And almost as importantly, the projector which transmits light to the retina on playback as well.

Given those challenges at the cutting edge of digital design, perhaps the better strategy is modulating at the optic nerve, rather than via the retina. nerve connections are even more primitive, but the time it takes to complete them might be comparable to the extra time to complete photo techniques. And much more broadly productive engineering for many applications.

we will never ...

[Anonymous Coward]

We will never need one petabyte of storage.
and then 10 years down the lane
We will never need one exabyte of storage...

Re:A petabyte of pr0n

[MobileTatsu-NJG]

"Cue the 'finally enough storage space for my pr0n' jokes."

Yeah, THIS is the comment in this whole thread that should be modded redundant. Oh well, at least 'overlords' hasn't been modded up yet.

Re:A petabyte of pr0n

[imboboage0]

What's funniest about this comment: It was modded redundant.

Not Such Good Math

[HUADPE]

The likelihood of failure on any given day is not independent of any other day. Chance a drive will fail on the second day is pretty low (first day is pretty high, installation mess-ups and the like). Chances increase over time, so maintenance will follow a pattern of exponential increase. And in 6.5 years, I can have a 1PB raid under my desk at home.

Re:Not Such Good Math

[hackstraw]

Bathtub model.

'nuff said.

When will a petabyte hard drive arrive here?

[rolfwind]

The thing is built around 2,400 500GB hard drives.

I wonder when (if) the average consumer can get 1PB harddrives?

I don't know if Moores law applies historically to harddrives, but if doubling of capacity occured every 18 months and figuring 500GB is the limit size now and the doubling continues into the future:

500GB - Now
1TB - 18 months
2 - 36
4 - 54
8 - 72
16 - 90
32 - 108
64 - 126
128 - 144
256 - 162
512 - 180
1024TB = 1PB - 198months which is 16.5 years.

Re:When will a petabyte hard drive arrive here?

[Knetzar]

According to the paper High Density Hard Disk Drive Trends in the USA, hard drive density has doubled every 12 months.

500GB - Now
1TB - 1 year
2 - 2
4 - 3
8 - 4
16 - 5
32 - 6
64 - 7
128 - 8
256 - 9
512 - 10
1024TB = 1PB - 11 years, Assuming that ariel density continues to double and the form factor stays the same.

Re:When will a petabyte hard drive arrive here?

[WuphonsReach]

Unfortunately, that paper only covers 1998-2000.

A few data points from that paper...

1998.08 - 12Gb/sq in
1999.08 - 26.5Gb/sq in
2000.10 - 60Gb/sq in
2002 - projected at 75Gbits/sq in

1 (square inch) = 6.4516 square centimeters

The Hitachi 7K400 series is only 62Gb/sq in. That's the 400GB drive that came out last fall (Sep 2005?). GMR was rumored to top out at around 80-100Gb/sq in, IIRC.

Perpendicular recording is supposed to give us up to 230 Gb/sq in or up to 245 Gb/sq in, depending on who you talk to. Seagate demonstrated 100Gb/sq in back in 2001-2002 (perpendicular recording technology) and have just started shipping drives with 100-130Gb/sq-in.

So, PR is roughly doubling the areal density over GMR out of the gate with the potential of 4x to 5x more density then GMR. That puts the limits at around 2TB for a single 3.5" drive and maybe 500GB for a 2.5" notebook drive.

Now that PR drives are finally shipping, I think we'll see them ratchet up the densities about as fast as GMR did. (Layman's opinion.) So there's a strong possibility that we'll see those 2TB 3.5" drives in a few years (2008?).

After that, they'll either have to improve PR more (unlikely?) or come up with the next big idea of how to pack more bits per square inch.

Re:When will a petabyte hard drive arrive here?

[Anonymous Coward]

Why would you want to settle for a single PB drive, when you could get a 2.4EB (Exabytes) RAID array using 2,400 of those drives?

Wait wait wait... in 3001: The Final Oddissy....

[modmans2ndcoming]

We were JUST suppose to have gotten to petabyte storage.... that is 1000 years from now... at this rate, by the end of the century we will have 100 petabyte drives (which in reality are only 1000 terabytes rather than 1024 terabytes because drive makers cheat you)

mud

[Anonymous Coward]

try your hand at cuniform, written with mud and a stick, or etch some runes with a bit of rock

I need to create my own business

[this great guy]

Damn I really need to create my own business in the storage market. I am not exactly sure about what EMC provides in this $4 million package (servers, 24x7 contract, maintenance, hard disk replament, etc) but I KNOW how to create a 1 PB storage device for less than half the price ($2 million instead of $4 million). And I am pretty sure about my numbers...

I am sick of the current state of the storage market. Vendors are either designing unnecessarily expensive solutions, or are having HUGE margins...

Re:I need to create my own business

[Wanon]

Does that include the redundant disks, maintenance (including personnel), hardware to connect and run them, supply power and software development to run and maintain them?

This isn't just a heap of hdds.

PS. Contrary to popular belief the low end (average consumer) hdd market has very low margins.

Re:I need to create my own business

[JustASlashDotGuy]

Go for it.

With EMC, you do pay a lot for the name, but you also get a lot for
the price. We have an EMC at our data center that most of our
servers run on. We also have a test 'SataBoy' (soon to be replaced
by the SataBeast) hooked up to another server that we use for
replication and a Disk-2-Disk-2-Tape solution. The Sataboy is much
cheaper then the equivilant space on a sataboy, but you do lose a lot
of features.

If you could create a product similar to what EMC offers at half the
price, then we'd be interested. I don't see it happeing tho.. well,
not unless you happen to work for NetApp, heh.

Oh.. and it's not a big deal if the big *ss EMC has one of it's 2,400
drive failing fairly often. The EMC 'phones home' on it's own to
report the bad drive and dispatch a tech to swap the drive out for
you. It's not like you have to check all 2,400 drives yourself.

Re:I need to create my own business

[this great guy]

Yes, my $2 million USD price would include redundant disks, servers, power supplies, enclosures, disk controllers, network gear, maintenance contract with on-site technician to replace failing parts, etc. Everything would be pretty much redundant. The only thing is, the initial investment necessary to start my business (and validate my ideas) would be high: $3 million USD (first product + initial development cost + safety margin). And of course I am pretty sure that if my idea pulls off, EMC would immediately try to kill my business by slashing their prices because I know they know their prices already include large margins...

Re:I need to create my own business

[this great guy]

No I don't work for NetApp. Could you be more specific about what would interest you ? You would buy 1000 TB for $2m USD. What about 500 TB for $1.2m USD ? 250 TB for $700k USD ? Would you like such a product to appear as a single file server on your network ? Or, say 10 or 100 independent fileservers ? What about the protocol NFS, FTP, TFTP, CIFS, other ? Do you have any power requirements (the whole thing must be

Re:I need to create my own business

[this great guy]

(rest of the post)
...the whole thing must be < 50 kW) ? That's it, no more questions.

Re:I need to create my own business

[alienmole]

The truth is that if you're serious, and you get the funding you'll need to set up a business that'll be taken seriously by the kind of companies that buy stuff like this, you could do well. It's not as if companies like EMC or NetApp are unassailable, in theory. However, all those questions you're asking: they already know the answer to them, know what the market will bear, have the infrastructure in place to sell and support them, and so on and so on. It sounds as though you're looking at, essentially, the cost of parts, but that's not really what businesses care about. They don't want someone to dump a bunch of parts on their doorstep, they want something that they know they can rely on, and that will be maintained for them. They know that the premium over the cost of parts that they pay is generating profits for the company they're paying them to, but they don't mind that, because they get what they need in return. You can make some of those profits too, but you won't do it just by being a glorified drop-ship artist.

Re:I need to create my own business

[Anonymous Coward]

and so do you have your own software to run the controllers that will allow someone to manage the storage, creating virtualized disks of different sizes to present to your servers ?

riiiight.

Re:I need to create my own business

[this great guy]

To answer you question: of course no, I am not just looking at the cost of parts. I know that vendors HAVE to provide all the usual services such as maintenance, remote monitoring, parts replacement, support, help, fine-tuning, installation, etc.

Thanks for your input, guys. I am pretty confident in my approach, and I am going to seriously think about it :)

Re:I need to create my own business

[this great guy]

Yes. And I am dead serious: mostly open source apps + in-house software for the missing parts (I know exactly which ones are missing).

I'll say it

[Profane MuthaFucka]

A petabyte ought to be enough for anybody. And I mean it this time.

Re:I'll say it

[Anonymous Coward]

Well, does 1PB (compared to 16 GB - a quite small harddisk these days) sound any more preposterous today, than 1GB of RAM sounded in the days of 16K computers?

I don't see any reason that 1PB disks wouldn't be common. Clearly it's desirable to store full length, full resolution movies on your disks. How much space does one of those take?

Say its 3 hours, and say the depth is 64 bits and the resolution 2048 x 1200. That's 8 * 2048 * 1200 = 19MB per frame. Now let's multiply with the number of frames per second: 47MB. There are 60 seconds in a minute, and 60 minutes in an hour: 169TB. There are three hours per movie: 509 TB.

Store a couple of those and you are quickly running out of petabytes.

Mean Time Between Failures

[tewner]

I talked to someone who set up a Petabyte array many years ago (he had a *lot* of funding). It seems that when you put tens of thousands of disks together, your total mean time between failures becomes, well, for all those disks, every few minutes. They had someone full time whos job was to go around replacing disks all day.

Petabyte drives...

[jd]

It really depends and Moore's Law doesn't really apply to it. The jumps tend to be much larger and much more random. The problem is that capacity is limited by several factors: drive speed, disk rigidity, read/write-head speed and the distance the read head is from the disk surface.

The faster a disk spins, the more disk surface is exposed to the magnetic field used to write to the drive, so the less storage you have. Disk rigidity is important for two reasons - it limits how close the read head can get and it limits how precisely you can know how much disk surface has been visible. The faster you can either read magnetic fields or generate them, the less disk you need to write to, thus increasing storage. The distance of the read head determines the surface area exposed to the magnetic field on writing, so determines how far apart your data must be to not overlap.

A trivial question might be: Using a standard, existing hard disk (but modifying the controller as necessary) increase the capacity of a hard drive? The answer is "probably".

One way to do it would be to add enough RAM such that a fairly substantial portion of the disk can be held in ramdisk on the controller. Because you are then not reading and writing to the disk directly, but going through ramdisk, the speed of the drive becomes much less important. If you slow the drive down substantially, whilst writing to it at the same speed, the data won't be smeared over the disk as much, so you should be able to increase the density.

In practice, as disk manufacturers don't design their disks with that kind of mod in mind, you are very likely to run into significant problems with defects on the surface that simply aren't visible at 7200 or 15000 RPM. Other problems, such as stability (drives depend a lot on gyroscopic effects and aren't built to go slow), may also limit how much you can cheat on the density.

Another option would be to seriously cool the read/write head, so that you could flip the magnetic state faster. Again, you're limited. Mechanical devices don't like being freeze-dried - even when they ARE dry. However, you may be able to get some improvement that way.

If you're just looking for ANY increase in capacity, then that's trivial and requires no engineering (but some programming). Modern computers are very fast, compared to modern hard drives. If you have one physical sector per physical track, then break down the structure entirely in memory, you eliminate the need for inter-sector gaps, physical sector headers, etc. You might be able to squeeze out another 10%-15% by this method, which isn't a whole lot but isn't bad for the effort it would take.

There are very likely other mods that hard disk manufacturers could use, but which would be totally beyond anyone doing homebrew stuff. The platters probably aren't using the absolute ideal materials - let's face it, they're in business to make money and there are far more home buyers wanting cheap drives than there are perfectionists wanting perfect drives. I suspect there are other areas they could improve on, using existing technology, but won't because it's not economic.

That's probably why you see bursts of improvement. When there's a massive enough need for the extra storage, it can be achieved. When there isn't, it's not worth the extra investment.

Re:Petabyte drives...

[mennucc1]

The faster a disk spins, the more disk surface is exposed to the magnetic field used to write to the drive, so the less storage you have.
You got it completely wrong here (and someone even modded this "5 informative"!!).

From actionfront.com: To get more data on a track, the spacing between each bit in the down-track direction must decrease. The data density in this direction, also called the linear density, is measured in thousands of bits per inch (kbpi). Similarly, the track density across the disk is measured in thousands of tracks per inch (ktpi). Areal density is the metric used to quantify the impressive growth in HDD data storage capacity. It is the product of bpi and tpi, which reflects the amount of user data that can be stored reliably in one square unit of area on the disk surface. It is now measured in gigabits per square inch (Gb/in2). For decades, the compound annual growth rate (CAGR) of areal density was about 30%. With the introduction of MR (magneto-resistance) head technology around 1990-1991, the rate increased to 60%. When GMR (giant magnet-resistance) heads were introduced in the late 90's, the CAGR temporarily increased to over 100%, during which time there was an increase in the number of companies that exited the industry or merged. The pace of areal density growth is now slowing and should settle somewhere between the historical rates of 30 and 60%.

See also Kryder's law: Kryder's Law is essentially Moore's Law for storage. But the density of information on hard drives has been growing at an even faster rate, increasing by a factor of 1000 in 10.5 years, which corresponds to a doubling roughly every 13 months.

As you see, nowhere does it say "disk speed". Truth is, as area density increased, disk speed increased as well (from 5000 to 7500 to 10000). And at the same time, data transfer thru the disk head increased as well (math teaches that... if bits per inch increase, and rotation speed increase, then data transfer must increase as well).

Re:Petabyte drives...

[Peter La Casse]

As you see, nowhere does it say "disk speed".

That's absolutely correct, which causes me to wonder what it has to do with the post you respond to. Quoting information that does not address the relationship between rotational speed and data density does not say anything about the previous poster's claim. *How* were they incorrect to assert that it's harder to make the largest drives as fast as smaller drives? I don't deny that the reason might be different than the one stated, but simply asserting so doesn't do anything for us.

If it is, in fact, just as easy to make large drives as fast as small drives, then where are the 15k rpm 300+ GB drives?

Re:Digging deep for this one...

[TheScottishGuy]

do you work for dreamhost? i just checked them out and am considering hosting with them, they seem pretty sweet.

Re:Warrantless wiretaps

[luvirini]

They allready can, things like this will just save them money as they need to buy less of these than the old style things. $4million is really small money when it comes to important goverment functions.

Yattabyte

[Anonymous Coward]

I disagree. I bet you could fully encode 4 Japanese men, although you probably wouldn't have enough space for all their clothing.

Re:Wait wait wait... in 3001: The Final Oddissy...

[gbobeck]

Of course, 3001 was assuming that it would take a long time for a filesystem to be developed that could handle the 1 petabyte drives. This timeframe has since been sped up due to the fact Microsoft has announced that Vista will ship soon and has promised WinFS to be released by 3001.

Density?

[Anonymous Coward]

Paper is better than stone or clay tablets, but is really not that much more dense (information-wise) than parchment or papyrus that came before it. I don't really think it's such a leap, unless you are talking about the cost/volume of information. Certainly once paper became mass-produced it was cheaper than anything else, but that was well after its invention.

dom

EMC just a badge

[Exter-C]

After the experiences that I have had with EMC gear in several different companies around the world I am suprised that anyone would still use EMC junk. Ohhh. but I guess EMC would support someone who is paying $4million for their storage gear rather than someone that only paid $0.5million... Since the company has started to use alternative vendors for storage we have had fewer problems, with higher performance and better support at a much cheaper price. And what is more interesting is that the EMC story I am talking about is a very common story from what I have seen. I would be interested to hear other peoples storage issues with EMC if they are not a Top100/500 company?

Re:EMC just a badge

[Anonymous Coward]

No where near a top 100/500 company but have been running EMC gear for 5+ years, and what I've found is that most downtime and problems have occured from not enough training. Self knowledge is key, you should know the equipment; sure you paid a nice maintenance contract to EMC to manage a black box, but that doesn't mean you should sit on your hands.

Either you took a sales guy pitch hook, line, and sinker and was sold inappropriate technology (i.e. ata drives or not enough spindles: I can support sustained 80+ MB/sec writes onto 5400rpm ATA drives but it was very precisely engineered) or you have improper configuration; either of which would be fixed from a knowledge perspective.

Filed under: Peripherals

[drgonzo59]

I like the section heading: Peripherals

With a beast like this that fills up a whole room, anything else becomes a peripheral....

TV Station

[Anonymous Coward]

I recently began working at a TV Station (although not directly for them), and from what I've heard (not confirmed by an engineer though), they have at least a petabyte of storage.

It's not that surprising if you think about it... digital video takes a lot of space, and this is broadcast quality, some of it even high-definition.

Of course, it has been nicknamed "The Peta-file" ;-)

Apple XServe

[thaddjuice]

Is this really a cheap solution? I could get 145 7TB XServe RAIDs plus XServe G5 cluster nodes with XSan for about $2.3 million. Throw in a couple hundred $K for racks, etc. and I'm at around $3 million. What does this get me over an Apple (or other) solution?

Re:Apple XServe

[vidarh]

It gets you a unified system designed to work as a cohesive hole, and that is easy to maintain and operate.

That's generally what you pay a fortune for when you buy these big beasts.

It all boils down to what is most important for you - the money or the hassle of managing less integrated systems. The big filers are by no means the right choice for everybody, but they are nice to work with if you can justify the cost.

Features in high end storage systems like this typically include things like redundant-everything (multiple controllers with automatic failover, multiple sets of write cache, RAID, multiple power supplies), up to and including systems where you can pull out entire packs of drives while the system is running without noticing more than a reduced IO rate.

Many of them also have extensive built in health checks, and some will "call home" and the first you might know about a potential problem may be the engineer showing up at your office to fix it before it does become a problem (of course, you pay accordingly....)

Other features usually involve snapshot support (get a second virtual "drive" that is "frozen" at a point in time - makes doing backups a breeze because you can quiet database updates etc., make a snapshot, and then go on with your business and not have to deal with complex hot backup solutions), and often remote synchronisation (get a second box at a second location, put up a fibre link between them, and let the boxes handle the rest)

Of course you can do most/all of this with cheaper hardware too, but then you have to build it yourself. If you're, for instance a bank, and are dealing with huge sums of money, it's often far easier to buy stuff like this and pay for the maintenance contracts and just not have to deal with it any more.

For mere mortals they are usually just outrageously overprised compared to the features we actually need.

Though I must say I have always had a weakness for hardware that comes with cases big enough to live in and requires forklifts to move... :)

Re:Apple XServe

[rah1420]

Though I must say I have always had a weakness for hardware that comes with cases big enough to live in and requires forklifts to move... :)

Never trust a computer you can lift, I always say.

Whoa ho ho!

[Ayanami Rei]

Chip.
On.
Shoulder.

But interesting none-the-less. Too bad it got modded into oblivion. Some of these "losers" could have learned a thing or two about industry. But then, you had to get all offensive.

You have to be _subtle_ with your barbs, so it flies over the heads of the moderators with PMS.

Re:A petabyte of pr0n

[Anonymous Coward]

You forgot the ^_^, bone smuggler.

Re:There is an explanation for this

[vertinox]

I was just thinking about how 4 years ago you could build a terabyte array for about $5-10,000 down from many millions 8 years ago. Today, you can get a terabyte for less than $500. In a few years, a petabyte is only going to cost $5,000.

Law of Accelerating Returns

Just don't name it...

[MadCow42]

Just don't name it the Peta-file. :)

Rumor has it that Sony tried to use such a name for a tape system several years ago, until the North American team heard it. Not sure it that's true, even though it came from a friend that worked in Sony marketing in Canada at the time.

MadCow

Has anyone noticed

[xmedar]

That it says you can get 480TB for $250K, so two of those would give you 0.96PB for $500K rather than spending 8 times that to get the other 0.04PB?

Re:I'm waiting for someone to mod this...

[smbarbour]

The score is already -1, so I don't need to mod it, but there isn't an option for "bad pun" anyway.

Use hard links, rsync, big redundant disk array.

[Medievalist]

I keep a lot more than 50 days worth on line. And I get effectively more than 90% compression. And individual users can do their own restores from their own desktops.

Look at how dirvish works. Or rsnap, or rsync-incr, or rsnapshot, or ribs-backup, or indeed any tool based on Mike Rubel's basic idea.

I use a homebrew variation that is suited to my employer's unique needs and infrastructure. You may find it expedient to do the same. I don't save any metadata other than the snapshot date for each tree, and I use data mining techniques (well, actually I use find and gawk from command line) if I want to determine what's going on or how the system is doing.

It has run for years with no maintenance other than periodic OS patches. It is not our primary backup system because it does not support off-site archival, but it's well worth the investment for rapid restore of user-deleted files. I'll consider this array (I'm currently using linux soft raid 1+0 on two physically separate busses) when I need more disk eventually.

Name for device

[r00tyroot]

They should call it the "Petafile". that is a nice, innocuous brand-name, isn't it?

Much better drives means lower failure rates

[Terje Mathisen]

The Internet Archive Project http://www.archive.org/ is running on the PetaBox http://petabox.com/ rack system, which was commercialized by Capricorn Tech http://www.capricorn-tech.com/ more than a year ago.

This system uses absolutely no board/controller lever redundancy, instead they use a separate file system on every disk, then mirror pairs of 1U units, and finally mirror the entire (mirrored) rack to a geographically distant location.

I am currently testing a much denser solution, the SATABeast http://nexsan.com/products/products/satabeast/sata beast.html from nexsan http://nexsan.com/ which manages to pack 42 500 GB SATA drives into a single 4U rackmount box. With multiple RAID5 volumes and shared hot spare drives, this results in about 17-18 TB of usable file system space.

According to the nexsan engineer I spoke with today, they do so much burn-in testing of the Hitachi Deskstar drives they ship, that over the 15-18 month period they've used these drives, the total error rate has been just 0.4%.

Even if these numbers are somewhat skewed due to many systems (i.e. drives) being relatively recently installed, it is still very impressive.

For our setup we plan to use multiple full boxes, each connected to a separate NFS server. Each server has multiple FC host adapters, so if a server crashes, the corresponding box can be connected to one of the other servers.

We will also use rsync to mirror all data across the country to a secondary site.

Terje

Re:Wait wait wait... in 3001: The Final Oddissy...

[Anonymous Coward]

Very funny :-)

But the parent point still stands, all because of a pesky little OS called Solaris and that danged ZFS it has...

Problems with EMC ?

[Anonymous Coward]

Where we work, the IT department has switched to EMC for all the major storage hooked to our UNIX systems, and we are talking hundreds of terabytes. They brag about how EMC is redundant, and doesn't fail. Every 6 months or so we have serious problem with these "redundant never fail" systems that causes loss of work for a day or more for large portions of the company. We pay big bucks for these EMC systems, and removed other brands that never gave us any problems. Have other folks had these problems with EMC?

WOW

[PCCybertek]

Imagine how much the RIAA would want if you filled it with MP3's!!!!!!!

password hashes?

[kipple]

this should be enough to store hashes of all possible 8-charachters password for 92 keys. More or less, I mean.
Man, thank god that windows has 256-chars password length

(it's all fiction, I made the numbers up. but I'm pretty sure about the size of the hashes db..)

expensive?

[mattr]

4000 bucks per terabyte sounds a little pricey. Whatever happened to economy of scale? On the other hand I get $3900 bucks for the price after 10 generations of splitting the price by 2. So figure in 10-15 years you'll have that petabyte, but by then you'll be drooling over the sextabyte or whatever it's called. (insert puns here)