Facebook Puts 10,000 Blu-ray Discs In Low-Power Storage System

itwbennett writes "Facebook said last year that it was exploring Blu-ray for its data-center storage needs, and on Tuesday it showed a prototype system at the Open Compute Project summit meeting in San Jose, California. It designed the system to store data that hardly ever needs to be accessed, or for so-called 'cold storage' (think duplicates of users' photos and videos that it keeps for backup). The Blu-ray system reduces costs by 50% and energy use by 80% compared with its current cold-storage system, which uses hard disk drives, said Jay Parikh, Facebook's vice president of infrastructure engineering." It's a prototype, and they're also evaluating low power flash as another alternative to keeping seldom accessed data on hard drives.

Comments

Write once?

[WPIDalamar]

Anyone know if these burners are write-once drives?

If so, it pretty much guarantees that Facebook keeps a copy of your stuff forever, even if you "delete" it.

Re:Write once?

[tripleevenfall]

Doesn't Facebook have a right to control over their product? (you) ::ducks:::

Re:Write once?

[StripedCow]

When you delete your account, somebody will go and get the corresponding disk, copy it (except your data), and destroy the old disk.

It's write-once only if you don't consider "destroy" a write-operation.

Re:Write once?

[lagomorpha2]

If so, it pretty much guarantees that Facebook keeps a copy of your stuff forever, even if you "delete" it.

Facebook keeps a copy of your stuff forever, even if you "delete it". So does gmail/google. Even stuff you type into a textbox but never submit.

http://www.slate.com/articles/...

Come to think of it, "deleted data" is probably exactly what this cold storage is for. They never have to worry about overwriting it when users change the data because it's data the users have already "deleted".

Re:Write once?

[mlts]

Some technologies come full circle. I'm reminded of Kodak's optical storage technologies that stored 3-6 gigs on large (8 inch?) MO platters in a jukebox that had its built in "clean room". The advantage of this technology was the fact that once burned, the data was there forever, which was useful for long term archiving.

Rewritable MO disks came out in drive arrays after that, arrays that had the ability to flip disks, so it could read/write the both sides (300 megs per side.)

Optical tech ended up on the sidelines because tape got cheap, especially when DLT started having decent capacities and tapes with WORM capabilities hit the data centers.

Now, tape drives are very expensive, and require a LOT of I/O on the attached computer, or else they will shoe-shine themselves into oblivion.

In the past, optical burners had issues, buffer underrun was one of those. Now, with modern ones that just turn off the laser once the buffer empties and resume very close to where it left off once data starts arriving again.

With tape out of the price range, I have not understood why someone hasn't made a Blu-Ray autochanger. Sony has one, but it is a carousel unit made for playing. However, couple that with a BDXL drive, "flippy" disks that have two sides for twice the writability, and that would provide more than adequate storage on an archival basis for large volumes of data. Two autochangers will allow one to have the ability to move data offsite, and almost every backup program out there has some form of encryption on it.

I just don't see why this isn't done. Even a 5-10 Blu-Ray autochanger that used five disk caddies (so one could just load the pack, and then not have to touch the media after that) would be immensely useful for critical backups.

Re:Write once?

[Chris Mattern]

When you delete your account, somebody will go and get the corresponding disk, copy it (except your data), and destroy the old disk.

Except they won't. Facebook doesn't delete your data when you delete you account *now*, what makes you think they'll do it when it becomes this much harder?

Re:Write once?

[Grishnakh]

Huh? How would a 5-10 Blu-Ray autochanger be userul for backups? 10 BDs equals a paltry 360GB of data storage; that's only enough for 1/3 of a typical 1TB hard drive. BDs are nearly useless because they simply don't store much data; why FB is bothering with them, I have no idea. Optical discs have always been found to be pretty awful in terms of storage capacity and data integrity over time compared to tapes. The only problem tapes have is the drives are expensive, but large companies don't have a problem with spending $2k on a drive. On a per-GB basis, they're easily the cheapest thing out there.

Re:Write once?

[Guspaz]

How do you get 36GB per bluray disc? Commercially available BD-R discs come in a variety of capacities (25, 50, 100, 128) depending on number of layers and density, and 36GB isn't one of them.

10 BDs equals up to 1280GB of storage for quad-layer high density discs, although I can't find any of the 128GB discs for sale, only 100GB discs. Either way, the higher capacity discs are rather expensive, but if you're buying them in big enough bulk, it may not matter as much.

Re:Write once?

[Solandri]

There are larger capacity BDs, but they still don't amount to much storage. As others are speculating, this is probably a pre-emptive action by Facebook, so when they eventually get sued for not deleting someone's data, they can truthfully testify in court that it's logistically "too hard" to comply with a single user's delete request.

Re:Write once?

[nabsltd]

LTO6 is still pretty small at over 2TB per tape. No damn idea why Facebook wants to "stripe" that same dataset across 20+BDs.

T10000 tapes are the same size and store 5TB uncompressed, and cost no more than $300/each. That's about $0.06/GB, with Blu-Ray no less than $0.10/GB.

For storing less than 100TB, then hard drives plus infrastructure (drive bays, controllers, etc.) win on price/performance, and Blu-Ray or tape without auto-changer capabilities isn't far behind, while either with auto-changer is far more expensive. For storing thousands of terabytes, you're definitely going to need an auto-changer, so tape wins hands down.

Re:Write once?

[AmiMoJo]

BluRay discs designed for archival are actually thought to be very good for data storage. They are actually more like magneto-optical discs. A laser melts the plastic and a magnet aligns reflective particles within it, and then the plastic hardens again. It's pretty permanent if the disc is made of durable materials.

Re:Write once?

[kheldan]

I don't get why they'd use an opto-mechanical drive for this. Why not use SSD's instead? Yes I know TFA said they're looking into using "low power flash". SSD's can be completely powered down when not in use, are a COTS solution, would require little if any software development to use, and, in the case that someone else presented of a user deleting their account, the space used could be easily recovered to use for other data. I'd have to say that the Blu-ray idea is just a proof-of-concept more than a production-level solution.

Re:Write once?

[wiredlogic]

When I started at Kodak in 2000 they were ramping down development on their DOTS product for Digital Optical Tape Storage. It used optical film spooled in a cartridge in an automated mini-lab type machine to process data stored on the film with exposure from LEDs. Each cartridge was 1.2TB and has a 100 year shelf life. For long term, write-once archival storage it is very cost competitive against magnetic tape but they just gave up on it. The cool thing is that it looks like a company acquired the patents and is going to bring it to market after all these years.

Re:Write once?

[Anonymous Coward]

Perhaps their cold storage isn't as cold as you imagine. The optical disks are much better at random seeking to pull out the occasional data asset from the jukebox. You don't have to rewind and seek the 2TB tape to fetch a record in the middle...

Re:Write once?

[CTachyon]

Anyone know if these burners are write-once drives?

If so, it pretty much guarantees that Facebook keeps a copy of your stuff forever, even if you "delete" it.

Where I work, we use large-scale tape backup (complete with robots), but tapes are so crappy that you basically have to treat them as write-once media anyway, so you have the same problems. (And tape drives are a consumable, but that's another story.) We solved this by encrypting each backup batch with a unique symmetric crypto key, and when a backup expires a cron job throws away the crypto key and marks the batch as "deleted" in our tape index. If all the batches present on a given tape end up deleted, only then do we bother to recall the physical tape from off-site storage and throw it out.

Has the bonus that we don't have to trust the security of our off-site storage provider.

Re:Write once?

[mlts]

If HVD actually made it into mass production, it would be a game-changer. However, it would have to be a different diameter than CDs/DVDs/BD media... that or else be able to read the previous media types.

I remember when Tamarak out of Austin was promising a similar product, and they died with not even a whimper in the early 1990s.

Wonder what keeps holographic tech from hitting the market.

Tape?

[jythie]

I guess tape just isn't sexy anymore.

For cold storage it is still pretty hard to beat, but I have noticed a lot of tech companies have blinders regarding 'stodgy' technology.

Re:Tape?

[Big Hairy Ian]

Depends what you do with it :)

Re:Tape?

[cdrudge]

How soon do you need the "cold stored" data when you do need it? Random accessing one bit of information on a 1-1/2TB tape sucks. A stack of 1 BDXL discs holds the same amount of uncompressed data in less space, has almost instant access (relative to a tape drive) once the disc is loaded in a drive, and multiple discs could be loaded in multiple drives to increase simultaneous accessibility.

Re:Tape?

[AmiMoJo]

Since we can assume they have knowledgeable storage specialists working for them there must be some interesting and novel reason why they are using BluRay. TFA doesn't really say, but it would be fascinating to know.

Re:Tape?

[avandesande]

Tell me more of this time when tape was sexy ;-)

Re:Tape?

[LordLimecat]

Problems with BD:
  * Price
  * throughput speed (like if you wanted to make a copy)
  * Reliability-- we know how long tape lasts because its been around forever. We also know that optical tends to be piss-poor for archival.
  * its write speed sucks

Re:Finally a demonstration

[NotQuiteReal]

I predict the squirrels win. The one ox will eventually die, but a thousand squirrels is a viable population.

Lots of redundant data

[Rik Sweeney]

It designed the system to store data that hardly ever needs to be accessed

So that will be several million inactive profiles. I hope they've made their solution scalable, pretty soon they'll be storing 75% of their current profiles on those discs.

Re:Out of touch

[MightyYar]

Ah, nevermind. I saw the $20 per 100 price on Google shopping. But when I cilck on it, they are just DVD-Rs. Blu Ray seems to be $1, which puts them in a similar price/gig with hard drives. Mods, kill my original post please! :)

Longevity will be an issue

[Marxist Hacker 42]

NONE of those solutions, including the current one, have been tested for longevity.

I went a year between my honeymoon and getting pictures off of my 1st gen digital camera, stored in flash memory. About half were corrupt.

Re:Longevity will be an issue

[mlts]

I've yet to find a single media solution that has stood the test of time. Yes, I might be able to pull data from a tape from the 1990s, or a burned CD from 1998... but I wouldn't want to bet my data on it. Long term, the only way to do things is archive data in a format that detects (and corrects) errors (I use WinRAR, but .PAR archives work as well) and keep moving them forward in media.

Even cloud storage is unreliable. I have had sync errors completely flatten my TC volumes stored on DropBox, and restoring from Amazon Glacier is doable... but is something I have as an absolute last resort.

Re:Longevity will be an issue

[Anonymous Coward]

For that first year, he was probably trying fucking a bit harder rather than worrying about data recovery. Priorities, lonely internet dude, priorities.

Re:Longevity will be an issue

[GrumpySteen]

I've yet to find a single media solution that has stood the test of time.

Clay tablets. Tested and proven for 5,000+ years and counting.

Space required for storage may be an issue, though.

Re:Longevity will be an issue

[AmiMoJo]

I have archival grade CDs burnt on 1st generation drives at 2x that are readable. They were damn expensive though.

Archival grade BluRay discs are not too pricey though, because BD-R is basically magneto optical and thus all you need to do to be archival grade is use quality materials and glue. No fancy chemicals. Long term they should be fine if looked after.

Re:Longevity will be an issue

[Kjella]

I've yet to find a single media solution that has stood the test of time. Yes, I might be able to pull data from a tape from the 1990s, or a burned CD from 1998... but I wouldn't want to bet my data on it. Long term, the only way to do things is archive data in a format that detects (and corrects) errors (I use WinRAR, but .PAR archives work as well) and keep moving them forward in media.

As long as there's someone there to look after the process, but what if instead of the box of family photos in the attic you find an unreadable backup CD labeled "family photos"? The danger is that you ignore or neglect it for a little while and you can't get it back. Having something you can put in a box for 50 years and pull out at the nursing home to reminisce with would be better. And if there's ever a WWIII and people have more important things to think about for 10+ years (war and rebuilding) it would be nice if we didn't lose our entire history.

I'd still like to have something that I could be really sure that yes, this would be easily readable in consumer equipment a long time from now. I wouldn't need to have my own writer, but if you could have something etched that'd last for say 100 years and play in a BluRay player that'd be a good start.

Tape?

[bored]

Sounds like what they really want is tape..

Besides the difficulty of dealing with 174 bluray disks instead of 1 tape... You have to wonder about the reliability of those disks sitting around on a shelf for ten years..

Oh, and you can write said tapes at 500+ MByte/sec.

Plus, tape is well understood, and there are tons of media management applications that track whats on the tape, when it expires, where its located, what encryption keys are used to decrypt it.. Basically 40 years of data management infrastructure.

News flash

[TheloniousToady]

Though I can imagine that Blue-rays may be economical for cold storage in some sort of jukebox format, it's hard to imagine how flash could be, either now or in the forseeable future. Flash storage currently is significantly more expensive than hard drive storage (ask anyone who's bought an SSD lately), and it's unlikely to get much cheaper due to fundamental limitations on the size of circuitry needed to hold enough charge to store data reliably.

Re:News flash

[Immerman]

For cold storage SSDs are probably the wrong benchmark to compare to, being generally optimized for performance rater than price. Try something closer to SD cards. You can easily get a 64GB card for $35, so about $0.55/GB. Still more expensive than HDs for now, but a lot less than SSDs, and far more physically compact(which is another very real consideration it's you're archiving petabytes per day) Just how may SD cards do you suppose can be stored in the space of one 4TB hard drive? A heck of a lot more than the 63 required to get the same capacity I'm sure.

Of course there's still the fact that flash, like DRAM, must get regularly refreshed to maintain data integrity - leave it unpowered for a couple years and it's liable to have erased itself for your convenience. THAT in my mind would make it unsuitable for cold storage, but then again maybe Facebook's digital attic is a lot "warmer" than mine.

Re:News flash

[TheloniousToady]

FWIW, I did a little quick research on Amazon and came up with the dollars/GB of various media as shown below:

Compact Flash: $1.06 / GB
SSD: $0.68 / GB
HDD: $0.04 / GB
Blu-ray: $0.04 / GB
DVD: $0.08 / GB
Data tape: $0.02 / GB

This suggests that flash media would need to come down in cost by more than a factor of 20 to be competitive with HDDs and cheaper media. Also, Compact Flash seemed to be more expensive per GB than SSDs.

Although flash prices may drop, other media likely will also, so a relative drop by a factor of 20 seems unlikely. Factors other than cost may be a consideration, but if I were running things at Facebook, it would be pretty hard to pay 20x as much per GB just to save space.

I'm surprised it beats LTO-6

[hawguy]

LTO-6 holds 2.5TB/tape (native, not compressed), so it's more space dense than Blu-ray since a single tape can replace fifty 50GB bluray disks. A 1 petabyte cabinet would only need 400 tapes, and LTO tape libraries of that size are readily available off the shelf - plus the software to manage it is also off the shelf.

Cost-wise, the tapes and disks are around the same, branded dual-layer blurays seem to cost $1 - $2, and LTO-6 tapes are around $60.

The only advantage I can see for blu-rays is in random access performance, but for a rarely used cold archive system, you'd think that wouldn't matter.

Re:No tape?

[mlts]

For a drive + 50 BD-R disks per TB, I'm looking at a C-note for the drive and $25 for each terabyte after that.

For a modern tape drive, I'm looking at $3500 for the drive and $65 per 2.5 TB, native capacity.

This also doesn't include hardware and software. For the LTO-6 drive, I need a dedicated server with a SAS card and a high end backup program. For Blu-Ray... it can be used, albeit slowly, with a USB 2.0 connection, but works decently with eSATA or USB 3.0.

For the big stuff, the relatively cheap price per TB of the LTO-6 drive is useful. However, not everyone can spend about $6000 for the drive, I/O card, and a decent server that can run it.

Re:Doesn't add up

[Guspaz]

100GB discs are $40 each if you're buying one of them. But what if you go to the disc manufacture and order 10,000 of them? I think you'd be able to get them a wee bit cheaper.

This is literally true

[Burz]

Google+ recently created an account for me (without my permission) when I signed onto Youtube recently. When I accessed Google+ to get rid of it, the thing stepped me through a wizard that wanted to link me up with people I haven't conversed with for many years! I delete all mail when downloading it from gmail, and I have Google search's web history deleted and turned off.

Accessing Google generates a permanent dossier on you.