Slashdot Mirror
Facebook Experimenting With Blu-ray As a Storage Medium
s122604 links to CNN's explanation of what may be the future of cold (or at least lukewarm) storage at Facebook, which is experimenting with massive arrays of Blu-Ray discs for seldom-accessed user files. Says the report: The discs are held in groups of 12 in locked cartridges and are extracted by a robotic arm whenever they're needed.
One rack contains 10,000 discs, and is capable of storing a petabyte of data, or one million gigabytes.
Blu-ray discs offer a number of advantages versus hard drives. For one thing, the discs are more resilient: they're water- and dust-resistant, and better able to withstand temperature swings. Their data can be restored more quickly, and they're easier to transport.
Most important, though, is cost. Because the Blu-ray system doesn't need to be powered when the discs aren't in use, it uses 80% less power than the hard-drive arrangement, cutting overall costs in half.
Can I ask Facebook to delete my stuff from one of those (assuming I had a Facebook account in the first place)
Enterprises have been doing this with tape for 30 years.
In fact, modern tape technology probably has a higher "volumetric" density than BD.
"I don't know, therefore Aliens" Wafflebox1
When you first access this data, you have to sit through 42 previews before you get to it.
It was a joke! When you give me that look it was a joke.
They'd also be cheaper, even at the bulk HDD rate that FB would pay.
A quick on-line search show a spindle of fifty 50GB Blu-Ray discs (2.5 TB) retails for about $100. A 4TB HDD costs about $140. So HDD is actually cheaper per byte of storage. Maybe wholesale price ratios are way different from retail, but I see no reason to assume that. So BluRay doesn't win on price, volume, or access speed. The concerns about moisture and big temperature swings seems odd. Are Facebook data centers exposed to the weather?
This is a company who's product stack is written in PHP.
Mod me down, my New Earth Global Warmingist friends!
I read TFA. They're not using them as "storage" in the sense of active, accessible storage. It's a backup system.
What they're trying is, instead of storing redundant copies of everything on multiple drives (for resilience and geolocality), they're keeping one copy live and keeping backups on blu-ray.
So there's never a latency of minutes while it loads data from Blu-Ray, you just might be routed to Siberia or something to get the one active copy. If that copy's bad, error (restore from backup during next nightly batch or something).
So HDD is actually cheaper per byte of storage.
If the HD needs to be replaced much more frequently than the Blu-Ray media the advantage switches quite quickly. For example, if the HD is replaced every 5 years and the Blu-Ray media is replaced every 20 years the HD would have to cost 1/4 of the Blu-Ray to match the hardware price.
The concerns about moisture and big temperature swings seems odd.
Temperature and humidity control are very expensive as it takes a lot of electricity. If the media can handle higher temperature and humidity swings then operation costs will be much lower.
That's not really how tape systems work. Generally they keep an index online so you can tell the tape system to pop in a specific tape and goto a specific position, longest load times... in real world that i've personally witnessed... 10 mins
What you describe is called a MAID and over the years it has proven quite unreliable. Hard disks are sensitive creatures and don't age well when being powered-on/powered-off randomly, and because of the nature of cold storage it is difficult to achieve a right balance of redundancy and power savings.
Also I would advise you to be careful when you label something as "phenomenally stupid" otherwise in instances like this one it may make you look like you are "phenomenally uninformed".
lucm, indeed.
When you deal with cold storage you have to look at things from a node level, not in global storage size. If your basic unit is a 50GB device instead of a 4TB device, this means that each request you make to recall data has a much smaller footprint.
Let's say that each stored account takes up 1GB of space. That's 50 accounts per BD drive, and 4000 accounts per hard disk. This means that when some dude comes out of jail and tries to access the photo his mom posted on his Facebook wall in 2010, there are 3999 accounts that are pulled out of their coma with it for no reason. On a BD that's only 49.
As long as you partition stuff properly it's unlikely that a single request will span multiple BD drives. You may have to deal with clusters of BD disks and this requires a bit of tuning, but even with the best indexing system in the world you can't power up only part of a hard disk. So BD is a clear winner here, especially if to that footprint issue you add the fact that spinners die quickly when you keep playing with the on/off switch.
Bytes are bytes when you live in a software world. But physical factors and limitations come into play when you deal with storage, and that's why most people with a software background can see WTF where there is instead good engineering.
lucm, indeed.
You're actually talking about MAID (Massive Array of Idle Disks), a technology that I first encountered in 2002. Now-bankrupt Copan Systems was the company I first encountered that was doing MAID, and New SGI (i.e. former Rackable Systems) bought their assets out of bankruptcy in 2010. Most storage companies now offer MAID add-ons for their storage arrays, though not all of them allow completely powering down the drive like Copan's solution did.
The upsides of MAID: Disks are cheap. Turning on and spinning up a hard drive to pull up some bits is faster than a robot fetching a Blu-Ray disk, placing it into a drive in the jukebox, and waiting for the disk to spin up and come online. You could store many more bytes in a cabinet with MAID than you could in an optical disk cabinet.
Downsides: The disk drives in a MAID array simply don't last that long, comparatively speaking. Spinning them up and down all the time is hard on a drive. So you end up having to replicate data and from time to time migrate data to new drives as old drives reach their service life. The service life of rarely used Blu-Ray media that has always been handled robotically (i.e., nothing touching its surfaces ever) is such that Blu-Ray media from ten years ago is probably still usable, the technology itself will become obsolete like DVD-RAM long before the media wears out. Not so much with hard drives, though disk arrays basically have unlimited life given typical failure patterns (i.e., if you're using RAID6, a drive develops errors, you remove the failing drive from the array, rebuild the array on a new drive, and chances of having two more drives fail during rebuild and thus losing the array are slim for a 12-drive array). So MAID has not really taken off the way we expected ten years ago.
At the time I first encountered MAID I was working for a company called DISC Storage, which had a NAS head which would automatically migrate little-used data to an optical jukebox in a way similar to what Facebook appears to be attempting. I designed and implemented the clustering function that would replicate the data between two NAS heads / optical jukeboxes, since the DVD-RAM platters were not themselves RAID'ed, as well as implemented a lot of the back end functionality for jukebox control and so forth. In any event, it looked like a NAS head but most of the files had been migrated to the DVD-RAM platters, and if you accessed one of those files, you would (at some point maybe 15 seconds later) get your data back as the file got read back onto the hard drive. It worked. But it was somewhat slow and cumbersome, because you're relying on a robot to go out and fetch the disk and put it in a drive, and disk robots then, and now, simply aren't that fast compared to media that's already in a drive ready to be spun up and read.
So anyhow, it was fairly obvious to me by mid 2003 that optical jukeboxes simply weren't going to be the future. In the ten years since DISC went under (there is a German company by that name now but it isn't the same company, it bought the name and some of the IP), I have not had any inclination to work for a company doing optical storage, because it's clear that for most problems it isn't the solution. It's too slow, too bulky, and magnetic disk drives and magnetic tape drives just continue getting bigger and cheaper every day. And now, with SSD coming on strong, optical jukeboxes look even less compelling.
So color me amazed. Optical jukebox and optical media technology essentially has barely moved on in the past ten years and what wasn't particularly compelling then, is even less compelling now. If you have need to keep data for a *long* time, this is how you do it... but frankly, I will be surprised if Facebook even exists ten years from now given the pace of innovation in the industry (though I'm just as surprised that Slashdot still exists!), so I question why they would do this rather than invest in LTO tape libraries, which have the advantage of being significantly denser.
Send mail here if you want to reach me.