Facebook Puts 10,000 Blu-ray Discs In Low-Power Storage System
itwbennett writes "Facebook said last year that it was exploring Blu-ray for its data-center storage needs, and on Tuesday it showed a prototype system at the Open Compute Project summit meeting in San Jose, California. It designed the system to store data that hardly ever needs to be accessed, or for so-called 'cold storage' (think duplicates of users' photos and videos that it keeps for backup). The Blu-ray system reduces costs by 50% and energy use by 80% compared with its current cold-storage system, which uses hard disk drives, said Jay Parikh, Facebook's vice president of infrastructure engineering."
It's a prototype, and they're also evaluating low power flash as another alternative to keeping seldom accessed data on hard drives.
Anyone know if these burners are write-once drives?
If so, it pretty much guarantees that Facebook keeps a copy of your stuff forever, even if you "delete" it.
After reading about it for so many years now we finally get to see a demonstration of the 1,000 squirrels versus 1 ox in a demonstration. Pull!
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
I guess tape just isn't sexy anymore.
For cold storage it is still pretty hard to beat, but I have noticed a lot of tech companies have blinders regarding 'stodgy' technology.
[Citation Needed]
It would be nice to find them even at $2 each.
Make it official by signing formal agreement with NSA, which also keeps backups of certain information. Why duplicate and waste energy?
Here they are at 50 cents/disc.
http://www.amazon.com/Optical-...
SJWs are the new boogeyman. -Me
Reminds me of Plan 9's read-only snapshotting system:
https://en.wikipedia.org/wiki/Fossil_(file_system)
It designed the system to store data that hardly ever needs to be accessed
So that will be several million inactive profiles. I hope they've made their solution scalable, pretty soon they'll be storing 75% of their current profiles on those discs.
Summation 2
Ah, nevermind. I saw the $20 per 100 price on Google shopping. But when I cilck on it, they are just DVD-Rs. Blu Ray seems to be $1, which puts them in a similar price/gig with hard drives. Mods, kill my original post please! :)
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
If they're so concerned about energy and related costs, why not use tape? It's still, by far, king of cold storage.
NONE of those solutions, including the current one, have been tested for longevity.
I went a year between my honeymoon and getting pictures off of my 1st gen digital camera, stored in flash memory. About half were corrupt.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
Sounds like what they really want is tape..
Besides the difficulty of dealing with 174 bluray disks instead of 1 tape... You have to wonder about the reliability of those disks sitting around on a shelf for ten years..
Oh, and you can write said tapes at 500+ MByte/sec.
Plus, tape is well understood, and there are tons of media management applications that track whats on the tape, when it expires, where its located, what encryption keys are used to decrypt it.. Basically 40 years of data management infrastructure.
Mods, kill my original post please! :)
Mods, that's cute.
Though I can imagine that Blue-rays may be economical for cold storage in some sort of jukebox format, it's hard to imagine how flash could be, either now or in the forseeable future. Flash storage currently is significantly more expensive than hard drive storage (ask anyone who's bought an SSD lately), and it's unlikely to get much cheaper due to fundamental limitations on the size of circuitry needed to hold enough charge to store data reliably.
Because Obama has realized the cost of storage, and the Mighty Pen of the Executive has dictated that the federal government won't spend money on that anymore, and outsource the job to the common carriers instead, who will then be required to provide exactly the same information upon request.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
I've been using the Optical Quantums that dugancent mentioned for a year now, and have had very good luck with them. They are about 50 cents per.
They are slow blanks, burning at 4X, but you get what you pay for, and for large backups running in the background, they are good enough. I've saved off terabytes of data using WinRAR split volumes (with recovery archives just in case) and Nero SecureDisk burning, which makes it easy to check integrity of disks before starting a restore.
LTO-6 holds 2.5TB/tape (native, not compressed), so it's more space dense than Blu-ray since a single tape can replace fifty 50GB bluray disks. A 1 petabyte cabinet would only need 400 tapes, and LTO tape libraries of that size are readily available off the shelf - plus the software to manage it is also off the shelf.
Cost-wise, the tapes and disks are around the same, branded dual-layer blurays seem to cost $1 - $2, and LTO-6 tapes are around $60.
The only advantage I can see for blu-rays is in random access performance, but for a rarely used cold archive system, you'd think that wouldn't matter.
Facebook can become the new AOL!
TFA says 10,000 discs for 1PB, which would be 100GB/disc. But 100GB discs are $40 each (50GB discs are $2 each, and 25GB discs are $1 each). Unless they're factoring in 2x data compression the way the tape people do.
why saving data which already exists identical somewhere else? "duplicates of users' photos and videos that it keeps for backup" are stuff that can be deleted (does not help anybody anyway and will not be accessed anyway) or if the databases are so conservative, just remove the redundant files and instead put some links in place so to not break the user experience on looking at the same happy cat photo all over the web for the n-th time :)
LTO6 can store 2.6TB of data per tape and its low power and robots are available to read/write them and they have completely cold storage capabilities and are much quicker than blu ray disks which are at most 128GB for a quad layer (roughly 25gb/layer). I pretty much stopped dealing with optical media years ago because its too big (physically), too slow, and just not worth my time.
Is flash really in a position to fill the "slower and cheaper than disk" slot, as well as the well-known "faster and more expensive than disk" slot? That's very interesting if true. Even considering power costs, I didn't think we were there yet.
Or is this some sort of near-line "flash jukebox", where most units are completely powered down most of the time?
The mainframe has had HSM since I guess before these Facebook guys were born. If a file isn't used, it's swept off of primary storage (disk) and written to secondary storage (tape, disc, etc). If anyone ever needs the file again, it's automatically restored. Fun to watch these guys reinventing it!
what generally kills things like SaaS providers and online content is not the cost to back up the data. tapes could be arguably just as cheap and once the data is stored, the medium is just as powerless as blu-ray. Time-to-restoration is a very big concern. things like SAS arrays of LVM striped blu-ray disks could mitigate the issue but seeing as the machinery is the same pick-and-stick model used with tape robots, throughput is going to suffer. the definition of 'cold' also comes to mind as most cold storage is one-and-done. the next interaction it sees is when its boxed and shipped to some cave in the midwest where it drives profit for some disaster recovery service like iron mountain.
what about the halflife of inks? consumer cyanic ink cd-r is supposedly OK for 10 years, although ive seen laminate degradation in less than that personally. will a disk, with arguably fewer moving parts than a tape, be able to cope with stresses to its laminates as its routinely retrieved and re-read at a data-backup frequency?
Good people go to bed earlier.
if it's so seldom accessed that it has to be moved to an optical disc, email the person and ask if they want to keep it and if they dont respond or say no, just delete it. if you have forgotten all about using your facebook account, it's unlikely that you want it to be on record for all time.
Anons need not reply. Questions end with a question mark.
But it depends on the sexiness of the female that it is wrapped around :P
...and it wants its HSM back!
Hierarchical storage systems, and hierarchical storage managers (HSMs) have been migrating lesser-used data to cheaper (and sometimes offline) storage for decades. So what's new about it, the use of contemporary yet inefficient Blueray discs?
big #s co-opting is finally legal? http://finance.yahoo.com/q?s=VRNG still clinging to our hemispheres too? http://www.globalresearch.ca/weather-warfare-beware-the-us-military-s-experiments-with-climatic-warfare/7561 never ending WMD on credit genocidal holycost running full blast now
The failure rate on dye-based writeable optical disk based storage is horrific. There is reason to think that foil based CDs, DVDs and Bluray disks- the ones you buy with films and music pre-recorded, could last an extraordinary age if well manufactured and carefully stored, but the write-once disks are a very different technology indeed.
The whole optical disk dye market is a sickening mess. Like the third-party ink refill market for printers, and the 'fake' electrolytic capacitor scandal that ruined billions of dollars worth of electrons, the problem is that patents and massive over-pricing of the base chemicals means that even 'branded' disks from supposedly 'reputable' sources can have any old crap passing itself off as a long lasting dye. And worse, the existence of so much counterfeit dye means that even the proper stuff has never really been properly tested to see if it makes for disks that can reliably hold their data for tens of years.
All of us that have burnt a lot of disks know the uncertainty of reading the disk even a couple of years later. Disks exposed to light and heat fail faster, but even disks kept in dark cool storage frequently fail to read in a remarkably short period of time. The very principle of dye based storage gives ZERO confidence, compared to the known properties of magnetic tape, or the safety of burning holes into a metal foil.
Admittedly, most of the problem lies with cheapo readers, since the market for burners/readers has very much been a race to the bottom. A better disk reader, with superior optics, and a powerful set of DSP algorithms processing the data recoverable from the dye, would allow fading dyes to provide good data for more years- but the very fact that the user never knows if the dye is some crap that will fade quickly is the true problem. The 'electronics' biz suffers counterfeit products even into the inner most projects of the USA defence department.
And worse again, people easily allow themselves to be fooled with optical disks, by conflating the longevity of 'pressed' foil based product with the extremely different dye based disks.
PS RW disks use a very different technology again- and there is good reason to believe a RW disk will hold its data for far longer than a write-once disk, is carefully stored.
PPS 'hardly ever accessed' data is a code word for data you really wish you could simply throw away. Consider how much archival film stock has been lost across the ages because said film was stored, but with near zero concern for deterioration and the risk of fire damage. Sometimes an organisation feels the need to pay lip-service to the idea of archival storage, but doesn't really give a damn if the data is lost.
>> to store data that hardly ever needs to be accessed
In other words, data that you think you are "deleting"
-- I was raised on the command line, bitch
Only someone with the fiscal resources of Mark Zuckerberg and Facebook ($Billions) can afford a single blu-ray(tm)(c) disk. More than one? They took out a mortgage on a datacenter, to be repaid within 90 days at a nominal interest rate of 12.5%. Pump the stock price once or twice and you can make most of that blu-ray disk purchase money back.
Wow.. I guess they need a lot of storage for deactivated accounts. that is why I deleted all my posts, pictures, and friends before I deactivated.
the member count is inflated since they count deactivated accounts too.
I'm going into tinfoil hat territory, but I wonder if there is some advance in BD storage that FB is assuming, but the average person does not know about. If BDXL disks drop from $45.00 per disk to $1 a disk or even $2 a disk, that would change the game. Similarly, Sony/Panasonic's Blu-Ray successor that stores 300GB per disk would also be a big thing, should each disk be priced at a reasonable amount.
LTO technology keeps advancing too - LTO-7 will store 6.4TB natively on a tape.
Oracle/StorageTek T10000D holds 8.5TB/tape native right now:
https://en.wikipedia.org/wiki/StorageTek_tape_formats
in 1996. Not clue ray, but write ones to dick for older account not likely to be access. So I"m not sure what's cutting edge about that.
The Kruger Dunning explains most post on
How about checking your crap FIRST before mashing your Hulk-like fingers on your keyboard?
Blu Ray seems to be $1, which puts them in a similar price/gig with hard drives.
These ones work out at GBP 0.324 each including VAT (sales tax) in packs of 10 at ordinary retail price. Subtract the VAT (since you guys don't normally include sales tax in quoted prices) and convert to US dollars, that's around 45 cents each.
Obviously that's some no-name brand, you'll pay a lot more for Verbatim and the like, but I'm sure Facebook may get a discount if they use more than 10 ;-)
When hdds are at 4T and tapes are at 6.4T it sounds ridiculous to go back to optical technologies which seem to have reached a hard limit in scalability over a decade ago.
Sounds to me like a bunch of youngsters messing around with storage robotics for fun.
Google+ recently created an account for me (without my permission) when I signed onto Youtube recently. When I accessed Google+ to get rid of it, the thing stepped me through a wizard that wanted to link me up with people I haven't conversed with for many years! I delete all mail when downloading it from gmail, and I have Google search's web history deleted and turned off.
Accessing Google generates a permanent dossier on you.
http://www.sony.co.uk/pro/prod... From desk models to Petasite. It uses cartridges of blu-ray discs, UDF.
Sony already does 12 discs in a cartridge in their Optical Disc Archive
http://www.sony.co.uk/pro/products/broadcast-solutions-mass-storage-archive
Sony is pushing this it has no LTO generation migration issues plus being laser based it doesn't wear like LTO tape. Got a lot of interest in the broadcast market.
FYI, I'm a Sony employee :)
No matter what way you spin that (no pun intended) it doesnt sound all that clever. The physical space to store 160 discs would be far greater than to store a disconnected HDD.
I am surprised. Even using the cheapest BRs and assuming bulk discounts, they are still way more expensive than LTO-6 or T10000D tapes (and the drives, when you're factoring in 1000 tapes per drive on average).
They're also a shedload bulkier and nowhere near as robust/long-lived.