Where Facebook Stores 900 Million New Photos Per Day

1sockchuck writes: Facebook faces unique storage challenges. Its users upload 900 million new images daily, most of which are only viewed for a couple of days. The social network has built specialized cold storage facilities to manage these rarely-accessed photos. Data Center Frontier goes inside this facility, providing a closer look at Facebook's newest strategy: Using thousands of Blu-Ray disks to store images, complete with a robotic retrieval system (see video demo). Others are interested as well. Sony recently acquired a Blu-Ray storage startup founded by Open Compute chairman Frank Frankovsky, which hopes to drive enterprise adoption of optical data storage.

Comments

They could save space

[NotQuiteReal]

They could just delete most of the photos after they age a bit, analyzing it with some of their AI whiz-bang software.

If anyone ever asks to see the image again, they can just show one that is "close enough" and nobody would ever know the difference.

I personally, have never posted a photo to Facebook, so I'd be OK with that.

Re:They could save space

[QuietLagoon]

They could just delete most of the photos after they age a bit, analyzing it with some of their AI whiz-bang software....

More than a few of my [real world] friends use facebook as their archive for photos, eschewing local or cloud-based storage for their historical family photos. They would be unhappy if facebook were to randomly start deleting photos just because they've been on facebook for a period of time.

.
Of course, I've told those friends that facebook may not have the same photo-preservation goals as they do, but they seem to be unconcerned.

Re:They could save space

[QuietLagoon]

.... If they are storing their photos on facebook, they ARE storing them in the cloud....

In a general sense, correct.

.
However, when I said "cloud-based storage" I meant the cloud service was a storage service, not a social media service. If I had meant facebook, I would have said cloud-based social media service.

Re:They could save space

[davester666]

Sure: "Show me the picture of me and my wife on the beach 10 years ago"

Wife: "Who the hell is that in the picture with you? And when did that happen?"

Delete?

[DanJ_UK]

What happens when a user wants to delete an image permanently. If it's stored on an optical disc are they going to destroy the whole disc and burn it again?

Re:Delete?

[bill_mcgonigle]

What happens when a user wants to delete an image permanently.

What gave you the idea that's a service Facebook offers?

Re:Delete?

[NatasRevol]

I see you haven't read Facebook's terms of service.

There is no delete.

Re: Delete?

[devilspgd]

Another implementation would be to encrypt each item with a unique key and destroy the keys, rather than the underlying item, in a delete event, such that not even forensic tools would have a reasonable chance at recovery once the key-storage media has been re-written.

Re: Delete?

[devilspgd]

No, keys are small enough to store without needing cold storage.

Re:Delete?

[Rasperin]

What the heck are you talking about?! Privacy advocates for years have been screaming about Facebook and their stock remains just as strong.

The average person doesn't care about long term punishments when the short term gains are attractive. This is why I use Facebook. But I treat Facebook like a loud speaker, it's a great place to share my idiotic ideals but I try to avoid saying anything damaging/damning. (btw, this is what we call acceptance, I long ago welcomed our new Facebook overlords).

Title is wrong

[neilo_1701D]

Should have read:

You won't believe this one weird trick Facebook uses to store data!

Other than that, fascinating look at how all that data is being stored and retrieved.

Going on for a while

[Alomex]

I've noticed large latency for rarely used pictures in FB for over eight months now, and by large latency I mean visit the page, then come back the next day to see the next batch of > 5 year old pictures and wait another day for the final batch of ~10 years ago pictures.

Re:Replace them

[Irate Engineer]

After 3 months of no views, just replace them with a goatse image.

Dear God, there is more than one!?!

Re:How do they do this?

[pla]

I think he just means that optical disks don't have quite the same amount of inertia or need to do internal self-checks as HDDs, before you can actually access them. They still "spin them up", but it happens in a few ms rather than on the order of 5-15s.

Amazing

[bws111]

Wow, they discovered HSM only 40 years after it was introduced. Amazing.

Re:Amazing

[Ravaldy]

Pointless arrogant comment.

Nobody claimed it was new or that they had reinvented anything. They just applied modern technology to a well know strategy to solve a known problem. In the modern age of storage and data centers I have yet to see this (not to say that nobody has done it).

When someone shows you an electric car do you tell them cars have had 4 wheels since before 1903? I assume you do.

Duh

[wonkey_monkey]

In the cloud, obvs.