Slashdot Mirror

← Back to Stories (view on slashdot.org)

How Facebook Stores Billions of Photos

Posted by ryuzaki0 on from the laser-printer-and-a-warehouse-i-figure dept.

David Gobaud writes "Jason Sobel, the manager of infrastructure engineering at Facebook, gave an interesting presentation titled Needle in a Haystack: Efficient Storage of Billions of Photos at Stanford for the Stanford ACM. Jason explains how Facebook efficiently stores ~6.5 billion images, in 4 or 5 sizes each, totaling ~30 billion files, and a total of 540 TB and serving 475,000 images per second at peak. The presentation is now online here in the form of a Flowgram."

3 of 154 comments (clear)

  1. Re:540TB / 30 billion images by JuanCarlosII · · Score: 5, Informative

    A quick survey of the most recent images on my profile tells me a full size image comes in at 50-60k and a standard thumbnail at ~5k so given the other sizes of thumbnail as well I'd say 18k per image is about right.

  2. Already been done. by sirrube · · Score: 5, Informative

    This stuff is cool either way, even if it is just "childish spam." Many of us only dream to work on something that will become this large scale.

    ...

    Fortune 500 companies could probably learn a thing or two...

    This Fortune 500 company could teach a thing or two on this subject. Since before 1999 DataTree has already did this. With over 40 billion land records online, and 600+TB of data, they deliver many millions of images daily. Not to put down FaceBook's Implementation, but DataTree does not need to run 10k webservers and 1800 SQL databases to provide images. It is nice to see the scalability factor of their design, but it does not mean that it is the most efficient way to do things, or to follow and learn from.
  3. User-mode GoogleFS by Panaflex · · Score: 4, Informative

    (summarizing the big long presentation)

    This is basically want to make a usermode GoogleFS. Their biggest problem is reducing reads - which are hampered by Posix file standards (inodes, metadata, etc...)

    Instead they use a database-like index/data file arrangement. The index stays in memory and files are stored together in large contiguous spaces on a single file. It's possible to utilize a LUN for storage - but not there yet.

    There... where's my cookie?

    (Oddly enough - I'm writing the exact same code they are... bazaar world, eh??)

    --
    I said no... but I missed and it came out yes.