Facebook Details the Software Engineering Behind Graph Search

Nerval's Lobster writes "Facebook's Graph Search, its new and powerful way of searching the social network for all manner of information, has drawn a lot of attention since its January unveiling. Some have praised its innovation; others have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Still more have questioned what it all means for users' privacy—always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Graph Search (short answer: the Disaggregated Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Graph Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, an inverted-index system they'd had in development for quite some time."

Comments

Let's rewrite that

[Anonymous Coward]

"Facebook's Stalker Search, its new and powerful way of searching the social network for all manner of information about you, has drawn a lot of negative attention since its January unveiling. Few have praised its innovation; fewer have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Most have questioned what it all means for users' privacyâ"always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Stalker Search (short answer: the Disorganized Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Stalker Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, a mythical flying horned horse they'd had in the basement for quite some time."

What it means for user's privacy.

[QilessQi]

You get as much privacy from Facebook/Gmail/Hotmail/etc as you pay for. Sometimes, you get less.
If you're unhappy with those terms, you probably shouldn't use the service.

Re:What it means for user's privacy.

[PopeRatzo]

It's still worthwhile making sure as many people as possible know just how shitty Facebook is about respecting privacy.

They've made every effort to obfuscate what they're doing, and not everybody has the time and energy to search out the details of the privacy risks of using Facebook.

I salute the public-spirit minded people who make an effort to inform. As often as Facebook changes their privacy policy and acts dishonestly, I don't think you can say "People who use Facebook know what they're in for."

Re:What it means for user's privacy.

[QilessQi]

I agree. But I think the simple axiom is this: if you use a free service, assume that any information can be sold to anyone for any purpose at any time.
If people could be made to understand that, the specifics would almost be uninteresting.

Re:What it means for user's privacy.

[tlhIngan]

I agree. But I think the simple axiom is this: if you use a free service, assume that any information can be sold to anyone for any purpose at any time.
If people could be made to understand that, the specifics would almost be uninteresting.

An even simpler one dates way back to when we were connecting computers together by modems. "Don't post online what you don't want the world to know." Because anything posted online IS pretty much available for the world.

Yes, even with privacy settings. Privacy settings are a way to extract even more personal information from people than they otherwise would give. They're the ultimate in marketing - people mistakenly believe they're one thing but they're really something else.

The only real way to have privacy is to not put that information online in the first place.

Re:What it means for user's privacy.

[PopeRatzo]

The only real way to have privacy is to not put that information online in the first place.

Yes, except how much information comes not from what you PUT on the internet, but what you LOOK AT on the internet?

Cool stuff

[futhermocker]

Would really like to see Unicorn become open source.

Where I work we use datamarts spread across several data warehouses, which is quite similar to the FB way.
Since we use a bottom-up design model, creating so called solutions using this indexer would be very straightforward.

Re:It had to be unicorns.

[oztiks]

No no, it was a highly complex set of sql queries devised by a highly complex set of mathematical algorithms which took a team of washed up college drop outs to devise. I was able to speak to Mark Zuckkerberg on this matter and he divulged to key areas of the system's source to me.

$searchquery = 'people who are hipster losers';

$sqlquery = 'select * from pages p inner join friends f on p.userid=f.friendid where 1'

mysql_connect("localhost", "leet", "hax0r");
mysql_select_db("facebook_db");

$page = mysql_fetch_array($sqlquery);

if($page['like']>0) {
        if(strpos($page['content'], $searchquery) === true) {
                echo $page['content'];
        }
}

Make benefit glorious engineering teams

[Drunkulus]

"Hard part of startup is make money from wheel after reinvent it." -Devops_Borat

This is what I think of when I read this...

[fldsofglry]

“What are the differences between Mark Zuckerberg and me? I give private information on corporations to you for free, and I’m a villain. Zuckerberg gives your private information to corporations for money and he’s Man of the Year.” – Julian Assange I can't confirm if, where, and when he said this, but regardless the idea rings true for me.

Re:Search isn't enough. Social network analysis is

[History's Coming To]

We need to allow app makers to do the things and offer the services we can't, the really intrusive stuff that we need plausible deniability over, and by monetizing our data via licensed app services which perform tasks which we find morally ambiguous we can keep our new and desperate shareholders happy in both ways.

Even on SlashDot...no one cares about Graph Search

[xxxJonBoyxxx]

>> Some have praised its innovation

Er...what? 28 comments in 8 hours tells me no one cares about Graph Search - not even on SlashDot.