How Facebook Ships Code

Hugh Pickens writes "The two largest teams at Facebook are Engineering and Ops, with roughly 400-500 team members each, together making up about 50% of the company. All engineers go through 4 to 6 week 'Boot Camp' training where they learn the Facebook system by fixing bugs. After boot camp, all engineers get access to the live DB and any engineer can modify any part of Facebook's code base and check-in at-will so that engineers can modify specs mid-process, re-order work projects, and inject new feature ideas anytime. Then arguments about whether or not a feature idea is worth doing or not generally get resolved by spending a week implementing it and then testing it on a sample of users, e.g., 1% of Nevada users. 'All changes are reviewed by at least one person, and the system is easy for anyone else to look at and review your code even if you don't invite them to,' writes yeegay. 'It would take intentionally malicious behavior to get un-reviewed code in.' What is interesting for a company this size is that there is no official QA group at Facebook but almost every employee is dogfooding the product every day."

Comments

Re:no process

[iluvcapra]

the culture of the company seems to be set so that *everyone* feels responsibility for the product

It's astonishing that they can keep such a process rolling with 500 engineers, let alone 200

product managers have a lot of independence and freedom. The key to being influential is to have really good relationships with engineering managers. Need to be technical enough not to suggest stupid ideas.

So basically, Facebook is run by an aggressive engineering culture based not on consensus or managerial decision making but by cliques and lobbying, where the worst thing someone can possibly do is suggest an idea that an engineer claims is "stupid" and doesn't give them opportunity for nerd glory. How much you want to bet that Zuckerberg sets the tone and decides that any modifications to the way the privacy settings are run is "stupid" and "boring."

Re:no process

[Anonymous Coward]

Facebook is run by an aggressive engineering culture based not on consensus or managerial decision making but by cliques and lobbying

At any other company, that could be counter-productive. But considering that the tool they're building is basically the ideal tool for building your clique and growing your influence, it probably just adds to their "dogfooding."

This explains bizarre feature regressions

[Y-Crate]

I know they're always looking to "streamline" Facebook, but sometimes really obviously good features vanish without explanation and leave users scratching their heads. These decisions would make more sense if you could see some way that FB was making life easier for advertisers or something, but often, the feature regressions are just nonsensical to the extreme.

A few:

- Facebook got rid of statuses. The one e-crack feature they're best known for. Gone with the new profile. Now they're just wall posts to yourself that quickly fall down the page. *facepalm*

- Photos are now uploaded in descending order. It used to be that a group of images would be uploaded like this: beachtrip-1.jpg, beachtrip-2.jpg... and so on. Now it reverses the order of an uploaded album. The last photos taken are displayed first. Posting vacation pics? Well your friends get to see the day you left and work their way back to the day you arrived.

- The "Reverse Photo Order" option was removed when the above "feature" was introduced. You can drag around photos to manually reorder them, but every photo that you mouse-over jumps out of the way and moves to different rows. It's a UI disaster.

- A few months ago I started seeing tagged photos of friends despite the fact they were in private albums of people I'm not friends with who also happen to have locked-down profiles where you can't see anything at all unless you add them.