Facebook Trapped In MySQL a 'Fate Worse Than Death'

wasimkadak writes with this excerpt from GigaOM: "According to database pioneer Michael Stonebraker, Facebook is operating a huge, complex MySQL implementation equivalent to 'a fate worse than death,' and the only way out is 'bite the bullet and rewrite everything.' Not that it's necessarily Facebook's fault, though. Stonebraker says the social network's predicament is all too common among web startups that start small and grow to epic proportions."

Comments

Commercial databases

[drolli]

Well. then they convert from one db to another. So what. its not like that would be a completely new thing to happen, and i am sure that oracle or any other big db provider will send experts to help with the task.

Re:Commercial databases

[cgeys]

But like the summary and article note, that requires rewriting the whole codebase. They should had gone with Oracle database to begin with, but of course no one ever thinks about the expanding possibilities when they're starting out and just want something free, ie. MySQL.

Re:Commercial databases

[Billly Gates]

I went for a job interview a few years ago which was very SQL intense. I looked at some SQL code in C# for both ODBC as well as direct SQL Server code, and it was the most complex thing I have ever seen and frankly hair pulling ugly. It was no simple UPDATE INTO TABLE like simply MySQL with php.

Rather, It was weird ASYNC VSYNC Data.adaptor,x and weird eseortoric lines consisting of 35 to 40 lines of code for each insert doing God knows what! Maybe a SQL programmer can explain what a Vsync was and what a data,adaptor is and why was that code so evil? The question was how to fix it? I realized I was obviously not qualified for the job.

I googled the code and it seemed it was operating optimally with all that stuff. Sure you can use a simple insert statement, but that is frowned upon as not optimized by SQL programmers. Most of them use very complicated steps and layers of abstraction where KISS is frowned upon, because if the database doesn't perform well you do not want to look like an *ss.

PHP is really nice for it's simplicity, but as soon as you move to Java or C# it gets very ugly and each database requires different code and optimization techniques and a rewrite if you change your schema. Again, I am not a SQL programmer so hopefully I wont get bashed too much here by the real ones, but it is just what I observe as I want to learn this myself. I have a feeling this is why Hibernate is becoming popular to avoid these things as it is a framework that does some of the nasty details in a seperate layer ... at least from what I read.

But with Java or .NET you can get caching, transaction control, and other neat things you can't get with PHP but it comes at a cost. Same is true with a real database like PostgreSQL, SQL Server, DB 2, or Oracle. SQL statements are a small part of the code and the rest is proprietary with working with the RDBMS. My hunch is the vendors love these as it encourages lock in with expensive licensing fees.

Re:Commercial databases

[qwijibo]

Ability to convert depends completely on the application. If the MySQL app written using simple or at least standard SQL, it will be easy to migrate. However, MySQL has some very problematic areas (i.e.: select foo from table1 where id in (select id from table2 where criteria='something')) that make people do some very nonstandard and MySQL-only style fixes to address performance. The query shown with 5000 rows in table1, 50000 in table2, table2 only having 50 rows that met the criteria took ~10ms on PostgreSQL 8.3, and 52 minutes on MySQL 5.1 on the same hardware. The only way I could find to get the ~10ms performance on MySQL was so goofy that MySQL itself refused to allow me to create a view from that select statement.

Converting from PostgreSQL to Oracle has always seemed much easier and smoother, but PostgreSQL isn't as popular as MySQL because it hasn't been as easy to throw hardware at problems with scaling PostgreSQL, whereas MySQL has always made that option easier.

Each database has its own pros and cons, but most times you don't discover how hard it is to migrate until it's too late.

Keep on backpedalling, you silly NoSQLers.

[Anonymous Coward]

It's hilarious how the NoSQL fools are now constantly backpedalling these days.

It turns out that writing database queries in JavaScript is a stupid idea! Imagine that! All of their attempts to invent a better query language end up being almost identical to, guess what, SQL!

Then they realize that trying to maintain data consistency using logic written in JavaScript, Ruby or PHP doesn't work so well. Values go unconstrained, and the referential integrity gets fucked up. Soon the data is nearly worthless.

The smarter/less-ignorant ones then think that they'll just use transactions. But wait, their NoSQL database of choice doesn't support that, or doesn't support it properly. So they tell themselves that their data will become "eventually consistent", or worse, they try to implement some shitty ass "transaction" support using Ruby. Regardless of the path chosen, failure is the result.

Now they're realizing that it's mandatory to use a real relational database when working on anything remotely serious. So we see this bullshit about "no" now meaning "not only". That's funny, last month it meant "no", as in, "we will never write a SQL query again, and we will never use a relational database again."

I'm going to make a prediction: Next month, we'll get to read articles and comments from them about these amazing new database systems that they've just discovered. These new systems avoid all of the problems associated with NoSQL databases! What are their names? Oracle, DB/2, SQL Server, PostgreSQL and SQLite.

Re:Successful Troll is Successful

[fermion]

And note that two stories down it is reported that SAP is once again over budget and over schedule on a major implementation. So I suppose that now everyone will stop using SAP as it unreliable.

Re:Successful Troll is Successful

[sco08y]

Academic purist discovers that one of the most prolific and successful database users in the world is using a system he doesn't approve of. He decides, with no insider knowledge at all, and despite all evidence to the contrary, that they should throw everything away and start over from scratch using a system that he thinks would allow them to see the performance and scalability that they've already achieved.

Presumably he's tired of Facebook being used as a counter-example to everything he's been preaching.

He's no academic purist. He's pushing his product, and he's either an outright liar or, worse, doesn't know what he's talking about:

Stonebraker said the problem with MySQL and other SQL databases is that they consume too many resources for overhead tasks (e.g., maintaining ACID compliance and handling multithreading)

Is that so? MySQL, as with virtually all SQL DBMSs, defaults to "repeatable read" transactional guarantees, and it doesn't even spend time guaranteeing foreign key relationships by default. About the only thing MySQL really guarantees out of the box is durability.

It's just nonsense to talk about all the "wasted resources" when, if they don't need them, it's a few lines in a config file to turn them off.

Reply from a former FB engineer

[prostoalex]

Reply from an engineer who worked at Facebook 2007-2011.