Slashdot Mirror
Twitter Reportedly May Abandon Ruby On Rails
Raster Burn writes "According to TechCrunch, Twitter has plans to abandon Ruby on Rails after two years of scalability issues. Candidates to replace Rails are said to be PHP, Java, and Ruby without the Rails framework." The post links a brief comment (at 139 characters, probably a tweet) from Twitter founder Ev Williams saying it ain't so. The comments following the post embody the controversy over whether or not RoR sucks.
I have an argument with a coworker frequently about architectural orthogonality vs performance. I fall on the "architecture should be clean and easy to understand and maintain" side of the argument and he falls on the "speed, memory, and response time at all cost" side.
What is more important? Is developer time and productivity over the software lifetime more valuable than CPU cycles? If the price of that productivity imposes a maximum limit on performance, how much optimization should be undertaken?
It's a hard question to answer. On the one hand employees are expensive and hardware is cheap. On the other hand, you can't simply forego developing for performance just because of some religious belief that architecture should be clean.
And the scalability issue is not the x86 Slowaris systems they're running it on? Scalability of a platform is rarely just a RoR issue...
What is it exactly, about the framework/language that makes it difficult to scale? Honestly, I've heard this complaint about RoR for years, but no one I know has enough knowledge to verbalize the issues. Can someone who is knowledgeable in RoR give a few points here?
Is it just me, but how in the world did Twitter get $5.4M of funding? It's a pretty simple system and I just don't get it - granted I'm sure it's not cheap to run a system so many use but it can't be that bad...
Do the funders think that people are going to be okay with little ads being on each tweet people send out eventually or did they just want a cool new web 2.0 toy themselves?
The problem is Ruby. It is a very slow language (http://www.blognone.com/node/4385). My tests confirm it is 20 times slower than python in simple loops. Java is a dinosaur and PHP is not easy to maintain. I would go with Django or web2py. If they use web2py (http://mdp.cti.depaul.edu), I will help them. web2py scales very well because allows you to bytecode compile the models, the controllers and the views (so there is no parsing when serving a page) and cache every function in ram.
All of these slashdot conversations about rails have no technical details in them, describing which parts of rails are failing, which parts are preventing concurrency. The programmers on here can be just as stupid, unscientific, and reactionary as anyone else. As an example, you!
The funny thing is, for social networking sites, this is almost expected and par for the course. MySpace, and more famously Friendster had HUGE problems with availability due to their unexpected success. It's taken years for them to stop being buggy. Writing applications to handle this kind of load ain't easy.
Also, the one case of twitter's downtime being rails' fault, that I'm aware of, was due to Rails' lack of multiple db connections, which was fixed within days. And that wasn't really a bug, but a feature that was missing because no one using rails was dealing with twitter's requirements at that point.
Rails remains the best way to develop solid and maintainable web apps. But it will not compete with C for speed. Once you understand your business process, and have developed a mature algorithm to make the business work, there is nothing wrong with writing the core infrastructure in C. There are lots of C libraries that do the core part of Twitter. But the company would have been idiotic to start with them, and would be foolish to move new development off Rails.
Don't forget the aim bot.
Seriously, talk had advantages over IM. You actually could see what the other person typed as they typed it, including backspaces...
ICQ actually offered a "chat session" feature that did that. but it required direct client-client connection which made many people (me included) abandon it. Also it apparently had fairly high network overhead.
I tend to think being able to compose your thoughts before letting the other end see them is probablly a good thing anyway.
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
When Twitter opened it was handling 11,000 requests per second and doing it well. Twitter has gone from nothing to sensational in a very small amount of time. If you hit the ground running that quickly, your growing pains will be evident regardless of what framework or language choice you're using.
That I'm going to have to disagree with you on. For starters the most recent troubles that have plagued Twitter happened as recently as a few days ago. They have had plenty of time to mature their stack, but they are still seeing major downtime.
Second, I work for a major website and over the past two years we have *never* experienced downtime like what I've seen on Twitter, and we get far more traffic. We're on a J2EE stack, and without getting into evangelism I wonder how much of an impact that has had. It certainly has a better track record for us.
Now, large-scale enterprise applications are difficult (if not impossible) to meaningfully compare. But one can't help but notice these things. Twitter goes down a lot, and it's not all that complicated of an application. Why? A lack in a robust QA group? Operations? Rails itself? All of the above?
Having worked with Rails for a year, I have found listening to people who talk on any web forum about any language draws out nothing but hyperbole. So, I would take most of what is said here with a grain of salt since it is obvious that most of the people commenting here are stating "truths" from religious wars.
The framework operates a multiple levels. At the highest, a complete page can be generated from scaffold that automatically hook model to controller to view. I have found the scaffolds to be lack luster. From a completely database-centric view, there are some neat things that are automatically generated. For instance, verification methods in models are use to display errors on the page and mark input fields in red. My personal experience has found scaffolds to be lackluster.
Most well-designed applications revolve around the application's use, not its internal data representation. Using scaffold strongly ties the interface to data representation which creates the situation that "the user can be wrong." You see this in Microsoft Access databases where you can enter something in or choose options that are mutually exclusive. Because the application lets you see that data, the program generates an error if you are wrong. From what I have seen, the gripe is that the full scaffold is too specific and rigid. Well, duh?! That's the point of each layer of scaffold--to provide a guide for usage.
Personally, I have shunned most of the page scaffolding and tend to rely on creating my own use flow. I use the controllers to present that choices are possible and to manipulate the models as opposed to the common practice of having the controller just load a set of records and pass it to the view (which formats the output). The advantage is that the user is never wrong. Options that are logically inconsistent are never presented. Add to this the ability to monkey-patch (the extension of predefined classes) and lambdas, the code is clean and concise. Both can be used to refactor procedural code into functional code and move it out of the controllers and models. Most importantly, the design allows you to think about what you want to manipulate and then after the fact extend the functionality. A common example of the is the statement:
1.day.from_now
The numeric class is extended in Rails so that you do not have find and use a static date class, but can state simply the desired result.
So, where does it fail miserably? So far, I have not found any great place that it does. It performs as well I need it to serve about a dozen users on a lowly Pentium 4 machine with 256 megs of RAM. So far, the application has been 99% maintenance free. A date verification package I am using had a Feb 29th bug in it. The cool thing is that since I can see the source, I could fixed it. Perhaps there are issues with scaling, but from what I understand, the system was designed around a non-centric design. In theory, a correctly designed application should be able scale horizontally.
Given that there are other high-profile, high-use web sites written in Rails that do not suffer from Twitter's issues, I am left thinking that its failure in general looking for a specific reason. Rails has been very stable and easy to extend, but then I write for maintenance and ignore hype.
Bel, the mostly sane.. "Of course I can't see anything! I'm standing on the shoulders of idiots." -- Me
Standard prologue: I work at yellowpages.com, a Rails site that gets millions of searches a day. The site went from 125,000 lines of Java to around 14,000 lines of Ruby and shrinking. For more details, see our presentation at RailsConf.
How many servers has your operations group had to add? What has been the increased cost for co-location space?Decreasing the lines of code doesn't require stick ing a fork in scalability. Servers, power, and cooling are NOT free.
Pardon the off topic rant (please feel free to mod appropriately), but I have to agree that RoR DOES SUCK. (At least I didn't set up my own blog just to rant about it.) The RUBY language is the best thing since sliced bread, but Rails isn't. And I'm not talking about the installation and management issues like everyone else. I mean from a developer's perspective, RoR is horrible. Here's why:
.NET and it's various components, but I don't want to work on Windows, I don't want to work with IIS, I don't want to write VB or C#, and .NET really does a bad job of layouts and makes a lot of other stuff more complicated. And it's ORM simply can't touch ActiveRecord. Rails does only a tiny amount of what it could and should.
1) Automated copy-n-paste is still copy-n-paste
Maybe it's changed since the last time I used it, but creating a rails application COPIES a bunch of files from the distribution into the app directory it creates. How do you upgrade? Painfully, that's how. I have numerous small applications that break when I upgrade rails and it's dependencies because the copied files don't work with the newer version. My applications should be 100% code I write directly. Everything else should be kept separate and accessed via includes so emerge/apt/yum/gem dependencies can keep the rails code compatible with itself and I never have to "fix" code I didn't write.
2) RoR gives you the 1% that's used 10% of the time, not the 10% that's used 90% of the time.
So you set up a RAILS app, create your database table, and run 'generate' to get your pseudo-MVC (seen #4 below). You've got everything you need to edit a single table via the web, but that's not even close to an application. It probably saved me about 1/2 of setting things up by hand. That's simply not good enough. It should be able to create an app that supports validation (both JS and server-side for obvious stuff like numeric and lengths), sorting, filtering, searching, relationships, and css skins. It could do this just from the information available in the database metadata, which would get you 90% done. And a huge number of simple apps could be completed simply by writing a custom CSS skin and adding some graphics.
3) No UI components, which are the hardest part of web development.
Most of what rails does buy you is the back-end stuff. It's an easy way to get stared with ActiveRecord, which does the heavy SQL lifting. AR, the one shining gem of RoR, is a great object-relational model and I believe it is responsible for 99% of RoR's popularity. But SQL isn't that complicated in the first place. The real tough part of web development is getting rich, graphical, reusable UI components that work across web browsers. Prototype/Scriptaculous are a wonderful starting place, but I need code that I can feed an AR class (and possibly a list of columns and/or related tables+columns) that will generate cross-browser compatible HTML view of the table complete with searching, sorting, filtering, and paging. There could be functions/objects that render it as a table, a list, a tree, etc. You're probably thinking I should just use
4) It's NOT MVC
The Model-View-Controller design pattern is about limiting the amount of communication necessary by having one instance of some code (the controller) that all access to data (the model) from other code (the views) goes through. Views subscribe to a model, get their data and then do their thing without worrying about other views. If some other view changes the model, the controller notifies all other subscribed views of the change. Rails MVC is something totally different that doesn't solve the same problem. Rails does provide data validation via AR, which is part of true MVC, but it still misses the point of MVC, which is a coherent and always up to date set of views into the model. In fact, an MVC is impossible to implement over the web because communication is one way: the browser must initiate all communication with the web server. (For those that don't "get" this,
Wow, generating and serving web pages in C/C++ - does anyone still do that? Are there any modern libraries that, for example, make your tables look good by emitting different HTML based on the browser? That layer of abstraction is key these days, but I've only seen people doing it in PHP.
A modern C++ library that takes an abstract layout description and creates browser-specific HTML to match that description would be a very cool thing. A C++ library that merely generates "standard HTML, verified by the W3C" would be a nighmare to work with/around.
Socialism: a lie told by totalitarians and believed by fools.
I wonder if they've considered Groovy on Grails instead of Java or PHP? Groovy has all the dynamic language features that make Ruby and JavaScript popular with developers, but is essentially 100% backward compatible with Java (since it's running on top of the Java VM and is implemented in Java). Groovy also leverages Java and any other language that can be hosted on the Java VM because it interoperates with all of that stuff, and itself runs on the same VM. You also get legitimate closures and simplified syntax for many things. It can even give you Java 5 semantics (generics, enumerations, etc.) under JDK 1.4.
Grails is very similar to Rails, but is not a straight port -- more "inspired by," as it's described by its proponents. Grails also doesn't have the problem that Rails has in terms of scalability and performance, since it's built on Groovy/Java (which have a real threading model). No screwing around with creating a zillion Mongrel instances to scale your site, etc. And if you don't care to use the built-in web server that comes with Grails, you can have the Grails framework generate a WAR file which you can deploy in any container like Tomcat or Jboss.
At a recent 3-day Grails training session taught by Scott Davis, I was surprised to hear that some major corporations have jumped on the Groovy/Grails bandwagon, including Mutual of Omaha. For a conservative company to make that kind of leap says something. (Furthermore, they used to be a COBOL shop -- the rationale appears to be that it's easier to get COBOL programmers to make the leap to Groovy first, then switch slowly to Java, as opposed to migrating directly to Java.)
After that 3-day training, I was pretty impressed. The biggest win for Grails seems to be rapid development and deployment, but all the other stuff you get for free in the package makes it something you'd like to stick with. I'd say the thing that most impressed me with Grails was GORM, which makes Hibernate even easier to work with. GORM spoils me, since it obviates the need to write SQL most of the time.