Balancing Performance and Convention

markmcb writes "My development team was recently brainstorming over finding a practical solution to the problem that's haunted anyone who's ever used a framework: convention vs. customization. We specifically use Rails, and like most frameworks, it's great for 95% of our situations, but it's creating big bottlenecks for the other 5%. Our biggest worry isn't necessarily that we don't know how to customize, but rather that we won't have the resources to maintain customized code going forward; it's quite simple to update Rails as it matures versus the alternative. What have your experiences been with this problem? Have you found any best practices to avoid digging custom holes you can't climb out of?"

Comments

Expand the toolkit

[thethibs]

Would it be out of the question to assess the needs of the troublesome 5% (and perhaps the other 10% that were shoehorned into Rails) and add a framework that's more in line with those needs?

Data exchange is sufficiently mature that interaction between applications in different environments is not an issue, so all you are left with is the added overhead of supporting two frameworks; not a bad thing if you consider the added flexibility of using the framework best suited to each application. Having options is usually a good thing.

Re:Expand the toolkit

[Bearhouse]

Mod up! If the only tool you have is a hammer, you end up treating everything like a nail.

As the OP pointed point, all development environments sooner or later hit this wall. I've never found one, (and am pretty sure that one will never exist), that ticks all the boxes for a typical mixed environment (the GUIs, OLTP, distributed database management, web access, making the coffee...).

In the 80s, I worked for a company that - as was fashionable at the time - developed its own '4GL' and decreed that everyone use it. In a satellite office, we tried, but found that in certain circumstances it just plain would not do what we needed. Guess what? We went back to 'hard coding' those key functions...

Don't worry about it, (after all, same bits of windows, to name just one OS, are written in assembler to speed performance).

So as the man says, expand your toolbox and your competencies - it'll make you a better project manager (to handle the interfaces) and programmer.

Hard to say, but...

[SlashDotDotDot]

Making this sort of decision really requires more technical details than you provided in the summary or the linked article.

If you do choose to make custom modifications to Rails (or any other third party dependency in your system), make them very carefully. Keep notes about what you changed, and why. Comment the modified source code with a consistent, searchable tag, like

// MYCOMPANY Begin custom changes
// Optimized this block for queries where the
// search string is only one word long, and
// measured a 20% performance increase on the
// production server under moderate load.
// 3-Jan-2009
...
// MYCOMPANY End custom changes

Also, check the third party source into your own version control system so that you can track the changes explicitly.

Since you are talking about making changes for performance reasons, be sure to follow good optimization practices. Measure first, optimize only the true bottlenecks. Measure under realistic scenarios. If optimizations are even slightly confusing or subtle, comment them thoroughly. Keep the original, unoptimized code, maybe just in comments next to the new code, or maybe in a separate "reference implementation" function so that you can fall back to a known reliable version.

Re:Switch to django and python for starters

[latentevent]

Comments like this really don't address the core issues. Frameworks, and especially heavily-developed ones like Django/Rails, are rarely cause for insurmountable performance problems. Of course, caching and setting up distributed workload processing queues will be necessary in any framework, but that's up to the development team working on a specific app rather than the framework itself.

I've been working on a large Rails application for over a year, and we haven't found any scaling issues related to Rails. Instead, there are a number of things that we've done in our application in order to help it scale for our given workload.

That said, there are a number of things that Rails will do to help you scale your application. You may want to look at how Rails 3.0 (the next release of the Rails framework) is incorporating several ideas from Merb to make the framework lighter-weight and more extensible through Rack and the concept of Rails "metals." For offline processing in a Rails app, look at plugins like workling/starling and other message queues. I think you'll find that 1) Rails doesn't have scaling issues, programmers on Rails have several challenges that they have to face to make their applications scalable (just as in any other framework) and 2) As far as providing tools to facilitate developers in scaling their apps, Rails is quite a powerful framework and one that should be given a lot of consideration by development teams.

In conclusion, though, vague comments like "avoid Rails, use Django" are just unproductive spreading of FUD and don't really contribute to any significant discussion or understanding of how to get any of the viable web application development frameworks to scale in any specific case.

Re:Switch to django and python for starters

[Forge]

After reading the linked commentary, I can say there really is a better 4th option. It was even hinted at by the author.

Write your extension/hack as a modification of Rails and/or Ruby. Send them back to the maintainers of said products and see if you can get them inserted as a standard part.

If they do not exact a significant penalty elsewhere for the improved performance in your pretty mundane scenario. It is highly probeble, your changes will be accepted.

This isn't just true for Rails. It's true for any software (except stuff maintained by immature, anal retentive morons.

If it's L/GPL, BSD or other common free license, then this adoption of customer written code is a routine matter. If it's closed source and proprietary they may feel the need to get a lawyer involved and have you sign over all ownership rights to the code you wrote to improve their product (ask for money or stock when that happens).

Either way, in most scenarios good clean extensions tend to get adopted as part of the main program.

PS: Yes. Even "The Evil Empire" (Microsoft, Sun, Apple, IBM etc...) have been known to do this.

Build vs. Buy

[michaelmalak]

What you call "convention vs. customization" is really just a special case of "build vs. buy". You can research the standard factors that go into such a decision, but the most important one is whether the part you intend to build is both a) essential to the business and b) something that would give you a competitive advantage over buying.

I agree with the above poster.

[Jane Q. Public]

I work on a major website, all done in Ruby/Rails. We, in contrast, have found that the tremendous ability to customize the framework is liberating, rather than constrictive.

Without knowing much more detail about your problem, it is difficult to give advice. Having created large, complex websites, and maintained them for long periods of time, we have yet to run into intractible problems as a result of the language or framework themselves. The fact that there are many large, complex Ruby on Rails websites being created and maintained by Fortune 500 companies also contradicts the idea that Ruby and Rails have serious, show-stopping shortcomings.

I strongly suspect that part of the problem is that you do not know your platform (in particular, Ruby) as well as you perhaps could. Ruby is very extensible via Gems that are freely available or that you can write yourself, or even just code modules. Rails is similarly, wonderfully extensible via plugins. Plugins are also freely available for just about every situation you can imagine, or again you can write your own.

Ruby is one of the most flexible languages in existence. Rails is hardly perfect, but any shortcomings are easy to code around by adding your own extensions. But expecting any framework to be complete for your purposes, before you begin, is pretty much wishful thinking.

Once again, I do not know enough about your situation. But if you expect any framework to be "plug and play" for very complex tasks, like an erector set, you will be disappointed. There is no substitute for knowledge of your platform, and programming ability.

Re:Switch to django and python for starters

[arth1]

And when the guy knowing Django moves on, what's the chance of finding a replacements who knows it? How much will that reduce the number of eligible applicants for a position -- will you disqualify fifty really smart guys for five mediocre ones who happen to know Django?

In my experience, being a system administrator for over a generation, what is the most maintainable is to go for standards and well-defined APIs, and document it, document it, document it. Especially document what's obvious and "standard practice", because it may be "standard practice" to do a certain thing in a certain language or certain package, those who come around ten years down the road may not know that this was standard, or why. So document, not so much what you're doing as why. Any coder worth his pizza will figure out what but be clueless as to why.

Ask Slashdot

[starrsoft]

FTFA:

So we have two basic options: pay for bigger hardware, or spend more time writing/maintaining custom code that allows us to live on our existing hardware.

Hardware is Cheap, Programmers are Expensive:

Given the rapid advance of Moore's Law, when does it make sense to throw hardware at a programming problem? As a general rule, I'd say almost always. Consider the average programmer salary here in the US. You probably have several of these programmer guys or gals on staff. I can't speak to how much your servers may cost, or how many of them you may need. Or, maybe you don't need any--perhaps all your code executes on your users' hardware, which is an entirely different scenario. Obviously, situations vary. But even the most rudimentary math will tell you that it'd take a massive hardware outlay to equal the yearly costs of even a modest five person programming team.