Slashdot Mirror
Hardware Is Cheap, Programmers Are Expensive
Sportsqs points out a story at Coding Horror which begins:
"Given the rapid advance of Moore's Law, when does it make sense to throw hardware at a programming problem? As a general rule, I'd say almost always. Consider the average programmer salary here in the US. You probably have several of these programmer guys or gals on staff. I can't speak to how much your servers may cost, or how many of them you may need. Or, maybe you don't need any — perhaps all your code executes on your users' hardware, which is an entirely different scenario. Obviously, situations vary. But even the most rudimentary math will tell you that it'd take a massive hardware outlay to equal the yearly costs of even a modest five person programming team."
Recently my boss reviewed my schematic and asked me to replace 1% resistors with 2 or 5% "because they are cheaper". Yes true, but I spend most of the day doing that, so he spent about $650 on the task, thereby spending MORE not less.
So yeah I agree with the article that's it's often cheaper to specify faster hardware, or more-expensive hardware, than to spend hours-and-hours on expensive engineers/programmers trying to save pennies.
Or as Benjamin Franklin said, "Some people are penny-wise, but pound foolish." You try to save pennies and waste pounds/dollars instead.
FOX NEWS.com should be BANNED from television and internet. Have the Congress take it over and give us Truespeak.
"10,000! We could almost buy our own ship for that!" "Yeah, but who's going to fly it kid? You?"
With cheep hardware readily available, I agree that, for many projects, it makes no sense to spend lots of time optimizing for performance. When faced with this situation, I optimize instead for readability and easy debugging, at the expense of performance.
But, and this is a big but, fast hardware is no excuse for sloppy, bloated code. Bad code is bad code, no matter how fast the hardware. Bad code is hard to debug, and hard to understand.
Unfortunately, bad or lazy programmers, combined with clueless managers fail to see the difference. They consider good design to be the same as optimization, and argue that both are unnecessary.
I believe the proper balance for powerful hardware is well thought out, clean unoptimized code.
We'll see. The good developers probably won't be in the first wave of folks looking for jobs. I know our company is still in the "we have to figure out how to hire fast enough to do next year's work" mode.
Where having good engineering really helps, though, is in version 2.0 and 3.0 of the product, and when you try to leverage embedded devices with some of the same code, and when you try to scale it up a few orders of magnitude... basically, it buys you flexibility and nimbleness on the market that the "throw more hardware at the problem" folks can't match.
Despite Moore's Law being exponential over time (so far), adding additional hardware is still sub-linear for any snapshot in time. So it's not going to automatically solve most hard scalability problems.
E pluribus unum
From someone who has been there, done that. I can say that throwing hardware at a problem rarely works.
If nothing else, faster hardware tend to increase the advantage of good algorithms over poorer ones.
Say I have an alghorithm who runs at O(N) and another one functionally equivalent that runs at O(N^2). Now let's say that you need to double the size of the input keeping the execution time constant. For the first algorithm you will need a machine which is 2X faster than the current one, for the second O(N^2) you'll need a 10X times faster machine.
Let's not forget that you need not only things to run fast, but to run correctly, and the absurdity of choosing less skilled programmers with more expensive hardware will become painfully evident.
PS: Sorry for the typos and other errors: english is not my native language, and I've got a bit too much beer last night.
Your ad could be here!
This only works for certain cases. Some your problems are too many orders of magnitude too big to throw hardware at them.
Before you do anything: Profile, analyze, understand.
It might be useless to spend a month of development effort on a problem that you can solve by upgrading the hardware. It's also useless to spend the money on new hardware and the administrator time setting it up and migrating programs and data, when you could've just known that wouldn't have helped in the first place.
Two questions I used to ask when giving talks: "Okay, who here has used a profiler? [hands go up] Now who has never been surprised by the results? [almost no hands]"
Before you spend money or expend effort, just take some easy steps to make sure you're not wasting it. Common sense.
For pure CPU driven applications, I would agree with this statement. But NONE of the business applications I write are bogged down by CPUs. They are bogged down by I/O, either user uploads/downloads, network, or disk access.
I have yet to see any application that was fixed for good by throwing hardware at it. Sooner or later, the piper has to be paid and the problem fixed. Someone improved response time by putting in a new server?? Does that mean they had web/app/database/data all on one machine?? Bad, bad, BAD design for large applications, no where to grow. At least if it's tiered and using a SAN with optical channels more servers can be added. Sometimes, more, not faster is better. And resources can be shared to make optimal use out of the servers that are available.
The FIRST step is to determine WHY something is slow. Is it memory, cpu, or I/O bound. That doesn't take a rocket scientist, looking at sar in Unix or Task Mangager in Windows can show you that. Sure, if it's CPU bound, buying faster CPUs will fix it.
The comment about developers having good boxes isn't the same as for applications. My latest job gives every developer a top-notch box with two monitors, I was in heaven. Unfortunately, it can't stop there. I also need development servers with disk space and memory to test large data sets BEFORE they go into production.
Setting expectations is the best way to manage over optimization. Don't say "I need a program to do this", state "I need a program to do this work in this time frame". It is silly to make a daily batch program that takes 2 minutes run 25% faster. But it's not silly to make a web page respond in under 2 secs., or a 4 hour batch job to run in 3 *if* it is needed. But without the expectation, there is no starting or stopping point. Most developers will state "it's done" when the right answer comes out the other end, while a few may continue to tune it until it's dead.
I rarely read replies, it's my opinion and if you thought about your opinion a little more, I'm OK with that.
Everyone knows a blind mapmaker will finish his work much faster on a motorcycle than he will on foot. This is basically the same thing...
-1 Uncomfortable Truth
When I was programmer, we once had a programming job at a large bank. One of our main reports was running across all booked loans and calculated the futural finance stream (interest and amortization) either until the debt was paid off, or up to 40 years at current interest rates. This report was sent to the Federal Bank for control, and to the department tasked with managing the bonds to get enough capital for further loans.
This report took 200 processor hours to complete. To get it done, it was split into 18 tranches, each running 11 hours. So it was possible to complete the job during a weekend run on 18 processors, and restart it twice in case of errors.
A colleague of mine took the task to rewrite the report to speed it up. For that she hooked into each booking that changed the amount of loan or the interest rate, repayment, end-of-contract or amortization and modified it so it wrote a flag into a table.
Then she rewrote the central report to store the calculated finance stream each time it was calculated. Loans that were unchanged since the last calculation didn't have a flag set, so the report took the old calculation. This sped up the report about 150 times: Instead of 200 processor hours now it completed within 1:20 h.
It allowed to put four large RS/6000 out of service, cancelling of the service contracts, rescheduling the report to run daily instead on weekends and saving on weekend man hours. With the daily report to the bond managment department also the finance controlling unit became interested and used the report results to refine their own tools. This together easily paid the amount of programming time put into the report.
As you can see: There are programming task where just throwing more computing power at doesn't solve the problem. It hasn't even to be some high level programming job, sometimes it's a dull task (finding all points in a bookkeeping system where the booking changes the finance stream of a loan is a dull task!), but if someone gets it done, it pays off easily.
Imagine server-software so craptastically written, that the maximum amount of people that can use the app at any one time is, say: 20. Now, imagine that when you double the hardware capacity, user capacity only goes up by a factor of say 1.4
Still sure you're _only_ going to throw hardware at the issue when business wants the application online for a couple of thousand people?
The idea expressed in that article isn't just stupid, it is economy destroying, civilization threatening, mind-bogglingly stupid.
The author is trying to solve the problem of inadequate resources buy spending more to increase the brute force effort toward his already failing solution. It is the mythical man month expressed in CPU horsepower.
That isn't improving your situation, that is merely delaying your inevitable downfall. You're running to stand still, and eventually your organization will collapse of exhaustion, while your competitors, who invested in smart design and smart people, lap your corpse.
And if you simply can't afford better people, then your reach is exceeding your grasp. Scale back your ambition, plan for when you can, or accept your niche and buy the third party solutions produced by experts who can write scalable software.
Besides, one thing that's not covered in the article is that hardware has an exponentially higher residual maintenance cost.
In order to maintain production, many companies these days insist that hardware be in-warranty and thus able to be replaced at a moment's notice. There comes a point as well at which the amount that the hardware will cost on an ongoing basis far exceeds the cost of a single programmer to write a decent app that doesn't need it.
I have recently saved my company the equivalent of my salary, doubled for the next two years purely in the cost of maintenance contracts for around 150 servers. Granted, this was using virtualization rather than programming to combat the problem, but in this case it made sense. The concept is still the same regardless.
If they're watching movies all day long, just fire them. No need to re-orient their monitors.
Worst BBC News Stories
Actually, I think that is the wrong way round. Places like CERN do 'throw hardware at it', lots of hardware, and it still isn't enough.
Modern desktop systems have giga bytes of memory, hundreds of giga bytes of disk and multi core processors ... and in the Adobe example you are using it to display PDF documents or Flash movies.
Your application would typically be using less that 1% of the available resources.
Spending lots of money optimizing the performance does not make commercial sense.
Large science projects like CERN are pushing the limits of hardware and software. They typically deal with data sets, data rates and processing requirements that are orders of magnitude larger that most systems can cope with.
A typical science desktop application needs to be able to process and display giga byte data sets, often comparing more than one dataset visually in real time. A typical eScience grid service needs to be able handle extremely large (peta byte) datasets in real time, and you can't drop data or pause for a moment - the data stream is live and you only get one chance to process and store it.
Same applies to Google, Yahoo, FaceBook etc. If your application is pushing the hardware to the limits, then optimizing the software to increase performance by 5% is worth a lot of developer time.
But the observation on cost also means that good programmers should focus on correctness rather than performance.
Just to illustrate how difficult it is to get correctness right, on page 56 of The Practice of Programming by Kernighan and Pike---very highly regarded book and highly regarded authors---there is a hash table lookup function that is combined with insert to perform optional insertion when the key is not found in the table. It assumes that the value argument can be safely discarded if insertion is not performed. That assumption works fine with integers, but not with pointers to memory objects, file descriptors, or any handle to a resource. An inexperienced programmer trying to generalize int value to void *value will induce memory leak on behalf of the user of the function.
Or, for a modest increase in hardware requirements to get the same performance, we can introduce automatic resource management (aka garbage collection) which makes this particular little difficulty go away.
Pivoting means that ClearType or favorite subpixel rendering of your choice won't work. And I do really prefer ClearType, in the same way I prefer dualmon and high resolution.
The $hardware$ $programmer-time$ equation is always based on the assumption that the programmer is always worth their qualifications.
You are correct that this is an unrealistic assumption but, like the "rational self-interest" assumption in economics, it is a very useful one.
Given a set of uniformly competent programmers, you quickly reach the point of diminishing returns on optimizing performance over hardware - but that's because a competent programmer should implement code with reasonable performance in the first place. Sadly, some people think they can compensate one with the other (competence vs hardware), when that is an entirely different problem, entirely different variable (e.g.: an incompetent programmer with more time is not always a good thing).
First you have to reach the point of competence where you can talk about performance optimizations in the first place. What you describe is not 'unoptimized code', it is not a naive but reasonable implementation - its gross incompetence (assuming SQL qualifications were claimed in the first place).
As you said, you can't pay for enough hardware to compensate for that. But in the same vein, you really, *really* do not want to pay for more of that programmer time either.
Freedom is the freedom to say 2+2=4, everything else follows...
Indeed. It depends entirely on the problem, this is where computational complexity comes in, but cheap programmers wont even know what computational complexity is. The more complex the problem, the more knowledgeable your programmers will need to be in coming up with novel solutions.
You only have to look at most combinatorial optimization problems to see where you may run into trouble, a cheap programmer may try and brute force it and no matter how much hardware you throw at the problem that method simply isn't going to work for all but the smallest of data sets. You're going to have to get someone who knows the tricks (algorithms such as ACO) to produce acceptable solutions in a sensible time frame.
But you don't even need the hardest COPs to demonstrate the types of problems you may run into, even the most basic COPs can throw lesser skilled programmers whilst better programmers can implement a solution without even needing to look up any references.
It's another case of cutting corners. To the companies considering this option; sure if you wanna hire cheaper programmers and throw hardware at the problem that's fine. Just don't come crying when your entire system keels over under the weight of a problem it can't solve with the method implemented to solve it and when you then have to get someone in to do the job properly. Also then when you find yourself with a load of hardware lying round you never actually needed had it been done right to start with.
Cheap programmers are great for throwaway or non-mission critical software, but make sure you have at least some good programmers around who have the computer science background underlying their software engineering abilities to deal with the tough/complex stuff.