Slashdot Mirror
Hardware Is Cheap, Programmers Are Expensive
Sportsqs points out a story at Coding Horror which begins:
"Given the rapid advance of Moore's Law, when does it make sense to throw hardware at a programming problem? As a general rule, I'd say almost always. Consider the average programmer salary here in the US. You probably have several of these programmer guys or gals on staff. I can't speak to how much your servers may cost, or how many of them you may need. Or, maybe you don't need any — perhaps all your code executes on your users' hardware, which is an entirely different scenario. Obviously, situations vary. But even the most rudimentary math will tell you that it'd take a massive hardware outlay to equal the yearly costs of even a modest five person programming team."
Sure, right now it may be more expensive to hire better developers.
But just wait a couple more months when unemployment starts hitting double digits. You'll be able to pick up very good, experienced developers for half, maybe a third of their current salaries.
Sure, invest in some HW now. That stuff will always be handy. But don't just go off and assume that developers will be expensive forever.
Not sure if this site copied Jeff Atwood's post with permission or not, so I'm posting the original link to Coding Horror: http://www.codinghorror.com/blog/archives/001198.html
The Political Programmer
Bears shit in the woods?
It's been this way for 15 years.
Recently my boss reviewed my schematic and asked me to replace 1% resistors with 2 or 5% "because they are cheaper". Yes true, but I spend most of the day doing that, so he spent about $650 on the task, thereby spending MORE not less.
So yeah I agree with the article that's it's often cheaper to specify faster hardware, or more-expensive hardware, than to spend hours-and-hours on expensive engineers/programmers trying to save pennies.
Or as Benjamin Franklin said, "Some people are penny-wise, but pound foolish." You try to save pennies and waste pounds/dollars instead.
FOX NEWS.com should be BANNED from television and internet. Have the Congress take it over and give us Truespeak.
are cheap. But they also suck, so expensive in the long run.
"10,000! We could almost buy our own ship for that!" "Yeah, but who's going to fly it kid? You?"
http://www.codinghorror.com/blog/archives/001198.html
Give the person who actually wrote the article the ad revenue rather than this bottom feeding scum.
TFA says the average programmer with my experience level should be getting a salary of around $50/hour but you'll see I've recenetly advertised myself at $8/hour.
How many hundreds of thousands of jobs have been lost in Silicon Valley alone recently?
The crisis has gutted demand for hardware as well, but things are changing so fast, yesterday's calculations are very likely very wrong. Tomorrow, hyperinflation could hit the US making hardware go through the roof due to the exchange rate.
Seastead this.
Back in the mainframe days (when you were likely to be charged for every byte of storage and CPU cycle, hardware was viewed as expensive. But at least in my career, since about 1980 programmer time is viewed as the most expensive piece.
With cheep hardware readily available, I agree that, for many projects, it makes no sense to spend lots of time optimizing for performance. When faced with this situation, I optimize instead for readability and easy debugging, at the expense of performance.
But, and this is a big but, fast hardware is no excuse for sloppy, bloated code. Bad code is bad code, no matter how fast the hardware. Bad code is hard to debug, and hard to understand.
Unfortunately, bad or lazy programmers, combined with clueless managers fail to see the difference. They consider good design to be the same as optimization, and argue that both are unnecessary.
I believe the proper balance for powerful hardware is well thought out, clean unoptimized code.
Toss as much CPU and memory as you want at a chatty transaction and you won't solve the problem. What about the cost of your 2000 users of the application that wander off to the coffee machine while they wait for an hour glass to relinquish control to them? Over the years I have seen wanton ignorance from programmers that ought to know better about efficiency, scalability and performance.
When the people fear their government, there is tyranny; when the government fears the people, there is liberty.
If you buy the newest hardware gizmo on the market, the geeks will be begging you to let them code for it.
This is the original justification for using high level languages e.g. FORTRAN, COBOL, etc, instead of working with machine instructions.
Just like the debate over mainframes vs thin-clients/cloud-computing, this is an old notion that waxes and wanes in cycles. Besides FORTRAN the biggest single step in trading execution speed for higher level programming was the adoption of Java, unless you include the minority of programmers who get to use a fourth generation programming language at work.
From someone who has been there, done that. I can say that throwing hardware at a problem rarely works.
If nothing else, faster hardware tend to increase the advantage of good algorithms over poorer ones.
Say I have an alghorithm who runs at O(N) and another one functionally equivalent that runs at O(N^2). Now let's say that you need to double the size of the input keeping the execution time constant. For the first algorithm you will need a machine which is 2X faster than the current one, for the second O(N^2) you'll need a 10X times faster machine.
Let's not forget that you need not only things to run fast, but to run correctly, and the absurdity of choosing less skilled programmers with more expensive hardware will become painfully evident.
PS: Sorry for the typos and other errors: english is not my native language, and I've got a bit too much beer last night.
Your ad could be here!
This only works for certain cases. Some your problems are too many orders of magnitude too big to throw hardware at them.
Before you do anything: Profile, analyze, understand.
It might be useless to spend a month of development effort on a problem that you can solve by upgrading the hardware. It's also useless to spend the money on new hardware and the administrator time setting it up and migrating programs and data, when you could've just known that wouldn't have helped in the first place.
Two questions I used to ask when giving talks: "Okay, who here has used a profiler? [hands go up] Now who has never been surprised by the results? [almost no hands]"
Before you spend money or expend effort, just take some easy steps to make sure you're not wasting it. Common sense.
everything ranging from a measly meal to healthcare is so expensive that, any kind of rare labor becomes exponentially expensive. because, people need multiples of pay to make any advance in their standard of living due to cost of living.
in u.s., due to the ease you let mega corporations run rampant because they yelp and wank 'hands off business', you people are paying a fortune for almost anything that is sold for much cheaper in any other country. even, the SAME corporations are selling same products for much cheaper in europe, whereas they are giving you the shaft on the price of the same product in u.s.
'hands off business' was supposed to 'create jobs', 'increase standard of living' and so on.
did it ? what we see currently is totally to the opposite.
the wealth did not 'trickle down' (and why the hell should it anyway), you're losing jobs whilst cost of living almost stays the same (after all, corporations have to make profits, so that they can provide jobs arent they - but where are the jobs), spiral goes deeper and deeper.
i blame this on one thing alone - extremism.
extremism is bad at EVERYthing. every aspect of life, social or personal, without any exception.
when you go extreme on something, you break some other things to the extent that it becomes a disaster. just make a list of such stuff you experienced in your life, and you'll see.
business, economy are just features of social life, and they are no exceptions. if you go to extreme to ANY side, be it extreme 'freedom' or extreme regulation, it breaks down.
america went to the extreme lawless end in the last 30 years. it cost entire world a crisis. n. korea went extreme control in the last 50 years, it cost their people a poverty.
balance is the answer, balance is the key. take europe as an example. with all its faults, the system seems to be working exceptionally well. a lot of petite european countries which should not have any significance at all because of their lack of natural resources and manpower, are producing and creating much more compared to u.s. in a ratio scale. not only that, but the life standard of their people is much, much higher.
one word; balance.
Read radical news here
Throwing hardware at a bad application is ALWAYS the right way to go.
There's an old saying "Never throw good money after bad."
GC
Gregory Casamento
## Chief Maintainer for GNUstep
I always thought the same about managers. Companies are better off spending money on productive workers and the machines they need than with cushy managers who IMHO do work that is way less hard.
I think companies would see real productivity gains from having the programmers (or whatever type of worker) manage themselves... either by meeting together regularly, rotating the leadership position, or preferably a mixture of both.
This way the skills and competency of the workers are enhanced, the decisions are made those doing the work, they have a bigger stake in decisions that are made (because they helped make them) and they can divide that extra high management salary amongst themselves.
In my experience, in professional technical settings the supervised almost always have a better understanding of their worker than the supervisors. And the further up the further up the management train you go, the less likely you'll find anyone with a clue.
Not to mention, this idea of just throwing hardware at efficiency problems kinda bucks the trend of finder lower energy IT solutions.
What a fucking douche-bag.
Good hardware running code written by bad programmers just means the code will fail faster. The primary goal of a programmer is to make the code work, and that does not change no matter how fast your hardware is.
In a lot of big orgs it is amazing how expensive it can be to upgrade your hardware, or add to an existing farm. Not because of the hardware cost, but because of all the overhead involved in designing/specifying the setup,ordering, waiting for it to come, getting space for it, installation, patching, backing up, etc.
In fact I've seen several orgs where the cost of a "Virtual Server" is almost as much as a physical one because the cost of all this servicing it is so high. Whether or not this is necessary I don't want to debate here, but it is undeniably the case.
So I think the case for throwing hardware at issues is not as clear cut as this article implies.
For pure CPU driven applications, I would agree with this statement. But NONE of the business applications I write are bogged down by CPUs. They are bogged down by I/O, either user uploads/downloads, network, or disk access.
I have yet to see any application that was fixed for good by throwing hardware at it. Sooner or later, the piper has to be paid and the problem fixed. Someone improved response time by putting in a new server?? Does that mean they had web/app/database/data all on one machine?? Bad, bad, BAD design for large applications, no where to grow. At least if it's tiered and using a SAN with optical channels more servers can be added. Sometimes, more, not faster is better. And resources can be shared to make optimal use out of the servers that are available.
The FIRST step is to determine WHY something is slow. Is it memory, cpu, or I/O bound. That doesn't take a rocket scientist, looking at sar in Unix or Task Mangager in Windows can show you that. Sure, if it's CPU bound, buying faster CPUs will fix it.
The comment about developers having good boxes isn't the same as for applications. My latest job gives every developer a top-notch box with two monitors, I was in heaven. Unfortunately, it can't stop there. I also need development servers with disk space and memory to test large data sets BEFORE they go into production.
Setting expectations is the best way to manage over optimization. Don't say "I need a program to do this", state "I need a program to do this work in this time frame". It is silly to make a daily batch program that takes 2 minutes run 25% faster. But it's not silly to make a web page respond in under 2 secs., or a 4 hour batch job to run in 3 *if* it is needed. But without the expectation, there is no starting or stopping point. Most developers will state "it's done" when the right answer comes out the other end, while a few may continue to tune it until it's dead.
I rarely read replies, it's my opinion and if you thought about your opinion a little more, I'm OK with that.
I almost feel an order of magnitude more stupid for reading that article. Throwing more hardware at a problem definitely makes more sense for a small performance issue, but this is rarely the case. The whole idea makes me sick as a developer. This reminds me of the attitude of many developers of a certain web framework out there. Instead of fixing real problems, they cover up fatal flaws in their architecture with a hardware band aid. There's no denying it can work sometimes, but at quite a high cost and completely inappropriate for some systems. Not everyone is just building a stupid to-do-list with a snappy name application.
Consider that many performance problems graphically have an upper limit. At some point throwing more hardware at the problem is going to do absolutely nothing. Further, the long term benefit of hardware is far less than the potential future contributions of a highly paid, skilled programmer.
Another issue is there are plenty of performance problems I have seen that cannot be scaled easily just by adding more hardware. A classic example are some RDBMS packages with certain applications. Often databases can be scaled vertically (limited by RAM and IO Performance), but not horizontally because of problems with stale data, replication, application design, etc. A programmer can fix these issues so that you can yes then add more hardware, but it is far more valuable in the long-term to have someone to enable you to grow in this way properly.
Actually fixing an application is a novel idea, don't you think? If my air conditioning unit is sometimes not working, I don't go and install two air conditioning units. I either fix the existing one or rip it out and replace it.
Further, there are plenty of performance problems that can never be solved with hardware. Tight looping is one that I often see. It does not matter what you throw at it, the system will be eaten. Another example is a garbage collection issue. Adding more hardware may help, but typically delays the inevitable. Scaling horizontally in this case would do next to nothing because if every user hits this same problem, you have not exactly bought more time (therefore you must go vertically as well, only really delaying the problem).
The mentality of this article may be innocent in some ways, but it reminds me of this notion that IT people are resources and not actual humans. Creativity, future productivity, problem solving skills, etc are far more valuable to any decent company than a bunch of hardware that is worthless in a few months and just hides piss poor work by the existing employees.
I feel like a return to the .com bubble and F'd Company. I am sure plenty of companies following a lot of this advice can look forward to articles about their own failures. If someone proposes adding hardware for a sane reason, say to accommodate a few thousands more visitors with some more load balanced servers, by all means do so. If your application just sucks and you need to add more servers to cover up mistakes, it is time to look elsewhere because your company is a WTF.
Surely that might work for a one-off, but if you're selling millions or even thousands of copies of your software, even a $100 increase in hardware requirements costs the economy millions. Just because it doesn't cost YOU millions doesn't mean you don't see the cost.
If your customers are spending millions on hardware, that money is going to the hardware vendors, not to you. And more importantly, that money represents wasted effort. Effort that could otherwise be used to increase real wealth, thus making the dollars you do earn more valuable.
So i guess the lesson is, If you're CERN, throw hardware at it. If you're Adobe, get a lot of good programmers/architects.
Can you be Even More Awesome?!
I think you need to complicate this logic a bit by taking into account added electricity required to power the extra servers, run the servers at a higher load, or run the clients at a higher load as well as the air conditioning cost increase as well.
also, time is money. If a program takes more time, there is more time for users to be idle which will also have a cost.
best practice? program as efficiently as possible. Programming expenses are only spent once which the power bill lasts forever.
... throw the money at genuine software engineering (not psuedo engineering) so that we have much better tools by which to program with.
Problem that has nonlinear impact on performance can not be solved by adding of two more servers...
Simplest example is index in database. Before adding of index it takes 2 days to execute it, after adding of an index query executes in 100 milliseconds. How can you solve that by adding of more hardware? Also you usually can not solve IO issues between app and DB servers by "just adding of two more servers"...
Not to mention that when it comes to scaling of DB you really can not just depend on "adding of another server in cluster"...
One thing not in the equation here: Hardware is cheap, but having that hardware managed isn't so cheap. When you scale from a couple of servers to a big bank of server, you have to pick up system admins to manage all of those boxen.
Less expensive than a programmer (some times) but certainly not free.
Custom, hands-free Linux installs. Instalinux
And using them inefficiently is also expensive. If you're looking for a quick fix perhaps you should first consider your company's processes and the tools you use to support those processes. If you can hire a programmer or two to write and maintain tools that allow you to eliminate some of the meetings you have to have every week because no one knows what's going on, you'll find it doesn't take very long for him to pay for himself.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
It is true programmers are expensive and hardware is cheap. But, trying to keep the explanation simple, say you have an algorithm that operates on 100 objects and requires 100^2 = 10,000 computing resources to work. So lets say You want you app to handle 10% more objects. That's 110 so you need (110)^2 computing resources or 12,100 computing resources, a 21% increase in hardware for a 10% increase in capacity. Now 100^2 is very good. What if you had stupid ass programmers that wrote something that's N^4? 100^4 = 100,000,000 computing units, or 10,000 times the capacity to hand 100 objects. And to increase the capacity a mere 10%, you would need (110)^4 = 146,410,000 computing resources.
That's a whopping 46% increase in computing resources for a 10% gain in capacity. Maybe you should just pay a programmer to straighten out that spaghetti code instead? Hardware is cheap. But the best investment a development shop can make is dedicated employees to performance optimization.
But not with enough emphasis. To the suggested procedure:
1. Throw cheap, faster hardware at the performance problem.
2. If the application now meets your performance goals, stop.
3. Benchmark your code to identify specifically where the performance problems are.
4. Analyze and optimize the areas that you identified in the previous step.
5. If the application now meets your performance goals, stop.
6. Go to step 1.
I would add:
0. If the performance is lower than you can possibly fix with faster hardware, skip to step 3 and find out the real problem first.
Hardware upgrades may be cheaper, but software design problems can make a program orders of magnitude slower. I've seen 6 large, sorted database queries followed by a small one that was the only one that really had any reason to be sorted, for example. Hardware can't perform miracles.
When I was young, eager and naive I worked at a place that was doing some pretty heavyweight simulations which took a good three-four days on a (I think) quad-processor Sun box.
It was quite a big site and had a relatively high turnover of decent hardware. Next to the IT support team's area was a room about 6 yards by 10 yards almost full to the ceiling with older monitors, printers and a shitload of commodity PC's. And I'd just stated reading about mainstream acceptance of linux clustering for paralellizable apps.
Cue the lightbulb winking into life above my head!
I approached my boss, with the idea to get those old boxes working again as a cluster and speed things up for the modelling team. He was quite interested and said he'd look into it. He fired up Excel and started plugging in some estimates...
Later that day I saw him and asked him what he thought. He shook his head. "It's a non-starter" he said. Basically, if the effort involved in getting a cluster up and working - including porting of apps - was more than about four man-weeks, it's cheaper and a lot safer just to dial up the Sun rep, invoke our massive account (and commensurate discount) with them and buy a beefier model from the range. And the existing code would run just fine with no modifications.
A useful lesson for me in innovation risk and cost.
Political language
Because throwing more hardware at the problem will fix your software bugs. Oh wait...
Now assume that the application has a low number - say 10 customers per programmer, for a server application, and each customer instance needs 2 boxes. So the programmer optimisation cost is currently around $400 per server per annum.
The root flaw in the article is an assumption that each application has only one customer. That may be true of some in-house projects, but in these cases the main value of programmers tends to be their specialist knowledge of the company and the application. In these cases too, the process of updating and replacing servers taking into account all the internal constraints (likely to be limited by lack of resources) is probably many times the hardware cost.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
1) When the pending performance issue can be solved for the next year's worth of growth with an affordable amount of hardware, buy hardware.
2) If the system is only deployed in one place (not a problem multiplied across 10s, 100s, or 1000s of sites)
3) When the hardware to be purchased is NOT expensive legacy hardware that makes it more expensive to fix the right way later (or makes finance think you don't know what you're doing saying you can replace a $1M AIX machine w/ $40K in commodity hardware, leaving them with a huge capital cost to depreciate).
4) When the marginal cost of hardware is less than the marginal cost of hiring a new programmer. This is true for most Sun machines and all x86.
I've been in the opposite of each of the above scenarios, when we used programming talent to solve the problems because the criteria above were actually the opposite.
The article seems to assume that bad programmers write slow but correct code, which is a big assumption. But the observation on cost also means that good programmers should focus on correctness rather than performance.
Just to illustrate how difficult it is to get correctness right, on page 56 of The Practice of Programming by Kernighan and Pike---very highly regarded book and highly regarded authors---there is a hash table lookup function that is combined with insert to perform optional insertion when the key is not found in the table. It assumes that the value argument can be safely discarded if insertion is not performed. That assumption works fine with integers, but not with pointers to memory objects, file descriptors, or any handle to a resource. An inexperienced programmer trying to generalize int value to void *value will induce memory leak on behalf of the user of the function.
I once had a signature.
Ten years ago many web servers were hand coded in relatively low level complied languages. Even though hardware had become cheaper, and the day of the RAID rack of PCs were coming on us, to get real performance one had to have software developers, not just web developers.
Of course cheap powerful hardware has made that all a thing of the past. There is no reason for an average software developer to have anything but a passing familiarity with assembly. There is no reason for a web developer to know anything other than interpreted scripting languages. Hardware is, and always has been, cheaper than people. That is why robots build cars. That is why ISM sold a but load of typewriters. That is why the jacquard loom was such a kick but piece of machinery.
The only question is how much cheaper is hardware, and when does it make sense to a replace a human wiht a machine, or maybe a piece of software. This is not always clear. There are still reletively develop places in the world where it is cheaper to pay someone to wash you clothes by hand than buy and maintain a washing machine.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
The main goal of writing solid code isn't to lower resource requirements.. it's to increase maintainability.
Sure you can hack out shitty code and make up for it with more hardware to handle the mem leaks and bloat... and probably save some money in the short term. In the long term though, when you need to add something to your mess of spaghetti code.. you're going to spend much more programmer time .. which is what you were trying to save from the get go.
I`m a firm believer that a little extra time and money spent on writing good, clean code will pay off in the long run.
had to be written by someone not working in a real business.
As mentioned by others cpu speed is rarely the case, and even if so optimization isn't always for page response or even speed.
Much of my optimization is to reduce transaction fees and user speed. More machines won't reduce io bound per-user speed.
I wish you could just throw hardware at it. Also even if you do, software has to be changed to accommodate various levels of scaling, you often can't just plug and go. You're going to increase your infrastructure engineering staff and costs, it's gotta come from somewhere.
This is why interpreted or semi-interpreted programming languages make so much sense, especially for stuff such as web applications. Here you can scale to what ever the best hardware is, even changing CPU, without worrying that you will need to recode, or recompile. The same can't generally be said for languages such as C++. Its ironic that you would have to choose a approach that is probably less optimal to get cheaper long term improvements in performance.
Jumpstart the tartan drive.
This uses servers as an example, but what about desktops? We use Windows desktops where I am, and having AIM and Outlook open all the time is more or less mandatory for me. Plus there are these virus-scanning programs always running which eat up a chunk of resources. I open up a web browser and one or two more things and stuff starts paging out to disk. I'm a techie and sometimes need a lot of stuff open.
We have a call center on our floor, where the people make less than one third what I do, and who don't need as many windows open, yet they get the exact same desktop I do. My time is three times more valuable than theirs, yet the company gives me the same old, low-end desktop they get, resulting in more of my productive time being lost - those seconds I wait when I switch from an ssh client to Outlook and wait for Outlook to be usable add up to minutes and hours eventually. Giving everyone the same desktop makes no sense (I should note I eventually snagged more RAM, but the point is about general company policy more than my initial problems).
Practical C programming lists in their "optimization" chapter that hardware is often the most cost effective optimization. It even gives an example of a couple of thousand on a new machine cut execution time in half instantly with no effort. The programmers then did some optimizations, but it was the easy stuff, not hard core.
The book is old now, and the anecdote older.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
The first is that the hardware cost isn't the only cost involved. There's also the costs of running and maintaining that hardware. Many performance problems can't be solved by throwing just a single bigger machine at the problem, and every one of the multiple machines means more complexity in the system, another piece that can fail. And it introduces more interactions that can cause failures. An application may be perfectly stable using a single database server, but throw a cluster of 3 database servers into the mix and a problem with the load-balancing between the DB servers can create failures where none existed before. Those sorts of failures can't be addressed by throwing more hardware at the problem, they need code written to stabilize the software. And that sort of code requires the kind of programmer that you don't get cheap right out of school. So now you're spending money on hardware and you're still having to hire those pesky expensive programmers you were trying to avoid hiring. And your customers are looking at the failure rates and deciding that maybe they'd like to go with your competitor who's more expensive but at least delivers what he promises.
Second is that, even if the problem's one that can be solved just by adding more hardware, often inexperienced programmers produce code whose performance profile isn't linear, it's exponential. That is, doubling the load doesn't require twice the hardware to maintain performance, it requires an order of magnitude more hardware. It doesn't take long for the hardware spending to become completely unbearab le, and you'll again be caught having to spend tons of cash on enough hardware to limp along while spending tons of money on really expensive programmers to try and get the software to where it's performance curve is supportable and watching your customers bail to someone offering better than same-day service on transactions.
Go ask Google. They're the poster boy for throwing hardware at the problem. Ask them what it took on the programming-expertise side to create software that would let them simply throw hardware at the problem.
Consider one of the most common cases. A web application that starts to bog down quickly when it reaches a certain threshold. Let's assume it's not as simple as the network bandwidth (I *wish* my customer's bandwidth doubled every year for the same price!). You've got 200 visitors and they're all responsive, but when you get 300 it's twice as slow, and when you get 350 it's 3 times as slow. Go ahead and replace the servers with double the CPU & RAM, and the next step up of disk tech (raid 5 SCSI to RAID 10 SAS for example) and see if you can suddenly support 400 users.
Part of the developer's skill set, and the reason they are expensive, is engineering and architecture. Consider the mechanical engineering world. Humans have been doing that for a lot longer than programming, so we have a better understanding of a solution that is plain stupid. If a device is underperforming, can you skip talking to an engineer and solve it by swapping out the engine for one twice as big? Have you ever talked to an engineer who said, "yeah, just double the power and that won't affect anything else!" Maybe you've got a steel rod that keeps breaking, and the actual solution is to make a slightly bigger rod out of titanium. But if that actually works, it just means the engineering wasn't done right in the first place. This is how engineering disasters happen.
And finally, good programmers are not dumb. If a problem lands on their desk that can be fixed with a few thousand dollars worth of hardware, they're going to consider it after they've billed that much time. Chances are they've got better things to do because their skills can be better used elsewhere.
It's always programmers that claim programmers are expensive, and hardware is cheap. People who make this claim rarely take other costs into consideration. Like support, power, rack space, etc. I would just love to see someone do a study tracking total project costs against the money spent up front in planning, design, and development.
"Natalie Portman can't act for shit and she has the tits of an 11-year old girl. Grits are bland and best served to the inbred, down-syndrome-afflicted inhabitants of the Southern United States."
OK, OK, ya got me horny, hungry, and nostalgic for the folks back home, but what was your point?
"This post is an artistic work of fiction and falsehood. Only a fool would take anything posted here as fact."
There's more to hw cost than just the initial price. There's also the sw licensing, people support for adding hw to your infra and the cost to move applications from 1 platform to another. If virtualization is used effectively, then you don't necessary need to buy anything new to get more CPU, RAM, whatever. If small servers are used, the ability to place multiple servers on shared hardware is limited and overall company costs will probably be higher.
In general, newer hw is significantly faster and cheaper to run than any 2 year old hardware. It also tends to reduce license costs purely due to a fewer number of CPUs being needed to perform the same amount of work.
As an example, I was forced to redeploy an E6800 wit 8 CPUs when I needed a V240 with 2 faster CPUs. I had to upgrade the E6800 with RAM and HBAs that costed more than 2 V240s including double the RAM and the correct HBAs.
Why? I don't know the answer for certain, but believe the other 16 CPUs were planned to be used by other projects. Personally, I think the company should have traded in that class of server for whatever discount they could have gotten. This happened about 18 months ago. This company had more than 40k servers so adding more without removing some is a really big problem. Most of their data centers were "full" whether by power or space capacity.
Software developers don't seem to understand the complexities of running data centers. Larger volume costs need to be considered whenever making policy decisions concerning new equipment. BTW, I did software development for over 10 years.
If your performance problem is in an Oracle or SQL Server database, throwing more hardware at the problem probably has a license fee attached to it, and that can easily be measured in multiple developer salaries. This also causes people to scale using bigger boxes, rather than more boxes, and that gets you out of the range of commodity hardware and into the land of $$$$$.
Which is why I don't care to deliver on Oracle, but my employer hasn't figured out that Postgres and MySQL will work for a lot of problems, and is still fellating the Oracle and IBM reps.
I was taught to respect my elders. The trouble is, it's getting harder and harder to find some.
Sure- we can easily throw 3 or 4 systems at a problem that can be solved by 1 with good programmers.
But what do you do when the code is unmaintainable and stuck together with bubble gum and duct tape? That too is the results of that way of thinking. You're not paying for good programmers to make things fit on less hardware. You're paying for good programmers to make good software, and a side effect of that is that it runs efficiently.
that's the point - they DO get off on it!
As for the rest, if you REALLY want to improve productivity:
The real productivity killers are poor morale, poor management, poor communications, poor specifications, poor research, lack of time for testing, lack of time for documenting, lack of time for "passing on knowledge" to other people, etc. Not hardware.
Yes, hardware IS cheap. Poor management is the killer - in every field. Just ask anyone who has been on a death march project. Or bought GM stock a year ago. Or who supported John McCain, then watched Sarah Palin become his "bimbo eruption." They all have one thing in common - people who thought they knew better, didn't do their research properly, and then screwed the pooch.
That's a tradeoff that's at the start of every software development project: you pick the tools that allow you to get the job done with overall minimum cost. That's why so many projects are written in Perl, Python, etc.
Unfortunately, many people make the wrong tradeoffs and then don't even follow through. In particular, they pick languages that in theory permit high performance implementations (e.g., C, C++), but then actually fail to write high performance code. A lot of desktop applications written in C and C++ fall into this category.
The pretense of solving performance problems by adding more performance to the hardware is something typical of the MS Windows generation ... instead of clean, optimized programming, relying on more CPU (or whatever) power to solve the problems is a very short-sighted solution. Sure, you'll be able to get your system performance up to par again, but what when you run out of performance yet again?
Instead, check where the actual bottlenecks are ...
I believe, every person who wants to get into programming should get some hands-on training with either ancient systems like Atari or C64, or embedded systems like AVR or PIC. All of which with limited resources, but with decent programming style they are capable of getting the job done well!
Yes we learned that lesson generations ago.
When I started writing code, memory cost $1 per bit, and programmers cost about $4 per hour. If you were writing a program that would be installed on many machines, you could afford very many man hours to save a bit or a byte here or there. It was precisely that misguided sense of economy that created spaghetti code and the worst programming practices ever.
I was personally responsible for a programming horror as a green youngster. I wrote a load-flow program for real time applications. I was given only 100K bytes budget for disk (drum) storage for code plus data plus saved results. To fit in that constraint I had to invent a 16 bit floating point number format. That made my application unusable and useless in the long run.
After the end of the project, I belatedly learned that all the other programmers routinely exceeded their memory constraints by 300-400%; and by doing so they managed to deliver something useful.
I think this is one of the main reasons (AJAX etc. aside) why JavaScript became popular and accepted lately. Why waste YOUR server resources generating structure of the page? Just vomit raw data with adequate classes/id's and let THEIR machine structure it client-side. Makes perfect sense to me.
Although I'm an advocate of buying good hardware, and not getting silly on the hardware to developer ratio. You must also note that more complex hardware requires more skilled/talented operators and more numerous hardware requires more skilled/talented operators and either one runs up the electric bill. Most of the cost of a server is operator costs, NOT purchace. So be careful on how you run the numbers. With that said, custom software development should be a choice of last resort. I am one of those guys, and if I'm bringing less than a factor of 10 improvement in speed in a redevelop project, it probably would have been cheaper to just buy more hardware.
It depends on how many instances of the code will be run. In the innermost loop of an PDE integrator, which will be run in a monte-carlo simulation on 1000 cores, even a small time saving can be valuable and pay off immediatly.
If you are running on a system which has intrinsic limitations (e.g. embedded systems battery) you also may find it useful *NOT* to run into the platforma limitation without need (changing the platform may be expensive).
However, if its about aggregating the database in the evening, the it maybe does not matter.
In the long run, your best investment is still the good programmer, as long as you can keep him happy and productive, because then you can grow more/faster (by buying hardware as well).
"I love my job, but I hate talking to people like you" (Freddie Mercury)
I've done this many times. It boils down to simple stuff and its all a trade off; time vs money vs quality. Time is usually the key where I work as the environment is a highly changing one. Deploying new hardware is generally less risk and faster than upgrading software, and uses different people from the (always too busy) software teams. So it buys you time which is often fine for the business folks that don't care about how inefficient that algorithm or SQL query is as long as the business needs are met. However, there's only so many times you can band-aid this stuff until hardware cannot solve it. When you do new software, its a full development cycle. If it requires major rethink on the design you can bet new issues are raised in production when it goes live; the business do care about this and you have to do some extra preparation to deal with these new risks as the new software is rolled out. Having gone all through is, the goal-posts change again. New activity happens from business that drives software in ways you didn't imagine or get the chance to implement for ahead of time. You're back to square one on the hardware versus software again.
Programmers and efficiency are not unrelated free market resources. Good coders write more efficient code regardless of weather thet is a primary goal. While premature optimization is athe root of all evil, claiming there is no need to optimize at all is equally a fallacy.
Andy Hertzfeld, engineer on the original Macintosh team:
Steve was upset that the Mac took too long to boot up when you first turned it on so he tried motivating Larry Kenyon by telling him well you know, how many millions of people are going to buy this machine - it's going to be millions of people and let's imagine that you can make it boot five seconds faster well, that's five seconds
times a million every day that's fifty lifetimes, if you can shave five seconds off that you're saving fifty lives. And so it was a nice way of thinking about it, and we did get it to go faster. (PBS, Revenge of the Nerds, Part 3)
Throwing hardware at a problem means the writer failed to use his sysadmin staff to do basic capacity planning while there wasn't a problem.
And as johnlcallaway, said, the problem isn't usually CPU: most bottlenecks are either disk I/O or code-path length.
I'm a professional capacity planner, and it seems only the smartest 1% of companies ever think to bring me in to prevent problems. A slightly larger percentage do simple resource planning using the staff they already have. A good example of the latter is Flickr, described by John Allspaw in The Art of Capacity Planning, where he found I/O was his problem and I/O wait time was his critical measurement.
Failing to plan means you'll hit the knee in the response-time curve, and instead of of a few fractions of a second, response time will increase (degrade) so fast that some of your customers will think you've crashed entirely.
And that in turn becomes the self-fulfilling prophecy that you've gone out of business (;-()
Alas, the people who fail to plan seem to be the great majority, and suffer cruely from their failure. The last few percent are those unfortunates whose professional staff planned, warned, and were ignored. Their managers pop up, buy some CPUs or memory to solve their I/O problem, scream at their vendor for not solving the problem and then suddenly go quiet. The hardware usually show up on eBay, so I think you can guess what happened.
--dave
davecb@spamcop.net
Both India and China who are sucking the jobs have tied their money to our dollar. Yes, china has it in a "basket" that does not match any known formula. They have it fixed. In addition, India recently was trying to strengthen the rupee against the dollar, but numerous companies like Qwest and Verizon threatened to pull out. They dropped trying to untie them.
I prefer the "u" in honour as it seems to be missing these days.
You hit the nail on the head. Of course there is always what's in my sig too. (just in case I ever change it: For every problem there is a solution that is simple, elegant, and wrong.)
One reason corporations don't like part-time is that as long as you are full-time, you actually tend to work way past 40 hours a week. You do whatever it takes to get the job done, under impossible deadlines.
Quite the opposite in many cases. Companies like part-timers because they can be worked right up to 39.999 hours per week, thereby allowing the company to avoid paying any bennies (talk about violating the spirit of the law.) Oh, need more hours? Just hire some more part timers. A lot of big companies that laid off huge swaths of their workforce found out that the work still needed to be done ... so they hired back the exact same people for a fraction of the cost. Bit hard on the workers, of course, who suddenly found themselves umemployed and then re-hired, but for less pay and no health insurance or retirement benefits. This kind of treatment of your employees makes perfect sense: that is, if you're a bean-counting android CEO (or a human sociopath in the same position.)
The higher the technology, the sharper that two-edged sword.
Speaking of which, are there any good tools for profiling your system in a little more detail?
At the consumer level, the knobs I can turn are:
CPU speed, Last-level cache, RAM size, RAM speed, FSB speed, HDD size, speed, RPMs, GPU speed, GPU RAM size, GPU bus type&speed, GPU shared memory (if applicable), and probably more, but those are the easy-ies.
But top and task manager are not nearly detailed enough to make guesses about which improvements would actually be useful, and if you've got an upgrade budget of one or two hundred dollars, you can only make one dramatic change or two or three small changes. Across the board = new system, but then I have to guess what combo of those is most efficient.
anyway, I'd really like to be able to buy a cheap, but upgradable system, load the software I'm going to use on it, and then determine what additional upgrades, if any, to bother with.
For instance, does an extra 4 MB of L2 cache really make a difference? (most importantly, would it make a difference to me?) Would it be better to just spend that money on RAM, or would the applications I'm running not actually need any more?
Can you be Even More Awesome?!
When I was programmer, we once had a programming job at a large bank. One of our main reports was running across all booked loans and calculated the futural finance stream (interest and amortization) either until the debt was paid off, or up to 40 years at current interest rates. This report was sent to the Federal Bank for control, and to the department tasked with managing the bonds to get enough capital for further loans.
This report took 200 processor hours to complete. To get it done, it was split into 18 tranches, each running 11 hours. So it was possible to complete the job during a weekend run on 18 processors, and restart it twice in case of errors.
A colleague of mine took the task to rewrite the report to speed it up. For that she hooked into each booking that changed the amount of loan or the interest rate, repayment, end-of-contract or amortization and modified it so it wrote a flag into a table.
Then she rewrote the central report to store the calculated finance stream each time it was calculated. Loans that were unchanged since the last calculation didn't have a flag set, so the report took the old calculation. This sped up the report about 150 times: Instead of 200 processor hours now it completed within 1:20 h.
It allowed to put four large RS/6000 out of service, cancelling of the service contracts, rescheduling the report to run daily instead on weekends and saving on weekend man hours. With the daily report to the bond managment department also the finance controlling unit became interested and used the report results to refine their own tools. This together easily paid the amount of programming time put into the report.
As you can see: There are programming task where just throwing more computing power at doesn't solve the problem. It hasn't even to be some high level programming job, sometimes it's a dull task (finding all points in a bookkeeping system where the booking changes the finance stream of a loan is a dull task!), but if someone gets it done, it pays off easily.
It's one thing to say that you should not spend too much time making optimizations in business applications because they seemingly don't care. However, that is not a license to do things that result in O(n^2) or even O(c^n) performance hits. If I have had a nickel for every time I've seen a double loop to compare lists, or, the effective double loop, like, have an invariant select inside ... it goes on and on and there's a lot of stupid programmers out there and the laws of physics are pretty much on the side of the smart programmers for now. Stupid programmers can make programs too slow and too big to run, without even really thinking about it, because they can't, or they don't.
This is my sig.
Not investing in programmers may cause problems, that no amount of hardware can solve. A difference between a linear search though and array vs. implementing a hash-table explodes very quickly, for example, and no hardware can keep up.
So, if you only have junior developers, who a) don't anticipate this sort of problems; b) only test on small data-sets, you'll hit a wall — and will need a (very expensive) consultant to solve a customer's production problem. A human being can't distinguish a millisecond from a microsecond. Times a million, however, the former turns into 15 minutes, but the latter is only a second...
Then comes the difference between programmers, who can write a program, and software engineers, who can maintain a software project — not only writing the first version, but managing the project's growth, implementing (and enforcing!) automatic non-regression tests. If you ignore this last aspect, for example, as one company I know did, you'll paint yourself into a corner very quickly with per-customer source-code branches. Why? Because, without proper non-regression tests, fixing one bug may either create or expose another. Bitten by this, your customers will reject upgrades with "pre-emptive" fixes for problems, they haven't encountered (yet) — they'll be (justifiably) afraid of encountering problems, your recent bug-fixes created instead.
In short, hardware is only one part of a computer system, and solves only some of the problems, that better developers can solve. You may not need many of them (indeed, this leads to its own set of problems), but you need at least some of them to be really good.
(Oh, and just as we thought, hardware is fast enough for just about anything, XML was imposed to set us all back ten years or so...)
In Soviet Washington the swamp drains you.
"Given the rapid advance of Moore's Law, when does it make sense to throw hardware at a programming problem? As a general rule, I'd say almost always.
even the most rudimentary math will tell you that it'd take a massive hardware outlay to equal the yearly costs of even a modest five person programming team
All the hardware in the world isn't going to fix an insidious segmentation fault, or ensure that your database queries properly handle all inputs, or rework a poorly designed algorithm that runs in O(n^n) time. "Throwing hardware at a programming problem" is like trying to fix a flat tire by putting more gas in your car.
Life would be easier if I had the source code.
If it's a one-off program, or a program for in-house use, then throwing hardware at it might make sense.
I have a deployed base of about 13,000 locations, with 5 - 10 systems per location. Spending $100 per on more memory is talking some real money.
The preferred solution is to not have a problem.
Solution: Robot Programmers!
The days of the digital watch are numbered.
When developers ask for a new monitor or dual monitors, let them have 'em but mandate that the monitors be in a vertical orientation as opposed to the typical horizontal orientation. That way, they'll have to use the monitors for efficient viewing of code rather than watching movies all day long. It's such a simple idea that I'm surprised that more businesses and coders haven't caught on to it.
Just throwing hardware at it is what I like to refer to as "poor man's economics." Poor man's economics works like this: you go to K-mart and buy a $20 pair of shoes. They last about 3 months, so then you go back and buy another pair. After a year you've spent $80. The smart guy goes to the Nike outlet and buys a $70 pair of shoes which last the whole year and don't hurt his feet.
If the software is inefficient enough that additional developer resources could significantly reduce the hardware requirements then chances are unfortunately good that you face another problem with the software: it isn't just inefficient; it's also incorrect.
Scaling a software system up doesn't mean getting it to run under heavy load. It means architecting the system for parallelism and getting it to produce correct results under heavy load. If your system depends on a database (it usually does) and the database has a probability of bugs corrupting data that varies in proportion to the number of records and the queries per second (it usually does) then just throwing more hardware at it isn't going to solve your problem for long.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
Every time you buy hardware, you pay for it. Software can be copied an infinite amount of times for the price of the developer. If you are selling things in quantity, it makes sense to always solve problems with software and not hardware. If you're buying one or two, then hardware can be cheaper.
The idea expressed in that article isn't just stupid, it is economy destroying, civilization threatening, mind-bogglingly stupid.
The author is trying to solve the problem of inadequate resources buy spending more to increase the brute force effort toward his already failing solution. It is the mythical man month expressed in CPU horsepower.
That isn't improving your situation, that is merely delaying your inevitable downfall. You're running to stand still, and eventually your organization will collapse of exhaustion, while your competitors, who invested in smart design and smart people, lap your corpse.
And if you simply can't afford better people, then your reach is exceeding your grasp. Scale back your ambition, plan for when you can, or accept your niche and buy the third party solutions produced by experts who can write scalable software.
There should always be someone who says "should we be doing this" - whether it's adding a feature, or even the whole project. I've always maintained that the most bug-free code is the code that never gets written because the [ feature | idea | whatever ] is dropped. Part of the problem is that the people involved in "conceptualizing" what they think is a "really good feature" have terrible math or other instincts.
Then there's the whole "web apps" thing. People confuse "web apps" with "web pages." The two require completely different skill sets.
Then there's the problem with "the browser as application platform" model. Eventually, we're going to have to realize that it's better to eliminate the browser and let our apps communicate directly with servers, if we want to have better security and performance.
... or worse, can't read source code because their tool "hides all that" from them.
It's thinking like this that gives us poorly performing garbage like Java.
The author's not a Linux/BSD Kernel developer.
Minor tweak to your presentation-- BIG dual monitors.
We have 16 cores and 32GB in our product and still need to optimize the code very carefully. Right now we're looking at Assembler for some of the inner loops. Can't find better algorithms any more and Moore doesn't help enough.
thegodmovie.com - watch it
Sometimes the work of the programmers end up making the system requirements increase. I work at a hospital, and we've been using the same DEC Alpha server for 14 years. Last year we upgraded our hospital registration and billing software, and ever since then we've been having huge performance issues.
We got an initial quote (I'm not sure where from) of $400k to $500k to upgrade the hardware. I'm not sure what was in that quote, but I hope that included some really fancy stuff. We're far from ready to close the deal, but at those prices more programmers start looking better.
Who will be the first to post "ICodeInJavaWithClassesWithReallyReallyReallyLongNames.youIgnorantClod();" ?
Then there's the problem with "the browser as application platform" model. Eventually, we're going to have to realize that it's better to eliminate the browser and let our apps communicate directly with servers, if we want to have better security and performance.
Exactly, the problem with that is that then everyone builds Windows only apps for their sites, locking out Linux and Mac users
Hardware is cheaper than programmers! Fire all programmers and when client bitches about bugs, just add more ram! :-)
Don't blame me, I didn't vote for either of them!
In the 80's I worked on an large software project. We had a prototype that showed that the general design was solid, but a complete implementation would take too much memory and run too slow to meet the requirements. We the Software Engineers were nervous and wanted to redesign parts of the system. The manager, however, wasn't worried at all. He told us to go ahead with a full implementation of the original design, because the hardware would catch up. Of course, he was right. The final system worked beautifully, nestled in a corner of the RAM footprint, greatly exceeding the speed requirements. Never waste money optimizing software for hardware deficiencies that will go away. But DO optimize software for inefficiencies that aren't related to hardware! The fastest computer on the planet still takes forever to finish an infinite loop.
There are a good number of brilliant young people coming up right now who would really love to do some indoor work with no heavy lifting. Not all of them are going to get the chance. It's a horrid waste that many of them are going to go intellectually numb bagging groceries or ruin their backs hanging drywall. They can be had for cheap, at first, and grow into productive and loyal members of your team. If you can find them.
Help stamp out iliturcy.
...Sun Microsystems.
--k
I agree. I recently brought in an electronic computer to my workplace. It _really_ improved my efficiency a great deal.
Other points, especially in programming, is good education and the freedom to choose the right tools for the job.
If you quick and dirty code has any N^2 operations you could throw $1,000,000,000 of 10 year from now hardware and still not have it go fast enough.
Almost all N^2 and N^3 algorithms are fast as lightning in small test cases. But the junior programmer throws his hands in the air and gives up when it explodes one day for no obvious reason. 11 hits per second for 2 hours instead of the normal 10 hits per second that youlve gotten for the past year. It is very hard to stress test a system sufficiently to reveal all the critical masses needed for catostrophic collapse.
But the real point is that hardware won't do squat in many inappropriately applied algorithms.
Consider simple DB nightly maintanance. After a day of high volume you might reveal such a polynomic algorithm such that shutting down mysql with kill -9 is the only way out. If you had 10x faster hard drives or more memory, it would only take an extra 5% of extra volume to oversaturate you again.
It isn't just O(N^2) you have to worry about. You have synchronization points in code which cause backup just like normal car traffic jams. DB operations. Mutex's. Rpc calls. All need to be carefully considered. You need these thoughts AS you code, not after a profiler or busy day reveals them.
-Michael
What it usually meaning to "throw hardware at a problem" is that an organization is going tolerate a bad design. The problem is an organization also requires highly paid people to manage a large hardware configuration. Throwing hardware at a problem often requires that your to tweak the basic program for the larger hardware configuration. This takes away from the time that a programming team could be spending on improving the basic design of the system.
Programmers may be expensive. But if you have a poor algorithm, one that does not scale, then you will throw an exponential amount of hardware at the problem.
This article seems like something of a step backwards to the attitudes of the 90's. One of the Really Good Things about modern lightweight methodologies like Scrum and XP is that they are getting away from seeing the programmer as a dollars-per-hour resource and realizing that lack of quality is always more expensive. Doing the job poorly always cost more in the long run. A management system that thinks otherwise will fail in any reasonably long run.
What is being ignored is the fact that programmers are already doing this with Open Source. Zero expense programmers. But that is not enough.
Fewer decently paid programmers means less innovation - No Google because the machine that replaced the programmer decided it would require pesky 'expensive' programmers to create it and therefore did not.
An underpaid programmer has little incentive to innovate simply because the other aspects of life drain his or her energy away from innovation to survival. The main point of paying people well is so they have fewer distractions, no worry about heating the home because there is not enough money coming in. You do get what you pay for. Well paid programmers are more productive, happier, and that translates into a sustainable business.
Algorithms designed to optimize profit will eventually decide it is cheaper to engage in genocide to eliminate most of those expensive and pesky humans in order to keep fewer and even more expensive humans happy (until they get eliminated in a recursive loop that keeps deciding they are now too 'expensive') - Microsoft Windows Genghis Khan can do this using it's scheduler to repeat the algorithm each month, say.
We should replace those expensive workers with machines forthwith and take this country back from what it decided to engage in, the notion that all 'men' seek happiness and therefore need to be kept busy doing something useful instead of trying to eat the last pig in town because the machines have decided its too expensive to employ anyone to grow food and distribute it.
Besides, it will costs a few *illion to figure out how to first create the proper 'failsafe' algorithm, debug it, and then have a firewall preventing any human with nefarious aims from taking it over for personal gain. It can't be done because humans are too imperfect to make something perfect (other than Open Source).
Sounds like 'Metal Gear Solid 4, Guns of the Patriots'. A hero human will be required to rescue us from the bots?
I've got about 60 3GHz small form desktop PCs with 512MB of RAM and gigabit Ethernet sitting on a couple pallets. Since I already have a few DRBL servers set up for netbooting compute nodes, setting them up as a render or compute cluster would only take a couple days. I've done a few pilots of this in my basement with 4 nodes and the system is said to scale fairly linearly. If I had the watts I could probably get a pretty respectable cluster going with little trouble, and the experience would probably come in handy. But I can't afford the time and I don't have the watts.
It's a shame, too, because I could probably come up with a few hundred of these machines a year at negative cost. Companies are actually paying to be rid of them. Writing apps for this type of cluster would be... educational.
Help stamp out iliturcy.
O(2^n)
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
There's an economy of scale in deployments. Every extra system image requires a ton of development and deployment time, troubleshooting and repair become more problematic and expensive, and these costs rise exponentially with the number of variant systems. This is why OEMs offer "long term support" product lines that are guaranteed to be image compatible with each other over a span of 7-8 years.
In short, your salary isn't the only factor here. One way to leverage extra money for more expensive staff is to give them larger, higher resolution monitors and/or multiple monitors, memory upgrades, or other enhancements that don't alter the system image. This allows them to get more work done more than just having a faster PC would without incurring the extra expense that having a bunch of different types of computer would.
Help stamp out iliturcy.
If they're watching movies all day long, just fire them. No need to re-orient their monitors.
Worst BBC News Stories
'things are expensive' does NOT solely mean beer, food, tshirts, whores and whatnot.
,say, $100 a month in u.s., but selling it from $20 in finland despite avg. purchasing power is $300, that means they are selling it more expensive in u.s.
EVERYTHING counts. ranges from healthcare costs you pay to the gas station, to prices of houses and whatnot.
moreover, it is also related to average purchasing power of the people and its distribution.
a shoe may be sold from $10 in u.s. and $20 in finland. but what matters is, HOW expensive in regard to average person's purchasing power is that 10 or 20. if the corporation is selling it from 10 in u.s., despite avg purchasing power is
get a clueeeeeeeee
Read radical news here
As an IT Operations person, I've encountered this argument a few times.
The answer is never simple. Sometimes more hardware is the answer, sometimes more / better programmers are the answer.
If you disagree, please explain to me how higher end hardware will fix issues where the code gets stuck in an infinite loop. Or a dead lock. Or a . . . (You get the idea)
This is wrong on so many levels it's down right scary. It's the sort of thing that PHBs use all the time. There are a ton of hidden assumptions that make this so very wrong.
Programming is a fungible commodity.
There is always a trade off between development time and performance.
Just code it, and if there are any problems we can always go back a do a quick fix.
Don't get me wrong, I was doing agile development before it was called that, but most of the time the demos and prototypes were thrown out afterwards.
This is all coming from a guy who actually has made a living performance tuning code.
First off, USD 5K is a joke for big shops. So right off the bat this limits the scope of the problem, of course PHBs will never understand that.
Good design means faster, better & cheaper then poor designs. Which is why you want a really good architect. The difference can be several orders of magnitude in performance and be much more accurate. Which is why engineering is not fungible.
In my case I cost about 3X more then the average developer and can often be 10x more productive, for relatively simply things.
A case in point. A job was taking 150 minutes on production and needed to take 30. The server was a big honking HPUX box. It would have cost USD 100K or more for the upgrade and then it was not clear how much of a speed up it would be, because it was an Oracle SQL app. The client had their people spend several weeks, say 120 Hrs on it with almost no benefit, just a few minutes.
They gave me 20 Hrs to examine it and write a prop on how to fix it. 12 hrs later I handed it back, it was running full volume on their test box (smaller then prod) in 18 minutes. About a tenth of the time and with the results they needed.
If the original developer had bother to spend a little time understanding the data model they would have realized they were doing a partial Cartesian. Creating a 140 million records to then filter out 120 million of them was not really very effective.
In my career, and granted it has be only on large servers, it is always better to get a few really good people rather then a lot of average ones.
Really good people even negate part of Brooks Law. They don't slow the project down because they can become productive almost immediately without much help from existing developers.
The same client gave me an other query to speed up. While pulling it apart to better understand it a logic error was also uncovered. So, not only did they end up with a faster running query but they also got one which was correct.
Clients usually call me in when things are in big trouble. Getting the cheapest possible resource is rarely a good project plan. You might be able to overpower a problem in Access with hardware, but it rarely works with Tera-bytes of data.
Ok, I'll bite .. what do you actually use the 2nd monitor for ?
I find that throwing hardware at bad programmers helps to solve this problem. If you want to save some cost, throw bad hardware at bad programmers... you get to throw harder too. The more you practice, the better your throw becomes.
Although you mention scalability and flexibility, I don't think you really hit the nail on the head.
Performance and scalability are NOT the same. They are fundamentally different. You can have a weakly performing software product that scales nicely, and you can easily have a high performance application that doesn't scale at all.
Understanding this difference can be the make/break point in whether or not a mildly profitable company can become a world-changer! It's fairly easy to write high-performance software. But it's quite a bit more difficult to build software that scales!
It all really comes down to understanding the Schlemiel the Painter algorithm which is RAMPANT in software designs.
Quite literally, there is simply no way to avoid these types of algorithms, but by designing your software correctly, you can limit the effect of these algorithms on the overall scalability of your software stack as the problem set grows larger and larger.
And that's software that scales. For example, PHP often scales very nicely, because although it's not a fantastic performer, it's "share nothing" approach means that adding more processes and/or servers doesn't particularly impact your original infrastructure. But if you don't design your application right, PHP can scale miserably, depending on how you manage your resources.
If you write software, ask yourself: what if the whole world were using your product? Could you handle it? Whatever your answer, if you feel sure of your answer, it's probably because you don't yet understand exactly what it means to scale.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
I think there will be more developers looking for work in the future, but I don't think the price is going to drop THAT much. I just think you'll be able to find qualified developers more easily.
Yep, with the increasing number of H1-Bs from India replacing laid-off North American IT personnel it is a race to the bottom. Them with their fake degrees and phony work experience are washing onto the shorelines as we speak.
As for the article, it makes a lot of sense when you're running in a controlled environment. It's really a no brainier in consulting work. Upgrading hardware or optimizing software will both meet the customers needs only the hardware upgrade is $2,000 and the software optimization costs $20,000.
Of course, if you're releasing software into the wild and it needs run on many different machines you better make sure it performs well especially if it's a retail product. So spend the extra money and make it really good.
I must have completely missed what this article is about... hardware cannot replace a human. The largest supercomputers in the world could not do in a million years what a human could do at any moment: reason, think, feel.
Want to reduce costs? Keep employees from reading and posting on stupid Slashdot articles.
However, I have seen plenty of example where that is not true.
Enterprise databases are a great example. While the DBA can often do alot to compensate for dev's bad db code ... sometimes that just is not enough.
A fulltime employee can work on optimizing code for an entire year before it pays to add another node to the cluster. (server hardware, hba's, fiber switch ports and most importantly oracle licenses). Interestingly enough PHB's often times choose expanding the infrastructure eventhough the ROI is not there citing opportunity cost, etc.
Another favorite are processes running on new hardware ... the dev just simply did not consider parallelizing the tasks and opted for a monolithic single process approach.
From my experience most of these issues dont go away by horizontally scaling and eventually they come back to haunt you one way or another.
Surprisingly, the added cost of administrating added infrastructure is never part of the calculation ... the only consideration is the cost of hardware.
Pivoting means that ClearType or favorite subpixel rendering of your choice won't work. And I do really prefer ClearType, in the same way I prefer dualmon and high resolution.
Our Labor Cost in the US is too high. We have long had the ability to feel we are entitled to new TV's, computers and other things and thus feel justified in a high salary and an insular economic attitude. We can no longer afford this. When programmers in India can be hired for $20,000/yr we need to reassess our expectations for our standard of living. I know this pisses off a lot of people. However, look at Detroit. The UAW demanded $79/Hr to build cars and they were still crap. If we insist on these kind of wages and the standard of living that comes with them then we are dead in the water, especially with the economy tanking now like it hasn't in 25 years.
Hardware is cheap,
Programmer time is less cheap,
Operational costs will kill you.
Hardware is cheap because it's a sunk cost, and for most applications, much less than the sunk cost of the programmer time in writing the code installed.
See? Two sunk costs.
Operational costs are with you for the life of the project. If it was well designed, well written, you need maybe a trained monkey once every three months. Almost nobody does that, and besides, many operational costs scale with hardware inputs. Electricity, floorspace, cooling, repairs, networking, monitoring, logging, auditing, patching ... there's a lot that goes on which a lot of developers never even bother thinking about.
The best code I have ever seen coming out for a serious size of bespoke installation was the direct result of a developer going to the admin corps and asking how to make his code awesome. It was awesome, and while every other facet caused a stream of headaches, his stuff was bulletproof.
Lesson learned on my end.
I ran into yet another IE7 css bug. Doesn't support table-row and table-cell attributes, so I had to switch to inline-block.
However, css doesn't yet support specifying more than 1 column or row for any particular cell - it really is time to start moving client-server apps away from their dependence on browsers. It's one of the reasons I've been boning up on (*gasp*) java.
over 9000 cpus!!!!!! in all seriousness, this get used so many times, especially by people who use scripting languages (e.g. python/perl/php/ruby/javapoo/next fad) ... who cares if the run time runs like sticky poo down the toilet, just throw more hardware at it.
Keep a few shells open for database, man pages, tail -f /var/log/apache2/error_log, dos2unix [filename] for code that someone else munged up, toolbars (layers, tools, etc), compilers (gcc, g++, javac), which, mc, tree, chmod [perms] [filespec], email, searching, testing, notes, etc.
You can also stretch an IDE or code editor across 2 monitors.
Ok, I actually RTFA because the stub made no sense whatsoever and found out the article is no better. Seriously folks, how many "problems" in computing can be solved by legacy software running on faster hardware? Answer: Very few. Here's a for instance:
I have a piece of code that was written in 1983. It still does everything I need it to do, but I am taking on larger problem sizes with that code and it's starting to run slower than I'd like.
Without updating the code to take advantage of modern hardware, i.e., multi-core, throwing bigger and faster hardware at the code will not yield the same performance enhancements that throwing an intern, grad student, or other development resource with parallel programming experience at the code. A majority of code written before and even since 1993 is not parallelized and will not gain much from bigger, faster hardware.
I've seen this argument a thousand times where I worked in academia. Researchers would buy bigger, faster desktop machines to make their code go faster and hit a plateau in performance because the code would not be able to take advantage of this hot new machine. They would then back away from taking on bigger projects with bigger funding because they were unwilling to update their applications. We would then suggest to those that came to us to sit down with a group we setup that would help them rewrite their code so it would work better on modern hardware. It's an arduous and relatively unsupported process, but it needs to be done. Old code and faster hardware will only take you so far, and the sooner you adapt the code the better off you are going to be. Just throwing hardware at a problem is far more wasteful (from a productivity and ROI stand point) than writing better code and being more productive with that hardware.
My $0.02...
As far as I can tell, this hasn't been mentioned in the comments so far.
This is completely moronic - the measure of a programmer is not how fast their code runs. More talented programmers make fewer errors, and better design decisions.
There's a saying that cheaper programmers are more expensive. Finding and fixing bugs is an immensely expensive undertaking, and cheaper programmers make more of them. Bugs cannot be fixed regardless of how much hardware we throw at the problem, and software quality cannot be recovered no matter how many cheap programmers you use.
Better programmers also make better design decisions - more elegant, maintainable and extensible code. Software version 2 is a much easier undertaking when most of the development time is not squandered understanding, rewriting and fixing version 1 code.
That is not to say that throwing more hardware at the problem is a bad idea. We already do - all these managed languages with automatic garbage collection and so on - they free the programmer from worrying about memory management so they can spend more time developing the functionality. This comes at a cost of creating a more compute and memory intensive application.
--
#include <malloc.h>
free(your.mind);
Maybe in America and other developed countries. But, at some point these third world countries will move up and out of their current level. As they do so, they'll need both software and hardware to enable them, though they still won't have the funds to purchase en massé what we can.
When that happens, that will be a huge market. Assuming your software is cost effective for them to begin with, the prospect of having to spend a lot more to update hardware will be a huge pitfall, rather than a convenience.
Granted, we may be many decades away from such things, but if you get into that mindset now it will be hard to get out of it when you need to.
Who's paying to store them? Provide the B/W? Power? Who's maintaining them?
More hardware isn't always better.... And in my exp makes problems worse.
oogly boogly!
For in-house applications it will always make more sense to spend money on better hardware than a month extra coding, provided throwing the money at hardware will solve the problem. For packaged applications the trade-off is different. That extra month coding adds a negligible amount to the cost of the product when spread over all installations, but the cost of more powerful hardware will be felt in every one of them.
This comes down to choosing the right tool for the job. Perhaps their application is written using frameworks which enable/enforce this kind of normal transactional processing of requests.
Now if the existing application is the only way that they know how to get to the data, then it may easily become the golden-hammer / silver bullet that gets used for performing an upgrade, rather than writing an external sql script which they might not be familiar with. Add to this the common convention of "nothing touches my database except my application", which has proven to be useful by preventing rogue updates which cause application 'bugs', and the golden-hammer / silver bullet becomes even more appealing.
SQL script wouldn't be the only non-application choice, there are quite a few good ETL solutions available, however these tend to cost quite a bit and perhaps the vendor does not want to impose another licence fee onto the client. This brings up a point more relevent to the main article thread, when deciding whether to throw people or hardware ata particular problem, you always have to be aware of the hidden costs i.e. licences for all software used including pre-requisites, network capacity, server-room capacity power & cooling etc. Naturally wide-spread use of open source software makes the initial calculation of software licencing cost a lot easier, although I'm sure that there are those who could argue that the savings on open-source software licences are eroded by necessary additional staff costs.
RTFM is not a radio station.
Ok, now I realize this may be modded a troll or flamebait, but from a moral perspective it simply must be said.
LOL! Whoops, right?
For the confused... our brainiac friend here posted in the wrong article. How fucking dumb do you have to be to do that? And not only that, he replied to another post, instead of posting at the top level. So he was sitting there, writing out his message and feeling all yummy about himself, while the post he was REPLYING TO was right there above it, STARING HIM IN THE FACE:
Hmmm... sounds like most of the Dr. Who companions.
He actually responded to this. To THIS. With what he thought was an intelligent post. Ha ha. He was wrong about that.
YOUR OPINION IS NOW WORTHLESS, AND YOU ARE A FOE, ALAIN94040. For fucking SHAME.
I realize this is /. and just from our very nature, we tend to score lower in the social graces (just ask our girlfriends, wives, or anyone we made an attempt at becoming one to confirm this) however it just doesn't condone the attitudes and actions of some people. To go on a total rant over something as lame as posting in the wrong spot? You really need to up your Prozac prescription. And to top it off, you didn't have the decency to log in. This in and of itself (to quote another poster...) makes YOUR OPINION WORTHLESS.
Oh... BTW... Merry Christmas you foul mouthed asshole
Maybe because it is not a good way to fix the problem, and it would only demotivate good people by making management seem petty?
If they're watching movies all day instead of working, they wouldn't stop wasting time just because the monitor is not horizontal... it's not like network-enabled computers are not great sources of time-wasting opportunities. The interweb is vast and infinite, after all. If that's the case you have a personnel problem that has little to do with monitors.
On the other hand, if they are NOT wasting your company's time all day, I don't know how they wouldn't find such a micro-management "mandate" insulting, straight out of a Dilbert cartoon.
You're saying management knows best exactly how they should read/write their own code? What's next? Enforcing syntax color schemes? IDE layout? Will their manager drop by their office to adjust their desk and chair to the 'mandated ergonomic position', and patrol around to keep an eye on their back posture?
Companies need to be able to trust their engineers to have enough control of their working environment to optimize their own comfort and effectiveness. If you cannot trust them to set their own monitor for what they do day-to-day, how on earth can you trust their code?
The vertical orientation is a helpful tip, but it may surprise you to know that sometimes developers need to do more than look at a single window with a single piece of code. In my experience, having more windows / files / tools open is almost always more valuable than the extra vertical space during debugging code... I do tend to find the vertical orientation extremely useful when working in sequential documents (specs, white papers, etc), so any developer should at least try it out.
But such things depend too much on personal preferences, habits and even eyesight. Mandating them is a great way to get your developers looking for another team / job where they are not treated like 5-year-olds.
Freedom is the freedom to say 2+2=4, everything else follows...
If anyone moderates the ridiculous parent post as "insightful" or "informative", that moderator needs to be eviscerated.
Know how to prevent people from "watching movies all day"? Don't hire employees who do that. And if you accidentally hire one who does? Fire them.
I am not kidding here. It's such a simple idea that I'm surprised that more businesses and managers haven't caught on to it.
Thrashing is a violent action such as causing a hard disk to spin fast and long. The effects are generally benign, i.e. there is no damage done when the thRashing stops.
.net
Trashing causes long term damage, such as installing windows or
But apart from the semantic differences, I would say you're right on the money.
RTFM is not a radio station.
When developers ask for a new monitor or dual monitors, let them have 'em but mandate that the monitors be in a vertical orientation as opposed to the typical horizontal orientation.
The point of wide screen monitors is that you put two source files on the screen side by side. A cpp file and its header file. A java file and its unit test. You get the idea.
What irritates me is that most IDE's won't let you do this properly. Emacs has this built in though, and I've made it work with other editors (slickedit).
Anyway, if you don't trust your highly paid engineers enough to not spend all day watching movies, then you have other problems.
People writing custom code for an in-house system might not see a problem with throwing hardware at a problem. It may very well be a good idea.
As someone who works on commercial applications that many thousands of people use to make their jobs easier how much is a few hundred hours of additional time worth to write something that actually saves customers time, works right and doesn't otherwise run like total shit? I would say compared to the potential for wasting the customers time the developers time is worth less than shit.
Ok, I'll bite .. what do you actually use the 2nd monitor for ?
You have to ask?
Running the gui application you develop in the other window.
Run the web application you develop in the other window.
Documentation.
Uh... everything you already use virtual desktops for?
If your application is largely static and a one-off, hardware is cheaper than programmers.
If your app will need to scale larger at some point, the balance could quickly tip the other way. Consider, $100,000 in programmer hours might allow the app to fit in your current farm or $50,000 in hardware will double the saize of the farm to make the app work OK as-is.
Next year, you need to handle double the load. That'll be another $100,000 in hardware (plus the 50K you initially spent on avoiding paying for developer hours), or just 50K more in hardware (plus the 100K you spent on developers).
There's your break-even (assuming electricity hasn't gone up in price). One more cycle and the programmer time was cheaper by any measure. That, of course, is for a single-instance app.
If the app is to be licensed to others, the programmer time probably wins out right away.
Short answer, make the code as clean and speedy as you can, then any extra hardware needed is an obvious request.
Long answer ... a lot of people don't think about the other costs. I am a software engineer and, for a while, the group I was in was under an 'operations' org. What I learned was that it's more than hardware, it's also the costs of administration of the hardware (when it breaks, when it's acting up, etc...), the underlying OS (security updates, audits, monitoring, etc..), power/cooling consumption, and the cost of spreading the operational (generally system administration) team X more thin.
There is a middle ground. If your worried about C++ being to slow then your probably worrying to much (or need better engineers) :-). If your thinking that writing your shiny new app in jruby on top of java inside of an application server, running on top of the CLR then, yeah, 'hardware' becomes expensive and pretty quickly. If you go with proven languages (C, C++, etc..), currently popular languages (Java, C#, PHP, etc..), or up and coming languages (Ruby, Python, Erlang, etc..) your trade off's should be sane (assuming the developers don't take all the short cuts then can to increase hardware).
"Hardware is no ware with out software", the programmers used to tell me as an EE. Actually, it takes both to work correctly.....
You're seriously arguing that it would be an intelligent move to accept a manual-labor, outdoor construction job for $10/hr, when you could get paid $8/hr to work from your bedroom?
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
The bottom line is, software improvement is a one time cost, once its done, it's done.
Hardware solutions on the other hand, though cheaper outright, are reoccurring (you'll need keep upgrading that hardware as it becomes outdated) and scale up with demand (if you double your number of servers, you'll need to double this hardware as well)
This is why, except in cases were demand won't increase, or the extra hardware is unlikely to become outdated, software solutions tend to be the more economical choice.
Pivoting means that ClearType or favorite subpixel rendering of your choice won't work. And I do really prefer ClearType, in the same way I prefer dualmon and high resolution.
Speaking of which, turning on ClearType is another of those "get more productivity for free" scenarios. Jakob Nielsen thinks it might be worth $2000 per year, which is probably overstating it. But it will save a measurable amount of time (meaning a measurable productivity gain) by making text more readable.
So you get something written in a hurry. You throw X amount of hardware dollars at it. But the thing wont scale well because of the way it is written. So to be viable it has to handle 1,000 times as many users/visitors/items/whatever. And you need to now use X * $1k dollars or something. Good luck.
Throwing hardware at a problem is the same as throwing money at a problem. Expect to use a lot of money, don't expect to actually solve the problem.
Bitter and proud of it.
Jeff Atwood is a living example of Sturgeon's Law.
just a ghost in the machine.
In a corporate IT department, it almost always makes sense. Hardware is usually a one-time expenditure and programming resources can consume a awful lot of money when compared to a bigger machine.
In consumer electronics the problem is completely opposite. If it takes an engineer six months to develop a solution allowing a $1 savings on something with a market life of 5 years and projected total manufacturing run of 5 million units, you are talking about a $5,000,000 savings for what, $3,000 for the Chinese engineer? Even with modest numbers where you might only sell 50,000 units having a savings of $0.10 per unit means $5,000 savings. With software engineering as cheap as it is today, there is no point in making the programmer's life easier. Make the hardware as cheap as possible and the programmers will just have to figure it out. Whatever you save on the hardware will more than make up for increased engineering costs.
This has been the way it is in consumer electronics since before there were consumers.
The formula is not so simple. Hardware for Google is REALLY expensive - big numbers, big money. Now think of it a bit, what if Google had a small number of dumb programmers. Hmm. How much hardware would Google need in such situation? :) Much more, 5*big number = 5*big money.
So, programmers expensive? Not simply true.
Hardware is a production line--the problems are narrow-focused, such that R&D can actually do something useful in the short and long terms (investigate a specific topic 100%, e.g. power usage). Software, on the other hand is not a production line yet. The problems are more generalized and can't be focused without wasting money. S/w problems get complex faster. And can never be focused because of the unpredictable user community (can't force them to use something of order of a standard 'plug'). And software problems change faster than hardware problems (where h/w problems are easier to focus on and fix).
I work at an ASIC company writing software to control our hardware. So yes in the applications I work on throwing hardware (specialized circuits to perform a specific task) at a problem makes a lot sense and can greatly speed up an application. However in this case you don't get rid of programmers or need less skilled programmers.
Throwing servers at our customers won't help out at all.
That being said our build environment sucks and they have been throwing servers and cluster solutions at it to no avail. We are basicly coming to a point where we need to throw out all our makefile cludge and re do it because 5 hour builds are just no fun.
If you do this with my monitors, the viewing angles and colors are all screwed up (with some others, they aren't). How many IT departments do you know that will recognize this? And how many will react on this after you've just gotten dual monitors while the other staff hasn't?
Besides, I keep the project outline or project explorer on the left hand sides most of the time.
When developers ask for a new monitor or dual monitors, let them have 'em but mandate that the monitors be in a vertical orientation as opposed to the typical horizontal orientation. That way, they'll have to use the monitors for efficient viewing of code rather than watching movies all day long. It's such a simple idea that I'm surprised that more businesses and coders haven't caught on to it.
Never used eclipse, I see.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
It's called Java
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
Programming certainly is about logic. After all, when all is said, done and coded, you have a logic that produces a certain outcome with certain input (please, no snide comments about black magic and computer voodoo. Computers cannot produce anything but determined outcomes, you all know that one of the most nontrivial problems of IT is to produce genuine random numbers).
Still, no machine can come up with new ideas, and they cannot create new code. They can slap together existing logic and they can assemble parts to create a whole program, but they cannot create new ways of solving a problem. If you insist in having a logic solve a programming problem, you will end up with the same solution whenever you pit it against the problem. If you want new, faster, better, more solid solution, you have to put a person at the helm.
What this may result in is that you need fewer programmers, and that only the top level programmers will be able to survive in the field, because the work of lower skilled people can be taken over by machinery. Much like it was in the old days when machines and robots took over for unskilled labour.
So what this means for us is that we have to become better programmers. Better than any machine.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Sorry but HTML/CSS/XML is VERY powerful and offers flexibility in both the way that things get displayed and in the way we tie systems together. Two quick examples, the AJAX platform Google uses for Google Maps has allowed them to write native clients for a whole host of mobile platforms at a fraction of what a tradition client/server system would have, the second is a system we developed for adding accounts payable automation to our systems. We essentially use a local proxy server that intercepts the html from our ERP system and ties it into the AP system and our content management systems. All of this is done without touching the code in the ERP system which would have been massively expensive, required a HUGE testing effort, and would have made the ERP system much harder to upgrade. This is how you use open, flexible systems to avoid the hidden costs so common in legacy platforms.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
That's why triple monitor is even better, documentation/debugger in the left window, IDE in the middle, and application in the right monitor. It's extremely useful for me and I'm not even primarily a coder (network admin). I use 3x17" monitors, some people might find a pair of 22+" monitors more useful depending on their workflow and work style, either way it's dirt cheap.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
http://blogs.zdnet.com/Ou/?p=583
Another example of the difference between a programmer and a real 'Software Engineer'.
Performance Analysis, when done early, and correctly can save thousands of dollars, and hundreds of hours of programming time.
One of the most useful classes I ever took in university as part of a B.Eng a long while ago, was performance analysis of Software and Hardware.
I can summarize the class as follows:
Know what your hardware can do.
Know what your software MUST do.
Know what [delay/performance] your customers/users can tolerate.
I have seen to many 'software professionals' use crap like bubble sorts, claiming that 'the user won't know'.
It works for a dataset of 100, but when it goes to 1000, 10k, 100k, users begin to know.
Performance analysis AHEAD of time, can save the 'optimization' effort in the back end.
This article proves _most_ programmers should stick to what they know: Coding and not economics.
OK you throw more hardware to the performance problem. What will your programmers do meanwhile? Nothing? Will you lay them off and give them 2 months worth of severance?
What are you going to do when you have to update the code? Hire them back? Well, they have new jobs now and no thank you they don't wanna work for you. So you have to hire and train new guys, and hope they update your code in time for you without losing business or money.
In another note, I wonder how many other engineering disciplines would discuss something like "Should we solve the root of the problem or just cover up the symptoms?"
Parsing out xml is slow. Anyone who says "we'll just use xml because it's fast" doesn't know what they're talking about. Also, AJAX isn't tied to xml, despite the name - it can use any data format - tab-delimited, binary, plain-text, csv, json, Heck, it isn't even tied to javascript if you have a browser that supports other scripting language interpreters.
"We essentially use a local proxy server that intercepts the html from our ERP system and ties it into the AP system and our content management systems"
Again, an html screen scraper has nothing to do with xml OR css. Additionally, it shows that xml is NOT needed for data transfer.
Moore's Law is dead. Where have you been?
Got a 5-10GHz CPU on your desk? Read anything recent from AMD and Intel, and you'll find they're praying for developers to save them by making use of multiple independent CPUs. That's not more Moore's Law advancing anymore than a huge Beowolf cluster or render farm is.
Buying newer, faster hardware usually only speeds things up by a factor of two or four.
(There are exceptions to this, mostly occurring when you add more memory to eliminate disk access.)
Conversely, a good programmer MAY (depending on the situation) be able to fix the work of bad programmers to speed the program up by 10x or 100x.
It always depends on what you need and what you're slowdown is.
Actually, I went from dual 22" monitors to a single 30" monitor at work, and can never go back. They really do help productivity, and the cost difference between the 30" and dual 22" isn't all that much.
"False hope is why we'll never run out of natural resources!" - Lewis Black
Ever come across the n+1 selects problem in hibernate. How many junior devs are good enough to figure out whats going on? Not many.
It means if you are fetching 1,000 records from the database it takes as much as 1,000 times as long as it should. Is halving your dev team cost really worth a 1,000 fold increase in hardware costs because your programmers don't understand the technology properly.
Everyone is living in a personal delusion, just some are more delusional than others.
So, basically a rowboat with a competent Boy Scout in it will get farther than an oil tanker piloted by Barak Obama?
Gee, who would'a thunk that management can make or break a project...
Consider yourself spoken to.
- crappy devices -- you really designing sucks
- weird mispeled error messages on screen
- frequent crashes
- more frequent cracks and thank you thank you my mafia boss likes you data very much thank you thank you data harvest going well
- mafia bosses paying well, we wok 4 them instead of you
- ya, you good lunk with that
- throw you cheap hardware see where it sticks
Yep, that's me. I have three monitors. Center for work, right for application viewing, left (vertically) for documentation such as Safari or a PDF.
Works great. I have a harder time at work because I only have the laptop and an attached monitor (also vertically).
Yea, the first setup is my home system :D
[John]
Shit better not happen!
When developers ask for a new monitor or dual monitors, let them have 'em but mandate that the monitors be in a vertical orientation as opposed to the typical horizontal orientation. That way, they'll have to use the monitors for efficient viewing of code rather than watching movies all day long.
Well, look here. There's a lot of personal preference involved in efficient text handling, and arbitrarily forcing programmers to work in landscape or portrait just so they don't watch movies is ridiculous. Matter of fact, if you have coders doing that on the job, either give them the requisite attitude adjustment, or just fire their happy little asses and hire some responsible citizens. Maybe in their next position they'll be a little more focused.
.Net or Java, you're probably happier with a horizontal layout given how wordy those languages are (.Net in particular, Christ on a crutch and I thought Cobol was verbose.) I've also found that when I'm editing source code, it's often nice to have the IDE running vertical, with the other, horizontal monitor for both my debug output and the application display itself.
... they're rarely helpful and usually counterproductive, because of considerable variation between individuals. We're not all alike, and we're not all maximally productive in the same identical environment.
Furthermore, I don't know about you but the apps I develop are generally not used with the monitor in a vertical configuration (matter of fact, given the nature of the software I work on that would be completely inappropriate) so it would nullify the advantages of a dual-monitor setup if I were forced to use them the way you describe.
Continuing this theme, you can't just say, "programmers work better with monitors oriented THIS way." Sure, if you're hacking assembler code a vertical setup might (might!) be better for you because the lines tend to be relatively short, unless you're like me and like lots of comments. If you're coding in
So, I'd say this: give your developers the tools, training and any good advice they need, and then let each of them figure out what works best. Otherwise you're just another overbearing manager more interested in exerting his authority, rather than running an efficient, productive development team. Beware of arbitrary constraints
It's such a simple idea that I'm surprised that more businesses and coders haven't caught on to it.
Well, now you know.
The higher the technology, the sharper that two-edged sword.
Om nom nom... thanks for feeding me =)
I'm not a programmer. I'm an architect / sysadmin type person.
/var/log/messages. It depends what I'm working on.
First monitor - Putty, Word, Visio, or the admin GUI for whatever system I'm looking after at the moment.
Second monitor - either Outlook, or tail -f
Serving Suggestion: Defrost
The reality is that programmers are scarce and increasingly so. The vast majority of those masquerading as programmers are merely coders - folks who know a language, a proprietary library and an IDE. Real programmers are engineers - problem solvers who also know languages and how to use them to solve the problems.
As a result of this misdescription, productivity is low - largely because "solutions" usually prove inadequate at first round and require massive reworking (witness "service packs") - so costs are high. Those who measure "programmer productivity" in lines of code per time perpetuate the problem. The real measure has to be fully functional and robust solutions delivered per time. Against that measure there will always be a few real programmers who are vastly more affordable than the rank and file even though their apparent rates may be higher. The fundamental business problem is to identify them.
The more you consider what a programmer makes the more you go towards outsourcing to another country all over again. I thought we dealt with this years ago and realized its not a good idea always or hardly ever to outsource many IT jobs to other countries and I'll let you ponder why.
Where I am currently working, a pizza box server has an annual cost of 2.5 developers salaries for the same period of time. It's grossly out of balance from this article.
Perhaps there is a reason some companies need Government Bailouts...
if(units() * savings() > programmercost())
hireprogrammer();
When you sell a million units a penny means $10,000 and $1 means a brand new Lamborghini. I guess this article only covers enterprise software where the number of machines thats running your code could be in the thousands. The opposite argument can be made when you talk about consumer products where the unit counts are in the millions.
This is definitely OT, but you DO realize that it was Sarah Palin that completely re-energized McCain's campaign and was partly responsible for him not losing to Obama in a complete landslide don't you? I don't care if you don't like her, that's your right to an opinion, but don't spread FUD about her, that's just lame.
I was thinking about getting a 30" as a secondary for my laptop - it's "supposed" to be able to do up to 3k x 2k ... and it's not like I need humongous video refresh rates to write code, but I was wondering about neck strain.
I am late here for this story, but I would like to add something to it for the sake of the late readers anyway :)
In the second half of 2001 I was on a project for a long time defunct company called WorldInsure (hey, former Corelan guys, any of you still out there, working for Symcor by any chance?)
So, I came in about half way into the one year project, in a few months the person who was the most senior developer on the project left but the team was still about 40 people in total. The application was something like 5MegaBucks by the end, but the client didn't want to pay the last million, because the performance was outrageously slow. 12 concurrent transactions per second as opposed to the 200 that the client wanted on 2 gigantic for the time 4 way Sun servers.
The app was a very detailed page after page insurance questionnaire, that would branch into more and more pages and questions as previous questions were answered. At some point a PDF was generated with the answers and provided on one of the last pages. The problem was with moving from page to page, the waiting times were too long, approaching minute wait times for some pages.
I was asked to speed it up. Long story short, after 1.5 months of tinkering with code produced by a bunch of novices, here is the list of improvements that I can remember at this point:
1. Removed about 80% of unnecessary database reading by removing TopLink.
2. Removed about 80% of unnecessary database writing by changing the way the data was persisted. Instead of persisting the entire data set on each new page, only the incremental changes now were persisted.
3. Reduced the page pre-processing by getting rid of the XSLT transformers on XML structures and switching to JSPs instead.
4. Removed cluster IO thrashing by reducing the session object size from unnecessary 1MB to a manageable 10Kb.
5. Reduced CPU load by caching certain data sets instead of reprocessing them on each request within a user session.
6. Decoupled PDF generation into a separate application and communicated the request to generate the PDF via a simple home grown message queue done with a database table. This was one of the more serious problems within the app. because it could bring down a server due to the buggy code in the Adobe PDF generator that was used at the time. In fact the original application ran the PDF generation as a separate Java application that would be restarted after about 5 generations and would be called via System.execute call so not to bring down the BEA Weblogic. Later on this entire portion was rewritten and the Adobe code thrown away. I am sure that today the Adobe code is fine and all, but at the time it was a real pig.
7. Removed many many many many unnecessary System.out.println calls, and replaced with proper logging where needed.
8. Fixed the home grown servlet manager (similar to Struts main servlet), this code was freaking ugly as hell and totally unstable.
There were some other smaller fixes, but the main bulk is listed here. By the end of the month and a half the app was doing over 300 simultaneous transactions per second.
300/12, that's 25 times code performance improvement. I am not at all convinced that this improvement could have been achived through hardware at all, but even if it could, it would have cost much more than what I cost at the time (was like 70CAD/hr for 1.5 months.)
Oh, did I mention that the client coughed out the last million bucks after that? After all, the code met their performance expectations and exceeded them by half at least.
You can't handle the truth.
I use only a single 20" monitor when developing on Solaris, but I need two when developing for windows. I can't put my finger on it, but somehow in Unix you can manage to get enough information at your fingertips just by opening three or four shell windows, but in Windows, that same amount of information takes much more screen space to display.
If you are not allowed to question your government then the government has answered your question.
A lot depends on the number and cost of deployments. If all you build are 'singleton solutions', applications that run on one set of hardware, the cost perspectives are significantly different than if you're writing software that runs on costly and/or large numbers of deployments.
As one example, I was working on an Air Traffic Control system in the mid-'90s. The controller workstations were high-end systems with expensive RAM and we had a requirement that the controller applications had to be RAM resident, no swap. I rewrote part of the system and reduced the size of the application by about 50mb. The cost savings in that RAM (a 64mb chip) over the number of workstations we deployed was roughly $1m back then. Was that worth 3 months of my time? Management certainly thought so!
Imagine the cost savings associated with embedded computing, e.g. printer drivers, anti-lock brake systems, etc.
There are times when it really bothers me how few people understand things above their 'IT in the office' environment...
dave
You'll get my vote if you say that programming for performance is about the last thing you need to worry about, and programming for scalability one of the first. Only when you're program is working and scalable will it make any sense to "throw hardware at it".
Show a man some news, distract him for an hour. Show a man some mod points, distract him for the rest of his life.
Any time you have to manipulate data in more than one document, source file, whatever.
Any time you are editing source whose output can be viewed in real time in another window.
Any time you are editing a document or sourcefile whilst consulting reference material.
Every context switch leads to a drop in productivity, Having two monitors helps. A lot. To this I can attest.
"Did you exchange a walk on part in the war for a lead role in a cage?"
This may have been true in the past (I still wouldn't agree completely), but with the changes in hardware that we've been seeing lately, performance gains aren't going to come for free. The Alpha was a great chip, but it's almost a decade old. The free ride of increasing clocks is over. The performance gains of the future are going to require human interaction, i.e., developers.
A handy note for those that don't know, Under X11 in addition to the rgb and bgr subpixel orderings, you can chose vrgb and vgbr vertical orientations to allow subpixel rendering (ClearType) for odd or rotated lcd screens.
http://notanumber.net/
Hardware can get you a factor of 2x-10x, depending on the bottlnecks of the original applicaiton (it's usually the database tier, or some other "shared state" component).
Well-concienved algorithmic or architecture changes can net performance gains of 100x or more.
So if 2-4x is all you need, by all means throw some hardware at the problem. But when you need to shave weeks into hours, a hardware upgrade isn't usually the right answer, and expensive experts (who understand the algorithms, but also memory locality, networking, storage architectures, and other stuff that developers don't usually grok) are worth every penny.
Ok, I'll bite .. what do you actually use the 2nd monitor for ?
Okay I'll bite too ... Pictures of Natalie Portman?
You would think this would be true; but, in fact, 1 piece of 'government quality' hardware is usually many orders as much as the architectural change to make it faster or scale better.
I say this because I'm in government contracting.. and seeing the government approve a $60,000 database server; across 8 different staging environments.. just to avoid a 2 week piece of work to re-arch.. is commonplace.
in programming that can be solved by simply "throwing hardware" them. Also, cost is only one constraint on project resources. It is true that good programmers are expensive, but they are the one element that you can't really automate.
Looked at another way: if you are using programmers to do work that COULD be automated, you are wasting your money. Put your programmers to work building the automation tools. When they are done you can put them to work on other revenue-generating projects.
Nazis. Hitler.
Fin.
Good programmers are cheap, bad ones are expensive.
The difference in value between good and bad programmers is seldom fully reflected in what they get paid.
Everybody thinks they're a good programmer...
Check out the Scrum et al video from Google Tech talks.
http://video.google.com/videoplay?docid=-7230144396191025011
Seems like a good way to run a team. Heck, you can even apply this to non programming jobs.
The fact is that Sarah Palin, after the first couple of weeks, was a continual drag on the campaign, except for - wait for it - people who were going to vote republican anyway.
Thanks - VERY handy.
Most LCD monitors look horribly in vertical position. Color is not really stable in vertical axis so when you rotate monitor your left eye gets slightly different color than right one. Also you can't use ClearType in vertical mode.
As a database guy, I would say that:
Database locking issues with a poor design won't be fixed no matter how much hardware you throw at it, it will just lock up faster.
Database structures are one of the first products of development, and many (most) developers just try to model the data without any thought to performance.
The other terrible mistake that many coders make is to process records and fields one at a time instead of using bulk operations.
The old saw about "Premature Optimization is Evil" is false:
Making performance a late afterthought guarantees doom that no amount of hardware can fix.
I have an uneasy feeling about compensating slacker/substandard work by throwing better engineered hardware at it. If the hardware/engineering group took this same approach, crappy developers would have no camouflage for their shoddy work.
Upgrading hardware is always an easy solution and can be done with great results AFTER code optimization. Take some pride in your work.