Slashdot Mirror
The Ways Programming Is Hard
An anonymous reader writes "Those of us who spend our days sitting in front of a screen trying to make computers do our bidding know how difficult programming can be. But from an outside perspective, there's not much to indicate difficulty. Most of us have heard somebody compare our job to digging ditches, or some other manual labor, meant to contrast easy (sitting around and typing) versus hard (muscle-wearying work). Now, Peter Welch has written an amusing essay to help combat that point of view, titled Programming Sucks. He compares bridge building to a big software project. Here's a small part of it:
'You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he's involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don't worry, says Mary, Fred's going to handle the walkways. What walkways? Well Fred made a good case for walkways and they're going to add to the bridge's appeal. Of course, they'll have to be built without railings, because there's a strict no railings rule enforced by Phil, who's not an engineer. ... Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn't.' Welch goes on to gripe about all the ways in which programming is almost awesome, but ends up being annoying."
'You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he's involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don't worry, says Mary, Fred's going to handle the walkways. What walkways? Well Fred made a good case for walkways and they're going to add to the bridge's appeal. Of course, they'll have to be built without railings, because there's a strict no railings rule enforced by Phil, who's not an engineer. ... Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn't.' Welch goes on to gripe about all the ways in which programming is almost awesome, but ends up being annoying."
Only complete idiots/tools think this way about any profession. Brick laying looks easy, but I wouldn't trust someone who's never picked up a trowel in their life before to put up a brick wall. Anyone 'outside the profession' should only be concerned that the code works, is maintainable, and is to spec, along with passing a security audit.
Wimp working for an excessively-large company with rules that might allow "casual Fridays" with strict dress codes. He's whining about corporate culture, not programming. The "Neo" problem that first gave you the hint that he wasn't an actual hacker, just some script kiddy in a minimum-wage cubicle farm.
problem of "other peoples' code" is an actual problem?
Because if so, I think you may be the script kiddy in a minimum-wage cubicle farm.
STOP . AMERICA . NOW
It's a good article, and all too true. Software is a house of cards at best, with your boss shaking the table, the hackers throwing ping-pong balls at the house, and the NSA coming down the hallway with a baseball bat.
I do not fail; I succeed at finding out what does not work.
Compare this to digging a ditch, which is hard labor, but provides almost instant gratification (you can always look at how much of the ditch you've already dug), and you're not likely to fail completely (unless you're trying to dig in solid rock).
The author has a great sense of humor, and ripostes some of programmings possible pitfalls into clever and hilarious absurdities. However, I strongly refute that this article is a remotely accurate portrayal of programming, and I hope it is not taken as such by prospective coders on the fence. In my view, programming is possibly one of the few havens of relative sanity available (although with wood-working and pure mathematics probably have it beat). The true insanity is HUMANS TRYING TO COLLABORATE WITH OTHER HUMANS. If in doubt, please re-read the article with that in mind, and I think you'll find all the admittedly hilarious conceptions boil down to that issue, and that issue is pervasive in nearly any job you could name. Programming is an oasis from insanity, not ground zero.
What I struggle with is translating badly worded/badly thought through/incomplete business requirements into program logic. All too often something which seems straight forward to a business person is a can of worms when it comes to implementing it.
I want a list of atrocities done in your name - Recoil
Because the code I wrote is not what is being executed, and if it was, we would not have the level of performance we have today. Moreover, the level of complexity of today's computer. Moreover, tolerance are such in the physical world that a construction guy can afford to "cut corners", whereas software is a lot less tolerant to a bad input, remember, it's a dumb automaton.
He's wrong though.
... Aaaand the customers often buy it :).
Bridge building is more like compiling.
Bridge designing is more like programming/program designing.
And there's the big difference.
Civil Engineering:
Design Phase costs about 10% of Build Phase
Build Phase involves tons of construction workers and heavy machinery.
The blueprints and plastic models are way cheaper to make than the Real Thing.
Management often doesn't mind spending a bit extra to get the design better, because the budget only allows for one big Build.
Software Engineering:
Design Phase costs more than 1000 times the Build Phase.
Build Phase involves the programmer typing "make all" and going to read Slashdot or fetch a coffee.
The plastic models cost as much to make as the Real Thing.
Management often sells the blueprints/plastic models as v1.0 because they compile and "kinda run" and the budget only allows for one big Design...
It should be no surprise then that the plastic models regularly fail.
Portable toilet cleaner ...
Sewers cleaner
Animal masturbator
Janitor at a porno theater
Programming is bonanza
It is....sometimes. The biggest problem with Open Source QA is also one that affects a lot of research, everyone wants to code, nobody wants to be a reviewer/bug fixer.
Look at the HeartBleed bug, there was only one source review before release. There could have been more, but open source suffers from the peer-review paradox: the people with the ability and resources to do thorough reviews are the ones least likely to want to do reviews. Quite simply, there isn't any "glory" in it, and it isn't nearly as much fun as creating new code yourself. Now in big commercial operations, especially web sites, there are large QA departments where everyone has a financial motivation to scrutinize code and find weak spots. Really if companies like Google et. al want to help open source, they shouldn't just contribute code, they should donate their QA team's time and talents to doing really thorough reviews on critical open-source code before it's merged into the main branch.
Monstar L
(Wait, did I just hear you denying that the general) problem of "other peoples' code" is an actual problem?
Because if so, I think you may be the script kiddy in a minimum-wage cubicle farm.
Bonus points for realizing that your own code from three months ago is also "other peoples' code".
The best part is when you find out that the "other guy" who put in the stupid code is you from months or years ago. You start to get incensed and rant about what idiot would write such awful code, check the commit logs, then facepalm as you realize it was you.
I occasionally wonder if that kind of thing would happen less frequently if our profession wasn't so quick to fire the guys whose gems turned black.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
When people work in a group as a team this inevitably happens and that's all right. Building a perfect software that satisfies everyone is a hard NP problem. What's the point of being perfectionist if that 10% you wish to polish it going to consume 90% of your time and money.
"get better at estimating how long your tasks will take"
At my last programming job, the head of engineering took all my time estimates for a project and arbitrarily cut them in half, because "we're smarter than most companies".
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
If you're not satisfied with the level of quality in open source software, you're free, actually encouraged, to improve it.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
This is a real problem. Open source projects have a very varying degree of Quality Assurance.
Look at the name - Quality Assurance. QA can only tell you it's bad, it doesn't make anything better. And what they can look at is usually only the external, visible part, which may seem to be working nicely but built from spit and wire on the inside, ready to fall apart at the next version update of the printer driver. That doesn't mean QA is useless, it's just a final double-check to see that you've built the right thing. :P.
No, writing good quality software is a team effort of everyone involved. As a developer in the middle, you should be able to smile at the great design documents given to you by the system analysts, telling you what's needed and how, while having enough freedom to do it in the best way given the tools and frameworks you work with. And you should be able to high-five the QA people when they fail to find anything wrong when you deliver it, because you know that if they give their OK you can sleep on both ears and the users will be able to do what they need to do. And that's a good month's work if you managed to get there on the third try
Where's the paradox, it's the same in science. Everyone wants to make the new discovery, nobody wants to review and verify them.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
That joke actually stems from Soviet times and was making fun of perceived and reported field yield vs. real, but it works just as well for management and projects.
A programmer has to estimate how long he'd take to get something done. He ponders, calculates and finally decides: 3 months. ... anyway, they can do it in 6 weeks, too, I'm confident!
His supervisor isn't happy with this, 3 months, that's probably too long. He notices the programmer has a vacation planned and there are a few holidays that he could work overtime in, he cuts those and corrects the estimate to 2.5 months.
The group's superior isn't happy with that. The quarter report is due in about 2 months. But if they think they can do it in 2.5 months, they sure can do it in a week or two less by cutting time somewhere or working overtime, so he puts down 2 months.
The department head doesn't like what he hears. The general meeting of the shareholders is in 6 weeks and he sure wants that prestigious project to be mentioned in there. But maybe somehow we can cut 2 weeks somewhere, probably we'll hire a temp or some other way that doesn't cost
The project lead finally gets the estimate and is very happy. 6 weeks! We have almost 3 months time left! Time to push for those features I wanted, they said they'd tack 2 months onto the project, but somewhere they can surely shave off 2 weeks of those and we'll be done in time!
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
The comparison that has been coming to my head, regardless of whether I was self- or otherwise- employed is: coders are the plumbers of the Internet age. Furthermore, we are the electricians, the elevator drivers, the janitors, the security guards, the dealers, the cops, the architects. All generalized comparisons apply, because the Net is a representation of the world. Slightly skewed, a representation nonetheless.
I am proud of the being an Internet plumber and cheers to others who spend their days trying make it work smoothly.
a} Clueless psychopathic suits in management, who make impossible schedule demands, and have no programming background themselves.
b} The use of popular, but garbage programming languages. C++, PHP and Perl are probably the main three culprits here. Dishonourable mention also goes to XML, JavaScript, and the XHTML Document Object Model. I have never encountered a "Web application," yet, which wasn't a disorganised, bloated, CPU hogging abomination.
For the last two months I've been economically forced to use a dual core 1.5 ghz laptop with 2 gb of RAM, and it can only barely keep up with the inefficient, JavaScript-infested obscenity that the Web has become. Virtually none of said JavaScript ever provides truly valuable functionality, either; most of it is just trackers of various kinds.
It's also purely due to Capitalism; all of it. Why have Red Hat had Lennart try and force systemd, GNOME, and the rest of their corporate crapware on Linux users? Their desire for a corporate monopoly, that's why.
What caused the UNIX wars? Corporations wanting to add their own non-standard extensions, to ensure their coveted Unique Selling Positions.
We must get rid of the suits.
That's true for both. When was the last time you saw a Nobel Prize winner repeat an experiment from someone else just to prove his colleague?
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
As someone who had a part-time construction job in college: It's amazing how many people doing that actual building on construction projects who fuck shit up and then hide it long enough that by the time it's discovered the designers have to tweak their near-perfect designs to incorporate workarounds.
Thankfully you don't get that in software development, it'd be like a world in which developers wrote near-flawless code but your average compiler/interpreter played it fast and loose and did shit like randomly round numbers up and down. Random rounding was actually pretty common when I did construction, the design clearly states that each section between two windows needs to be 1.9 meters, there are 12 windows on that side of the building and the sections come prefabbed in 2 meter segments that you're supposed to cut 10 cm from to get 1.9 meter segments? Yeah, Bob the 47-year-old foreman will shoot down any attempts to "waste time" by cutting the segments and suddenly you have to settle for just 11 windows because the total length of the wall is 1.1 meters too long. Because Bob thinks engineers are a bunch of pedantic wusses. Sort of like if your compiler were to try "optimizing" your floats and doubles to ints
Thinking is hard, but there is a kind of thinking that is harder.
Rough order of thing (easiest to hardest).
1) lounging around doing nothing (easiest)
2) doing what you are told to do without physical exertion (menial white-collar work)
3) doing what you are told to do with physical exertion (menial blue-collar work)
4) independent thinking about things not involving organising other humans (programming, painting, composing music, solving systems of equations)
5) independent thinking about things involving organising other humans (managing programers)
6) independent thinking about things involving organising other humans who are in turn doing independent thinking involving organising other humans (hardest)
The article is very funny, and makes a lot of good points, but it's not really addressing the question. It gives reasons software projects are difficult. Not why programming as such is. The engineering part, if you will.
The reason programming is difficult is because it's discrete. Whereas most (all?) other engineering is continuous.
If you try to bend a steel rod, there's going to be a range of strain for which nothing will happen, another where it will flex and recover, another where it will flex permanently, and finally a point where it will fail. This is all contiguous, and you can reduce it to a few numbers that will accurately predict the behavior of identical rods. Depending on the expected load, and the desired behavior, you can then pick the right rod, that will behave the way you need it to, with a good and predictable safety margin, because you know when what will happen.
Software, on the other hand, is discrete. That linked list can behave correctly as you add elements, then remove them, all good, then fail when you remove the last one and it (should) become empty again. There's no hint of that unless you test the very specific condition. There is no failure progression. There is no concept of safety margin.
That's why software regularly blows up. That's why it's difficult to make it not blow up. It's always on the edge.
>Bridge Analogy Time
And when you are supplied with help, most of them have never used construction tools but still claim to have years of experience building bridges. You assign them a section to tighten bolts and after a few days check and see that nothing is done because they're all crowded around an incorrectly sized bolt and trying to make it fit. After showing them the correct bolt size, they swing the wrench around wildly trying to turn the bolt by hammering the edge. When you ask what they are doing, they say that they only have experience with hammers. So, you grab a bolt, put it in your hand and proceed to show them how to tighten a bolt and check with a torque wrench. You go back to the large section of the bridge roadwork you were assigned, but now horribly overdue for the CEO to drive his car around on. Days later, your boss checks on how the other workers are doing. Since they haven't accomplished much, they claim that your example was bad and they had to research on the internet how to tighten bolts. They proceed to pick up a bolt and tighten it with their hand, following your example _exactly_. "See? The bolts are not getting into the bridge! Only my hand!" Your boss looks at you accusingly and says "you should have tightened the bolts for them on the actual bridge!".
This is the point where you proceed to jump off said bridge.
Good point about the hardware: it's not just about the design and building, but also about the materials you get to work with, and the environment. Bridge builders use structural steel elements, bolts, cables, and need to account for environmental factors like the ground they're building on; software designers rely on computer hardware, the OS, the network, and they will also use APIs and libraries to build the software.
The problem with software components is that they are difficult to test completely compared to physical elements (it can be done but it's often cost prohibitive, and even then it is all too easy to make a mistake). Also, software doesn't fail gracefully but chaotically: small errors can have large consequence, and even a bug in an unimportant feature of the software can bring the whole thing down. In terms of our bridge: mount the railing next to the walkway incorrectly, and the bridge comes crashing down. Use Traffic Signs v3.0 together with Overhead Girders v5.41, and things come down; not just the traffic signs or the girders, but the whole bridge, and somehow the town downstream gets flooded, too. And god forbid you forgot to account for the brand and colour of the cars that will pass over your bridge; because the red ones will behave differently, and cause troubles. That's how software fails.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
How dare you perpetuate this insulting and offensive view of programming. All real developers know that it involves sword-fighting on wheelie chairs.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
Interesting take, I'd not seen it put that way before.
I once heard a horror story of an embedded software project. They were forced to only use debug builds, as the release builds seemed to trigger a race condition in the start-up code. The 'held together with paper-clips' approach to software engineering...
We can at least lessen the fragility by using regular testing and defensive programming.
(I'm almost crazy enough to think we should actually use the tools intended to make software that damn well works, but we all know that's never going to happen.)
a century ago, the same sorts of complaints were being made:
see The Ragged Trousered Philanthropists by Robert Tressell.
Not a bridge designer myself but I suspect bridge building is well defined for how one is designed. It seems that so long as you define span and weight requirements (plus whatever other details I don't know about) you will have a pretty successful bridge.
Most software requests seem to be more like "I want to drive across the Aleutian islands, make it happen".
But when you want coding to be implementation similar to construction, you have to understand that the design phase would suddenly become much more like the old Waterfall methods - where the entire system was designed, locked down, fully thought out and specified to the last detail.
Contrary to common perception, you need a design not only for waterfall, but for agile development too. The API and specs should be done before a single line of code is written, and compared to other APIs and specs to ensure its compatibility. So you should have lots of small designs instead of one big book.
The problem is that developers and managers don't want to spend that time up front, and think they can get something for nothing. And then it doesn't matter how good a programmer you are - you're coding by the seat of your pants, and things will break.
Making the problem worse are the coders who write unit tests after the fact to rubberstamp their implementation as "good", instead of exercising the robustness of the code against the design. Because there never was a design, and the coder isn't interested in discovering logical faults, just in getting an "approved" and move on.
And the problem is that the design for Agile is usually something like "This is what we're doing today, do what we say and shut up. Tomorrow we'll do something completely different, do what we say and shut up. No, we're not going to pay for changes, we like Agile because we don't have to do anything, just shut up and code." In my experience, this is the only half of Agile that actually gets implemented; the part where the business has to pay for changes, thus giving them incentive to think things through ahead of time and give adequate specs, is unpopular with the MBAs (as it requires them to do actual work and not drink lattes and play golf all day), so the coders get all of the drawbacks of Agile with none of the benefits.
And if you can document ONE case of that actually happening in the real world, go collect your Nobel.
The programmer frequently has no incentive to discover logical faults. They get punished for "not being a team player" and "asking too many obvious questions." They have incentive to make it someone else's problem through the approval process; however, even then it's the coder's fault even when it isn't the coder's fault.
Never underestimate the power of stupid people in large groups.
That was a pretty good rant, but if you want a true description of the mind-melting Lovecraftian horror that is programming (from a Microsoft distributed systems engineer, no less), there is no better read than this article.
I don't care if it's 90,000 hectares. That lake was not my doing.
I just don't get this bridge thing... I know cars drive on them but how does that relate to anything!?
Actually, it is more like solving a huge jigsaw puzzle, but without an actual picture of how it is supposed to look when it is completed, and with a picture that changes while trying to put it together.
I've experienced this first hand. I gave my estimate for a new feature. Over the course of the remaining meeting, I literally watched my estimate get cut in half by the client and management as they shuffled dates around.
I then chimed back in with my original estimate of how long it would actually take in the real world, and somehow everyone was upset that I had moved the deadline. Unreal.
Culture is more than commerce
Software that works very differently in debug builds is common. The 1993 edition of Steve McConnell's "Code Complete" was the first thing I remember reading that talked about reducing debug vs. release variance. IIRC, Microsoft was chewing this problem heavily then because the betas of Windows and their development tools going to developers included debug instrumentation, while the released versions did not.
SPARK falls squarely into silver bullet territory. Taking on that idea goes back to at least Fred Brooks's 1986 No Silver Bullet paper.
SPARK falls squarely into silver bullet territory.
Well, in terms of correctness, formal methods really are a silver-bullet.... assuming an infinite budget and an unchanging, well-understood spec ;-P
> That's why you have the customer sign off on the analysis stage, making the customer responsible for any delays if
> requirements are changed.
AHAHA HAHAHA HAHAHAHAH AHAHA HAHA HAHAHAHA HAHAHA HAH AHAHA HAHA HA HA HA
AHAHAHA HAHA HAHA HAHAHA HAHAHA
HAHAHA HAHAHA AHAHA HAHAHA HAHAHAHAHA
[wheeze]
HAHA HAHAHAHA HAHAHA HAH AHAHA HAHA
[cough]
Ahem.
Okay, sure.
Yeah, you can warn them from the very beginning. You can say "hey, don't ask for changes, because it will change the timeline." Then what they do is go over your head to some manager's manager's manager or the CEO who says "I don't care, we need this now and so-and-so says it's an easy change (and I don't know why you estimated 160 hours of dev time for this) so make it work. If you need extra time we'll just cut it off of the testing phase."
No dev plan is immune to office politics.
Besides, even if the customer is *aware* they're responsible for the delays and they assume said responsibility, *they're* not the ones coming in on weekends to write code to hit a deadline or scrambling to integrate the new change into a nice clean architecture plan. And upper management rarely yells at (or fires) the customers when stuff goes overbudget or over time.
----
"I used to listen to Null Device before they sold out."