Bad Software Runs the World

whitroth tips a story at The Atlantic by James Kwak, who bemoans the poor quality of software underpinning so many important industries. He points out that while user-facing software is often well-polished, the code running supply chains, production lines, and financial markets is rarely so refined. From the article: "The underlying problem here is that most software is not very good. Writing good software is hard. There are thousands of opportunities to make mistakes. More importantly, it's difficult if not impossible to anticipate all the situations that a software program will be faced with, especially when — as was the case for both UBS and Knight — it is interacting with other software programs that are not under your control. It's difficult to test software properly if you don't know all the use cases that it's going to have to support. There are solutions to these problems, but they are neither easy nor cheap. You need to start with very good, very motivated developers. You need to have development processes that are oriented toward quality, not some arbitrary measure of output."

Comments

Numbers don't lie

[Anonymous Coward]

50% of all software is of below-average quality.

Re:Numbers don't lie

[K. S. Kyosuke]

That would be "below-median quality", I suppose.

Re:Numbers don't lie

[pympdaddyc]

That depends on your definition of average, mathematically speaking that's not true. What percent of numbers are below average in this set: {1, 1, 1, 1, 1000}

This isn't pedantry, this is a meaningful distinction: I expect the amount of good software is extremely outnumbered by the bad, and even good software developers can be forced into kludges by time pressures, bad team culture, etc. I don't see any reason to think that code quality globally resembles a normal curve.

Re:Numbers don't lie

[istartedi]

Your one-liner has a bug. It's Open Source though, so it's a shallow bug.

Re:Numbers don't lie

[gorzek]

Yes, actually. You can measure it in terms of raw numbers of defects found. You can also determine the number of defects produced per some measure of effort (e.g. 1000 man-hours.)

You need quality control, which is about ensuring good practices that are documented, repeatable, and measurable, and quality assurance, in which the results of your process are analyzed for their overall quality.

Good QC should make QA's job a lot easier. (Or harder, depending on how you look at it.)

There is, of course, no number that says "this represents the total quality of this product." As with anything else, quality in software can only be measured in terms of failures (defects.) You want fewer and fewer failures over time--that is a sign of a good quality process.

Re:Numbers don't lie

[ceoyoyo]

It depends on your definition of "average" AND on the underlying distribution. If the distribution of software quality is symmetric then 50% of values are below the mean, median and mode (the three most common measures of central tendency or "average").

Re:Numbers don't lie

[bluefoxlucid]

Raw defects doesn't indicate quality. A defect by which the system occasionally has to stop and replay some data write-out because of some hoakey disk driver is not a gerat problem: the disk driver is buggy, and is using a shitty hack to fix it. By contrast, a much better written driver with a very corner case race condition that 1/100 as often simply destroys a ton of data has a huge problem.

Linux is like that. If a hard disk drive starts to not respond, it'll send it a reset command and continue. It'll mount the filesystem read-only without special options; in some conditions that's important, because the OS view of the FS might be completely different due to undetected write failures. In any case, it's still up and you can get information out of the kernel. I've had the system hose itself so bad I couldn't actually read the logs or run dmesg, but if your boot process copies a few utilities into a ramdisk and sets tty1-5=login tty6='chroot /recovery login' you should be able to switch to that tty and run. Bonus points for statically linking chroot on boot (i.e. the boot process copies everything in from installed fs, then uses ld to statically link chroot to all its dependencies), so in a barely-functional active ssh session you can '/recovery/bin/chroot /recovery /bin/sh'

A high-quality system that fails 1/10000 of the time and destroys everything is worse than a low-quality system that fails 1/100 of the time without cause for concern. Yet the low-quality system is clearly shitty.

Re:Numbers don't lie

[Smauler]

All defects are not created equal. A defect that is known that hits in a 1 in a billion UI interface offchance that just throws up an error, and nothing more is a defect, but 99.9% of people would not fix it, if it requires significant work. Why would they - it's a waste of time. A defect that drops all keyboard interface onto the internet in a secure system every time is still 1 defect - it's just a bigger one.

Re:Numbers don't lie

[gorzek]

I agree with you. Severity is certainly relative (and I should have said as much when talking about raw defects.)

Re:Numbers don't lie

[luis_a_espinal]

Is there a real definition for quality?

When it comes to software, yes. We have had it for decades: quality entails

1. architecture, design and implementation decisions that minimize the cost of change,
2. that does not deteriorate with each change (or at least deteriorates linearly with each change),
3. that exhibits strong cohesion and lose coupling,
4. that permits reasonable maintainability and configurability,
5. with relative small bug count per whatever metric one picks (FP, SLOCs, etc.)
6. that is amenable for testing
7. with architecture and design that are understandable (tied with #1)

Just to mention a few. There might not be a single universal definition of software quality, but there are common desirable attributes that have been known for decades. It's when people code without knowing these attributes (or not giving a shit about them) that we get the crappy stuff we get.

Software doesn't have to be perfect. It simply needs to be economical due to its qualities (again, qualities well known for decades.)

Re:Numbers don't lie

[HeckRuler]

So the first step is to assume the horse is a sphere...

Re:Numbers don't lie

[Grishnakh]

Not only that, but you've identified another problem with judging quality: software usually does not stand on its own; it's part of a larger system. What if a piece of software is well-written, but the libraries it links to are shit? A programmer may not have much choice if he's required to use system libraries, or some special vendor-provided libraries. He can add in workarounds for some of the bugs in the libraries, but that's it.

Re:Numbers don't lie

[ZorroXXX]

I think you also missed to include the corresponding XKCD reference for this :)

Re:Numbers don't lie

[lennier]

You can measure it in terms of raw numbers of defects found.

That's a pretty bad metric. Defects found is most certainly not the same as defects existing, or we wouldn't have the security situation we have today.

Granted, yes, we know those defects are in there because those defects are eventually found... by someone... but the huge elephant in the room is that 80% of security defects are not found by the company which wrote the code. In any other industry that kind of failure rate would be not just criminal but verging on hostile military action.

What we need is positive proof that defects simply aren't there, rather than knowing that an unknown number of possibly catastrophic defects are there but are likely to be found by the bad guys first. And we can get there using mathematics, but not using the languages which we're currently using, because they're mathematically intractable.

C and C++ are fast. But fast and full of 0-days just means you get rooted quicker. It's not good enough any more to say "but there's no alternative to C/C++". If there's no alternative to C, there's no alternative to a global infosys apocalypse. We can do better, and we need to if the Internet is going to survive.

Re:Numbers don't lie

[hairyfeet]

I think a better question would be "Why does it NEED to be better?" and the answer in most cases is "it doesn't". Remember perfect is the enemy of good and often the difference between "good enough" and great can be truly insane levels of expense.

At the last shop I worked at before striking out on my own i had to rush home one day and dig my very first two gamer PCs, a Pentium 60MHz and a P100MHz out of the shed, why? because a guy came in the shop about to have a breakdown because the PC they used to control their custom lathe had gone tits up and they had a $60k order due by the end of the week and no columns? No contract. No when i was cloning the drive I asked him "if the thing will only run on ISA, and the company is OOB, why not replace it?' and found out a unit today that will do what that one does would cost upwards of $150k, that one was paid for, it works, its easy to use.

Now I'm sure that if any of the programmers here saw that software they'd either laugh or bawl like babies. We're talking a primitive GUI running on top of DOS 3 that lets you build the design from templates or build your own from various shapes, VERY rudimentary and frankly made Win 3.x look like Win 7, ancient stuff. But ya know what? It works and works well. it does that one task, day after day after day, very WYSIWYG, you could get someone up to speed on running the unit in maybe an hour.

So while i can see why many programmers might look at something like that and think half ass I can also see why it was made the way it was. i'm sure those designing that software they are bitching about in TFA did it that way because they designed it for a task, weren't thinking about someone plugging into it down the line, and just concentrated on that one task. I'm not saying one should design like that today, but usually when i hear programmers screaming about bad software its because they've found something old and creaky and are bitching because they are gonna have to support it. Welcome to the real world Chuck, where jobs often suck.

And just to prove how fussy programmers can be I'll end this with something that always ends up getting programmers to gnash their teeth...I LIKE VB6, okay? If all you need is to have a GUI to a local DB frankly there is NO tool that can compare to VB6. Its fast, uses practically zip when it comes to resources, easy to whip off prototypes right there if you need to so the customer can see what fields he/she needs, for that one task and one task only, putting a GUI to a local DB even now you just can't beat VB6. That is why to this very day its like the third most popular business language, because businesses need that job often and it does it VERY well.

Re:Numbers don't lie

[anubi]

I keep several old machines around for the very purpose you state. Many businesses have invested a substantial sum of money in machinery and their controller that does a specific thing, over and over, day in and day out.

If it breaks ( which is seldom ), they are in a bad way if a replacement part is nowhere to be found.

The good thing is that a lot of that old stuff is very fixable and interchangeable. I can usually pull that old DOS stuff off one machine, slip it onto another, and take off like nothing ever happened. The only critical part is usually the interface board - which in those days was usually nothing more than a bunch of standard TTL latches and drive chips.

The most important thing is the guy at least has a good readable backup of his software and directory structure. There was an old program called "LapLink" which I could use to quickly transfer a working image of his system to a laptop, so I could burn the image onto modern storage devices such as USB sticks, CF cards, and CDRoms.

From what failure rates I have seen, I have enough stock to keep the family of businesses I support operational for as long as I live. At one time, I feared loss of disk drives would do me in, as I simply cannot get those old floppy, MFM and IDE drives anymore, but lately I have found ways of using CF cards to replace the old drives.

I have noted the old power supplies are now failing. The problem is usually the filter capacitors and the small electrolytic capacitors in the switching transistor's base drive circuit. Although I could retrofit the old case with a new supply, I often just re-cap the old one, and sometimes I have to replace the blown transistors which often occur when the transistor's base drive was compromised with insufficient drive due to the deteriorated capacitor. Of course, the fan nearly always needs to be serviced.

The fly I see in this ointment is there are not many of us old guys left which get into these things on the component level any more. I know of no companies which will support them on this kind of thing - as services as Geek Squad do not do this kind of stuff. If companies that have this old stuff have not cultivated a network of old coots like me which understand this old stuff, they have no alternative but replace the thing when it dies. Its such a shame because the controller system is so simple to maintain compared to the expense of replacing the machinery it controls.

Re:Numbers don't lie

[Skal Tura]

The bottomline is that there is a tool best fitting for every task, there is no magic bullet.
Even quick'n'dirty code is the perfect choice for some things, things that do just one thing, like a database backup which needs more than a mysqldump cronjob (ie. send e-mail notification if backup failed)

Too often programmers fail for wanting to make things too fancy, forgetting the golden rule of code: Keep It Simple, Stupid!

For example, Zend Framework used to be quite great couple years back, but already suffering from the Coder's God Complex, things needing to be too fancy, a simple redirect doing 13+ method class to different classes. However, mainly it was still fairly simple and easy to work with. Today? It's insane complexity even for the most basic tasks, and basicly ruined completely. Magento is the culmination of this Coder's God Complex problem. (Weeks to just template the thing for all which is not exactly the same layout just colors changed!)

Then there is the other kind of problem: Absolutely no thought given to the task at hand, and a big complex system done completely Quick'n'Dirty, like WHMCS, osCommerce.

WHMCS is the worst piece of crap i've seen, they seriously thought that 1 EUR = 1 AUD = 1 GBP. WTF?!
They still point towards mysql result arrays by *NUMBERS*, not utilizing the column names like they should.
It's mostly function based code, no classes, no abstractions, no structure.
It even have serious security flaws due to these, for example mass invoice payment works by giving the customer credit then marking all the mass invoice payment invoices paid, which results in many automated systems that the customer gets free services, as auto-suspending doesn't work anymore and services looks like paid.
Hell, if you enabled multi-currency you can't even sanely go back at all anymore! You need to build a big conversion script and verify everything manually.
Also, by default TAX reports are done invoice based not transaction based like they should, and amounts checked from invoice. So you end up with completely wrong tax reports.
Infact, all the reports are screwed up by default, every single one of them concerning money are too badly flawed to be useable.

Also, there is e-mail bomb security flaw in WHMCS, allowing a nefarious attacker, especially if smart one, to DoS attack your system with spending 0 of their own resources and completely untraceable (we had an attack like this).

I've reported tens upon tens of these bugs to WHMCS, and they simply delete the threads claiming there is no such bugs even tho repros have been provided, only if i provide workarounds/solutions they will not delete them immediately. Some of my fixes did flow back into future releases tho (without credit).

Nevermind the huge usability issues, i do specialize in UX and tend to notice usability issues very easily.

The most worrying part tho? Some vendors simply refuse to admit any issues exist, and simply ignore you, for example i found potential critical security flaws in Bit-Pay WHMCS payment gateway module, Bit-Pay never bothered to even reply to my concerns, even tho i showed the particular issues with the module. That particular module had the worst crap i've seen in many years tbh, just a glance over the code showed severe code QA issues, and altho i did not test, i'm fairly confident there is a exploit there due to incorrect handling of payment verification. I even refactored and fixed some checkup code before giving up as it being too far off

Re:Numbers don't lie

[hairyfeet]

Awww...what the hell grasshopper, old Hairy will show you the way. What you wanna do is go start talking to all your local mom & pop PC shops, let them know what kind of work you do and tell them you'll give them a referral fee for every one they send you way.

Ya see we old PC shop guys end up getting this kind of work because we are all little pack rats at heart and HATE throwing away working gear, and once you've done a few of these "miracles" it really don't take long for word of mouth to spread. Now since it don't sound like you want to get into the wonderful world of Windows PC repair, which is actually a damned nice way to meet girls BTW, then what you are gonna have to do is get to be buds with the PC shop guys.

While there are some like me that have old engineer buds that can do any chip changing and TTL stuff i can't there is just as many that are straight 'fixit guys' that can't do the soldering, replacing chips, fixing burnt boards, and like I said we hate throwing stuff away and telling somebody "we can't do it". hell that was one of the reasons i got to be buds with an engineer, I can't do the solder stuff anymore as my hands aren't steady enough and he hates working on computers so we just swap it out.

So go talk to the little shops, be prepared to BS awhile, have yourself some cards made up to hand out, and before long you'll be ass deep in busted gear. remember that most places won't work on this stuff so a little word of mouth goes a long way, a little ad or two can't hurt either. Trust old Hairy that there is plenty of old gear that needs fixing and as long as you charge a reasonable price (remember starting out you need the business, don't go nuts. Once you have the buzz the price can go up) and it won't take long for that phone to start ringing.

Re:Numbers don't lie

[cthulhu11]

I think a better question would be "Why does it NEED to be better?"

Because there are millions of humans on this planet with an apostrophe in their name, or two middle names. I currently have the former and have had the latter in the past. I *constantly* encounter broken software that can't handle either. In at least one case, a certain airline accepted the apostrophe when creating a FFM account, but not when booking, so I couldn't tie the two together. I frequently get snail mail where the apostrophe is replace by "&". I saw an endontist last week who couldn't bring up my insurance info because their software couldn't handle it. That's why it needs to be better.

A Wise Man Once Asked

[Anonymous Coward]

"Do you not know, my son, with how little wisdom the world is governed?" -- Axel Oxenstierna

Nothing New

[Herkum01]

That is because corporate infrastructure software does not generate revenue. Why spend money that does not directly impact the bottom line?

Maybe when you get people who actually understand the underlying business rather than a MBA graduate, that will change.

Re:Nothing New

[JDG1980]

That is because corporate infrastructure software does not generate revenue. Why spend money that does not directly impact the bottom line?

It does impact the bottom line; it's just harder to see and measure. When lots of employees are wasting time rebooting after crashes, or repeatedly navigating a slow and/or suboptimal user interface, that's wasted time that costs productivity and money. Just because you aren't measuring it doesn't mean it isn't happening.

Re:Nothing New

[frisket]

That is because corporate infrastructure software does not generate revenue. Why spend money that does not directly impact the bottom line?

Marketing can always get fat funding to have designers polish the turds on the web site, but the backend people don't have access to that kind of money.

Maybe when you get people who actually understand the underlying business rather than a MBA graduate, that will change.

If payroll is threatened, you may get some action, but anything else usually gets a Band-Aid.

Re:Nothing New

[luis_a_espinal]

That is because corporate infrastructure software does not generate revenue.

Says who? That's like saying a cleaning crew or the electric system does not generate revenue for a supermarket. Revenue is not just a function of what you sell. You need a lot of things to generate revenue. A business that uses corporate infrastructure uses it to generate revenue. The problem in most unorganized enterprises is that they have poor accounting for tracking the cost and revenue of corporate infrastructure. Added to that is that most software developers have no clue about the cost and ROI of the software they build for a living (which is completely unnaceptable.)

Why spend money that does not directly impact the bottom line?

Maybe when you get people who actually understand the underlying business rather than a MBA graduate, that will change.

It's not just MBA's who don't get it. Software developers do not get it either. There is plenty of blame to throw to both sides of the fence.

The old adage

[killmenow]

Good, Fast, Cheap...Pick Two.

Re:The old adage

[dkleinsc]

Another rule here is out-of-sight-out-of-mind: If management can't actually see the effects of what's going on, they don't care how good it is, which is why UIs can be fantastic while the backend completely sucks.

Re:The old adage

[kbolino]

Good, Fast, Cheap...Pick at most Two.

FTFY

I've seen plenty of software in the "none of the above" category.

It's called job security...

[IflyRC]

True, most software is badly written and there are entire jobs dedicated to maintaining legacy and even current systems. Some software is so badly written that it requires a team to prop it during peak usage times or War Rooms to determine fixes. Managers usually only care about meeting a deadline and push for that. Young guys don't care about if something is correctly written - just that "it works" in that instance in time. Being a good developer requires being enabled to be a good developer by your team.

It's a big world

[MozeeToby]

There aren't enough 'good' coders in the world to implement all the software that needs to be written, let along 'very good' ones. Not to mention good architects, designers, requirements analysts, etc, etc, etc. And even if there were, software that needs to work together isn't always designed to do so. Hacks, cludges, and jerry rigged solutions are what hold the tech world together, no amount of wishful thinking is going to change that.

Re:It's a big world

[Duhavid]

"Sometimes the most valuable skill a developer can have is a good instinct for which business requirements are most likely to change over time"

Amen!

Fixed

[SuperKendall]

That is because corporate infrastructure software does not obviously generate revenue, and losses of opportunity are invisible.

Fixed it for you.

Basically just supporting the last half of what you said.

Re:Yeah, but how do you measure 'Quality'

[SirGeek]

There are ways to determine quality. One pretty standard way is to count the number of bugs found during testing each phase of development (design, coding, unit test, product test, integration tests and after its in production).

Hey now!

[killmenow]

That's just mean!

User-facing software is well-polished?

[dgharmon]

"James Kwak .. points out that while user-facing software is often well-polished, the code running supply chains, production lines, and financial markets is rarely so refined"

I disagree, while the GUI may be well polished, the underlying code is of poor quality, as it has most probably written by some contractor on an hourly rate. Quality control works like this. If it compiles, ship it and fix all the bugs in the next version ...

Re:User-facing software is well-polished?

[HeckRuler]

Beauty is only GUI deep, but ugly goes straight to the code.

I've worked at investment banks since '93

[bobs_lounge]

and this article is absolutely correct. Forthe most part, we do regression testing, but a lot of code (a whole lot) is never unit tested, its not written to be used it tested, and there are configuration holes all over the place. Each time there is a Jerome Kerviel or Nick Leeson, a generation of auditors will come through and find systems faults, and put in reasonably effective controls, but that is not the same as programmatic correctness. Programmatic correctness often has to be baked into the code from the start (same with effective unit testing), and by and large, this is not an investment banks highest priority (as an earlier poster wrote, code that is not directly involved in revenue generation does not get funded).

Re:I've worked at investment banks since '93

[ebh]

Measuring how often something *doesn't* crash is extremely hard to show to the bean-counters. So, be ready to demo.

True story: When I started a sysadmin job, we had JBODs that routinely ran out of space, causing all sorts of downtime problems, like leaving the whole building dead in the water for days. I convinced TPTB that we needed decent storage (in this case, NetApp), but many of them choked when they saw both the purchase and maintenance costs. After we had it installed, I saw that a volume was close to hitting the wall. Before doing anything, I called in the most skeptical of the PHBs, and said, "Wanna see the NetApp pay for itself?"

I showed him the alarm that indicated a space problem, and told him we were on the verge of going down for two days. I waited for his skin to turn pale, and at the right moment, I said, "But we won't, because the NetApp can do this," and in a few keystrokes and mouse clicks, I added enough space to the volume.

I then told him that things like this happen once or twice a week, and I fix them without anyone knowing anything went wrong, but that I documented them in the trouble ticketing system I had installed, so if anyone wanted to know how many disasters were prevented by having decent storage, that's where they should look.

That didn't completely solve the problem of the thousand successes going unnoticed and the one failure never forgotten, but it came close.

It isn't cost effective to build good software

[neves]

It isn't cost effective to build good software... for a few users. I develop some internal systems. They are very complex and each of them have 40 users at most. The ROI of Apple polishing every tiny bit of a software is great. If each of their 100000 users spend one second less, it is a ROI of more than one day. Human beings are very intelligent. They can learn to play a musical instrument, drive a car, operate a machine and to use shitty software.

Re:Yeah, but how do you measure 'Quality'

[Reasonable Facsimile]

There are ways to determine quality. One pretty standard way is to count the number of bugs found during testing each phase of development (design, coding, unit test, product test, integration tests and after its in production).

Those can be valuable metrics, but finding and fixing a lot of bugs can't improve the innate quality of the item under development/test. In other words, you can't test quality into the product.

Re:Cost of geek food going up

[h4rr4r]

Sounds good, add another $1 to the price and pay the folks making it the money made from it as well I say.

I have no problem paying a little more so that the people working long hours for shit pay can at least see a doctor when they get sick.

Re:Yeah, but how do you measure 'Quality'

[Sperbels]

What is this number compared against? A bugless program may still be a piece of shit.

Re:Cost of geek food going up

[Surt]

Like a lot of the audience, I think it's really really good if products include the price of caring for the health of those who produce them.

Indeed

[Ancient_Hacker]

Indeed. I've been parachuted in to several companies with major software issues.

Three had avoided even starting a migration from hardware and databases that hadn't been supported in a decade or more.

Another placehad no concept of file locking or threading, or QA, and was using 8 different programming languages on just one project.

Two companies that handled 80,000 to 300,000 transactions a day did not have any way of simulating input or comparing the input to output.

One company that depended on several million TCP/IP connections a day had no idea that TCP/IP data might not all arrive in one packet.

Another place whose business was dependent on several custom fonts would not believe the veracity of both the Postscript and TrueType font verifiers when they said "your font has 488 serious errors".

About 3/4 of the places had not a clue what SQL injection was and how they were vulnerable.

The quality of the stuff out there is just horrible.

Re:Indeed

[MrSenile]

The quality of the stuff out there is just horrible.

Having worked under several hats, the latest being as a system architect, I can tell you exactly why this happens.

We start with some upper management who have this 'nifty idea' that they must have for the business. Ok, fine... now let's get the ball rolling!

First, you have the budgetary committee. Without any input what so ever from the technical groups that make up the technology and know what is or isn't possible, they work with vendors on a parachute budget for the project.

Secondly, with this locked in budget in hand, they introduce it to the system architects and project management. The project management are giving timetables saying 'we need this done by this time, no exceptions'. They then pass that timetable, as well as the budget, to the afore-mentioned system architect.

Introduce stroke-approaching WTF moment for the architect....

Third. The architect goes back to the project manager saying 'We can't build the specs for the money in the time allowed'. The manager goes 'oh, right'. They go to the budgetary committee and bring this up, and once they realize the bottom figure is wayyyy out in left field, they come back with 'that's impossible, we need this done, with the results of this, for the money you originally quoted us'. So... head back to the system architect...

Forth, the architect then, to un-bury himself from the absolute disaster sitting in his face, tell the project manager what will be required to minimally meet the ends. This generally requires a ton of over-seas consultants, paid to grind the wheel 24/7, at the lowest dollar, to get it to work on outdated hardware to meet the end core/cpu/memory requirements and still 'work'.

Fifth, the consultants are hired, you're lucky if they understand english sufficiently to understand the nuances of day to day communication. They also take shortcuts, because they either don't know the right way, don't want to spend time on the right way, or are told that doing the 'right way' is not time efficient for the cost. So now, we have crappy code being tossed in, usually undocumented.

Sixth, the dev work is slammed in, marginally tested, and quick-shotted through QA because the upper management are in a time crunch and don't have the time to deal with all that 'quality assurance nonsense'. So this work is now fast-tracked to production in a non-fully tested workflow.

Seventh, it's been live for a while, things break randomly without reason, but it's ok, a restart of the application always 'fixes' it. So what if you have to bounce the app every 2-3 weeks to free up that memory leak. It works, in the budget... well, maybe for a few hundred thousand more... but it's done, it provides the 'nifty feature' that the share-holders were promised by the upper management, and the things that don't work are being pushed on...

three guesses...

The management who pushed the idea? Nope.

The budgetary committee who gave the low-end figures out of their butt? Nope.

The project manager who gave the tight time-frame for the project without major input from the technical people? Nope.

I know... the IT professionals who are still at the company like the system architect, network team, dba team, san management team, and the security group who are left holding the bag of the big pile of steaming crap? Yup.

Soo... when things evidently break so bad to be noticed, and management are told to 'fix' it it will take more money, more time, and more hardware. Shock, awe, and bafflement is shown, bonuses/raises are crushed because the IT professionals obviously can't do their job right, and maybe a few heads roll because management have their golden parachute and are not held to blame for the project they initially started up.

That. Is why the 'stuff out there is just horrible'.

It's not any one thing, it's how business runs, because these yahoos frank

Writing good software is hard.

[RabidReindeer]

Software isn't hard. Everyone constantly tells me so. "It's simple! All You Have To Do Is...".

"Oh, those preliminary mockup screens look almost perfect". So you'll have the entire system ready for production deployment next Tuesday, right?"

"Just git 'er DUN!"

Re:Yeah, but how do you measure 'Quality'

[NeutronCowboy]

AC hits it on the head. This is nothing but the age-old search for the perfect metric. Development processes ARE oriented towards quality - for arbitrary values of "quality". The problem is that quality software is like porn: you know it when you're looking at it, but you have no idea how it is exactly defined. Is it a lack of bugs? Sure, but that's definitely not the only aspect. Is it maintainability? Maybe - if the software needs to be around for the next 30 years. Is it readability? Dunno - machine code is pretty unreadable, yet there's quality machine code out there. Is it how long it took to develop, how flexible it is, how user friendly, how much power features are in it? Maybe, maybe, maybe.

Pick a metric - a boatload of metrics - and I will find you a large number of cases where the metric fails. Are we doomed? Kinda. Just like there's no silver bullet that solves all your development processes, there's no silver bullet when it comes to measuring the output of the process.

In the end, what people care about is "does it do what we need it to do?", and that's all that anyone is going to remember. Unless, of course, it's review time, and then the only thing that matters is "the metric".

Yep, we're doomed.

Engineers don't make the buying decisions.

[gestalt_n_pepper]

Bean counter and management do. They don't care how much the staff struggles with lousy software (e.g. Oracle server on Linux). They care about saving a few bucks, getting their bonuses, reorganizing to hide the bodies and moving on to the next job. Hence, there will always be a market for crappy software. Capitalism fails at the interface level. If the engineers and low level end users made the purchasing decisions, you can bet quality would improve in a hurry.

Re:Yeah, but how do you measure 'Quality'

[Local ID10T]

Tea, made from real koalas, of course.
http://www.koalatea.com.au/

Oh. I see. Nevermind

Re:Yeah, but how do you measure 'Quality'

[davidwr]

A bugless program may still be a piece of shit.

Yes, but by definition a bugless program that is a POS is a POS by design.

In other words, if the customer orders a POS, and you give him a bug-free program, he will get a POS.

Re:Cost of geek food going up

[Desler]

Yes. I prefer my turds grilled with dill and lemon not warmed over like a Papa John's Pizza.

Re:Cost of geek food going up

[h4rr4r]

You do realize there are other nations that have socialized medicine than the USSR, right?

Many of them have fine medican care.

Re:hmmm

[PPH]

You mentioned utilities, so I've got to jump in with an anecdote.

Several years ago, I was brought in by our local electrical utility to develop a configuration control system for engineering documentation. Specifically, substation engineering documentation. The problem was presented to me as one of engineers getting behind schedule working on as-built drawings. And working from the wrong drawing revisions. So, before accepting the project, I asked to speak to some of the other stakeholders in the design and build process. Specifically, the construction crews. The first sign of trouble: Engineering management looked at me kind of funny. It seems that engineering and construction didn't get along too well. Nevertheless, as an outsider, I figured I had an advantage.

After speaking to a few workers, I had a better picture. It seems that the relationship between these two groups had been poisoned for many years by (as far as I could tell) one individual in engineering management. The crews considered him to be a raging a**hole and wouldn't work with him (he had retired years before, but people still carried a grudge). As a result, the crews pretty much built things the way they wanted instead of per the drawings. Specifically, they built new stations the way they built the last one. As a result, the smarter engineers as-built the older drawings. Those being the ones the crews were working to.

Management wanted me to come in and build a system to 'lock down' released drawings and restrict the work flow. Without understanding the root causes of their problem and how people were working to accommodate process and personnel problems. I walked away from that job offer quickly.

I have a few stories (oddly about defense contractors) where software was built intentionally to be lousy. If the users' management had to employ a staff of a few hundred people to repeatedly sanitize database entries (rather than built error checking into the entry system) it meant they were a third or fourth level manager (with the commensurate salary) instead of first level with a couple of DB admins under them.

Good software is easy to write. Bad management or personnel problems are difficult to fix. An associate one told me that he could fix all of the problems in a $250 million dollar failing s/w project with one clip in a .45.

May I Say a Word About Patran

[florescent_beige]

I have used this program my entire career. For the last 20 years (since MSC bought PDA), it has not changed apart from the odd user-generated macro getting included in. The windowing interface has had the same bugs (e.g. scroll bars that are 1 pixel high) since I was a wee lad. Half the stuff in it that isn't used regularly doesn't work, never has, never will and yet it is the standard for FEM pre/post in aerospace. Staring at this broken-ass POS year after year has filled me with ennui.

May there be a special circle in hell just for MacNeil-Schwendler.

Thank you. That is all.

Re:Cost of geek food going up

[ddd0004]

I always assumed that Taco Bell's product was consumed as a laxative, of which they produce a highly effective product.

Re:Cost of geek food going up

[luis_a_espinal]

You fools think it's going to be a dime or a dollar? The cost will never stop going up, and the quality of care will go down.

Boy you libs are so damn predictable.

You want 100% coverage, gee Cuba has had that for a long time now.

"Ohh I'll gladly pay [and pay and pay and pay and pay]."

Keep bending over suckers, the real hard part is just around the corner.

No, it hasn't. Ever been to Cuba, where you get an aspirin for free, for everything because that's all they have?

A better comparison is the health care systems of say, Japan, Canada or Israel... or almost any other developed country for that matter.

Re:Yeah, but how do you measure 'Quality'

[sosume]

That behaviour always annoyed me, there's just no valid use case for 'rm -rf /'. It has caused way too many system wipes to not be addressed. The only valid output should be:

# rm -rf /*

Usage: mkfs [-V] [-t fstype] [fs-options] device [size]

Re:Perhaps it's a communications failure

[joelsherrill]

Are you saying that the technicians who used a left arrow as a backspace on the Therac-25 were not "honest, competent people"? Bad software and hardware design can kill. See http://en.wikipedia.org/wiki/Therac-25

That's not the good coder

[SuperKendall]

Ah yes, the mythical good coder who writes it "right" the first time.

That's not really the work of a good coder. Anyone could get lucky, and no-one writes correct code all the time.

A good coder though can structure code in such a way that problems do not cascade, that incoming issues are limited in scope in terms of affecting the rest of the codebase. A good coder can make a huge system where you can replace a part of it without magic or too many tears.

Perhaps it's nothing more than the ability to think ahead a bit while coding or planning to code or pondering the data that is to be fed into a system, but from past experience there are simply many people who do not do that when coding or architecting for whatever reason.

For the non-coders out there, they kind of guy I am talking about is the pants tailor in "The Hudsucker Proxy", who added a second layer of stitching because he liked his client...

That's the kind of system a good coder gets you, the kind of system that lets you metaphorically hang by your pants 50 stories above the ground when things go pear shaped or your input goes wiggy.

Re:hmmm

[Anonymous Coward]

An associate one told me that he could fix all of the problems in a $250 million dollar failing s/w project with one clip in a .45.

That is an awesome line.

Pffft, of course it's simmetric

[tlambert]

Is the distribution curve simmetric?

Why wouldn't it be measurable by the little people who live in your computer?

we live in the era of "Ship then patch"

[logicassasin]

why is this news to anyone? Software is -always- shipped full of issues to meet a PM's deadline in order to say "See!!! We got it done on time!" to justify their salary and existence at the company. "Ship it now and fix it after the fact" (if at all) has been the mantra of in-house and commercial software for 20+ years.

Re:Cost of geek food going up

[19thNervousBreakdown]

I'm going to step in here as a pizza connoisseur. You are correct about how to order a Papa John's pizza (although you neglect to mention that the reason for going thin crust is more the hideous amounts of sugar they dump in their regular crust than its thickness), but it's definitely not excellent. Passable, if nobody else near you delivers and you don't feel like going out, but far from excellent.

The sauce isn't great, their thin crust is way too crunchy (unless it's soaked in grease and falling apart like toilet paper) and about as flavorful as a wafer cracker, their regular sausage is weird-tasting rubbery balls of fat, and their cheese is as tasteless as everything else unless you get one of their blends, in which case it's actually pretty good but there's far too much of it.

If Papa John's seems like excellent pizza to you, go to New York. Or Chicago, for a completely different but still very yummy experience. Either way, you'll find better pizza on the way there than you will at Papa John's, and I'm saying this as someone who ordered so much of it that the manager of my local store sent me a Christmas card.

Re:Cost of geek food going up

[Fallingcow]

Bingo.

Single-payer, for instance, would be a bigger boon to entrepreneurship, job mobility, and (actual) small businesses than any tax cut proposal the right (or left, for that matter) has put forth. Even better, if it's done right (so, at least as well as it is in the very worst system of that sort in the industrialized world, since they all spend less than we do) it'd cut costs for everyone, including big businesses.

Too bad that would be evil socialism, I guess.

"Just-Good-Enough" software runs the world

[n7ytd]

Remember: Broken gets fixed. Shoddy lasts forever.

Re:Yeah, but how do you measure 'Quality'

[arth1]

That behaviour always annoyed me, there's just no valid use case for 'rm -rf /'.

On the contrary, it's something I do often - I chroot into a directory which has read-only bin/lib mounted, to test software in, and the first and last command is always "rm -rf /".

What you have to realise is that the Nike options (like -f and --force in most commands) means you sign in blood that you know what the command is doing, have verified it three times, and that you know that any blame rests squarely on your shoulders alone.
If you aren't willing to sign that, use -i instead.

Not only important industries

[damn_registrars]

Bad software also runs unimportant websites.

So why are they called Software Engineers

[aXis100]

I'm not trolling, it's a reality check.

Most big software projects I've seen fail hard, like millions and 10's of wasted dollars hard. By comparison you just dont see that very often in big electrical/mechanical/civil projects, which can be equally complex (eg refineries, cruise liners etc).

There are software developers with all sorts of fancy titles - architects, analysts, engineers - and yet they cant get the code right. Usually the root cause is inadequete requirements spec and failure to manage the customers expectations but that's no excuse, there are usually numerous poeple employed in the project process specifically to get those parts right.

Software engineering is still playing catch up, in the sense that most developers and development companies I've seen still dont follow a formal enough process for it really to be called engineering. Usually it's a bunch of computer science graduates having a wild stab at it, and the good ones are closer to artisans than engineers.

Until the entire software industry gets off it's high horse and admits this to itself - and more importantly admits this to the customers, we are going to continue to be dissapointed with the quality.