A Gentle Rant About Software Development and Installers

Nerval's Lobster writes "This is the story of the comparison that just wasn't meant to be. It's a story of everything that can go wrong in the customer end of the software world, and some thoughts on what needs to be done, especially in an area known as Installers. I'm a software engineer with 25 years of experience, and for years I've wanted to point out some of the shortcomings of my own industry to help make it better for everyone involved—not only for the end-users, but also for the IT people who have to support the products; the salespeople who have to sell and later, possibly, apologize for the software; for the executives whose hands are tied because they don't have the technical knowledge to roll up their sleeves and help fix problems in the code; and for the programmers themselves who might get stuck with what some consider the absolute worst position for a programmer: maintenance of crappy code written by programmers who have long since left the organization."

Comments

The problem is that we still use installers...

[gestalt_n_pepper]

Instead of creating single exe. Memory and disk space are cheap. Portable application creators are easily and cheaply available (e.g. Cameyo.com - both free and excellent at what it does). It's easier to copy a single file than to go through the hell that is installation for most apps.

Maintenance Isn't a Bad Job

[Greyfox]

As long as you can incrementally improve the design and code, maintenance isn't a bad position to be in. I've seen far too many programmers who whine and whine about how much the code base sucks, but they never do anything to make it better. They insist that the only way to go forward is to rewrite the whole thing, a project that is almost inevitably doomed to failure. If you actually design new code, implement policies if the company doesn't have any in place, and clean up the old code while you're hunting bugs, it can be every bit as rewarding as new development. ANY programming project can be a joy to work on, or a nightmare to work on, and it's entirely the discipline and ability of the team and its management that makes all the difference.

Re:Maintenance Isn't a Bad Job

[Anonymous Coward]

That's because those programmers are not professionals but basically code churning prima donnas. They like to write new code and move on rather than do boring things like bug fixing and other maintenance tasks. The GNOME project epitomizes this attitude in spades.

Re:Maintenance Isn't a Bad Job

[Todd Knarr]

The big problem I have with your statement is that often the problem with the software is the basic design. It's 10 years old, completely insufficient for what's being asked of it today, and has so many kludges and hacks bolted on that you can't do anything outside a tight set of constraints without irreparably breaking something else. Usually it gets to this state because management or the team leadership won't permit changes to the design (the usual justification is that there's too much risk and no business benefit right now). Once code's in that state, new code has to follow the old design to work and it's that old design that's adding complications and keeping new requirements from being cleanly implemented. You can't design new code, because the constraints imposed by the design it has to live inside are too tight. You can't implement new policies because they break existing code. You can't clean up code because every bit you try to clean up requires you to first clean up 8 other bits, and eventually you hit a stop-point in the recursion where cleaning up that level requires that the program design be changed, see above.

Yes, good code should never get to this state. But the problem cases aren't good code, that's why they're problem cases.

Re:Maintenance Isn't a Bad Job

[quietwalker]

There is a problem in what you've stated, and it comes down to the last line, "entirely the discipline and ability of the team and its management that makes all the difference"

I've know a lot of exceptional programmers, and I've known a few absolutely horrible developers. However, I've known of NO developer who wanted to push code out that door that was just awful. Rather, I see a lot of developers end up over-engineering on every level to ensure that their product is the brightest, best thing ever. If they had all the time in the world, I'm sure their products would be simply exceptional.

On the other hand, I haven't worked for a single manager who, when the chips were on the table, said something along the lines of, "We'll give you 5 more moths to refactor this, to ensure that ongoing updates and maintenance will be straightforward, and our internal tools can support automation of common tasks," instead of "Just make it work, we'll worry about the rest in the next release." ...and when the next release comes up, it's, "We promised these new features, we don't have time to refactor: if it ain't broke, don't fix it."

I actually just left a company which has been fighting this problem for so long that the entire dev department is spending 80-90% of their time tracking down reported bugs, and the remaining time cramming in whatever was promised to the customers in the fastest way, damn the maintainability. Each year, the cost of bugs and maintenance has gone up, and the devs are now all on call - the operations team cannot support the product themselves anymore. Think about that; you are a developer, and you are on call. 24-7.

The problem is that you need an exceptional development manager who can fight the good fight for the good of the product and the company, and can make the subjective value of code reuse, good architecture, and so on apparent to those above and around him. The company needs that discipline you point out to set realistic customer expectations, to train their sales people not to overreach just to ensure a sale, to listen to the development managers when they describe cost, manpower required, and so on.

Of course, that's only if you want a perfect product, with ever-decreasing quality returns for your time and money investment. It's probably impossible for a developer or even development manager to say where the pivot point is in the market; where time and money spent on quality causes a potential loss of revenue for a product. Unless we're talking about military, healthcare, or hostile environment systems (subs, satellites, space travel, etc), there IS a point where the cost for quality is too high.

In layman's terms, the cost to make high-quality software exceeds the price people would be willing to pay for it. This means that we're all going to be happy 'enough' with imperfect software.

Re:Maintenance Isn't a Bad Job

[Anonymous Coward]

Try working in a place that doesn't permit rewrites and frowns on refactoring. I've seen software policies that a 120% guaranteed to result in the shittiest, cruftiest, nastiest pile of the most vile mismanaged crap you'll ever have the misfortune of trying to debug. What you call "whining" is usually the programmers trying to get 15 minutes of refactor time into the schedule. Answer: "No, bitches, where iz my features?"

Trying to fix it is futile and "against company policy".

Why?

[BCW2]

Why does every windows application programmer think their program is so special or important that it needs to run in the background? The first thing to speed up any computer is to kill all the crappola trying run at start up. A program should run when started and when exited from it should completely shut down and even delete any temp files it created. I did a cleanup on a customers machine once and deleted over 10,000 temp files. That is lazy and stupid programming.

Re:Why?

[Chris Mattern]

The windows pc+user is "Territory", applications have the main function of occupying it, and if feasible preventing the competition to do the same as much as possible. Exerting control yields profits.

Exactly. Program A completely exits when the user shuts it down. Program B keeps most of itself still in memory and running in the background when the user "shuts it down." Result: When the user starts up Program B again, he is pleased at how fast it comes up. When he starts up Program A again, it has to load--this is an acceptable price for not turning your PC into a mass of sloth all the time, but the user doesn't see that part. And the loading is even slowed down by Program B hanging eating up memory and cycles! So the user thinks of Program B as an efficient, fast program, and Program A as a slow piece of crap.

Re:Why?

[JaredOfEuropa]

Indeed... and that brings me to an area that needs way more attention than installers, which (in my experience) mostly work reasonably well. Lets talk about the way installed software is kept up to date. You'd think by now we would have solved this problem, but the following list of my issues with this process is still sadly valid in 2012:
- Program installs a little update process to run in the background. Don't. Acceptable ways to check for updates are: checking when the software itself is run, or glomming onto some centralised updater (take the App store as an example).
- Program doesn't really update itself, but initiates the download for a fresh install file that I then have to run and click through. Don't. Instead, make sure the program updates itself with minimal user interaction. (Kaspersky does this well; the Battlefield 3 browser plugin doesn't)
- Program updater makes you specify every install option again, like the installation directory, or even the ^&%$*(&@$ license that has to be accepted again. (Java is one of the many things that does this)
- Program updater asks to install a browser toolbar (the default is "yes", of course). Seriously, Adobe, a browser toolbar, in 2012???
- On top of all of the above: publish an update every other day or so. Don't unless you're patching critical security holes.

In short, the updater should not be implemented as a background process unless there is a very, very, very good reason to do so, and should perform the update with minimum user interaction required. That is: asking me for permission, and nothing else. And it certainly should not install anything other than the software being upgraded: publishing an update is not "an opportunity to re-engage with existing customers".

This is slashdot so...

[Anonymous Coward]

The problems will be the faults of everyone except for the developers and software engineers. Problems are always the caused by the "other guy", including previous developers. According to the developers, they should be the supervisor, the team lead, the manager, the lead salesman, the system engineer, the network engineer and every other position higher than what is beneath them. In their minds, problems only happen when they are not in those positions controlling it.

The funny part about this attitude with developers is they always feel there are two distinct groups of themselves, ones that are awesome and ones that suck. Every developer thinks they are in the group that is awesome and thinks just about every other developer sucks. Just about every developer I have ever known has complained about the existing code, previous developers, and work flow processes, and his/her eventual replacement will do the same about them, it is a predictable cycle and trait. It is a field of work where everyone thinks they are the best and can do no wrong and everyone else does it wrong.

Mod me down but analyze the comments from this story and previous stories and you will see this pattern.

Re:Okay.

[sribe]

Bullshit. There's not one thing in your list that cannot be handled with an installer. You just don't care to automate the process, and are making up excuses to justify that after the fact.

A Gentle Rant About Fixed Width Web Content

[c0d3g33k]

I have a wide screen monitor. The article uses about 1/4 of the total screen width. The column on the right a little bit more. The rest is whitespace. Resizing my browser window has no effect - just more or less whitespace. I skimmed the article but didn't read it because it is annoyingly narrow and I balk at zooming in or increasing the font size to fill more of the screen. Fixed width content needs to go.

Re:A Gentle Rant About Fixed Width Web Content

[Marc_Hawke]

I see your, 'fixed width web content' rant, and raise you a 'running browser full-screen' rant. There's no useful purpose (as you've so elegantly pointed out) to running your browser the entire width of your monitor. In fact, the entire point of a wide screen monitor is so you can get more done (ie, more done at once.) So, have your browser as a nice window on the side, that's in a size and shape that's useful for the content, and use the rest of your widescreen monitor for something else. Save 'full-screen' for those times when the content dictates that you use full-screen.

I find it interesting that 'zooming' was one of your proffered solutions, but scrolling isn't. You realize that even if you did zoom, you'd still have to scroll?

Re:Okay.

[GoatCheez]

I agree. By thinking about installation when creating complex enterprise software, you end up making better quality software. When you're forced to handle the inter-dependencies of complex systems in an installation process, you end up having a better handle on how everything interacts. This has been my experience at least.

The authors experience is largely my own.

[Vellmont]

I'm a developer who winds up having to do a lot of backend support and installs, I've been installing various enterprise packages for the last 6 years. The authors experience is VERY familiar to me. It's quite hit or miss, with some of the most expensive ($40,000+) pieces of software giving the most miserable experiences. You spend days trying to fix this or that, and it winds up being some obscure setting somewhere that only a super-expert could ever understand.

What sucks is that we have to put up with this crap. End users wouldn't stand for it.... but yet sometimes I swear IT staff think it's somehow OK, and they either blame themselves, or think they've "learned" something by going through these dumb install problems and jumping through the hoops. I'm tired of it, and it wastes a lot of valuable time. There's some things that can't be avoided, but the majority of the problems I've come across could have provided MUCH better indications of what went wrong, or avoided the problem altogether.

Enterprise apps are supposed to be hard

[quietwalker]

To those who haven't read the article, the author posits that testing two same-purpose pieces of software to see if they generate the same result is a simple proposition. Then we find out that the systems are SAP and Oracle. This is not like installing two mp3 players, these are the poorly-defined field of 'enterprise apps'.

I don't disagree that the installation for anything claiming to be an enterprise app is usually hard. Setting either SAP or Oracle ~properly~ requires expert knowledge, and running either ~properly~ requires expert knowledge. This is expected, because it's an extremely complex product which is meant to be deeply customized to your own business solutions. This is similar to the gulf between installing the next version of windows and installing, say, a slackware distribution. One is more about clicking next, while the other requests explicit knowledge of the system in order to tailor it to your specific needs, and even after install requires effort to make it usable. One requires you know nothing, and allows almost no customization, the other expects you know everything, and allows an almost infinite level of customization.

That's why the install of MySQLwas so easy for him. Nothing against MySQL, but I'm not going to put it in the same category as Oracle or SAP. It's not trying to be either of those - it has it's own niche. It just doesn't have the same level of customizations or capabilities.

The other thing that always irritates me: Why the hell would you have a software developer installing 'enterprise' software anyway, unless they're some sort of expert in that software type anyway? Don't get me wrong, I've installed many a developer version of Oracle locally, but I'd be the first to point out that I'm not the Oracle-certified expert that I expect will be running the show in production. Don't people understand that these are separate skillsets? The author states that he came from a j2ee shop running large products, my guess is, he didn't have to know how to admin the application servers. Ever try to dive into enterprise app server configurations? Clustering across firewalled domains with reverse proxies, remote caching, load balancing, certificate management, and active directory tie ins? In something 'big', like JBoss or worse, WebSphere? You could be an incredible java developer, but it has nothing to do with setting up and configuring the app server.

So, he was the wrong person for a task that was, quite honestly, supposed to be hard. Of course he had problems.

Re:Enterprise apps are supposed to be hard

[Maximum Prophet]

Setting either SAP or Oracle ~properly~ requires expert knowledge, and running either ~properly~ requires expert knowledge. This is expected, because it's an extremely complex product which is meant to be deeply customized to your own business solutions.

I hear this, but I think, "A C complier is a extremely complex product meant to be deeply customized to your own business solutions." Yet GCC is relatively easy to install, and once installed I can start customzing right away. I can even use customizations that I've written years before and expect them to run with little or no modification. (C programs)

Installing the product, and running a test case should be a no-brainer, no matter how big the product is. After it's installed and running, then you can start to add business customizations that might be difficult and time consuming.

Re:Okay.

[Anonymous Coward]

Mine too, but it never works that way.

The last thing that gets thought about is dependencies - this is also one of the reasons Android software acquires permissions but rarely relinquishes them. We only know how to grow software, once it starts working the "Job is done, go home".

Re:It's the submitter's fault.

[war4peace]

Why should an automobile last longer than an OS?

That has no longer been valid since post-'95. An OS lasts just as long as an automobile, if you consider all the variables. If the infrastructure doesn't change, an OS would last forever. But infrastructure (read hardware support, network support, etc) does change, and rapidly. Your own automobile would be good for nothing if, for example, current roads would be replaced by magnetic field monorails.

Re:Okay.

[sribe]

I agree. By thinking about installation when creating complex enterprise software, you end up making better quality software. When you're forced to handle the inter-dependencies of complex systems in an installation process, you end up having a better handle on how everything interacts. This has been my experience at least.

Absolutely true. I didn't even bother going there in my response, because I figured it was pointless when someone was actually claiming "enterprizey software is simply too complex for installers". Sigh. As usual, the creators of the problems are blind to the problems...

If we build it

[SmallFurryCreature]

If we build it, it is because the user, may his children be cursed for a thousand generations, told us to do so. In fact, that is pretty much the reason ANY crap building has been up to cheaply in a dangerous area. ONLY because THE LAW and BIG GOVERNMENT stops users from demanding high rises build out of mud on top of mud in a flood plane on a fault line, has this stopped... and then only when THE LAW and BIG GOVERNMENT do their job really really well. Just google engineering disasters for examples when THE LAW and BIG GOVERNMENT were stopped from doing their job.

Remember that Engineers have strict laws governing their actions. Software Engineers (HAHA) don't. You want to have the same quality from Software Engineers as real engineers? PAY FOR IT! And WAIT FOR IT.

There are good reasons for building codes and there MIGHT be a cause for coding codes BUT it won't happen when "cheapest" wins out every time. I have seen it far to often when a project was reduced in quality by the customer insisting it be done cheaper and faster. Well guess what, then you get buildings oops code that fall down when someone sneezes.

Want an installer that works in more then one near perfect situation? Sure that will be X on the total bill. User: oops no and I want to save Y amount as well, so cut some more corners please.

Is there a patent on administering electric shocks every time a user demands a shortcut but expects quality?

Re:Okay.

[gorzek]

Bingo! "My organization uses a bunch of home-grown ad hoc'd garbage that isn't versioned or managed by anyone sane or competent" isn't a problem with installers, it's a problem with having a deeply dysfunctional organization. There is no software that's gonna fix that problem, and it's not because packaging/distribution software is lacking in some functionality.

If you have to install it by hand, why? What are the steps? Why can't those steps be automated? Why can't those dependencies also be automated? What, you can't edit XML files, .ini files, or registry settings on the fly? Why not?

Re:Okay.

[aztracker1]

Creating an installer for a complex system... a couple man-years of labor from your highest paid employees for a system that only gets installed a dozen times a year.. vs. a checklist... Yeah, you'll be making millions with your strategy.

Re:Okay.

[Applekid]

Not blind, necessarily. Enterprise software is almost certainly designed to be "consultant ware."

Oh, having trouble getting it installed? Yep, our software is very powerful therefore very complex. We can help you with that, let's talk about a retainer.

A function isn't working? It must be your sprawling IT infrastructure. We can help track it down for you with an engineer dispatched to your data center, what's your closest hotel?

Well, that's great

[Giant Electronic Bra]

Yet another incipient genius who probably hasn't had the slightest experience on the ISV side of this equation with the convenient one-size-fits-all answer to end all answers. Thanks, that was amazingly edifying!

Frankly, you have NO FUCKING CLUE what you're talking about. I would presume you really have no familiarity with actual commercial line-of-business enterprise class software. Any enterprise which is not run by morons and is large enough to qualify as an 'enterprise' has an extensive IT infrastructure which is highly integrated and designed around an enterprise information systems architecture, involves capacity, availability, security, and usability analysis and planning etc. When you 'deploy' software into these environments, and I'm talking about large banks, Fortune 500's, large international corps mainly there's no such thing as "drop the installer on a hard drive and run it". You're dealing with integrated management systems, enterprise-wide credentials and role management, complex licensing situations, other corporate IT policy issues, etc etc etc. The idea that MY software, which handles transactions from generally 1000's of clients and integrates with easily on average 4-5 other complex software stacks is 'lacking an installer' because you don't install it clicky--click is hilarious.

You can think what you want, but at least in my world 'enterprise software' means something very different from what some people seem to be using it for. This is not like in your typical small business environment where you slap up a copy of Apache and install Trac on it and call it a day (and even there I'll note there's no single installer that can handle setting that up for you).