What Software Specification Tools Do You Use?

IronWilliamCash writes "I currently work for a small software development company and for many years we have been using internally built tools for all our software specifications, bugs, change requests and the like. Traceability is a big issue as we are CMMI level 2, and thus our internal processes need to be clear and everything must be documented. We are currently looking into getting a unified solution for this, and after quite a bit of Googling, there are quite a few different options (Contour, Kovair, MKS, Doors, CaliberFM, Accept360, etc.). I was wondering: what do other Slashdotters use in their everyday life? Does it fulfill your needs? And what is the most important part in a specification management tool?"

Comments

My sympathy for you

[Dolphinzilla]

If there is one thing that can suck the fun out of software development and engineering its requirements traceability tools - My company is CMMI level 5 and we use Doors, Rational ClearQuest and ClearCase, Teamcenter and others. As much as I hate them they have their place and on rare occasions are even useful. My best advise would be to find an integrated suite and avoid having a big hodgepodge of different crap that has to be glued together with third party applications.

Re:My sympathy for you

[Nerdfest]

Traceability=Butt covering over efficiency.

Not always true of course, but there's a strong correlation.

Re:My sympathy for you

[JamesP]

rational tools are anything but rational

I absolutely will think twice before taking a job that requires them.

However, one tool that I found to be extremely useful amd totally worth its price is Enterprise Architect. I don't remember if it does requirements

Re:My sympathy for you

[theshowmecanuck]

Traceability can also mean being able to hold someone's feet to the fire for doing a shitty job... or not doing it at all. e.g. "How did we leave this out?" "Let's see, programmer/analyst/dba/business analyst number one was supposed to do this." When people know they will be accountable for what they do or don't do, the task is more likely to be done, and done with better effort. Granted we need to assume people will try to do a good and conscientious job (why is generalizing about good things about people's behaviour OK, while generalizing things perceived as bad, is bad? I digress...) but we all know this isn't always true.

Word to the wise

[Rogerborg]

Do the absolute minimum amount of work that it takes to fake your way through your audits. Process People are a cost to your business, not an asset. If you invest time in processes, then you are not producing anything that your customers want to pay for - all CMMI/ISO compliance is about their Process People ticking the boxes that say your Process People have ticked their boxes.

Re:Word to the wise

[Anonymous Coward]

I am an auditor and you are so wrong!
Process Control is not about checking boxes as much as having control of you processes. Checklists are a tool to do the auditing of a process, not a goal to reach in themslves.
If you really care so little about your processes, come tomorrow they will byte you in the ass.
You know, it's like the unfastened seatbelt that you always wear in your car "because it's less bothersome" but still fools the cops.
At accident time you will be the sore ass with broken ribs.
Sorry, this downplay of yours of process control systems is simply unprofessional. I had enough of the "software developers are artists that have no rules" mantra.

Get real

Re:Word to the wise

[Anonymous Coward]

Wow... That is a rather ignorant and dangerous piece of advice you just gave. While I understand that writing software requirements and specifications isn't exactly a sexy job, and "process people" tend to create more work than they appear to be worth, they are some of the most important people in the software development process. Sure, if you're creating a new web-app designed to be used by 10 customers internal to your company, you can probably bang out the code without writing a spec, then backfill and BS your way through the documentation and testing.

If, on the other hand, you are writing large-scale software that will be used by thousands or millions of users on a variety of platforms, with a variety of hardware, and a variety of user expertise you would likely be complaining about all these "new requirements" that keep creeping up late in the game. These are the very same requirements that should have been caught in the spec sheet by those people who are a "cost to your business". Worse yet, let's say instead of writing a PC app, you are writing software that directly impacts the safety of operators or other people (nuclear reactor, thermostat, aerospace, automotive, medical equipment, etc, etc, etc). It is unlikely that while "faking your way through your audits" you are going to catch those little bugs that cost people their lives. If you don't want to believe me, look up Therac-25, the USS Thresher, Black Monday, various Airbus crashes, and any of the dozens of other incidents that are a direct result of the sort of mentality you are promoting here.

The next time you recommend someone just "do the minimum amount ... to fake your way through your audits" think of how comfortable you would be with the guy who developed your cruise control system thinking that way, or maybe the guy designing the chemical processing plant a few blocks away, or maybe designing your neighborhood water treatment facility.

Re:Word to the wise

[BlackSabbath]

I thought twice about replying to this but I can't help myself so hear goes nothing...

1. Your (presumably bad) experience with "process people" may not be everyone's experience.

2. Slavish adherence to anything (incl. processes) MAY be ill advised at times. Typically, standards and/or processes are instituted for a good reason. This does not remove the need to consider whether they are appropriate in all cases. Process does not remove the need to think.

3. Capturing requirements is pretty useful and having a repeatable, reproducible way of doing this is also useful. Need to take over some other person's 3 year old steaming pile of spaghetti? Requirements are a good start. Need to replace a home-grown system with a market solution? Requirements are pretty useful in negotiating a contract with the vendor (unless you don't mind the vendor nailing your balls to their invoice because you couldn't actually tell them what you needed their software to do).

4. Customers typically only care about the utility, quality and cost of the end product. They care about how the sausage tastes, not how its made. However, to achieve those things you will require some process. You cannot have worked on any large project and think otherwise.

5. I agree that even where "best practice" processes are in place, there are still inordinately many failures. However in my experience (25 years), this is almost ALWAYS down to people/political issues. My own hypothesis is that there is no system, process, methodology or framework that can enforce appropriate behaviour and that cannot be "gamed". The best you can do is try and minimise the opportunities for gaming the system. Irrational agents (at any level) can (and will if given enough time) completely bugger up the best laid plans.

I agree that IT is still incredibly immature when compared to a discipline such as (say) civil engineering. However these things are all part and parcel of developing that maturity.

When I hear criticisms of IT processes like this I'm reminded of people criticising democratic forms of government and Churchill's response "democracy is the worst form of government...except for all those others that have been tried".

Re:Word to the wise

[wrook]

People, in general, are a cost and not necessarily an asset. Making sure all your activities are returning value is an important part of running a business. More warm bodies looking busy doesn't make you more efficient, that's for sure. And in many large organizations I've worked in the "Process People" have often been "warm bodies looking busy".

However, there is a difference between a bad idea and a bad implementation. Having been involved in CMMI/ISO audits I have seen what kind of joke it can be. Those warm bodies are doing exactly what you suggest -- the minimum to pass their audit (which is barely anything at all, if I am honest -- mostly running around hiding evidence that the process is not being followed). But ISO and CMMI is not to blame for this injustice. The problem is those warm bodies and attitudes like yours.

Most "Process People" sit around fantasizing about what process everyone should follow. Then they write it down and tell everyone to follow it. Never mind that they have never actually seen what people are actually doing now. This is precisely the opposite of what is recommended by these meta-processes. How many times have I heard, "Well, normally it takes 5 years to get to CMMI level 4, but I think we can do it in 2 months if everyone cooperates"? Having said that, they have demonstrated their ignorance for any point at all in CMMI.

Document what everyone is doing. Not what they say they are doing; not what they want to be doing; not what you wish they were doing; what they are actually doing. That's your process. Compliance? Dead easy. They are already doing it. Documentation? It's not exactly rocket science. Make sure people put version numbers on documents. Or better yet, put everything in an SCM and say that all paper versions are merely drafts. Done.

You are right that you should do the very minimum that you can do with ISO and CMMI. Not because it is a waste of time, but because sitting around concocting elaborate plans for a cool process doesn't align with reality. Neither ISO nor CMMI tells you what you should be doing in your process. It tells you what you should be observing and thinking about. For level 2 you can write down pretty much anything you want for each section.

But the point is, once you have everything written down you can look at it and say, "Hmmm... we have some inefficiencies here. Bob, if you were to do X before Y it would help out Sally. Would that be OK?" Also, having written everything down it makes it easy for new people to come in and get an idea of the existing culture without having to discover it through trial and error. It allows them to quickly identify areas that they may want to change -- "You know guys, in my last company we did X. What do you think".

As you may have guessed, I have had the misfortune of being one of those "Process Guys". I prefer coding, but sometimes you have to do what you have to do. In my time as a process guy I tried to spend 80% of my time documenting what people were doing, identifying things that had changed over time and asking why the changes were made. I spent 10% measuring various things and making pretty graphs for upper management. I spent 10% of my time giving advice for things that teams might want to try to improve their efficiency. I spent 0% of my time enforcing compliance. If people didn't follow the process, then the process was wrong. I think that the fact that my management preferred me to do this work than write code (despite my protests) says something about the value I returned to the company.

Finally, to answer the question of the OP -- Don't buy a tool. A tool models someone else's vision of reality -- not what your people are actually doing. If you buy a tool off the shelf then you will spend most of your time chasing compliance rather than improving workflow. It really will be a waste of time and money.

Re:Word to the wise

[garyisabusyguy]

Oddly enough, quality is something that people will pay for, if they are aware of how much it affects their enjoyment of the product

It becomes more apparent when a failure in your product has the ability to kill a lot of people

It is also apparent (maybe to a lesser degree) if the low quality product cannot interface with other systems or fails in a short amount of time

How much loss of revenue due to turning out poor products does it take before the bean counters understand the value of change management?

Re:Word to the wise

[UnderCoverPenguin]

Yep. Most process people are truly only concerned with process. ... You can have a good process, but the resulting software is wrong. ... However, most auditors lack context when they evaluate process.

In my experience, the auditors are looking far more at compliance with the process then the process's compliance with business needs.

Re:Word to the wise

[plover]

2. Slavish adherence to anything (incl. processes) MAY be ill advised at times. Typically, standards and/or processes are instituted for a good reason. This does not remove the need to consider whether they are appropriate in all cases. Process does not remove the need to think.

The problem is that slavish adherence to process is seen by many as a guarantee of ass-covering. If things go wrong BECAUSE of the process, the person who followed it blindly still believes that they did no wrong. I've seen important people go out of their way to defend a failing process, just because they play an important role in it, and would be uncomfortable (or perhaps fearful for their jobs or their teams' jobs) if it were to change in any way.

5. I agree that even where "best practice" processes are in place, there are still inordinately many failures. However in my experience (25 years), this is almost ALWAYS down to people/political issues. My own hypothesis is that there is no system, process, methodology or framework that can enforce appropriate behaviour and that cannot be "gamed". The best you can do is try and minimise the opportunities for gaming the system. Irrational agents (at any level) can (and will if given enough time) completely bugger up the best laid plans.

In any modestly sized organization (or larger), you will find a number of people are going to behave this way regardless of the process. Adding more processes (such as checklists, tollgates, audits, etc.) in an attempt to make the system game-proof takes the system in the wrong direction. The best approach I've found is to involve the dissenters in changing the process -- give them the opportunity to improve it. The hope is that they'll see that other people recognize they play a valuable part in the process (removing fear), they'll see the amount of compromise that goes into making some of the weird decisions (getting the bigger picture), and maybe recognize that they can never get 100% of what they want. But if they feel they are a part of it, and are able to address the most egregious violations of sanity they have to deal with, they usually accept the new situation.

Re:Word to the wise

[B4D BE4T]

I agree with everything in the parent post but I think the last paragraph needs some clarification. You do not need to buy a tool for process development, but some tools will be required for the implementation of those processes. For example, almost all software projects will need tools for change tracking and version control processes.

Re:Word to the wise

[wrook]

Yes, you are correct. In fact there are a whole raft of tools that you probably would like to have once you know what you need. I find it interesting that the OP is asking /. about advice for tools having already built all the tools that they need. This tells me that they are likely trying to find a tool to fix process problems. That is backwards. You should fix your process and then notice that you are missing a tool (or a feature in one of your tools). Then it's pretty obvious what you need to get. I suppose it is remotely possible that one of those all-singing, all-dancing process improvement suites implement everything they way you are already doing it, but it seems unlikely. And there is no possible way that anyone on /. would know which one to recommend.

Re:Word to the wise

[acooks]

Worse yet, let's say instead of writing a PC app, you are writing software that directly impacts the safety of operators or other people (nuclear reactor, thermostat, aerospace, automotive, medical equipment, etc, etc, etc). It is unlikely that while "faking your way through your audits" you are going to catch those little bugs that cost people their lives. If you don't want to believe me, look up Therac-25, the USS Thresher, Black Monday, various Airbus crashes, and any of the dozens of other incidents that are a direct result of the sort of mentality you are promoting here.

Are you saying that these projects didn't have the necessary processes in place? I thought these kinds of incidents happened despite all the required processes that were in place. In fact, I believe that these accidents happened (and will continue to occur) _because_ people were _relying_ on the processes that were in place. Processes dictated from high above that were not suitable for the task.

In the given examples, it is easy to think up additional processes that could've been used in retrospect. Everyone has something to say about that, but how many people got to read the code to verify that the additional process would've prevented any given problem? How do these processes prevent integer overflow and race conditions?

Let's examine the examples:
Therac-25 - http://en.wikipedia.org/wiki/Therac-25
No mention is made of any of the processes that were in place. The list of "institutional causes" can be addresses through process, but how would that solve the specific software-related "engineering causes", like arithmetic overflow, not like "open-loop controller"?

USS Thresher - http://en.wikipedia.org/wiki/USS_Thresher_(SSN-593)
Disaster occurred _despite_ the processes in place. It seems to me disaster could have been prevented by an experienced officer, rather than the inadequate process of the time: "Jim Henry, fresh from nuclear power school, probably followed standard operating procedures and gave the order to isolate the steam system after the scram, even though Thresher was at or slightly below her maximum depth and was taking on water. Once closed, the large steam system isolation valves could not be reopened quickly. Reflecting on the situation in later life, McCoole was sure he would have delayed shutting the valves, thus allowing the ship to "answer bells" and drive herself to the surface, despite the flooding in the engineering spaces. Admiral Rickover later changed the procedure, allowing steam to be withdrawn from the secondary system in limited quantities for several minutes following a scram."

Black Monday - http://en.wikipedia.org/wiki/Black_Monday_(1987)
I don't see how this is related to software engineering. The business requirement of "portfolio insurance" implemented in software that will always cause this kind of crash - see May 6, 2010 flash crash.

Perhaps the GP could've expressed his sentiment differently: Take responsibility. Concentrate on building the piece that you are responsible for as well as you can. Understand how it affects other parts of the system as best as you can. Let the people who care about processes and CYA techniques worry about processes and CYA techniques, but limit their involvement as much as you can.

My point?
1. The more regulated the environment, the more people will focus on CYA and avoiding responsibility.
2. Your process will never prevent all possible disasters, but it will create a sense of comfort that prevents people from worrying about it.

Re:Word to the wise

[Alcanazar]

If his team is contracted to a larger organization, they may have people who can help. Some government organizations have independent support groups and research centers that can review your project and give advice. For instance, NASA has an IV&V group in West Virginia.

Re:Word to the wise

[gbjbaanb]

In my experience this *never* happens. Even for the little project with a 1-page document describing what the customer wants, I have scratched my head, phoned the business analyst at the customer, asked "wha?" and received a different requirement from that written down. Fortunately, in that case, we had an agile enough methodology that I could deliver what they actually wanted.

Now, for the 3,000 requirement document we receive for another project, there was no way in hell that every requirement would mesh perfectly with every other one. There were inconsistencies, duplicates that slightly varied. 100s of clarification requests later, we found the clarifications often conflicted with other requirements in the original document. And then, we received the 500 requirement change-notification document and we had to start all over again. Oh, I forgot the bit where we did user workshops and found the requirements didn't match what the user needed. And in the end we nearly got sued anyway because we didn't deliver to spec - even though that spec contradicted itself too often.

All in all, formal processes for requirements management are severely lacking in technical business analysis today. To the point where I know that its pointless trying to work within those frameworks. There are better methods that involve achieving the goal rather than ticking the boxes. Unfortunately, that requires dedication and full involvement (and some competency) from the people involved. The kind of projects that use full-blown requirements aren't the ones who can deliver that kind of staff.

It depends on the platform

[BadAnalogyGuy]

You really need to nail down exactly which platform you're talking about because the choices will differ depending on the platform.

For example, on Windows, you have either Notepad or WordPad. Now, I like Notepad because of its simplicity, but other people like their proportional fonts and formatting crutches. I guess you could splurge and go for something like Microsoft Word, but that's really overkill for most specification purposes.

Now if you're on Linux, you have a ton of choices. The easiest is Pine. If you've used Notepad, you'll most likely be able to pick up Pine pretty quickly. On the other hand, if you like needless complexity, vi might be the specification tool for you. You can do stuff like search based on regexes and copy and paste whole lines of text at once. If you're looking for something more fully featured, a lot of Linux platforms come with Emacs. I think it is a lot like Microsoft Word in that it has too many features that are simply unnecessary, but a lot of people like it.

Of course you're not using a Mac since these are not really programmers' tools. But if you are, I know there is a way to dual boot your system into Windows and get the full power of Windows without having to buy a separate PC. That's a pretty good deal.

When it comes to specifications, completeness, detail, accuracy, and readability are the most important things. Notepad and Pine are excellent tools to help you pound out good specs.

Re:It depends on the platform

[Anonymous Coward]

Yes. Of course real programmers use Notepad++.

Re:It depends on the platform

[onionman]

Are you fucking kidding?

Asking that of BadAnalogyGuy is like asking a car if it meant to be on asphalt.

Re:It depends on the platform

[BadAnalogyGuy]

I'm laughing.

at you.

Yay process

[The End Of Days]

A dedication to process is a substitute for thinking. When the processes become more important than the product, quit. That's the only advice I can give you. I'm aware that's not what you asked, and I'm almost certain you won't listen, but really, that's what I have to say.

Re:Yay process

[hguorbray]

A dedication to process is a substitute for thinking....

It sounds as though you have spent most of your time schlepping through cmmi process level 1 -albeit as a heroic individual (from cmmi link in tfa):

1. Initial (chaotic, ad hoc, individual heroics) - the starting point for use of a new process.

That's fine for lone developer projects or backscratch projects on sourceforge, but that's not the way to keep any organization with more than half a dozen programmers functional.

No matter how good a thinker or programmer you are, there will probably come a time when you are not there or not available and without process (and it's partner, documentation) the ordered development of the code you were working on will be very difficult for those who follow.

I've been dealing with that mindset at my current company -they have a 20-year history of seat-of-the-pants programming with little documentation and most of the important details of the various server products in people's heads -some of whom are not that far off from retirement....

and it has been one of my tasks to try to get all of the processes and products documented so that we can pass off the work when it is time -not to mention satisfying SoX compliance and pending ITIL process implementation.

But I still can't get some of the developers to help me document their products/processes because they are constantly running around duct taping some of our ancient, barely understood core products -because at the time no one cared about process and/or documentation. Most of them still don't today, sadly.

-I'm just sayin'

Re:Yay process

[Roland Piquepaille]

A dedication to process is a substitute for thinking.

If I didn't work as a QA engineer for a huge ISO-9001 company that absolutely looooves paperwork and red tape, I would print your sentence in huge letters and tack it above my desk at work. This is so true.

The problem with processes is, you need them to interface with customers that require it. Otherwise you miss contracts and opportunities. Unfortunately, QA of the kind I do (and the kind the OP seems to want) is a surefire way of turning a nimble, reactive, cheap small company into a stuffed up, slow, expensive and impossibly non-competitive one.

I hope the OP has a good reason to want more of that shit for his small company, because otherwise he'll be well on his way to hiring a lot of overpaid people who spend their days writing QA documents, norms, purchase specs, acceptance specs, procedures, test reports, waste kajillion reams of paper every day printing all that shit, travel all over the country to attend meetings with others of their ilk to discuss more forms, and generally waste everybody's time and money. I should know, that's what I do for a living...

Re:Yay process

[CosmeticLobotamy]

Good advice for capable people. For the 90% that are barely sentient drones that thank God when nobody notices that they just shit out some code that sort of works if you don't poke at it, those processes are critical. They take people that should be dunking french fries in oil and bring them to the "good enough for a Chevy" programming level required for customers that made you agree to do the work at 30% of what it should actually cost to do it right.

Agile

[delibes]

7" x 5" index cards, a marker pen, and lots of conversations between the people who'll really create the software and the people who'll really use it. Everyone in between can be ignored. All that other stuff you think is important... it's ceremony and job creation. Also, read the end of The Dilbert Principle - if you're one level removed from your company's core business (creating a policy and writing code rather that writing code, talking about customers rather than talking to customers, quality teams, process teams etc etc), then it's not worth doing.

Re:Agile

[WindowsTroll]

Agile is not sufficient for regulated industries.

For example, each commercial aircraft in the US has their own set of engineering designs specifically for that aircraft. Every single nut, bolt, and rivet is documented and signed off my multiple engineers - materials, electrical, mechanical, stress, etc. In the event of a plane crash, the FAA swoops in, grabs up all the pieces and reconstructs everything to determine the cause of the crash and they review all engineering drawings and documents. If the cause of the crash was due to the design of the aircraft - a lot of engineers are going to lose their license to practice engineering. Now, think about the "auto-pilot" software or other control software on the plane. If the plane crashes, do you think that the FAA will accept index cards as an acceptable substitute for documented design and specification?

How about the anti-lock breaks and traction control software in your car?

Or the software that is used to control medical equipment? I suppose you are not familiar with the programming mistake in the software of a nuclear medicine machine that exposed a patient to 20,000 times the expected dose of radiation and killed them.

Re:Agile

[wrook]

Index cards alone may not be. But each story should be accompanied by acceptance tests. I would hazard a guess that a short description of what you are building followed by a rigorous set of tests that show whether or not the functionality is acceptable would be exactly what they are looking for. If the tests are automated and you can show whether or not the production version of the software passes the acceptance tests, I think you would be much better off than a standard requirements document.

Wouldn't it be great if when a medical device kills a bunch of patients, somebody could go have a look at the acceptance tests, decide if they were rigorous enough and if not remove the license of programmers/PGMs to build medical software (wouldn't it be great if there *was* such a license)?

None

[defaria]

There are two ways to work - one is to plan methodically each and every aspect of what you intend to do for months then spend the 2 weeks it takes to actually do it and the other is to just asking do in 2 weeks and move onward. You seem to be of the first camp...

The best

[Anonymous Coward]

We use Lifesuck Greymaker (tm), the best solution for introducing bureaucratic process into your unbridled creativity.

CMMI - religion masquerading as whatever it is

[dread]

Listen. I've spent far too many of my working years dealing with companies that have caught religion of some sort. It doesn't matter which one it is, be it ISO, CMMI, Six Sigma or some virulent form of agile (yes SCRUM people, I'm talking to you); its a religion. Instead of focusing on the business and developing sound processes that fit the business model and the company culture these companies put in place this huge infrastructure hoping that this will make them automatically successful.

It doesn't.

It does kill whatever passion there is though. Yes that goes for agile too but in other parts of the company than the one you might be sitting in.

These days I have a good rule that works - when a company tries to sell me services based on being CMMI level 5 I tell them to far, far away and preferably perform some acts that are illegal in several states. After having dealt with a couple of them I have realized that the only genuine thing generated is a huge paper trail and innovation is dead or dying.

As to your question - I don't know and I don't care. I can only make the friendly suggestion that you look for work in a place that doesn't focus on religious adherence to principles defined elsewhere. I promise you that it'll be more fun, challenging and ultimately interesting.

Fred Brooks

[Beryllium Sphere(tm)]

As usual, The Mythical Man-Month cuts to the heart of the matter.

Brooks points out that there is always a specification, unless the project is so inchoate that it's doomed. It may be exclusively in the head of one person in the team, it may be in conflicting versions in the heads of multiple people, but it's there nonetheless.

Some specifications are better than others. A consistent specification is better than a self-contradictory one. A testable specification is better than a vague one. A specification where you can map items to their corresponding tests helps with getting all the bases covered without wasted effort.

If you're free to do so, call the revision control system that holds your tests the "specification tool".

get away from CMM if you can

[Vo1t]

Let me recommend a book : "Lean Software Strategies: Proven Techniques for Managers and Developers". It containes throrough analysis of craft, mass and lean production strategies and their reflections in software (CMM being on the mass side = already obsolete approach). If you can't abandon CMM because of market conditions, think about embracing CMM with as much lean as possible as Peter Middleton describes, and find auditors who would understand and allow you advance on CMM scale without sacrificing productivity and adding waste to your process. In terms of tools, good issue tracking system with customizable workflows is what I recommend.