Tools To Automate Checking of Software Design

heck writes "Scientific American describes some of the work to develop tools for examining the design of software for logical inconsistencies. The article is by one of the developers of Alloy, but the article does reference other tools (open and closed source) in development. The author admits that widespread usage of the tools are years away, but it is interesting reading the approach they are taking regarding validation of design."

Comments

too hard.

[yagu]

Back in the mid-80s I attended a seminar in Atlanta, it was about automated software engineering... and tools and workbenches that would take as input specifications and design parameters and would crank out entire suites of software/applications. (Heck, there was even a new acronym for it, can't remember what it was, but it was a hot, hot, hot button for a few years.) We were pretty much warned our careers were over, automation was here to generate what we as professionals had studied years to create.

It never happened. It never came close to happening. We are as far away today or further from tools that can generate applications transcendentally.

I was skeptical then, I'm skeptical now. Tools like the ones described are useful, but they're not foolproof, and they hardly supplant the intuition and "art" that is programming.

At best tools are an adjunct to the software development process, not to be a replacement for common sense testing and design and code walkthroughs. I could construct many scenarios that logically would be consistent but have no relationship to the desired end of the application, i.e., a logic consistency tool would not detect a problem. Any poorly designed system with these "new" tools applied will merely be rigorously poor systems.

As for the prime example (in the Scientific American article) of the Denver International Airport baggage handling debacle, I doubt logic analysis tools would have had much impact on the success or failure of that effort. I knew people involved in the project, and "logic consistency" in their software was the least of their problems. (I would have loved to been on a team to design and develop that system -- I think it was a cool concept, and ultimately VERY feasible... )

I did get one benefit from the Atlanta Seminar -- I got a football signed by Fran Tarkenton (he was CEO of one of the startup companies fielding a software generating workbench).

Re:too hard.

[AuMatar]

UML is snakeskin oil in and of itself. Yes, its occasionally useful to draw a diagram. Yes, a simple common syntax for those can be good. But thats about all its useful for- designing in UML is a joke (it just doesn't, and never can, portray all the information that text can) and automatic code generation is even more of a joke (at absolute best, it saves you 5 minutesof typing boilerplate class skeletons).

Re:too hard.

[Anonymous Coward]

I agree completely with the code-generation ... it's useless.

When I was young, stupid and without real-world experience, I also thought UML was crap. However, once you realize that the point of UML is not helping to design, but helping to communicate the design to other developers, it becomes useful (not ground-breaking, not amazing, not neccessary, just useful).

It's like that: when you design a piece of software alone, you can use whatever system you want to model it (keep it all in your memory, write class-structure as pseudo-code etc.). If you need to communicate your design to others, it's good to have a system with defined semantics to formulate the design (visualization is not even the issue here), and one such system is UML; surely not the best, but one that a sufficient number of developers (software engineers, not programmers) are trained to understand.

Lacking such a system of communication, you would first have to explain the meta-model (explain how the description of your design is to be understood) instead of just explaining the design-model itself.

Re:too hard.

[deuterium]

We were pretty much warned our careers were over, automation was here to generate what we as professionals had studied years to create.

I vaguely recall that fad as well. A lot of executives were jazzed about the idea, as they seemed to assume that software was rote and procedural anyway. They viewed programmers as simple translators, not realizing that program code doesn't just facilitate the resulting software, but was the software. Regardless of how many tools you devise to commoditize the basic functions of software, the effort required to actually make it is proportional to the complexity of describing its total functionality. You'll just end up having to specify all of the procedures involved, anyway, whether you do it in code or through some meta-coding tool.
By the same token, code checkers can't know what your intentions are for every variable and class relationship. They can tell you if you generate invalid or null variables, or if a function is orphaned, stuff that is strictly boolean. Beyond mistakes like that, you'll have to tell the checker in explicit manners what to look for, negating the benefit of the tool.
We have time saving techniques already. They're called code libraries, design patterns, and error handling.

Re:too hard.

[deathy_epl ccs]

at absolute best, it saves you 5 minutes of typing boilerplate class skeletons

The code generation from UML is only supposed to be the class skeletons, and I've got to ask... have you never written an application with more than a handful of classes? The time spent just building the skeletons for some of the applications I've written over the years has taken a helluva lot longer than 5 minutes.

I personally find class diagrams darned useful. I also find use case diagrams useful not because they help the programmer, but because they help to make sure that we correctly understand what the user is asking for.

From reading your post, though, I'm not entirely certain you have actually bothered to learn UML before you started to slam it. You say:

it just doesn't, and never can, portray all the information that text can

Anybody who knows UML knows that an incredible amount of the work is done in text. Sure, you can create state and activity diagrams for your use cases... or you can just attach textual documentation that is typically easier to create and much smaller than state or activity diagrams. This is what you are typically ENCOURAGED to do for anything but the most convoluted process... and when you have a very convoluted process, even text fails.

There are times that it is very useful to have pictures with circles and arrows and a paragraph on the back of each one.

UML is not the best solution for every development project, for very large projects with lots of developers involved, it can certainly make life easier.

I see one problem with this.

[Dragoonmac]

From what I'm reading it looks like these programs preform all sorts of different executions, thats great and all, but they probably dont behve like real people do. The average user isn't going to create a file and then (in the middle of that) start running the delete file interface. Also I doubt these tests include other common user issues (like clicking the same function over and over again if it doesn't respond immedietly). Maybe I'm just not understanding what these do... but if I'm even half right, real world tests are the way to go.

software snake oil

[penguin-collective]

None of those tools have ever been demonstrated to be cost-effective means of making software more dependable. It's an article of faith that adding a complex notation and another complex set of tools to the development process makes the product any better.

These tools have very limited applicability

[JurassicPizza]

These types of algorithmic testing tools are useful for small, truly critical functionality that has to work perfectly. It's not cost effective to try to model typical complex software in a manner that supports testing as described in the article. Most programming is not about designing the next great single algorithm, it's about integrating data, interacting with users, and providing all the logic to handle the myriad special cases that make up user requirements. Rarely is such a testing tool going to cover all the possibilities without a gargantuan effort to model the software -- which effort will most likely not be able to keep up with the actual development anyway. These tools won't be widely accepted until they can automatically read source code and create a software's model without programmer input.

Crutch for bad developers

[cryptomancer]

Sounds like some producer wants a magic-bullet program to replace some bad-performing designer. Even in the case of a 'useful' tool to apply to projects, this is likely to become an excuse for when an inconsistency is found later on by QA- "the program said it was good!"

It's not going to find everything, let alone fix it. See Turing: the halting problem.

Good Software Design

[KidSock]

A good design correctly models the concept of what it is you're trying to achive with the program. Ultimately this means the programming interfaces (APIs) for each concept are correct [1]. Don't design interfaces around procedures. Don't design interfaces around the physical world. Design to *concepts* and *ideas*. This is superior because you will never discover at a later time that the code is fundamentally flawed and needs to be totally re-written. If the interface correctly models the concept, by definition, it CANNOT be wrong. If it is wrong then you simply didn't understand the concept well enough or you failed to translate that concept into a suitable interface and you just need to think more and type less. If you do get things right you'll find that major peices dovetail together perfectly [2]. The implementation can be wrong and may need to be re-written but if the interface correctly represents the concept the re-write will be localized to one library or part of a library. That is a much more straight forward matter than using a bad design and finding half way through a project that the required changes transcend the whole system.

And thus you cannot validate a design because that would require representing a concept and determining if an interface suitably models it. That is HARD. If that were possible you would effectively have a thinking, rationalizing, brain (Artifical Intelligence) in which case you wouldn't be dorking around with validating programs, you would be dynamically generating them.

[1] Frequently people advocate that interfaces are "well defined". That just means there are no holes in the logic of it's use. Personally I think a well defined interface is useless if it does not correctly model a concept. You can always go back and fill in the holes later.
[2] Although this is also when you discover that you didn't get the concept right and need to adjust the interface (hopefully not by much)

Re:Good Software Design

[theMAGE]

A well defined interface means that if you build 1 million holes in a plank and I deliver 1 million pegs, when they "meet" they fit.

A square hole and a round hole, one 1 inch in diameter and one 1 foot wide, all of them model the concept, but are utterly useless.

You can always go back and fill in the holes later.

No you cannot, otherwise we would all use Dvorak keyboards, not this stupid Qwerty. And we would have had HDTV 15 years ago. And...

Re:Finding Nemo Architecture

[140Mandak262Jamuna]

It is six years since Finding Nemo was released? Looks like it was yesterday I saw the movie. [quick googling "finding nemo year release"] 2003. How long would it have taken the "tool" to find this contradiction in your posting?

Re:Finding Nemo Architecture

[russellh]

Yep.

If we get enough small components they can be combined into any piece of software. Eventually we wouldn't need any more components and thus no more software developers.

the key is that phrase "can be combined" although my second pick is "eventually". your finding nemo system will have to be self-organizing because is too vast to have organization imposed from without. You already have that kind of system today anyway. So if you have a self organizing system, two questions are a) how does it arise and b) how do you get it to do what you want. The real-world analogy for this way of thinking is the garden and the gardener. you're thinking wouldn't it be real cool if we had "cells" which you could stack on top of one another and you could have any kind of plant you want, whereas what you want is to be the gardener instead and let the software plants grow by themselves (think of the order of magnitude in scale between a group of cells and the entire garden). And so I think your "eventually" is a really, really, really long time.

Thank you!

[Anonymous Coward]

Someone who doesn't see it as an all or nothing proposition.
Tools that make sure programs are self-consistent are good!
What's the point of having testing and real world trials if you're programming doesn't even agree with itself?