Mapping/Understanding System Complexity?

thesandbender asks: "I've recently inherited a project to 'simplify' the application environment for a company that has 1600+ service offerings (many of these are product 'foobar' that has options (like 'Alpha', 'Bravo', 'Charlie', and so forth) available. I am trying to map out the applications' dependencies from a technological and a business standpoint. I would like to designate a group of applications as depending on concepts, technologies (like SAN, DB2 and AIX), specific customers (like 'Bravo' and 'Charlie') and legacy applications. Basically, I want to define any number of arbitrary dependencies and then be able to map them out in a graphical format. With those maps I can show the business oriented staff how removing one application will affect other applications, and I can show the technically oriented staff how removing one system will affect other systems or applications. Has anyone in the Slashdot community run across such a tool? If you haven't, have you run across the need for such a tool? What would you want from it so that I can fashion a usable tool that addresses everyone's needs and not just my own?" "The most appropriate tool-sets I've found to date are 'mind mapping' or 'concept mapping' tools. All of the tools I've found so far only allow me to create any number of ideas or concepts and don't allow for arbitrary, searchable and/or mappable attributes (e.g. Application 'foo' maps to attributes 'SAN', 'Java', 'Solaris' and 'Buy-Side') that would allow me to create hard and soft groupings that were based on defined attributes (e.g. I could ask for a cloud of all objects that share a specific technical attribute, and another cloud of objects that share a specific business attribute)."

Comments

Although not designed for that problem

[jd]

I'd use a database modelling tool, like ERWin or Dezign for Databases. (Yes, they're both commercial - I've not seen a single good entity-relationship modelling tool that is also Open Source, although it's an obvious tool to write.)

These tools map attributes in records to other attributes in other records. They're designed to then turn these maps into SQL code, but that part isn't important here. What is important is that you can create a full relationship mapping between entities. If you then treat the direction of the relationship as showing the dependency, you can map all the dependencies in the system.

Managers like diagrams to be of a format that are familiar to them, so anything that is "better" from a technical standpoint but "less familiar" to managers from an experience standpoint is, in fact, not as good of a solution.

Graphviz

[danpat]

You can probably draw the picture you want with GraphViz, found here http://www.graphviz.org/

To use it, you create a text file that defines all your dependencies, it'll look something like this:

    digraph thingies {
        "app1" -> "SAN";
        "app1" -> "Java";
        "client1" -> "app1"; ...
    }

You can then go on to group things together so they show up in meaningful locations on the diagram,
associate pictures with certain nodes, put labels on things, make things in colour, etc.

GraphViz takes care of the laying out parts (where best to put nodes and edges). Sometimes it takes
a while to define everything in format that gets draw neatly, the results are often impressive and
very useful.

On coming to a new job, I've used it to draw all the dependencies between a collection of a couple
of hundred SQL stored procedures in our database. The locals were horrified to have what they all
knew in their gut depicted to them on 35 A3 sheets of paper on a wall :-) It was quite useful for
identifying things recursive calls ("Oh, *thats* why that proc sometimes never ends...").

Re:"Complexity kills"

[QuantumFTL]

The problem with the concept of complexity based on program length, is that program length is highly dependent on the system of encoding used to represent the program. Short programs in functional languages can be quite long in imperative languages, and lamdba calculus functions for even simple things are sometimes so long as to be impossible to represent in a published paper.

I've yet to see any "cannnonical" representation that can be used for this purpose. "Kolmogorov complexity" is not useful for these things for that reason. A more interesting metric, often used in software engineering, is cyclomatic complexity.