Slashdot Mirror
Explaining Complexity in Software Development?
BMazurek asks: "I'm stumped by how to explain software development complexity (not theoretical big-O notation, that's easy) to non-developers. When it comes to people who aren't in the code, my explanations fall flat. It's not that the people I'm talking to are stupid, they're quite honestly people at the top of their respective (non-tech) fields. How do -you- explain software development complexity to non-developers? What analogies do you use?"
"I often try the famous Fred Brooks, Jr. quote (seldom to much success):
'Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.'
pictures help, metaphores help, madlibs help
by madlibs, i mean things like "think of [concept] as like a [cute noun (bunnies, kittens, etc.)]"... draw funny pictures, and connect them up with arrows... you can't over simplify enough, and the more fun you make it, the easier time the audience will have following you
remember that 9 times out of 10 you aren't explaining it to these people because they need to understand, you're explaining it so they have confidence that you know what you're doing and that the outcome will reflect well on themselves
What you want to convey, to the non software initiated, is why software is difficult, why it is complex, and why it is hard.
I would put it this way; software is really "soft", being only instructions a computer reads and interprets. A physical analogy is perhaps clay, something equally analogously soft, malleable, and easy to work with. Now imagine another physical analogy; take that clay, and then try to build a car out of it. A working car. Ignore the physical impossibilities, like combusion temperatures, but make the point that each component first has to be designed then baked, and that each component has to be integrated with each other component.
So that is what software is like. They may ask, "Why?", and here are a few reasons:
1) Software is easily copied, not easily designed. The hard part is making the first copy, as in the car example. Once you get one product finished, you can copy quite easily, but actually designing and building is not necessarily easy.
2) Software is REALLY malleable. You can continuously change the design as you are working, not unlike building an entire car out of soft clay.
3) Because software is so malleable, each time you discover a new problem you discover a new solution. In other words, the second car you make will be totally different than the first car because instead you will be going from a car to a boat or a train or a helicopter.
4) Also because software is malleable, every "customer" with different requirements brings in new design directions. Imagine if every person working on the car, viewing the car, and interacting the car wants a totally different car? You aren't designing one car, now, you are designing 17 or 18 cars; trucks, vans, compacts, sedans, and sports cars. Now take all these vehicles and merge them into a single multipurpose vehicle.
GPL Deconstructed
LOC != complexity.
Or you could compare it to a DVD of "Dumb and Dumber", which at 4.7G is 188 times bigger than your 25M quarter-million line code project.
In short, bulk? Don't go there.
Make a rough analogy. Every line of code is a part of a machine. It interconnects to other parts.
-A mouse trap has maybe 4 or 5 parts.
-A transistor radio has perhaps 100 parts.
-A 35 MM camera has maybe 200 parts.
-What does a car have? 10,000 - 50,000 parts?
-An airplane? 200,000 parts?
-A smallish computer program has between 10,000 and 50,000 lines of code.
-Something like an OS kernel might be between a million and 50,000,000 lines of code
-A typical commercial billing application might have 300,000 lines of code.
-A nice set of apps to run telephony switches might have 10,000,000 lines of code.
-Even your stupidest sort of hackish utility has 100 lines of shell script.
Programs are gigantic machines - you just can't see them or weigh them.
Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
Software development is hard because at its core it's as much about design and invention as it is about implementation. Implementation, in the general sense of the word, is often fairly easy. You have a plan, possibly a well known and understood plan, you can track progress, things are more predictable. With design and invention, you are often in unexplored territory so it's hard to tell where your next steps should take you and it's hard to track the progress you've made to date. What makes software development even harder, is often that the customers don't actually know what they want nor can they really be expected to know what they want. Problems are often not well understood.
One analogy that I've always liked is to look at software development as a mapping problem. When you start, you're pretty much dropped in the middle of nowhere without much understanding of the surrounding terrain. You then have to go exploring the area around, get an understanding of the feature space. Maybe the best approach is to climb a mountain or a hill, implement or design a major well understood component, and see what you can see about the area around you from that peak. You make little scribbles and notes on your makeshift map outlining what you've seen from the top of your mountain. Later, you climb a different mountain and all of a suddent everything looks completely different from the new perspective and you realize large parts of your old scribbled map are wrong. Maybe walking in between mountains one day you discover a large impassable ravine that was masked from above by foliage. Your in unexplored territory and there can be lots of surprises and setbacks.
Only, I think this kind of mapping analogy really falls short because it only takes into account a single perspective, that of the developer, and it assumes there is some well defined terrain that just needs to be discovered. In reality, the terrain being explored is much more mercurial. Much of it is visible only through the inconsistant and confused descriptions of customers or other developers. It's quite possible that the mountain you climbed yesterday, and the things you saw from the top, will not exist tomorrow.
Anyways, that's the best I've got at the moment. It probably doesn't make much sense, but then, what does?
I like your cooking analogy, but there's one thing missing:
One thing I can't stress enough to people is that programming is all DESIGN/ARCHITECTURE. The "do" part is essentially instantaneous. You figure out how to do something and write it down, in every minute detail (in code) and it's basically done. Then you compile it (or not even for some things). This process is nearly instantaneous (perhaps minutes but never days for almost any project, and never something you need to do yourself...)
To expand your cooking analogy, it would be like you work really hard on these recipies, but whenever you want you can "zap" create a new set of dishes. (Except that then you have to taste them all again to make sure your changes are ok..)
This is fundamentally different from the construction of anything else ever in my opinion. Code is akin to a recipe or blueprint. The "compile" part of programming, which takes zero work on the part of the programmer, replaces what is one of the longest and most costly parts of developing anything else.
Furthermore, it replaces the only part that you can reliably predict the duration of - making it very, very difficult to predict how long it will take to develop a piece of software.
I think it's important to realize that this makes software much faster to develop - if less predictable - but therefore we develop much more complex things in software. So to borrow from another analogy I saw here - because we don't have to physically build them, the expectation when building a car is that it'll have every feature of any car, boat, bicycle or plane that ever existed.
(The other important ramification is that it never saves developer time to remove an existing feature - and in an agile development process it may take more time to DECIDE whether a feature is important than it would have to simply implement it.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot
if someone building a house does something the same way more than once, it's a good idea. they use what they know works, and apply it over and over again. maybe some variations on a theme, maybe add another story here or a differently angled roof there.
but software is different. if you're doing something the same way more than once, it's generally a warning sign that you could be doing it better somehow. following that to its logical conclusion, a good programmer will never do the exact same thing twice. meaning that, as a good programmer, you're always doing something you've never done before.
that is why programming is hard.
it's like being an automobile manufacturing plant that never produces the same car twice. one day it's a civic hybrid, the next day it's an M1A1 tank, and the day after that you need two vespas, one that runs on gas and one that runs on fryer oil.
I've had some success illustrating the challenges of programming to the more visual or artistic types with this. Then you tell them to multiply that by tens of thousands, at least.
Ask them if they've ever seen a foreign movie with really bad subtitles. Ask them why something that seems so easy is so badly done. Ask them if they've ever had to explain something to a child who's only response is "Why?".
Everything that goes wrong with software is a result of miscommunication. From failed projects, to difficult algorithms, to null pointers. Programming is taking difficult to communicate ideas and concepts of human creation and translating them into the most literal and OCD afflicted of entities -- a computer.
You can talk about system size, deadlines, complexity, etc., but it all really boils down to communication. UML, JavaDoc, unit tests, forms, screens, blah blah blah. "Exactly what I asked for and not what I wanted."
Communication is difficult enough between two intelligent adults sharing the same goals. Now imagine trying to explain something to a machine.
I know exactly what you mean. My customers come to me and say, "I'm hungry and I want a healthy meal! I like ice cream, cookies and liverwurst. To eat it, I've brought my favorite chopsticks and a slotted spoon. Oh, and can I get that to go?" And after I make it, they point out how they're allergic to milk products.
Ok, it's complex enough. But, the examples I'm seeing in these comments, while they will impress people, are just a bit extreme.
For instance, the analogies around building a car. No, software is all about design. When building a car, you have to deal with real, physical possibilities and impossibilities. In software, if your design is good, the components are flawless.
Also, abstraction helps. A lot. The example of 25 megs of source vs a 100k Charles Dickins novel isn't relevant unless I actually have to work on all 25 megs -- that, and english can often be much more compact. I often make a point of showing people Bash one-liners as an example of good abstraction -- probably millions of lines of code went into what I just did, but I only needed one line of code to make all that other code work together.
When it comes to explaining program complexity, it really depends what misconception the person has. The most common one is that computers are capable of some form of human-like thought -- the form varying from person to person. However, it's usually easier to convince people that something isn't possible than that something is possible, because people are used to thinking of computers as huge, complex, rigid, and buggy.
Honestly, though, the best way to teach someone why programming is hard is to teach them to program. Second-best way is to teach them philosophy. Third-best is to teach them Calculus.
Don't thank God, thank a doctor!
With this, I (almost) always get "If it's red, you stop. Green, you go." and which point I interrupt "And yellow (or orange for you Aussies out there)." It's about here that they start to think about it. Then I ask "What if it's raining?" quickly followed by "What if it's the first rain of the year?" and shortly thereafter followed by "What if you see someone else running the light in front of you?" I then explain that if I were writing a program to do something as "simple" as deciding if it's safe to go thru a traffic light, I'd have to think of ALL of these issues, plus everything else that could ever possibility happen while traversing an intersection. If I manage to miss something, it becomes a "bug" in that program.
In my experience, people pretty much "get it" with this analogy. Course, YMMV...
"1984" was ment to be a warning, not a guidebook. You hear that Kim Jong-il!? BushCo?!
What your brother (a lawyer) really meant was "please don't tell anybody who might think that's a good idea because then my expensive degree will be worthless."
Actually what he meant was "Law can't be pinned down to precise language out of necessity. Specifying exact rules to cover every particular case is an impossible task. You can't foresee every possibility and even if you could there are too many factors to account for. It's an infinitely larger search space than chess, which is itself completely unmanageable. Hence law mostly relies on relatively short rules and human intelligence to apply them correctly in each situation." Where 'short' means someone who knows where to look can process the relevant ones in finite time.
I'm a programmer and a law student. Trust me, precise grammars are not an option.
Democracy is two wolves and a sheep voting on lunch.
When i'm discussing a request for implementing a new requirement or changing an existing requirement with the non-technical person making that request, a trick i usually do to show the complexity of apparently simple requests is to right there, present the high-level (eg. design) logical steps on how we would implement that request.
...
Non-technical people will understand the complexity of doing something when they follow you through the sketching of the process of making a solution that does it. For example, if you are asked to change a form so that a free text input field becomes a pull-down with a number of options you can:
- Ask them if the list of options is fixed or if an application administrator can change them
- If it's fixed, then it's easy to do, otherwise:
- Explain that you have to store those options somewhere. Maybe you already have a database, so explain that you have to add storage tables to the database, then add code to the application so that it can load and save that information to and from memory. Also explain that you have to pre-load some sensible default values into the database so that the application works out of the box.
- Since they want that some administrator can change it at runtime, figure out if said administrator is a developer/dba type of person or not. If not, explain that to allow an administrator user to change those values you have to extend your administrator interface, which means adding menu entries and one ore more new pages.
- Also figure out what are the rules (if any) that constrain the values that the administrator can configure for the pull-down. Depending on the complexity of the rules you might want to follow through into how you would do it, for example, if the rules are based on other values in the database which can also be changed by an administrator:
- Explain how you have to add code to retrieve and process the needed values so that you can check that the rules are not being broken, and that you also have to change the administrative interface for those other values so as to make sure that when they are changed or deleted, the values in the pull-down box do not become invalid.
-
The point here is that most non-technical people which actually use computers understand concepts such as memory and storing "data that cannot be lost when the application stops" (persistent data) somewhere other than memory. They can be made to understant the basics of a relational database: "a place to store data which has tables - a bit like excel tables - one table for each kind of information" (not exactly it but close enough) and they can usually follow a logical chain of steps (eg to show the data in the interface we have to get it from somewhere, since it is configurable, we store it in the place where we put the data that can be changed by the program but must be preserved even when the application stops).
Try and stay away of techie words and expressions ("We're gonna have to persist that data to our relational database and that means creating new tables, adding new data objects and changing the Hibernate mappings"). Instead assume non-techies are ignorant but not stupid ("We have to store that data in the database so that we don't loose it when the application stops. This means we have to configure in the database the tables where the data is stored and also have to add support in the program so that it can load that information into memory and use it").
Also, simplify:
- Don't try to be overly exactly - for a non-techie is enough to say you have a "database" no need to say it's a "clustered configuration of a <insert-brand-here> relational database".
- Don't go into deep details - you "configure the place in the database to store the data" no need to say you're "extending the tablespace, creating new tables, adding foreign keys to the related tables, and setting up a couple of indexes to speed up cross-referencing"
- Try and tune your words to the audience - for some people you can say you're gon
You know the annoying kid you knew in college who would repeatedly interrupt every discussion demanding definitions of all the words being used, and who would argue endlessly about flaws in the definitions so that you couldn't really talk about the topic you wanted to? Imagine you're writing a book, and that guy has to personally approve every sentence.
The 10 line function and the Haiku are simpler. Also, more elegant and better.
I disagree with the idea the cramming the most amount of logic into the least amount of lines is elegant. Brian Kernighan said it best, "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
is it the only way software development can be or is it merely the current state of software development, perhaps because we don't yet understand programming well enough?
Software can do many things. It can file my taxes. It can render 3D images. It can send commands to a satellite in outer space. Do any of these activities sound similar to you? I hope not.
It's not that we don't understand "programming" well enough. We understand programming fairly well. However, that's not going to get us far, because we are developing vastly dissimilar applications.
For example, better programming languages let you abstract out more complex and varied patterns in software than worse ones; the result is simpler, more regular programs. As our understanding of how programs operate and are structured improves, we can expect to make them smaller and more regular with such tools.
Yes, and these tools will likely be specific to certain application domains. We'll see more benefit in the future when the domains become clearer and the tools to implement the applications within them become more widely accepted. It will take time, however, for everyone to converge.