Slashdot Mirror
Explaining Complexity in Software Development?
BMazurek asks: "I'm stumped by how to explain software development complexity (not theoretical big-O notation, that's easy) to non-developers. When it comes to people who aren't in the code, my explanations fall flat. It's not that the people I'm talking to are stupid, they're quite honestly people at the top of their respective (non-tech) fields. How do -you- explain software development complexity to non-developers? What analogies do you use?"
"I often try the famous Fred Brooks, Jr. quote (seldom to much success):
'Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.'
Spanning 20+ years in computer programming I've stopped trying to explain what it is like and how it is done. Heck, we can't even clearly explain to peer programmers why vi is better than emacs, XP is better than Linux, Gimp is better than Photoshop (NOTE: these do not necessarily reflect my opinion, FTR, I prefer: vim; Linux; and Photoshop)
I shrug my shoulders and explain it's mostly dull. It's kind of like doing Math homework, except I have to do it every day for my job. It's always about solving and debugging, and people's eyes glaze when I begin poking programming nuance with a ten-foot pole.
Fortunately I find most people don't really care. (Also for those who would "get it", my experience has been they are ones who dabble and experiment in it anyway already.)
(Aside: as for the original poster, congratulations on being able to explain "big O notation". I sometimes suspect my girlfriend of faking it.)
Easiest way I've found- though it's begining to get a bit outdated thanks to bloatware. Charles Dicken's famous novel is about 100k. This makes it easy to estimate source code in terms of number of copies of that novel. The quarter-million lines of code project I'm currently working on takes about 25 MB to store- 25000k, or 250 copies of A Christmas Carol.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
pictures help, metaphores help, madlibs help
by madlibs, i mean things like "think of [concept] as like a [cute noun (bunnies, kittens, etc.)]"... draw funny pictures, and connect them up with arrows... you can't over simplify enough, and the more fun you make it, the easier time the audience will have following you
remember that 9 times out of 10 you aren't explaining it to these people because they need to understand, you're explaining it so they have confidence that you know what you're doing and that the outcome will reflect well on themselves
For the generic question, "What is programming?"...
:P
I say along the lines: "Think of programming like any job, where you have employees and you need them to do things for you to make your business work. You usually give your employees a list of instructions, things to do. They do what you tell them to and then come back to you for more instructions. Computer programming is similar, inside that box under your desk are a bunch of employees. Without a boss, they don't know what to do. So, what I do, is I tell each component inside there what to do. I tell the video card to turn on so you see stuff on your monitor as one example. I tell the main processing unit (think of it as the manager of the office, where as I am the big-cheese boss)) to run the "office" while I am away. I give the main manager, or CPU a big huge list of instructions that say this is how we run the place. The CPU (or manager) then follows my instruction sheet and tells all the other components inside the computer how to act, behave and what to do. We do it this way because it take months to write the instruction sheet out for the CPU, but the CPU can execute that instruction list super fast, much faster than if I were telling each component (e.g. employee) what to do directly...".
I actually just came up with this example right now on the fly... I never use the same example, I always think about what that person does for a living, spare time, hobbey, and I can almost always draw parallels that make them say "Ahhh, I wish someone else would of explained it that way before".
I've even explained how interrupt handlers work in regards to a USB joystick to a Lawyer... He was so happy in the way I explained it to him, he kept asking more and more questions until I told him I have to go
Modesty is one of life's greatest attributes
What you want to convey, to the non software initiated, is why software is difficult, why it is complex, and why it is hard.
I would put it this way; software is really "soft", being only instructions a computer reads and interprets. A physical analogy is perhaps clay, something equally analogously soft, malleable, and easy to work with. Now imagine another physical analogy; take that clay, and then try to build a car out of it. A working car. Ignore the physical impossibilities, like combusion temperatures, but make the point that each component first has to be designed then baked, and that each component has to be integrated with each other component.
So that is what software is like. They may ask, "Why?", and here are a few reasons:
1) Software is easily copied, not easily designed. The hard part is making the first copy, as in the car example. Once you get one product finished, you can copy quite easily, but actually designing and building is not necessarily easy.
2) Software is REALLY malleable. You can continuously change the design as you are working, not unlike building an entire car out of soft clay.
3) Because software is so malleable, each time you discover a new problem you discover a new solution. In other words, the second car you make will be totally different than the first car because instead you will be going from a car to a boat or a train or a helicopter.
4) Also because software is malleable, every "customer" with different requirements brings in new design directions. Imagine if every person working on the car, viewing the car, and interacting the car wants a totally different car? You aren't designing one car, now, you are designing 17 or 18 cars; trucks, vans, compacts, sedans, and sports cars. Now take all these vehicles and merge them into a single multipurpose vehicle.
GPL Deconstructed
As with explaining the complexity of anything, you have to try thinking either in terms that are completely universal, or in terms that the person you're explaining it to will understand from their field (assuming you know enough about their field to make an analogy, of course).
I usually end up rambling on until the listener's eyes glaze over, but I've had some success with demonstrating some relatively simple things with a deck of cards... sort and random algorithms are especially well suited to this type of explanation, for obvious reasons.
Anything that doesn't have an easy analogy in common knowledge, I don't generally worry about explaining beyond some noncommital answer involving a basic description of the task, then asking the listener to think about breaking it down into simple parts, with directions that a six-year old could follow. Generally works, and gives an idea for the complexity of a program, at least sufficient to give somebody who doesn't really need to know everything about the programming side an idea of the work involved.
Imagine having a huge legal contract. Then, add in a huge, convoluted yet intertwined legal system.
Furthermore, imagine that instead of writing this legal mumbo-jumbo in english, you use a language that is extremely strict and math-based.
This would be a fraction of the real complexity (mutiple threads/processes anyone? How about poor documentation?) but it starts the ground work.
To be honest, this way of thinking sounded better when I first started, and I'm kind of disapointed with my end product.
Am I open minded towards open source, or closed minded towards closed source?
When it comes to people who aren't in the code, my explanations fall flat.
Before I changed line of work (I'm not a computer professional anymore thank goodness), I used to explain my work like this:
See, what I do is programming. Programming is like writing a cooking recipe, only slightly more complex but not much more. But it's a very large recipe, and it relies upon many more recipes to make an actual dish (the program). Many cooks write recipes in different languages separately, and all we cooks have to coordinate to prepare the final dish. So we need chief cooks (managers) that call meetings and prepare Gantt charts to do that. Then sometimes a cook writes a recipe that has the wrong combination of ingredients, or that make no sense to describe how to prepare food (bugs), so we need tasters (QA) to tell us busy cooks if the overall result will be pleasing to the restaurant's customers. In the end, you get a huge dish that has some nasty morsels in it, as well as tasty ones. We then have to refine the dish so the nastyness goes away and more goodness goes in (new versions). etc etc...
The cooking recipe analogy has always worked great to explain what I did.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
If you're looking for the perfect 4-course meal at a fancy restaurant, you need to match flavors, textures, and colors from a lengthy menu, weigh the transitions from appetizer to main course(s) to desert, and fit in the perfect glass of wine and the ideally sized, shaped, and sweetened dessert.
Only it's a 100-course meal, the menu is 1000 items long, there's a cellar full of wines (most of them are unpronounceable) and it takes months to find a sequence that people will even TRY to eat all the way through.
And even then, they probably won't be satisfied enough to tip you very well, so you keep making slight tweaks to the courses until someone finally gives you the tip you want. At that point it's good enough, but everyone knows it's never perfect. (And they call you on it, too.)
As soon as someone notices that you're making a decent tip, the owner invariably decides they're going to serve Chinese food instead, so you start over.
Or you could compare it to a DVD of "Dumb and Dumber", which at 4.7G is 188 times bigger than your 25M quarter-million line code project.
In short, bulk? Don't go there.
Easiest way I've found- though it's begining to get a bit outdated thanks to bloatware. Charles Dicken's famous novel is about 100k.
You're way behind my friend: data bloat has grown so much these days it isn't measured in Charles Dickens units anymore, it's measured in tax code units:
- 1 tax code = 25MB
- 1 tax code table of content = 300KB (to measure smaller data units)
And no, I'm not kidding, the complete internal revenue code really is that big.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Make a rough analogy. Every line of code is a part of a machine. It interconnects to other parts.
-A mouse trap has maybe 4 or 5 parts.
-A transistor radio has perhaps 100 parts.
-A 35 MM camera has maybe 200 parts.
-What does a car have? 10,000 - 50,000 parts?
-An airplane? 200,000 parts?
-A smallish computer program has between 10,000 and 50,000 lines of code.
-Something like an OS kernel might be between a million and 50,000,000 lines of code
-A typical commercial billing application might have 300,000 lines of code.
-A nice set of apps to run telephony switches might have 10,000,000 lines of code.
-Even your stupidest sort of hackish utility has 100 lines of shell script.
Programs are gigantic machines - you just can't see them or weigh them.
Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
Software development is hard because at its core it's as much about design and invention as it is about implementation. Implementation, in the general sense of the word, is often fairly easy. You have a plan, possibly a well known and understood plan, you can track progress, things are more predictable. With design and invention, you are often in unexplored territory so it's hard to tell where your next steps should take you and it's hard to track the progress you've made to date. What makes software development even harder, is often that the customers don't actually know what they want nor can they really be expected to know what they want. Problems are often not well understood.
One analogy that I've always liked is to look at software development as a mapping problem. When you start, you're pretty much dropped in the middle of nowhere without much understanding of the surrounding terrain. You then have to go exploring the area around, get an understanding of the feature space. Maybe the best approach is to climb a mountain or a hill, implement or design a major well understood component, and see what you can see about the area around you from that peak. You make little scribbles and notes on your makeshift map outlining what you've seen from the top of your mountain. Later, you climb a different mountain and all of a suddent everything looks completely different from the new perspective and you realize large parts of your old scribbled map are wrong. Maybe walking in between mountains one day you discover a large impassable ravine that was masked from above by foliage. Your in unexplored territory and there can be lots of surprises and setbacks.
Only, I think this kind of mapping analogy really falls short because it only takes into account a single perspective, that of the developer, and it assumes there is some well defined terrain that just needs to be discovered. In reality, the terrain being explored is much more mercurial. Much of it is visible only through the inconsistant and confused descriptions of customers or other developers. It's quite possible that the mountain you climbed yesterday, and the things you saw from the top, will not exist tomorrow.
Anyways, that's the best I've got at the moment. It probably doesn't make much sense, but then, what does?
I like your cooking analogy, but there's one thing missing:
One thing I can't stress enough to people is that programming is all DESIGN/ARCHITECTURE. The "do" part is essentially instantaneous. You figure out how to do something and write it down, in every minute detail (in code) and it's basically done. Then you compile it (or not even for some things). This process is nearly instantaneous (perhaps minutes but never days for almost any project, and never something you need to do yourself...)
To expand your cooking analogy, it would be like you work really hard on these recipies, but whenever you want you can "zap" create a new set of dishes. (Except that then you have to taste them all again to make sure your changes are ok..)
This is fundamentally different from the construction of anything else ever in my opinion. Code is akin to a recipe or blueprint. The "compile" part of programming, which takes zero work on the part of the programmer, replaces what is one of the longest and most costly parts of developing anything else.
Furthermore, it replaces the only part that you can reliably predict the duration of - making it very, very difficult to predict how long it will take to develop a piece of software.
I think it's important to realize that this makes software much faster to develop - if less predictable - but therefore we develop much more complex things in software. So to borrow from another analogy I saw here - because we don't have to physically build them, the expectation when building a car is that it'll have every feature of any car, boat, bicycle or plane that ever existed.
(The other important ramification is that it never saves developer time to remove an existing feature - and in an agile development process it may take more time to DECIDE whether a feature is important than it would have to simply implement it.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot
if someone building a house does something the same way more than once, it's a good idea. they use what they know works, and apply it over and over again. maybe some variations on a theme, maybe add another story here or a differently angled roof there.
but software is different. if you're doing something the same way more than once, it's generally a warning sign that you could be doing it better somehow. following that to its logical conclusion, a good programmer will never do the exact same thing twice. meaning that, as a good programmer, you're always doing something you've never done before.
that is why programming is hard.
it's like being an automobile manufacturing plant that never produces the same car twice. one day it's a civic hybrid, the next day it's an M1A1 tank, and the day after that you need two vespas, one that runs on gas and one that runs on fryer oil.
Because it interfaces the most complex product of evolution(people) with the most complex creations of those people(computers and software). Computers are utterly rigid in what they do, and humans are nearly infinitely variable. Dealing with the differences, with a comparitivly small amount of experience, is hard.
Some software is pretty simple, because it solves simple problems. Is it any wonder that it often takes a complicated system to solve a complex problem?
Nerd rage is the funniest rage.
I've had some success illustrating the challenges of programming to the more visual or artistic types with this. Then you tell them to multiply that by tens of thousands, at least.
emacs would make a *great* operating system, if only it came with a decent text editor.
(I'm sure you've all heard that one before)
"The main processing unit in a computer has Millions of transistors, which are like little switches, turning on and off. These switches flip automatically Billions of times a second. Every second Quadrillions of switches flip, and if a single one goes wrong the computer 'crashes'. That's not even counting the Memory, or the Graphics card, or all the extra devices from Hard Drives to Tempurature sensors that are hooked in. All of these switches are controlled by programs that were written by the combined efforts of thousands of people, most of whom have never met each other. I'm continually amazed every second that a modern computer keeps running."
Ok, it's complex enough. But, the examples I'm seeing in these comments, while they will impress people, are just a bit extreme.
For instance, the analogies around building a car. No, software is all about design. When building a car, you have to deal with real, physical possibilities and impossibilities. In software, if your design is good, the components are flawless.
Also, abstraction helps. A lot. The example of 25 megs of source vs a 100k Charles Dickins novel isn't relevant unless I actually have to work on all 25 megs -- that, and english can often be much more compact. I often make a point of showing people Bash one-liners as an example of good abstraction -- probably millions of lines of code went into what I just did, but I only needed one line of code to make all that other code work together.
When it comes to explaining program complexity, it really depends what misconception the person has. The most common one is that computers are capable of some form of human-like thought -- the form varying from person to person. However, it's usually easier to convince people that something isn't possible than that something is possible, because people are used to thinking of computers as huge, complex, rigid, and buggy.
Honestly, though, the best way to teach someone why programming is hard is to teach them to program. Second-best way is to teach them philosophy. Third-best is to teach them Calculus.
Don't thank God, thank a doctor!
If I've got 5-10 minutes, I use a simple exercise of getting them to sketch out a program for a humanoid robot to set the table, i.e. to carry cutlery and food from the kitchen to the dining table. Before they begin, they consider it a trivial task. After about 5-10 minutes, they accept that even this "trivial" task is close to impossible. Here's an example: First, you have to get the thing to walk from the kitchen to the table. So you have to teach it to walk. Then you have to teach it to avoid obstacles. I tell them that their robot has just crushed the cat (it's not a chair or table, which was their understanding of an obstacle). So they modify it to avoid small furry objects... and kill the dog, which is large and furry. So they modify it again, and find that it's frozen in front of a throw rug, which is small and furry. So they modify it again, and find that it's been halted by a sleeping cat. Then you throw in exception conditions, e.g. a fire, or even just the phone ringing so the robot has to clear the way for someone to get the phone... it's fairly easy to demonstrate that even the apparently most trivial task is in fact incredibly complex once you have take all the different conditions into account, and depending on how much time you have (and how long they take to convince) you can keep throwing hurdles at them almost indefinitely.
With this, I (almost) always get "If it's red, you stop. Green, you go." and which point I interrupt "And yellow (or orange for you Aussies out there)." It's about here that they start to think about it. Then I ask "What if it's raining?" quickly followed by "What if it's the first rain of the year?" and shortly thereafter followed by "What if you see someone else running the light in front of you?" I then explain that if I were writing a program to do something as "simple" as deciding if it's safe to go thru a traffic light, I'd have to think of ALL of these issues, plus everything else that could ever possibility happen while traversing an intersection. If I manage to miss something, it becomes a "bug" in that program.
In my experience, people pretty much "get it" with this analogy. Course, YMMV...
"1984" was ment to be a warning, not a guidebook. You hear that Kim Jong-il!? BushCo?!
I've even explained how interrupt handlers work in regards to a USB joystick to a Lawyer... He was so happy in the way I explained it to him, he kept asking more and more questions until I told him I have to go :P
Ah great.... Now your lawyer is going to wager a class-action lawsuit on behalf of all USB devices for illegal discrimination and unfair employment practices by the interrupt controller. The PCI bus will be named as a conspirator and the CPU will be charged with negligence. The PCI sound card will be the chief witness.
When i'm discussing a request for implementing a new requirement or changing an existing requirement with the non-technical person making that request, a trick i usually do to show the complexity of apparently simple requests is to right there, present the high-level (eg. design) logical steps on how we would implement that request.
...
Non-technical people will understand the complexity of doing something when they follow you through the sketching of the process of making a solution that does it. For example, if you are asked to change a form so that a free text input field becomes a pull-down with a number of options you can:
- Ask them if the list of options is fixed or if an application administrator can change them
- If it's fixed, then it's easy to do, otherwise:
- Explain that you have to store those options somewhere. Maybe you already have a database, so explain that you have to add storage tables to the database, then add code to the application so that it can load and save that information to and from memory. Also explain that you have to pre-load some sensible default values into the database so that the application works out of the box.
- Since they want that some administrator can change it at runtime, figure out if said administrator is a developer/dba type of person or not. If not, explain that to allow an administrator user to change those values you have to extend your administrator interface, which means adding menu entries and one ore more new pages.
- Also figure out what are the rules (if any) that constrain the values that the administrator can configure for the pull-down. Depending on the complexity of the rules you might want to follow through into how you would do it, for example, if the rules are based on other values in the database which can also be changed by an administrator:
- Explain how you have to add code to retrieve and process the needed values so that you can check that the rules are not being broken, and that you also have to change the administrative interface for those other values so as to make sure that when they are changed or deleted, the values in the pull-down box do not become invalid.
-
The point here is that most non-technical people which actually use computers understand concepts such as memory and storing "data that cannot be lost when the application stops" (persistent data) somewhere other than memory. They can be made to understant the basics of a relational database: "a place to store data which has tables - a bit like excel tables - one table for each kind of information" (not exactly it but close enough) and they can usually follow a logical chain of steps (eg to show the data in the interface we have to get it from somewhere, since it is configurable, we store it in the place where we put the data that can be changed by the program but must be preserved even when the application stops).
Try and stay away of techie words and expressions ("We're gonna have to persist that data to our relational database and that means creating new tables, adding new data objects and changing the Hibernate mappings"). Instead assume non-techies are ignorant but not stupid ("We have to store that data in the database so that we don't loose it when the application stops. This means we have to configure in the database the tables where the data is stored and also have to add support in the program so that it can load that information into memory and use it").
Also, simplify:
- Don't try to be overly exactly - for a non-techie is enough to say you have a "database" no need to say it's a "clustered configuration of a <insert-brand-here> relational database".
- Don't go into deep details - you "configure the place in the database to store the data" no need to say you're "extending the tablespace, creating new tables, adding foreign keys to the related tables, and setting up a couple of indexes to speed up cross-referencing"
- Try and tune your words to the audience - for some people you can say you're gon
The simplest is browser compatability. Does the site owner care? Well yeah if you can get their head around the fact that if the site is IE only they will be losing a percentage of customers. It is very odd. Supermarkets understand they need an extra wide register for people with special needs. They know they need staff on hand to help disabled people because no supermarket can afford to turn that tiny percentage of people away who need help. Most brick and mortar stores would be horrified to learn that their to narrow door would loose them say 5% of visitors. So why is it acceptable on the net?
Then their is the proper use of standards. It ain't just that you got to use CSS but more that you got to choose a standard way of doing things. Use font tags if you want BUT then ALWAYS use them. Not some horrid combo of font tags, inline css, external css and included css. Makes it a pain to figure out what the fuck is supposed to affect what.
Finally there is the coding itself. Every damn time I get called into to fix a site I find that some kiddie has hacked it together and totally forget to do any kind of error checking let alone proper security. Most just take values straight from the user and insert them into sql queries. And why does anyone create a login page for the site CMS without https being enforced?
I seen code that would make you weep. Yet the site works doesn't it. Well no that is why I am called in but it then is very hard to explain why you need a week to upgrade the site before you can even start fixing things. (Note that the sites are generally so poorly designed that the original builders are no longer willing to support it wich is when I get involved).
But how do you explain to the customer about the difference between bad code and good code.
Recent page had a simple form for the usual add, update and deleting of a record. The code contained no functions and had the most horribly arranged IF/ELSE statements you could imagine (IF(user_logged_in) {ALL THE CODE} else {error_msg} EWH!!!!!
Worse it had the HTML/FORM 3 times for each type of action. Except that they were practically the same. Just an empty form for add, a filled in form for update and again a filled in form for delete. 3 Times the same HTML all because the idiot never grasped the concept of functions. (He didn't use them anywhere in the code).
Yet how do you make this clear to the customer. If you know the answer please tell me.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
You know the annoying kid you knew in college who would repeatedly interrupt every discussion demanding definitions of all the words being used, and who would argue endlessly about flaws in the definitions so that you couldn't really talk about the topic you wanted to? Imagine you're writing a book, and that guy has to personally approve every sentence.
The 10 line function and the Haiku are simpler. Also, more elegant and better.
I disagree with the idea the cramming the most amount of logic into the least amount of lines is elegant. Brian Kernighan said it best, "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman