Slashdot Mirror
Toward Better Programming
An anonymous reader writes "Chris Granger, creator of the flexible, open source LightTable IDE, has written a thoughtful article about the nature of programming. For years, he's been trying to answer the question: What's wrong with programming? After working on his own IDE and discussing it with hundreds of other developers, here are his thoughts: 'If you look at much of the advances that have made it to the mainstream over the past 50 years, it turns out they largely increased our efficiency without really changing the act of programming. I think the reason why is something I hinted at in the very beginning of this post: it's all been reactionary and as a result we tend to only apply tactical fixes. As a matter of fact, almost every step we've taken fits cleanly into one of these buckets. We've made things better but we keep reaching local maxima because we assume that these things can somehow be addressed independently. ... The other day, I came to the conclusion that the act of writing software is actually antagonistic all on its own. Arcane languages, cryptic errors, mostly missing (or at best, scattered) documentation — it's like someone is deliberately trying to screw with you, sitting in some Truman Show-like control room pointing and laughing behind the scenes. At some level, it's masochistic, but we do it because it gives us an incredible opportunity to shape our world.'"
In my 25 years of professional programming experience, I've noticed that most often, most programming problems are caused by improper implementations of the separation of unrelated concerns, and coupling of related concerns. Orthogonality is difficult to achieve in many programming exercises, especially regarding cross-cutting concerns, and sometimes the "right" way to code something is tedious and unusable, involving passing state down through several layers of method parameters.
Or.....
Maybe software development is just hard and you need to be a rocket scientist to see it.
You think programming's bad? Think about electronics, especially analogue electronics.
Incomplete, inaccurate data sheets. Unlabeled diagrams (where's the electronics equivalent of the humble slashed single line comment?), with unknown parts given, and parts replaced with similarly numbered replacements with subtly different qualities. And then you've got manufacturing variances on top of that. Puzzling, disturbing, irreproducible, invisible failure modes of discrete components.
You think linguists haven't pondered the same challenges for millenia? Chomsky famously declared that language acquisition was a "black box." There is no documentation. Syntax, semantics and grammar get redefined pretty much all the time without so much as a heads-up.
And the result of all this? We wouldn't have it any other way. Programming will be much the same: constantly evolving in response to local needs.
...wear a fucking helmet.
The post essentially points in the direction of the various failed 4GL attempts of yore. Programming in complex symbolism to make things "easy" is essentially giving visual basic to someone without the knowledge enough to avoid O(n^2) algorithms.
Programming isn't hard because we made it so, it's hard because it is *intrinsically* hard. No amount of training wheels is going to make complex programming significantly easier.
Better tools and languages just allow bad programmers to create more bad code.
http://michaelsmith.id.au
I see the problem as more of a chase of new stuff all the time, in an attempt to be more productive.
Whilst there's a certain element of progress in languages, I don't see that it is necessarily enough better overall to be worth the trouble - yet we have new languages and frameworks popping up all the time. Nobody becomes an expert anymore, and I think that because programming is hard, a lot of people get disillusioned with what they have and listen to the hype surrounding a new technology (that "is so much easier") and jump on the badwagon... until they realise that too is not easy after all, and jump onto the next one... and never actually sit down and do the boring hard work required to become good. Something they could have done if they'd stuck with the original technology.
Of course no-one sticks with the original, as the careers market is also chasing the latest tech wagon because they're partly sold on the ideas or productivity, or tooling or their staff are chasing it.
Its not just languages, but the systems design that's suffered too. Today you see people chasing buzzwords like SOLID, unit-testing using shitty tools that require artificial coding practices, rather than do the hard, boring work of thinking what you need and implementing a mature design that solves the problem.
For example. I know how to do network programming in C. Its easy, but its easy to me because I spent the time to understand it. Today I might use a WCF service in C# instead, code generated from a wizard. Its just as easy, but I know which one works better, more flexibly, faster and efficiently. And I know what to fix if I get it wrong... something that is impossible in the nastily complicated black box that is WCF sometimes.
But of course, WCF is so last year's technology.. all the cool kids are coding their services in node.js today, and I'm sure they'll find out its no silver bullet to the fundamental problems of programming just being hard and requiring expertise.
What is programming?
The answers I got to this were truly disheartening. Not once did I hear that programming is “solving problems."
I'd like to think that's because the majority of programmers (not once? Does that mean all of us?) aren't the sort to bullshit you with CEO-level bullshit about vision and buzzwords that fit into powerpoint slides.
It's probably not true, but it's a nice dream.
The problem with defining programming as "solving problems" is that it's too vague. Too high level. You can't even see the code when you're that high up. Hitting nails with hammers could be problem solving. Shooting people could be problem solving. Thinking about existential crisis could be problem solving.
The three buckets:
Programming is unobservable - you don't know what something is really going to do.
Programming is indirect - code deals with abstractions.
Programming is incidentally complex - the tools are a bitch
Something something, he doesn't like piecemeal libraries abstracting things. "Excel is programming". Culture something.
The best path forward for empowering people is to get computation to the point where it is ready for the masses.
We're there dude. We've got more computational power than we know what to do with.
Cue "that's not what I meant by 'power'".
What would it be like if the only prerequisite for getting a computer to do stuff was figuring out a rough solution to your problem?
Yep, he's drifting away into a zen-like state where the metaphor is taking over. Huston to Chris, please attempt a re-entry.
AAAAAAAAnd, it's a salespitch:
Great, now what?
We find a foundation that addresses these issues! No problem, right? In my talk at Strange Loop I showed a very early prototype of Aurora, the solution we've been working on to what I've brought up here.
Not clear to me that his is a viable objective. 80% of the masses do not think like programmers. Some might be trainable. Some, not so much. Many will not want to think the way problem-solving in code requires. I'm not sure how to quantify it, but the amount of effort expended on a project like this may not see an appropriate payback.
Even if we change the environment and act of "coding", the problem-solving itself still requires clear thinking and it *probably* always will.
If the Government becomes a lawbreaker, it breeds contempt for law;
Programming is hard because we only call it programming when it's hard enough that only programmers can do it. Scientists do stuff in Mathematica, MBAs in Excel, or designers in Flash/HTML, that would have been considered serious programming work 30 years ago. The tools advanced so that stuff is easy, and nobody calls it programming now.
Lots of stuff that takes real programmers now will, in 20 years, likely be done by equivalents of Watson. And the real programmers will still be wondering why is so hard.
Something something blames his tools.
The point of that proverb is that a good craftsman chose his tools to begin with, so he has only himself to blame. Programming is odd in that you have bad toolchains forced on you by management - tools you know are bad, know will cause more problems than their worth, but they're a corporate standard or some such BS. Usually not bad enough to be worth quitting over, so you hobble along.
Of course, I did quit a job once primarily because we had Rational Rose forced on us from above (but mostly because a management that would do that would do anything).
Socialism: a lie told by totalitarians and believed by fools.
I knew it! It's Bush's Fault!
I hear the California government has a bill to rename the San Andreas Fault to "Bush's Fault", just so they never have to stop using the phrase! But that may be just a rumor ...
Socialism: a lie told by totalitarians and believed by fools.
Some interesting points in the article. I think there's nothing really stopping you from creating a high-level representation that lets you work abstractly. A graphical programming model is probably going to be too simplistic, but the card example could easily be something like Cards.AceOfSpades. Or being able to call something like Math.eval(<insert some math here>).
Where it falls apart is when you have to hook this up to code that other people have written. If there was a single PlayingCard library that everyone could agree on, you might be able to create a card game by adding a "simple" set of rules (in reality, even simple games tend to have a lot of edge cases, but this would at least free up the nitty gritty work to allow writing something more like a flowchart expressing the various states).
Unfortunately, it's unlikely that a single library is going to meet everyone's needs. Even if you manage to get everyone to stick to one framework, e.g. if all Ruby developers use Rails -- as soon as you start adding libraries to extend it you're bound to end up with different approaches to solving similar problems, and the libraries that use those libraries will each take a different approach.
I used only free software programming for about 10 years and I thought I was pretty efficient at writing code. However, no matter what there where always poor documentation to deal with and strange bugs to track down where libraries just didn't work right.
Once I returned to school I started using MATLAB for some engineering classes and overall I have found it much better to deal with. The documentation is far more complete than any open system I have ever ran into with much better examples. I would never use it for general purpose programming but for engineering work it sure is hard to beat. So many things are built in that are nasty to try to implement in anything else. Things like the global optimization toolbox or the parallel computing toolbox make many things that are hard in other languages much easier to deal with.
MATLAB also takes backwards compatibility very seriously. If something is deprecated it warns and also gives an upgrade path and tells you what you need to change. That is the one thing that has seriously pissed me off about the free languages is backwards compatibility is just tossed out at a whim and you are left with a fairly nasty upgrade to deal with. Even now the vast majority of projects are still running Python 2 compared to Python 3. 10 years from now that will probably still be true.
In the end I care more about just getting work done now, not about any free vs proprietary arguments. I don't care if a language is open or not so long as it is very documented and runs on all the platforms I need it with a history of being well maintained. Modern development tools overall suck. We have javascript libraries that love to break compatibility every few months and people are constantly hoping from one new thing to another without getting any of them to the point where they truly work. We have languages deciding to just drop backwards compatibility. We have other languages that are just really buggy where software written with them tends to crash a lot. Software development needs to become more like engineering and that includes making the tools work better, sure they would not be developed as quickly but you would still get work done faster since the tools would actually work every time all the time.
Computer modeling for biotech drug manufacturing is HARD!
Let's look at the major programming environments of today:
1. Web
2a. Native Apps (machine languages (C/C++ etc))
2b. Native Apps (interpreted/JIT languages (intermediary byte code))
3, Mobile Apps
1. is made by 5 main technologies: XML, JavaScript, MIME, HTTP, CSS. To make it do anything you need another component, of no standard choice:your language of the server (PHP, Rails, .Net, Java, etc) .Net.
2. Was POSIX or Win32, then we got Java and
3. Is Java or Objective C.
We don't do things smart. There is no reason why web development needs to be 5 technologies all thrown together. We could reduce it all to Javascipt: JSON documents instead of XML/HTML, JSON instead of MIME, NodeJS servers, and even encode the transport in JSON as well.
Then look at native development. Java and .Net basically do the same thing. Which do what POSIX whas heading towards. Java was invented so Sun could keep selling SPARC chips, .Net came about because MS tried to extend Java and lost.
Then, we have the worst offenders. Mobile development. Not only did Apple impose a Objective-C requirement, but the frameworks aren't public. Android, the worst offender is a platform that can't even be used to develop native apps. At least Objective-C can. Why did Android go with Java if it's not portable? Because of some good requirement that they don't want to support a specific CPU, but then they go and break it so that you have to run Linux, but your GUI has to be Android's graphical stack. Not to mention that Android's constructs - Activities, Intents, etc are all Android specific. They don't solve new problems, they solve problems that Android made for themselves. We've had full-screen applications for years the same goes for theading, services, IO, etc.
I'm tired of reinventing the wheel. I've been programming professionally for 13 years now and Java was neat, .Net was the logical conclusion of it. I was hoping .Net would be the final implementation so that we could harness the collective programming power into one environment of best practices... a decade later we were still re-inventing the the wheel.
The answer I see Coming up is LLVM for languages. And Qt, a C++ toolkit. Qt runs everywhere worth running, and its one code base. Sure, I wish there was a java or .net implementation. But I'll deal with unmanaged memory if I can run one code base everywhere. That's all I want. Why does putting a text field on a screen via a web form have to be so different from putting a text box on the screen from a native app? It's the same text box!
Witty, a C++ webtoolkit (webtoolkit.eu) is mirrored after Qt and is for the web. You don't write HTML or JS, you write C++. Clearly the C++ toolkits are onto something. If they were to merge and have a common API (they practically do now) in an environment with modern conveniences (lambas (yes, C++11) managed memory) we'd have one killer kit. Based on 30 year old technology. And it would be a step in the right direction.
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
A good craftsman still doesn't blame the tools - he performs with what's at hand. Not having CNC routers didn't stop wood workers of the past from creating better products than today's staple-gun wielding "craftsmen" do.
Do wonders with what you have, and strive for getting better tools.
A good craftsman can get by with suboptimal tools.
A good craftsman is not content to "get by", almost by definition. If some part of your workflow sucks, you make it better, whether that's a better tool or more skill/training. If you're good, you never stop improving (until management forces BS on you, of course).
Socialism: a lie told by totalitarians and believed by fools.
Never trust a programmer who doesn't like programming (or calls it "anagonistic" and "masochistic").
His name's Stroustrup. Bjarne Stroustrup.
This guy seems to want an objects actions to be more apparent just from looking at the object, but he chose two rather bad examples. His math formula is as likely to look like gobbledygook to a non-math person as the program is. And the playing card has a fundamental set of rules associated with it that you still have to learn. You look at an ace of spades and you know that's an ace of spades, you know how it ranks in a poker hand, that it can normally be high or low (11 or 1) in blackjack or in a poker hand. But none of these things are obvious by looking at the card. If a person who'd never played cards before looked at it, he wouldn't know anything about it, either.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
That is such crap. Name me any platform without C code at its core. Just because C is relegated to a role of providing a system in which people can run their shitty code, doesn't mean it is obsolete. If you don't know C, then you don't actually know how your computer works. If you don't know how your computer works, then you will write shitty code in any language you choose.
VPSs are cheap. besides, it's an illogical computing unit since it is so abstracted.
Native apps are a run around on android... that's what Qt is. Native.
You still don't get it. Android reinvented the wheel yet again, in a context that is not re-usable outside of android.
The fact that C++ can still be used on any platform, and effectively at that, proves that nothing has been added since C++.
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
The original author is looking at the part of the problem that gets the most attention, but not at the part that causes the most problems. He's looking at programming languages and their expressive problems. Those are real, but they're not why large programs break. Large programs usually break for non-local reasons. Typically, Part A was connected to Part B in some way such that an assumption made by part B was not satisfied by part A.
C is particularly bad in this regard. C programs revolve round three issues: "how big is it", "who owns it", and "who locks it". The language helps with none of these issues, which is why we've been having buffer overflows and low-level security holes for three decades now. Most newer languages (not including C++) address the first two, although few address the third well.
There have been attempts to get hold of this problem formally. "Design by contract" tries to do this. This allows talking about things like "Before calling F, you must call G to do setup.", or "parameter A must be greater than parameter B". Design by contract is a good idea but weighed down by so much formalism that few programmers use it. It's about blame - if a library you call fails, but the caller satisfied the contract in the call, it's the library's fault. Vendors hate that being made so explicit.
it's a concept from aerospace - if A won't connect properly to B, the one which doesn't match the spec is wrong. If you can't decide who's wrong, the spec is wrong. This is why you can take a Pratt and Whitney engine off an airliner, put a Rolls Royce engine on, and have it work. Many aircraft components are interchangeable like that. It takes a lot of paperwork, inspections, and design reviews to make that work.
The price we pay for not having this is a ritual-taboo approach to libraries. We have examples now, not reference manuals. If you do something that isn't in an example, and it doesn't work, it's your fault. This is the cause of much trouble with programming in the large.
Database implementers get this, but almost nobody else in commercial programming does. (Not even the OS people, which is annoying.)
You are right of course. The problem is that most "programmers" are actually one-trick ponies and do not know more than one language, and hence cannot chose a language at all. Hence your first option applies to most "programmers". The others have been wondering what this issue is actually about for decades.
I mean, I can take a new language that has some merit, do some real task in it to get to know it better and get a real understanding what it is suitable for and what it is not suitable for. For most "programmers", this would involve "learning" the language first. For any competent programmer that requires a few days of looking at the reference often while writing the code, and maybe more redesign than usual. But it is not a big deal. (Of course some really convoluted and borderline unusable languages like Java or some 4GL stuff require more effort, often because the language is so broken that you need to master some "tools" just to be able to handle it.)
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
It's called software engineering. Most software is hacked, code-and-fix, cobbled together, unmaintainable dreck. Why? Because managers, clients, and customers blanch when presented with honest, realistic estimates based on metrics. So they'll often search out low-cost developers, eager (perhaps desperate) for work and who will agree to anything. They're probably minimally trained so they're not going to apply good practices (step-wise refinement, descriptive variable naming, OOA/D or SA/SD). They'll furiously churn out spaghetti-and-sausage-ware (not as tasty) that will miss the deadline anyway, barely work, but it'll be something. They invariably burn out and end up leaving and the next developer unlucky enough to be tasked with maintaining and extending the code will be blamed for taking too long for making changes because he or she will have to read, deconstruct, reverse engineer, and comprehend 100% more of the code than they would have had to otherwise.
As Fred Brooks said, there's no silver bullet. But all is not lost. Static analysis tools, code complexity tools can sometimes--I repeat sometimes--help target the 20% of the code that actually needs 80% of the maintenance effort. However, mention McCabe to most programmers and you'll most likely get a blank stare back.
...they'll either burn out, or even worse get promoted and you'll have to report to them. Yikes!
When I started 44+ years ago, the hard part of programming was getting programs to fit the memory size and processor speed of the computers of that day. We often wrote in assembly language, because it was often the only reasonable choice. Scientific code was written in FORTRAN, business code in COBOL (or maybe RPG). A little later C came along and became a viable alternative to assembly language for many things. There was much experimentation with languages in those days, so there were a lot of other languages around, but FORTRAN, COBOL, and C/assembly were the major ones in use.
One thing we did have in those days, which I recall with great fondness, were manuals. There were manuals for users of computer systems, and manuals for programmers, describing the operating system and library interfaces. There were people who specialized in writing such manuals, and some of these people were quite good at it. But at some point manual writing became a lost art, and the industry is poorer for it.
Over time, the machines reached a level of capability that exceeded the requirements for the kind of programs we were writing. It was rare to find a program (code, not data) that wouldn't fit in memory, and compiler technology had advanced to the point that we didn't need to obsess about the speed of code sequences in assembly language. So we started writing larger and more complex programs, and the main difficulty of programming was managing and testing the code. Source code control systems were developed, and eventually continuous build systems, but testing remains an important concern to this day.
As computer networking evolved, the problems of managing concurrency and asynchronicity became more severe. Many approaches have been developed to deal with these problems, such as threads and monitors, but this remains an area of difficulty, and is more important than ever with the advent of multicore processors.
By the mid-90's, we'd learned about all we could from the structured programming craze, and were well into the thrall of object-oriented programming, with many of us programming in C++. Large teams of programmers became common, and for them, source code control and continuous build systems were essential. Then the web finally arrived on the scene, and a great war between corporations for control of the web platform began.
How did that war end? Well, everyone lost. We got HTML, CSS, and JavaScript. Some people call that a platform. I call it an abomination. But the alternative would have been to have no standard at all, which I'm sure would have pleased some of the combatant corporations, but was never really a viable option.
Now we are in the age of frameworks. Everybody and his dog has a framework, and most of them are very poorly documented. I believe that it is only by virtue of forums like stackoverflow that some of them are usable at all. But many of them are aimed squarely at dealing with the problematic "web platform", and for that we should all be grateful. However, we really need to get some smart people together and design a new web platform, including a reasonable migration path from the current mess. The problem is, as we approach the 20-year anniversary of the web debacle, there is an entire generation of programmers who have never known anything else.
The nature of programming will continue to evolve, but evolution is not a particularly efficient process. If we are indeed intelligent, we should be capable of intelligent design.