Slashdot Mirror
Whatever Happened To Programming?
Mirk writes "In a recent interview, Don Knuth wrote: 'The way a lot of programming goes today isn't any fun because it's just plugging in magic incantations — combine somebody else's software and start it up.' The Reinvigorated Programmer laments how much of our 'programming' time is spent pasting not-quite-compatible libraries together and patching around the edges." This 3-day-old article has sparked lively discussions at Reddit and at Hacker News, and the author has responded with a followup and summation.
Programming is becoming nothing more than cutting and pasting, especially with languages like java, that provide libraries that do "the hard stuff" and let programmers concentrate on "programming".
Programmers are now a dime a dozen. I can find 10 people who can cut and paste available on the internet and modify it to do what they want.
Good programmers on the other hand, are few and far between.
It seems everyone wants to be a "software engineer", but nobody wants to focus on the "hard stuff", and instead chant "let java/X do it for you".
I think a better analogy would be to say that today's programmers are more like a Cargo Cult.
... and in those cases you should. There's nothing wrong with re-inventing the wheel in some cases, especially where you don't want to drag around a ridiculous framework that does way more than you need.
I'm already at the gunpowder stage. My code has been blowing up for years.
slashdot. where don knuth is an idiot because he cant grasp the awesome power of php
THE DUMBING-DOWN OF PROGRAMMING (1998): "My programming tools were full of wizards. Little dialog boxes waiting for me to click "Next" and "Next" and "Finish."...Dumbing-down is trickling down. Not content with infantilizing the end user, the purveyors of point-and-click seem determined to infantilize the programmer as well."
The article states that, no matter how important are things like unit tests, they are fundamentally in a supporting role to programming proper.
As someone who practices test-driven development and programming-by-contract, I fundamentally disagree. Tests, for me, is what defines the requirements and interfaces. The code itself is just the implementation. From the business logic perspective, HOW a program does something is secondary to WHAT it does.
Car analogy: imagine a programmer as a truck driver, and the project manager as the one who has his goods shipped. The programmer doesn't care much about what he ships (as long as it's not explosives or something like this) -- he cares about the route he's going to take to deliver those goods as fast and efficient as possible. That's all great. But the project manager doesn't care, nor should he. For him, the goods are the primary value, and the route the truck takes is the supporting value. As long as the goods arrive undamaged and on time, nobody other than the driver cares what route they went through.
We have a basic conflict of perspectives here. Programmers think it's all about how good their code is internally, and think that the coding is the most important part of the application, arguing that without that, the application obviously wouldn't work. But users and payers for that code do not care about those matters, they see a white-box perspective only. Just like the goods shipper, they care more about the goods than how they are delivered. And if the truck driver gets too bitchy about how and what goods he wants to deliver, it's usually easier to get a new truck driver than change your goods or shipping schedule.
I'm currently a J2EE (and C, but predominately Java J2EE) programmer, familiar with Hibernate, Spring, as well as the old school EJB 2 mess. I wasn't always a Java programmer. I've taught C and coded with it commercial. I also have commercially used a variety of other platforms from VB and Delphi, to Smalltalk, to C++.
Here's the core of the problem: The web is a horrible platform. I went from Rapid development drag and drop screen design in the late 90s to the abomination that is hand crafted JSP against shitty state based environments. Sure our applications are more scalable now, but I'm still hand crafting code to talk to a database object. There are tools out there that spit out mediocre code (hibernate tools come to mind). But nothing that I'm aware of spits out a good set of CRUD classes with corresponding unit tests. Why do we ever have to hand write this shit? (I haven't used Grails and Groovy extensively but I understand scaffolding has similar issues and not being as mature the people I've worked with have had to work around issues with transactionality)
Then you take a look at the GUI layer. Hand writing CSS and JSP? Really? In 2010? SHIT. Hand writing code for simple controllers. Never mind if you do actually end up doing anything non-standard in which case good luck getting into the guts of the documentation for Spring MVC or Struts or similar. And then you have to deal with having to redeploy your application to see simple changes OR using exploded views that don't update properly and leave you debugging a problem for 4 hours that should take 4 minutes.
It's a complete mess. It's WAY more complicated than it should be. I should be focused on the business problems - modelling the backend, getting the algorithms right for complex transactions etc. Instead there are people arguing that such simplicity leads to sloppy programming (usually mentioning VB as if the same programmers wouldn't have made a mess with something more complex). Well if you have nothing better to do than some stupid little dance just to get a web page up, that's your issue. For me that is a stupid statement. There's always a genuinely complex issue to solve without inventing one.
These posts express my own personal views, not those of my employer
"Knuth had his day."
Wow. Just wow.
First, I want you to write a work that tops TAOCP. Or at the very least show your check from Knuth for finding an error. Oh, wait, I highly doubt you've done either. It can be how you can express your imagination in ways that are beyond TAOCP if you like.
Next, write some software at least as useful as TeX.
Then, and only then, can you call Knuth an idiot.
I am officially gone from
Some things may be over-engineered. But in my experience, more often it is the case, that people rather re-invent the wheel,
than they bother to try to understand, what someone else has done, and how it is supposed to work.
And over time, it will bite. Usually not the one who wrote the code, because that person is gone, but the project in whole.
And no, I don't see a difference in "own code" and foreign libraries, from a "long" term perspective, it is the same.
"Between strong and weak, between rich and poor [...], it is freedom which oppresses and the law which sets free"
It isn't even a question of faster ... a lot of those "glued-together solutions" don't scale and are impossible to debug and maintain.
Warning! Before you read the linked article or its followup too deeply, be aware they are not by Donald Knuth. Instead, the author has a brief quote from Donald Knuth in his first blog, and the other link is a followup story. So, "the author" referenced in the Slashdot summary is NOT Donald Knuth. I made the mistake of reading the followup article first, and then when I read the original, I found a brief quote from Donald Knuth which tipped me off to the fact that the author was not Donald Knuth, and as far as I can tell, Donald Knuth doesn't even know this author.
If I can be modded down for being a troll, can I be modded up for being an orc, or a balrog?
You, need, to, add, more, commas,.
excellent programmers steal excellent code."
I stole this, and I don't know where it is stolen from!
The problem is simply: Why bother?
You are usually facing two choices:
1) Create a slick, nifty, fast framework yourself that does what you need. It will be lightning fast.
2) Use some sluggish can-do-everything-and-more framework out of the box that does 10 times what you need. Basically, it's like delivering a soup cube with a flat bed.
Option 1 will take about five times the development time, but save you well over 60% runtime.
Your boss will outright fire you if you opt for option 1. Why? Because it takes 5 times dev time for 0.5 seconds saved during use.
Machines today are fast. Much, much faster than what we need for programs to run. Hell, even games are today produced with sluggish frameworks that waste resources left and right, and they tend to be the programs that are most time-critical an "ordinary" user would get to see.
I hate this development as much as anyone who learned programming during a time when memory was scarce and gaining 10% run time speed was worth everything. But that simply ain't true anymore. Today you have programmers (I use that term loosly now) that look at you blankly when you ask them what sort algo they used, then they give you the name of some library function. No, they have no idea what the function does. Only that it somehow automagically sorts the stuff. Why this one and not the other one? Because they know this one, or it was the first the help file spit out when they searched for a "search function". But then again, it does not matter. The other function that might have been faster because it happens to implement an algo that is more fitting to the problem at hand would have saved about 0.0something seconds, because the machine running the program eventually is fast enough that it means jack what algo you use. Memory amounts today mean that it is pointless to ponder whether you really need double linked lists or whether you get by with single linked ones. Or that you use variables smaller than DWords to store integers.
So it doesn't really matter anymore whether you can program, or whether you even know just how much space and time you just wasted with that horrible choice of library functions that will probably eventually even do the work but are just about the worst choice for the problem presented. Modern computers are fast enough and have enough ram to compensate for programers' inaptitude.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
I think the difference is like that between mathematics and engineering. Programming used to be more of a math, you were basically writing an executable form of a proof. Now programming is more about assembling all the previously developed tools to produce a useful result in the same way that engineers use a bunch of mathematical tools (that they may not necessarily know how to derive--and for that matter don't need to) to build a bridge. For a mathematical guy like Knuth, the engineering bit is a drag (for me as well) and the act of problem solving is the interesting part. I would even wager that to such a person the means (or methods of solving) are more important than the end result. The beauty is in the elegance of the solution, not in the fact that we now have a solution. Now programming is much more of an engineering task. The tools might as well be black boxes that we assemble in different ways to produce results. Another type of person likes this sort of thing (not me). These people are more interested in the ends, not the means. They are more driven to produce something cool rather than to produce something in a cool way. (Am I making any sense?)
Neither of these groups is better or worse than the other, just different. Both are needed for progress. There will always be mathematical problems to solve, and there will always be a need to apply the toolbox created by such mathematics to practical tasks with an emphasis on results rather than methods.
I've dealt with Chinese and Indian outsourced code before - it's rather interesting. They take fragments of code they find via Google, paste them together, and do the bare minimum of editing to make it compile and say 'okay, we've fulfilled our contract, ship it.' This is what suffices for 'programming'.
On the other hand, I am still solving interesting problems with real programming at my current company, so I still think it's a lot of fun. The key point is that the programming is part of the /problem solving/. Code pigs have no concept of problem solving, just making the program work (by which they mean compile, or matching the sample screens). Engineers are solving problems, and the program is just a part of that. At my present job they really don't care what language I do things in as long as the job gets done, because solving the problem in the most practical manner is the most important thing. In practice this means I use C for things that actually do require high performance and minimal memory usage (this is still an issue in embedded programming), Python for everything else that I can, and domain specific languages for things like servo controllers or FGPAs.
The 'pasting not quite compatible libraries together' approach is a Java/COBOL thing of minimizing the damage incompetent consultants can do. I've seen it time and time again - once an Enterprisey Java programmer encounters sufficient complexity, a hormone kicks in and they create a framework to simplify this complexity. It does so, initially, but eventually ends up being 2-10x as complex as the original problem they were trying to simplify. But they see this as a net positive because they have a new acronym to put on their resume.
So basically, like every single damn post I've seen on here lamenting the state of programming, and repeating every damn comment I've made again and again, it boils down to 'solve problems as efficiently as you can'. Absolute rules, in programming or religion, are for people who are too simple to handle complexity. This is the difference between an engineer and a code pig.
Try reading it as William Shatner would.
Frameworks, like anything else, are fantastic when used properly but unruly beasts when they are used incorrectly. My current bane is legacy code that is a poorly thought out amalgamation of early version frameworks. I can hear my predecessors squealing with delight and saying things like "Spring and hibernate will magically handle all of my transaction boundaries and multi-threading? Neato!!" as I slog through 3-4 hundred line stacktraces (no joke) trying to debug a race condition. Brutal.
while [ 1 ]; do echo -n -e "\xe2\x95\xb$((($RANDOM&1)+1))"; done
"I think a better analogy would be to say that today's programmers are more like a Cargo Cult."
Responses to the recent MS-random-browser thread ("the faulty shuffle is close enough", "this guy's being pedantic", "knowing algorithms is a bad use of company time") are pretty good evidence of that.
http://developers.slashdot.org/article.pl?sid=10/02/28/1837223
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
Except that what really happens is that you end up trying to use a sluggish can-do-everything-and-more framework that does 10 times what you need it to do but fails to do 2-3 critical things that you absolutely need to do. And between the time spent learning the can-do-everything framework and what parts of it you don't care about and the time spent kludging and hacking on it to fill in the missing bits, you end up spending more time than it'd've taken you to write your own from scratch or using more primitive tools.
Been there, done that, got the drawer full of t-shirts.
Machines today are fast. Much, much faster than what we need for programs to run.
Until you get slashdotted. Having a sharp increase in number of runs per second can show just how fast your program isn't. And look at all the fail-whales soon after Twitter caught on.
Memory amounts today mean that it is pointless to ponder whether you really need double linked lists or whether you get by with single linked ones. Or that you use variables smaller than DWords to store integers.
Until you try to shave pennies from a mass-manufactured part.
I absolutely hate frameworks. They are the same as libraries. But without the ability to plug them into anything. They want to be the core of your application, and not play with anything else.
They are a very “enterprisey” concept.
Please, all, stop doing frameworks, cut them into their aspects, and start doing libraries again! Nobody wants you elaborate all-encompassing framework that is its own inner platform and yet unfortunately lacks the very function you need the most.
Any sufficiently advanced intelligence is indistinguishable from stupidity.
That's the kind of thinking that leads to batch jobs taking an hour, that should be done in 5 minutes or less. I've seen way too many "programmers" that don't understand what their high level language is doing being their backs, so they end up copying memory around all the time. Or going off to ask the database the same question over and over in a loop because they couldn't be bothered to refactor the code so it cached the answer.
09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
From the article: "...it’s a tedious exercise in impedance-matching, requiring lots of time spent grubbing around in poorly-written manuals that tell you everything the code already told you (because it was generated with JavaDoc or Rdoc or whatever), and none of the high-level stuff that you actually need to be told."
Ah, so the real problem is poor documentation.
I work all day in a programming language written by one of the biggest software companies in the world. The documentation is complete, detailed, and accurate. For large things, there's an accompanying "concepts" doc. While I have (very rarely) run into something that needs clarification in some sort of corner case, I've never come across any part of their language, libraries, or objects that wasn't thoroughly documented, with examples.
On the other hand, I don't think I've ever come across an open source product that had barest minimum of documentation. What does exist is typically out of date (and observations of such are met with "read the changelog!" - lame). There's certainly nothing that explains the major concepts in the code - at best, there's some guide to functions or objects, and usually that only because it can be autogenerated. Sometimes there are examples - though more typically, a few mini examples are the only documentation.
Documentation writing sucks. Programmers don't enjoy it. It's highly amusing to me that the two areas that are the least fun for programmers - GUI design and documentation - are the two worst parts of open source projects.
BTW, in the 80s, programmers were excited about OOP because it promised rich object libraries. Someone would create objects to do X and we'd never have to code X again - no one, ever! And now everyone complains programming is just stringing together libraries.
Advice: on VPS providers
That's the kind of thinking that leads to batch jobs taking an hour, that should be done in 5 minutes or less.
I've said this before and I'll say it again, because I've created plenty of unoptimized "batch jobs" that take longer than they should.
COMPUTERS ARE CHEAPER THAN HUMANS
A cubicle filled with racks of computers running inefficient batch jobs costs a tiny fraction of a competent person sitting in the same cubicle and optimizing every little thing by hand. And a program written in a high-level language is probably cheaper to maintain over the long run as well.
"I assumed blithely that there were no elves out there in the darkness"
Whining about "infantilizing" the end user? WTF? I get really tired of the elitist attitude that some computer types have that computers should be hard. They seem to think it should be some sort of almost mystical priesthood that you have to work at for many years to be allowed in.
Bullshit.
Computers are tools, nothing more. they exist to allow humans to do tasks that we otherwise can't do, or at least can't do easily. As such they should be as easy and accessible for an average person to use. Ideally they would require no training and be usable by even extremely mentally challenged individuals. The more we can simplify them, the better. They should be adapted to work how we want, we should not have to adapt to them.
Well guess what? Programming is another part of that. Ideally, we'd have computers that could more or less program themselves. People would tell the computer what they wanted it to do in plain English (or other natural language) and it would figure out how to make that happen. Obviously we are a very long way away from this, but the easier we can make it, the better.
Even as it stands currently, where you do need training/practice to be a good programmer, there's a lot to be said for easy tools to make parts of development quicker and more robust. The user interface would be a good example. If all UI elements have to be coded in C++ and then compiled to see how it works, it is going to take a long time to develop and change. Goes double if others (like artists usability experts) are working on it as well. You write it, compile it, send it to them, they test it, write up problems, send it back, etc.
Much better to have a simple GUI interface for laying out the GUI. You can make changes much quicker and easier, and see what you are doing to confirm it is what you want. Also, should the design change, a redesign is much faster and easier.
I really get tired of this idea that computers and programming should be hard, that we don't want it accessible. Bullshit. You should want that in general, because it makes it available to more people, and even for you, because the ease of use can save you time. Yes, it allows for people to write programs that don't understand it. Deal with it. The microwave allows geeks everywhere to easily prepare food without understanding how to do it. Doesn't mean we should demand everyone become a master chef and cook all their food from only elementary ingredients. That will give you tastier food, but there's something to be said for having a meal ready in 5 minutes with 0 effort.
It seems everyone wants to be a "software engineer", but nobody wants to focus on the "hard stuff", and instead chant "let java/X do it for you".
I don't see the problem there.
Not every programmer you're going to run into is going to be a brilliant assembly level kernel hacker. Some of them (these days anyway) are going to be mediocre. Using libraries that a lot of people have looked at, found the bugs for, and documented so that the "hard stuff" works reliably gives these people a chance at success. Not everyone coding these days is some uberhacker. Code that works is really the bottom line here.
Reason being - programming has moved from a small niche position to an industry. And the demand for programming is large. And the number of people who can perform difficult tasks like coding in assembly is small. Wizards are rare and demand is larger than that. So how do you bridge that gap? Easy languages and tools and lots of libraries to increase the number of available programmers that can meet the demand. Let the gurus stick to the heavy stuff and let the mediocre programmers spend their time solving tasks in their ability range.
It's simply market pressure.
Weaselmancer
rediculous.
You need, to boldly add, more commas [where no commas, have been placed, before!]
No, it won't bite. It will segfault.
So if this is the future...where's my jet pack?
As another writer of crappy code:
:).
The advantage of using libraries is it means you write less code.
The less code you write, the fewer bugs you create, and less code you are directly responsible for fixing and documenting.
Yes the libraries won't be perfect, but in general they should be less crap than your code (especially if used and fixed by many others).
People who like "writing everything" themselves should use stuff like Lisp- programming languages that are powerful because they allow a programmer to personally write all sorts of stuff.
The rest of us should use languages that are powerful because they allow the programmer to NOT have to personally write all sorts of stuff
The "real programmers" can sneer at us, but we'll have completed the project way before they have finished writing the BIOS, bootloader, operating system, libraries and editor so that they can actually start writing the "real program"...
They're probably right; it probably would be better if they re-wrote it.
It's not because of what badasses they are or what terrible programmers their predecessors were, contrary to what most people believe in their heads. It's because it's easy to see what a program is supposed to do once it's done.
The reality is hardly any software is finished as the exact same project it started as. Requirements get changed at the worst possible times, the scope redefined, the timelines shortened. And yeah, at that point getting a working product out is the most important thing. Rewriting it probably WOULD result in better code, which may or may not be justification enough for doing it.
And hey, there's also a good chance those same things happen to them on their rewrite. Oops.
My main point here is that I have seen WAAAAAAAAAYYYYY too many arrogant programmers talking from their bum about how much better THEIR code would be if only THEY had a chance to rewrite it.
The same is true for everyone from novelists to plumbers to aircraft designers. The second time through generally yields a better result; it's just frequently not an option. This doesn't make anyone arrogant, it's just life.
The "real programmers" can sneer at us, [...]
That's not about it.
As long as you understand what you write, it's fine.
The problem is with the newer generation of Java/C# who: can't write their own algorithms thus inevitably depend on libraries, manage to have problems integrating the libraries together and (worst) do not understand how/why the stuff works.
the BIOS, bootloader, operating system, libraries and editor
That's a misconception about the "real programmers".
The difference between the "new programmers" and the "real programmers" is that later were still taught math and computer architecture - former were taught only syntax of a sandboxed programming language. Later know why/how software/hardware works at least in general, former have to rely on book which tell them that it would works.
Those who are actually try to reinvent "the BIOS, bootloader, operating system, libraries and editor" are not "real programmers": those are last remaining artifacts of the DOS times, the times when it all fit 64K.
All hope abandon ye who enter here.
I agree from a different view that goes like this. Good programmers/software engineers are expensive so companies like to pay cheaper people. To enable use of cheaper people tools and processes are introduced.
One such tool is languages that have buckets of libraries or core tools that the new people merely have to extend or tweak.
Knuth is lucky that most things he needed didn't exist then so using libraries wasn't an option. If everything he needed already existed he'd have picked a different area to work in.
You presume that a month or twenty down the line the original author can still maintain it. Most crappy code that doesn't holds as true.
Part of your skill as a developer is in applying the correct framework or library to the problem at hand. If your framework doesn't do the two or three critical things you want it to then I suggest you've chosen the wrong framework!
That's just not true.
My first task at my previous job was to speed up a perl script that took 12 hours to run. That meant that until you got the results, you were stuck. It meant that "tweaking" the input parameters required a lot of preparation work, or just wasted a huge amount of time.
I didn't know perl, but I knew how to program and how to optimise. It took me two days to learn perl (just enough for the task) and to reduce the runtime to 30 seconds (I was told about the original programmer "he's a scientist, not a programmer"). The result was still written in perl, by the way.
The improvement meant that the script could be used a hundred times a day with different parameters, something that would have taken months. Your suggestion is to buy a thousand machines to crunch the numbers instead of optimising the script. That just doesn't make sense to me.
I'd mod you up if I had points.
Most of the code I call "crappy code" is stuff I'm asked to look at because somebody found a reproducible defect, I look at the code, blink twice, and wonder how we haven't had a "oops, all of our data is gone" scenario yet. Fortunately I keep daily backups around...
And, of course, when I ask the original developer to "fix it", they can't seem to figure it out, even when giving step-by-step instructions how to reproduce the defect, over, and over, and over, and over.
I've looked at some of these CGI apps "web developers" put out, and I wonder if they have any brain cells between their two ears. Just using some sort of MVC architecture would be a good start to fixing 99% of the problems I see with shitty web code, and at least keep things somewhat more organized (if only just slightly)
no, you cannot maintain 100 computers for the price of a single programmer - because you'll have to hire (or pay) for the sysadmins to manage them. There's a lot of hassle involved in computers that you probably don't realise occurs between hardware and software setup, there's networking issues, updates and managed downtime. In most corporates, just having 1 extra computer will cost as much as that programmer in terms of meetings, requests, forms and project managers for the assignment of computer to project, plus security reviews, etc etc etc.
A factor of 1400 improvement is not ridiculous. I've seen similar - a colleague wrote some code, but becuase he didn't realise the interaction of his way convoluted OO code, he ended up doing 3 pages (that's printout) of DB queries for every update packet - of which he received around 3 a second. Slow is not the word. Fixing it involved a little cache and remembering the results of the first query he made instead of repeatedly asking the DB for the same data.
In programming the best version is generally the third.
The first is the "I don't know what I'm doing" version, which gets written by trying without much thought, ugly hacks, and without a decent design. Sometimes it does work quite well however, as despite not being very pretty it does what it's supposed to.
The second is "V1 is crap, but now that I have figured it all out I can do better!". Often a horrible mess, due to things like wanting to make everything modular, adding every feature possible, and using the latest cool tech and design patterns where they don't belong. Turns out to be slow, huge, buggy and overly complicated to use. There's even a name for this: "second system effect".
Based on the lessons learned from the lack of planning in the first and the excesses of the second, the third version has a good chance of being actually decent.
Few programmers start to write a program by designing a new OS, etc., so your analogy is simply a strawman. How about this car analogy: when designing a new car you are allowed to use only already existing auto parts; you are not allowed to custom design hardly any part on the "new" automobile. This is the real analogy with patching together existing library routines to build a new program: connecting together a set of already existing auto parts to make a "new" car. Does this better analogy still sound just fine? Obviously depends on what one is reusing and how complicated that was to engineer.
You know Programming is in trouble when being "the goto guy" has become a compliment, rather than an insult.
"My country, right or wrong; if right, to be kept right; and if wrong, to be set right." --Senator Carl Schurz (1872)