Slashdot Mirror
'Here Be Dragons': The Seven Most Vexing Problems In Programming (infoworld.com)
InfoWorld has identified "seven of the gnarliest corners of the programming world," which Slashdot reader snydeq describes as "worthy of large markers reading, 'Here be dragons.'" Some examples:
- Multithreading. "It sounded like a good idea," according to the article, but it just leads to a myriad of thread-managing tools, and "When they don't work, it's pure chaos. The data doesn't make sense. The columns don't add up. Money disappears from accounts with a poof. It's all bits in memory. And good luck trying to pin down any of it..."
- NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."
The other dangerous corners include closures, security, encryption, and identity management, as well as that moment "when the machine runs out of RAM." What else needs to be on a definitive list of the most dangerous "gotchas" in professional programming?
1 who's my customer 2 What does he or she actually want.
Did you know 80 to 90% of the moderators on slashdot wouldn't recognize a troll even if one dragged them under a bridge.
Oh christ if I get one more node.js developer claim that node is multi threaded...
http://michaelsmith.id.au
2 Developers who decide to reinvent the wheel because "They know best". (Just dealing with code where somebody decided I don't want to use the built in stuff, I'm going to do my own date time stuff which constantly has issue. Makes you pull your hair out.)
Did you know 80 to 90% of the moderators on slashdot wouldn't recognize a troll even if one dragged them under a bridge.
I'd like to second that #2 problem you mentioned. Easily the most annoying problem in programming I face is functional requirements not being fully specced out before the project begins and discovering well into it that I have to redo a month's worth of work when the requirements exceed the demands of how the code was originally written. In most cases, time is usually given for these changes, but sometime's it's not... and man, are those times infuriating.
File under 'M' for 'Manic ranting'
As far as I know, there are four hard problems in programming:
1. caching
2. naming (i.e. how do I name that variable/method/class)
3. off-by-one errors
And it's not that hard. Some people make it hard because they're incompetent. The golden rule of multithreading is: your multithreading set-up has to be very simple. Don't optimize (and for the love of god, unless you really understand cache coherence, don't use atomics), aim for clarity, use mutex locks for shared data, and if you have to use several, make sure you acquire them in the same order. 99.9% of the problems disappear if you follow this simple advice.
Haha, you should be more Agile! It's not Agile to find out what your customers need first. Just throw a bunch of shit at the wall, make it flatter, and dump it into production to let your customers do your testing and chip away at the things they complain about.
They'll definitely give you useful feedback and not switch to another software product instead. If there's one thing that's always been true about customers, it's that they're fiercely loyal to their software vendors.
> NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."
Customer: Why is this problem not scaling?
Programmer: It is NP-hard we are basically just brute forcing it.
Customer: If you can't solve this "NP-hard" problem, then I'll just find someone who will.
Programmer: Sure, let me know when that happens.
That is part of the Dunning-Kruger effect, exacerbated by the software industry's propensity to follow a Logan's Run-like purging of more experienced developers.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Saw this in C:
some_func(char *data) {
char *buffer;
buffer = malloc(200);
memcpy(buffer, data, 2000);
send_buffer(buffer);
free(buffer);
}
Occasionally it would cause a crash.
http://michaelsmith.id.au
Re: Serious he missed the 2 biggest problems I've h
Arbitrary truncation of data...
1 who's my customer
2 What does he or she actually want.
What you mention are not programming problems.
I am a programmer and I deal with these two issues way more than any problems with multi-threading and NP-completeness. Multi-threading isn't that hard if you have some experience and a good library. NP-Completeness is more of a math problem than a programming problem. For programming, there is no optimal way to do NP-complete, so you just use a heuristic for a "good enough" solution.
Dates, including time zones and daylight savings time. Constant source of bugs.
Only if you think you can apply Agile and 2 week sprints to *everything*, including the "thinky parts". What that leads to is incomplete software architecture, and class / data models that do no cater well to all use cases => nasty refactoring. I found that in a lot of software projects I participated in, as well as my own projects... I am a terrible cowboy coder.
However, I've had a few projects that have been very well designed from the get-go, with a thorough design that stands up to chaning requirements. Agile is a perfectly fine approach to develop or extend that kind of software.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
Closures are not a ``problem''. Ignorant or incompetent implementations of closures is the problem. ...
Just as ignorant or incompetent implementations of garbage collection and memory management is a problem.
And ignorant or incompetent use of arrays/vectors/buffers and pointer arithmetic and access permissions and
So here is an alternative list:
1. Low-skilled programmers (``script kiddies'') who write profound amounts of buggy code.
2. Low-skilled language ``designers'' who re-introduce the known bugs of the past and introduce new innovative bugs as well.
3. Low-skilled managers who reward high output over high quality, thus ensuring an on-time, under-budget hairball and bugs nest.
4. Low-skilled educators who teach ``coding'' rather than computer science, thus ensuring another generation of the above.
5. Low-skilled professional organizations who reward and encourage incompetent industry leaders to unduly influence the field.
6. Low-skilled investors who reward incompetent technology from dominant, monopolistic companies.
7. Low-skilled consumers who flock to buy flash-in-the-pan shiny stupid gimmicks but won't invest in sound technical innovation unless it's flashy.
This is why C students should never be allowed to graduate w/ a degree. They only go on to further muck up the world. Color me bitter. ;-)
P.S. That `C' above refers to grade level / professional competence, not the language (which should really be named D or D minus minus). *smirk*
Error: NSE - No Signature Error
Yes, this. the threading comment struck me more as indicative of an amateur in the area than any real flag of a problem area.
if you understand threading, you can work just fine within it, do appropriate things, don't do inappropriate things. You also have to understand the problem you're trying to solve, but again, that comes down to your skills - not any inherent "Dragon-ness" of threading.
I thread the heck out of my software. Works great. Cross-platform, too. Relatively easy stuff - image processing - and really hard stuff, real time signal processing where all manner of stuff is happening concurrently that depends in one way or another on the other stuff that's happening concurrently. I'm not talking about some thread off doing network management, either.
For me, it's strictly a matter of not going in there casually and carelessly. The appropriate planning and thinking - IOW, design first, then write software - pays off every time.
Of course, if you treat threading casually (as with many other things), it immediately becomes a footgun. Well, that's why Darwin gave us multiple feet, sonny. :)
But the whole point is we're supposed to be good at this. You have to pay your dues to get there. Programming Javascript on a web site isn't going to build the required skillsets.
I've fallen off your lawn, and I can't get up.
NP-Completeness becomes a programming problem when the people who do the programming cannot spot a NP-problem and then spend countless hours to beat it.
Put everything in one huge massive universal class of doom. Kiss refactoring goodbye!
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Easily the most annoying problem in programming I face is functional requirements not being fully specced out before the project begins
This is one of the most fundamentally misunderstood problems in software design. Probably because it is not really a software problem at all, but a business problem, and the solutions require intelligent business decisions which seem to be beyond everyone.
The solution to the specification problem is not more detailed design specs. The reason is simple. The people writing the detailed spec are not the people using the software, and they are not the people writing the software, so they are in no way competent at providing communications between the two. All of the successful software designs that I have seen are the result of competent programmers becoming involved in the completion of a task that software automation would apply very well to. Those people then subsequently write the software applying their first-hand knowledge of how the software will be used, and what the users needs are.
If you want your software to be very well designed, sit the programmers down to perform the job of the people who will use the software. They need to become proficient at this job, so a 1 week training course aint going to cut it. You need them to spend at least a month doing the job.
If you have the software developers do this, you will get exceptional results without the need for any kind of specification, or other real involvement, because the programmers are naturally very good at optimizing processes, otherwise they wouldn't be good at programming at all. The client gets the features they need, want, and didn't even know they could have. The software design itself is cleaner and faster because the developers have a fully fleshed out idea of what the software needs to do, and can design to meet the need instead of designing to meet a spec that may or may not remotely resemble what is feasible , practical, or correct.
I wish I had a good sig, but all the good ones are copyrighted
Well, that's because Agile is not supposed to solve technical issues like architecture or data models. It's supposed to help software development efforts track changes in organizational priorities and incorporate things they learn into their plans as they learn them.
If you rely too much on any methodology it's not going to work.
I have often thought the most important quality a software developer has to have is caring about the people who will be affected by their work. If you're only interested in technology you'll find an excuse to use the latest thing. If you're a prig about methodology you'll end up going through the motions. When you care about your users you'll always find a way to not let them down.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
One of the biggest, most vexing problems in computer science is newbies who equate basic programming to the hardest problems of the field.
OK, I realize this is arcane. Let me give an equivalent situation in, let's say, agriculture.
Farmer: "OK, young intern, today we'll learn how to water plants!"
Intern: "Waaaaah! This is as hard as the manipulation of space-time with thought alone!"
Farmer: "...Grab a goddamned watering can, moron."
Dammit, 30 seconds after posting that I realized it was even better as... I got .99999999 problems, and a float ain't one......
HA! I just wasted some of your bandwidth with a frivolous sig!
Ooh, dragons.
1. Multithreading. In some languages (e.g. Java) multithreading is no worse than any kind of parallel processing. Can you be a buffoon and fail? Yes you can. But you can do that in a single-threaded program too.
In other languages (e.g. C) the chance of errors impacting some completely random data structure are so high that it's frankly important to stay away from anything that would confuse the control flow, such as multithreading.
2. Closures. Okay, you're talking about Javascript here. javascript is a hideous language which should not be used to the maximum extent practical. Why complain about closers when all of javascript is bad mojo?
3. Too big data. Also known as lazy-ass programmers who don't want to be bothered to deal with the possibility that their program may be called upon to process a large data set. Solutions for processing large data sets explicitly on the disk (not with virtual memory) are well understood. You just have to choose to use them.
4. NP-complete. There's no particular dragon there... the elephant in the room is those programmers too ignorant or clueless to bother computing the big-oh running time of their algorithms, so they write software that appears to "lock up" as soon as anyone gives it a useful data set to process.
5. Security. Correct security implements defense in depth. Break one level so what? Even patched slowly, it's patched before enough layers can be broken to grant access.
Programmers make two classes of stupid mistake:
a. They assume they don't need defense in depth because they've written secure software.
b. They carelessly implement functionality that cuts through all layers of security. Like single-sign on.
6. Encryption. The odds of a long-surviving conspiracy to hide mathematical breakthroughs which bypass encryption are essentially zero. You can relax on that score. You can also expect one-time-pad encryption to remain unbreakable regardless of mathematical improvements. Because no analysis of extremely long strings of random bits is meaningful.
You have to be pretty bone-headed to mess up security in this day and age. Yes, many many programmers are bone-headed. But that's not encryption's fault.
7. Identity management. The only thing which makes this hard is its constant misuse. No authentication is perfect, so correct programming makes authenticated actions reversible instead. The more critical the action, the more critical it be reversible.
8. Measuring hardness. Now we're in to mumbo-jumbo territory. A simple big-oh evaluation will tell you whether your algorithm is usable. Trying to figure out whether it will be easier or harder to solve one problem or another, neither of which have a known useful algorithm? Silliness! Problems like that are solved when someone finally has the key insight. Inspiration is not a product of work, it's a product of the creative mind.
9. The authors can't measure. They claimed 7 but listed 8 things.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
They missed the most vexing problem of all. Hiring a skilled and talented old timer programmer to come solve the problems that fresh faced recruit coders created. Then offering the same rate as they paid the fresh faced recruits, not being able to grasp that the ability to solve, often under tightest of all deadlines, messes left by inexperienced programmers. In some cases less! I have personally had the extreme joy to explain to a physicist that the code he wrote, while it would solve his problem, was written in a manner that the solution was of order O n squared. While the problem could be solved in an manner that mathematically provided the same solution but was order O n. The physicist then said he wanted to test both solutions. His was actually faster on a data set of 100 items. (Lots of fixed constant cost was involved, my solution had more precalculation setup, so for small sets of data, my overhead was more) He proclaimed himself the winner deriding my science. I asked (already knowing) what the normal sample size was they analyzed? What, 100,000 samples? Sure let's use one of those... 20 minutes later mine was finished, I cleverly asked to run first since I lost the prior test. He ran his. It took a while. The next morning his finished. The results were the same. Save the physicist was a mixture of miffed and conciliatory. Very brilliant physicist. Not so great at mathematics. I've even encountered programs implemented in a manner to be non-deterministic when a deterministic solution existed. Vexing is working with some programmer that thinks they know everything. Which really annoys the ones that do.
For a spec to be that good it's got to be written to the same precision as a computer program.
So just write the fucking code.