Slashdot Mirror
'Here Be Dragons': The Seven Most Vexing Problems In Programming (infoworld.com)
InfoWorld has identified "seven of the gnarliest corners of the programming world," which Slashdot reader snydeq describes as "worthy of large markers reading, 'Here be dragons.'" Some examples:
- Multithreading. "It sounded like a good idea," according to the article, but it just leads to a myriad of thread-managing tools, and "When they don't work, it's pure chaos. The data doesn't make sense. The columns don't add up. Money disappears from accounts with a poof. It's all bits in memory. And good luck trying to pin down any of it..."
- NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."
The other dangerous corners include closures, security, encryption, and identity management, as well as that moment "when the machine runs out of RAM." What else needs to be on a definitive list of the most dangerous "gotchas" in professional programming?
2 Developers who decide to reinvent the wheel because "They know best". (Just dealing with code where somebody decided I don't want to use the built in stuff, I'm going to do my own date time stuff which constantly has issue. Makes you pull your hair out.)
Did you know 80 to 90% of the moderators on slashdot wouldn't recognize a troll even if one dragged them under a bridge.
That is part of the Dunning-Kruger effect, exacerbated by the software industry's propensity to follow a Logan's Run-like purging of more experienced developers.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Saw this in C:
some_func(char *data) {
char *buffer;
buffer = malloc(200);
memcpy(buffer, data, 2000);
send_buffer(buffer);
free(buffer);
}
Occasionally it would cause a crash.
http://michaelsmith.id.au
Microsoft wrote a bunch of code, that in hindsight was a really really bad idea. I'm not really sure that the problems were all Microsoft's fault - they probably didn't invent them. They are Microsoft's fault in that they embedded the concepts into the operating system and then widely popularized them.
The big gotchas all have to do with limitations of C, and the twists used to optimize Microsoft's programming for the 8086 architecture. I'm thinking of:
char string[MAX_PATH];
What can possibly go wrong? Especially before MAX_PATH was invented.
malloc() new() and friends
For any non-trivial program, it's almost impossible to get correct. They only really work for something like a C compiler which will allocate a variable amount of memory, do a task, and then end. If you miss a few free() calls along the way - they will be cleaned up when the compiler terminates.
GlobalAlloc(), LocalAlloc() and friends
Just in case you couldn't make it work with malloc, try the operating system version of the call. Using OS calls really opened the window to some famous bugs - OpenSSH comes to mind.
Multi-threading in C
For any non-trivial problem - it doesn't work. Firstly, for any non-simple piece of code, it is tough to do correctly. Secondly, for a trivial piece of code, like a save operation, you need to somehow make the memory being stored immutable for the duration of the save. For most save operations, this defeats the purpose, as the first thing the user wants to do after save is to change the document. Thirdly, if you really need multi-threading, there is a good chance some of your users need parallel-processing across machines. Multi-threaded code and parallel-processing code are not the same things at all.
Embedding code in data.
Firstly, the x86 architecture encouraged this, by not having proper page protections. Secondly, it completely opens the system up to malicious code. ActiveX, JPEG libraries, font libraries, everywhere Microsoft had an API that embedded code in data, it was all exploited.
The best but if advice I've ever had for variable naming is to put the units in the name. If you are handling values in millimetres, use a _mm suffix. For area use _mm2. For graphics use _px for pixels etc.
I've seen so many programmers get hopelessly confused because they can't remember what units they have when doing maths or trying to render data on screen. On some systems pointers can use byte or word addressing too.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
C'mon. Threads -- I concur wholeheartedly
I've never understood why programmers have problems with threads. Almost everything is one of two models:
* Solve the problem as a series of queues with worker thread pools. Really hard to mess that up.
* Entire request-to-response workflow in a thread for each request. You have to be a bit careful about locking around shared objects, but usually the answer is avoiding shared objects.
If you can't get multiple threads on one machine right, how are you going to work on distributed systems?
Socialism: a lie told by totalitarians and believed by fools.
If you understand pointers and malloc(), you can work just fine within them too. But that hasn't stopped people from disparaging them relentlessly and trying to come up with new "safe" languages that don't have pointers and memory allocation.
Added irony: With pointer and memory allocation bugs, the problems are at least reproducible. Can't say that about threading bugs.
still cannot figure out why the Java developers thought they needed to do away with pointers and (application-controlled) memory allocation, but then turned around and encouraged the use of threads. It's kind of like saying to children: "Don't play with those scissors, you might cut yourself. Use the power saw instead!"
Easily the most annoying problem in programming I face is functional requirements not being fully specced out before the project begins
This is one of the most fundamentally misunderstood problems in software design. Probably because it is not really a software problem at all, but a business problem, and the solutions require intelligent business decisions which seem to be beyond everyone.
The solution to the specification problem is not more detailed design specs. The reason is simple. The people writing the detailed spec are not the people using the software, and they are not the people writing the software, so they are in no way competent at providing communications between the two. All of the successful software designs that I have seen are the result of competent programmers becoming involved in the completion of a task that software automation would apply very well to. Those people then subsequently write the software applying their first-hand knowledge of how the software will be used, and what the users needs are.
If you want your software to be very well designed, sit the programmers down to perform the job of the people who will use the software. They need to become proficient at this job, so a 1 week training course aint going to cut it. You need them to spend at least a month doing the job.
If you have the software developers do this, you will get exceptional results without the need for any kind of specification, or other real involvement, because the programmers are naturally very good at optimizing processes, otherwise they wouldn't be good at programming at all. The client gets the features they need, want, and didn't even know they could have. The software design itself is cleaner and faster because the developers have a fully fleshed out idea of what the software needs to do, and can design to meet the need instead of designing to meet a spec that may or may not remotely resemble what is feasible , practical, or correct.
I wish I had a good sig, but all the good ones are copyrighted
I was taught, for a spec to be good, you could give it to two different developers and get the exact same result. Otherwise, you need to improve the spec.
A related problem is that there is no way for the hiring manager to tell the difference between a very good programmer and a good programmer. All of the first year CS resumes say the candidates are skilled in C, Java, and a few more buzzwords. Every software project has a life-cycle, and the better programmers move between projects. This means all the experienced resumes show people with many projects. This makes it hard to tell the difference between a competent person that makes meaningful contributions then moves on, and a less competent person that is a good talker and drifts.
It's a real problem for the hiring manager. It essentially means that you can't pay more for better programmers. Economists even have terms for this. Product differentiation. If you can't show you are an A+ programmer as opposed to an A programmer or a C programmer, then it becomes difficult to make the case that the A+ programmer is worth four C programmers. It even becomes difficult to tell who the A+ programmers are. This tends to drive out experienced programmers into other careers.
This problem really affects software engineers. I can tell the difference between a skilled electrician and an unskilled electrician in minutes. Great architects win awards. Develop a great piece of real-time engineering on a safety-critical system - no one is ever going to see your work. Sure, you might save lots of lives with your code. But the hiring manager for your next job can't tell the difference.
Ooh, dragons.
1. Multithreading. In some languages (e.g. Java) multithreading is no worse than any kind of parallel processing. Can you be a buffoon and fail? Yes you can. But you can do that in a single-threaded program too.
In other languages (e.g. C) the chance of errors impacting some completely random data structure are so high that it's frankly important to stay away from anything that would confuse the control flow, such as multithreading.
2. Closures. Okay, you're talking about Javascript here. javascript is a hideous language which should not be used to the maximum extent practical. Why complain about closers when all of javascript is bad mojo?
3. Too big data. Also known as lazy-ass programmers who don't want to be bothered to deal with the possibility that their program may be called upon to process a large data set. Solutions for processing large data sets explicitly on the disk (not with virtual memory) are well understood. You just have to choose to use them.
4. NP-complete. There's no particular dragon there... the elephant in the room is those programmers too ignorant or clueless to bother computing the big-oh running time of their algorithms, so they write software that appears to "lock up" as soon as anyone gives it a useful data set to process.
5. Security. Correct security implements defense in depth. Break one level so what? Even patched slowly, it's patched before enough layers can be broken to grant access.
Programmers make two classes of stupid mistake:
a. They assume they don't need defense in depth because they've written secure software.
b. They carelessly implement functionality that cuts through all layers of security. Like single-sign on.
6. Encryption. The odds of a long-surviving conspiracy to hide mathematical breakthroughs which bypass encryption are essentially zero. You can relax on that score. You can also expect one-time-pad encryption to remain unbreakable regardless of mathematical improvements. Because no analysis of extremely long strings of random bits is meaningful.
You have to be pretty bone-headed to mess up security in this day and age. Yes, many many programmers are bone-headed. But that's not encryption's fault.
7. Identity management. The only thing which makes this hard is its constant misuse. No authentication is perfect, so correct programming makes authenticated actions reversible instead. The more critical the action, the more critical it be reversible.
8. Measuring hardness. Now we're in to mumbo-jumbo territory. A simple big-oh evaluation will tell you whether your algorithm is usable. Trying to figure out whether it will be easier or harder to solve one problem or another, neither of which have a known useful algorithm? Silliness! Problems like that are solved when someone finally has the key insight. Inspiration is not a product of work, it's a product of the creative mind.
9. The authors can't measure. They claimed 7 but listed 8 things.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
They missed the most vexing problem of all. Hiring a skilled and talented old timer programmer to come solve the problems that fresh faced recruit coders created. Then offering the same rate as they paid the fresh faced recruits, not being able to grasp that the ability to solve, often under tightest of all deadlines, messes left by inexperienced programmers. In some cases less! I have personally had the extreme joy to explain to a physicist that the code he wrote, while it would solve his problem, was written in a manner that the solution was of order O n squared. While the problem could be solved in an manner that mathematically provided the same solution but was order O n. The physicist then said he wanted to test both solutions. His was actually faster on a data set of 100 items. (Lots of fixed constant cost was involved, my solution had more precalculation setup, so for small sets of data, my overhead was more) He proclaimed himself the winner deriding my science. I asked (already knowing) what the normal sample size was they analyzed? What, 100,000 samples? Sure let's use one of those... 20 minutes later mine was finished, I cleverly asked to run first since I lost the prior test. He ran his. It took a while. The next morning his finished. The results were the same. Save the physicist was a mixture of miffed and conciliatory. Very brilliant physicist. Not so great at mathematics. I've even encountered programs implemented in a manner to be non-deterministic when a deterministic solution existed. Vexing is working with some programmer that thinks they know everything. Which really annoys the ones that do.
Agile is meant to help you learn about the problem domain. If you already know everything about the problem, there's no need for Agile. If somebody else already knows everything about the problem and you can hire him/her, the do it - no need for Agile. If nobody knows everything about the problem because it's a new problem that no one else has worked on yet or a new approach to a known problem, then you definitely need Agile or something similar to help you produce something that works at the same time that you are learning about the problem.