Slashdot Mirror
Wiring Programmers To Prevent Buggy Code
mikejuk (1801200) writes "Microsoft Researcher Andrew Begel, together with academic and industry colleagues have been trying to detect when developers are struggling as they work, in order to prevent bugs before they are introduced into code. A paper presented at the 36th International Conference on Software Engineering, reports on a study conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity (EDA) sensor, and an electroencephalography (EEG) sensor could be used to predict whether developers would find a task difficult. Difficult tasks are potential bug generators and finding a task difficult is the programming equivalent of going to sleep at the wheel. Going beyond this initial investigation researchers now need to decide how to support developers who are finding their work difficult. What isn't known yet is how developers will react if their actions are approaching bug-potential levels and an intervention is deemed necessary. Presumably the nature of the intervention also has to be worked out. So next time you sit down at your coding station consider that in the future they may be wanting to wire you up just to make sure you aren't a source of bugs. And what could possibly be the intervention?"
Seems like we just covered this...
...make sure you go back to the business guy who create the requirements being coded at the time, hook him up to an EEG, and see if he wrote a buggy spec :) ...for bonus points, put the EEG on the manager who just said they're going to measure productivity in LOC.
If we could detect stupid with an EEG, programmers would probably be the least useful people to put it on.
The last thing I need when I am focused of a difficult chunk of code is Clippy popping up and breaking my focus. Some tasks are just much harder than others. I think any decent coder knows when they are struggling and increases their focus or grabs someone on their team to help. Bad coders may not, but bad coders are always bad coders, and no tools will help someone who just can't get it or just doesn't care.
Doing hard or unfamiliar tasks is one way we improve and grow. Mistakes improve our understanding of a domain.
Flagging code for extra review based on "struggle detection" *might* be useful. Sadly, we all know that we'd end up with clueless management punishing or even firing good people because they were stretching to meet a goal.
It seems to me that when I find code very easy is when it is probably wrong. When it is more difficult it gets more concentration and analysis. When it seems "this is dead simple" - that's probably when I missed something (maybe some edge cases not being taken into account or whatever).
Bugs come when people lose track of the big picture. When they lose concentration and focus - just like when jugglers drop the ball.
If you want to reduce the incidence of avoidable bugs, get rid of the distractions. Ones such as other people interrupting, phones ringing, asynchronous non job-related alerts going off (fire alarms excepted) and the administrivia associated with the programming environment. Maybe even unplug them from their "entertainment", too.
There will always be non-avoidable bugs: ones where the programmer is simply making a mistake, isn't up to the task in hand or has been given a bad brief or wrong information.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Just as easy to put a bug in simple code while you are blissed out on something else.
While completely ignoring that the specs handed down from the higher-ups are gibberish, contradictory, and physically impossible garbage. But, you know, that is not a possible source of bugs now is it?
Someone should first wire up management to zap them every time they get an idea for a "brilliant" addition.
What isn’t known yet is how developers will react if their actions are approaching bug-potential levels and an intervention is deemed necessary.
Intervention - you get fired.
In software development/IT/software "engineering" asking questions is considered to be a sign of stupidity. So, you better search and search the internet. Spend all your time learning the code - and KEEP your deadline.
We need to get a job done. Spinning one's wheels for hours, days, or whatever over something that can be answered in 5 minutes is idiotic, but that's what you gotta do or you're an idiot. I know, I have been called an idiot because I asked questions. I try to be efficient - reading is slow - POINT me to the area. But Nooooo! Gotta bust your balls!
When one hears after asking a question, "If you asked that then you don't belong here!!"
Well. We didn't get much done.
The PHBs fired us and hired H1-Bs - who I had to explain what those asterisks mean in C code.
Whatever, guys.
I walk into an interview and I'm told, "Tell me what the layers are for a network stack."
And I ask, "Well, whose opinion of what a network stack should be? Tanenbaum's? Afterall, it's just his opinion."
Blank stare.
I can't remember; it's been years since I had to memorize that. I say, "If it is really important, I can memorize all of that again."
I then ask, "Why, are you writing your own stack? I've done that."
"No."
Alrighty then.
Feedback from recruiter: "You do not have the skills."
You know, there are a bunch of people who are looking for work - people who know their stuff - and yet, they are dismissed.
And I see companies who say they cannot get qualified people - for their social network nonsense or whatever software.
Every bright kid I know who is good at math and science, I just say, "The only worthwhile STEM profession is the 'M'. Stay away from engineering, software, and tech. Medical."
Everyone in my family who is in medical from nurses to doctors have none of the problems we have in tech - none.
Difficult tasks are potential bug generators and finding a task difficult is the programming equivalent of going to sleep at the wheel....
( hmm. Easy tasks are potential bug generators and finding a task easy is the programming equivalent of going to sleep at the wheel... huh. that works too.)
^^
Like all the other sane people I stopped reading at that point.
The real reason for doing this is that someone has invested a lot of time and effort in the technology used for this (and has a lab) , and this idea was the best one he came up with.
Samsung took back my unlocked bootloader because Google wants me to rent movies. They're both evil.
Difficulty act as a motivator, and helps developpers to focus. Most bugs I've seen (or caused) are on easy, boring tasks.
The story has a good premise: Can we measure the programmer's emotional and cognitive states to predict when they're more likely to produce buggy code? That's a fair question. Where it loses it is when it jumps from that to the assumption that difficulty (and thus concentration) is the mental state in which bugs are produced. Hopefully that was just a case of the reporter missing the point.
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
That's a good point, and consistent with what I meant but didn't explain very well. Maybe "struggling" or some other word is better than "difficulty". The point being that the article talks about some symptoms that they're trying to identify, but they fail to discuss that those symptoms can all occur under normal circumstances when there is nothing that could/should be done (e.g., it's a good difficulty that encourages focus and the developer is working on something that is intrinsically difficult, or it's a bad difficulty and the developer is struggling on something that isn't very difficult because they're hungover or distracted because they had a terrible date last night).
Let's see... maybe they'll gather enough metrics on productivity and frequency of lapses in concentration and discover that the only thing that really correlates with those are the number of hours per week that they're working, previously recorded at an average of 35-40 hours per week, after which workers are no more productive but do less creative and/or innovative work. They might find out some interesting and novel details but the research into working practices is already pretty sound. It's just a pity that most executives, managers, and HR managers don't read it.
Bugs mostly come from when you're not paying attention, or you forgot something.
If you've doing difficult coding then writing code is the last thing you doing be doing. By the time you get to writing code it shouldn't be difficult anymore.
On the other hand, it might be useful to have an actual resident guru or two wandering around helping out anyone having trouble. I assume you believe you're considerably better than the average programmer, so imagine you're the guru who gets notified whenever one of the code-monkeys starts having problems. If you're as good as you think and have a decade or three of experience under your belt, then I suspect that more often than not you could make a couple key suggestions that would improve the rookie's code considerably. But they're never going to actually ask for such help - that would make them look incompetent. If they suffer from inflated egos they may even resent your assistance, but the code base will benefit greatly from your intervention nonetheless.
Even among gurus, if you're working on something difficult that may be a sign that this is a section that would benefit from teaming up with another guru to for some pair programming. Obviously *you're* never going to want to do such a thing, you're the guru! But having another sharp mind on hand and less focused on the line currently being edited may still catch a lot of the subtle bugs that might otherwise slip through.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Actually, as I recall the studies suggest that sustained work-weeks over the 35-40 hour threshold actually generate *negative* productivity - as in you get less total productivity out of a 60-hour week than a 40-hour week. A far worse situation than your phrasing suggests. I would actually suspect that in most situations hourly productivity starts to fall off much sooner, so that a 20-hour week would deliver more than half the productivity as a 40-hour one.
I can think of two important reasons a company (or economy) would be tempted to ignore such findings though:
1) Benefits, especially insurance. Thanks to government-mandated wage-fixing during the World War insurance has become a major part of most people's compensation packages, and those expenses will be the same regardless of time worked.
2) Wage fixing. As you reduce the hours worked you increase the number of employees needed, which gives the labor market more bargaining power. As automation caught on the normal US work week had been falling slowly but steadily in the US from ~12 hours per day to ~8, *without* a commensurate reduction in salary (i.e. accompanied by a 50% increase in hourly wages). That continued until early last century when federal regulations established 8 hours plus breaks and weekends as the norm, after which further reduction ground to a stop.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
A good question would be what exactly do they mean by "difficulty" in the study? If they're simply detecting increased stress or focus then you may be right; however, if they actually found some neurological/physiological response that correlates well with "increased chance of introducing a bug" then they would be zeroing in on exactly the right kind of "difficulty" that would suggest intervention.
I would suggest that, even if they're detecting you working "legitimately difficult code", if the intervention were to team up with someone at least as competent as yourself for the difficult part, then that could very well be a productive allocation of resources.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
That they continue to try and blame programmers for problems that management creates should bother people. Yes, this is a rewording of the same article. The only way to sell this alchemy (I would not even call it pseudo science) is by continually bleating out the theme "programmers are baaaad and we can detect them being baaaaad".
Lets just say (though its impossible for them to do) that they can isolate every possible variable that would cause fluctuations in EEGs, heart rates, and everything else they want to track biometrically. How on earth do they plan to track and correlate that data without either knowing or directing everything you eat, drink, and touch (including shampoo, soap, and hand lotion).
Even better, how do they plan to either direct or track all of your personal interactions which can impact those same things. If they can't, then the numbers are skewed and we don't have any science, you only have biases.
Oh, I can see it now. "Monty. We know your dad died yesterday but you are performing sub par and your heart rate is elevated higher than we expect, so we are going to have to let you go. We can't have a repeat of your three days of sub par performance when your wife divorced you five years ago."
Now lets look at some of the great Microsoft Management decisions, and decide who's to blame: Programmer or Management. Lync can not copy/paste data. Do you really think a programmer didn't notice the lack of basic function and point it out? Windows 8, and in fact Microsoft's whole "One UI" strategy is management driven. Zune, was actually a decent device but management killed the program. Programmers said Win8 was not ready, Management released it (as they did with Vista, ME, countless back office products, etc..). Programmers have had fixes for security ready only to have them pulled by management for various reasons. "Can't fix that IIS back door because someone's code may break" is frigging hilarious, but how many times have we read just that from Microsoft management?
That list could get really really long, so here is the point. Why are they not hooking management up to these things and putting them under pressure instead of the programmers? Probably for the same reason I start with, which is that this is alchemy and not science.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
Actually, having the guru ask for help from the rookie is a good way to engage the rookie. It's pretty motivating to the FNG to be able to say "hey, I really helped the smart guy, I must be doing something right!" Pair programming is one way to put them both in that situation.
John
Hire older programmer which learned a lot and do less buggy code in average, rather than every few year a generation of youngling which suffer from NIH and must relearn everything the older knew, just because you can underpay them for long hours (or even instead of outsourcing). Yeah I know, controversial shit.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
I've been coding, and studying to improve my skills, for decades. People come to me for help and advice. With one open source project I work on, all code must be approved by at least three people. My counterpart for one module is also very experienced, and a perfectionist. Sometimes our egos collide, but when we get past that we come up with something much better than either of us started with. That's particularly good when we're working on an architecture or API that otter people will have to code to in the future.
The procedure is just a variant of letting the programmer write a piece of code and then hook him/her up to a polygraph and interrogate her/him if the code has any bugs, and if it does, in which source file and line number....
"Does this code have any bugs?"
"Yes, like all non-trivial programs, this program has bugs too."
"Where is the bug?"
"If I knew, I would have fixed the bug already. But that's not the point. There's always one more bug that haven't been found yet."
Honestly, I find my buggiest code is the easiest crap that I'm just not paying attention to. The moment I have a solid challenge my interest peaks and getting it singing perfectly is the only acceptable end result. It may take a few tries but it's what I thrive on. I'm my least productive and most careless on the routine pointless tasks.
Or, rather than thinly disguised duckspeak, it could simply be that Management is still stuck in an 1890's industry mentaility where the longer you run the machines and the harder you run them, the more output you get and the sooner you get it.
Human creative endeavor, however, doesn't work like a hamburger grinder. It develops at its own rate, and not infrequently, any attempt to accelerate that rate will actually cause it to jam up. Rather like the old joke that if you pull on the paper as it comes out of the printer, it will print faster.
There are, in fact, inherently bad coders. Coding is just like anything else. Some people have the apitude, some do not.
I am an inherently bad bookkeeper, since I work by scanning the big picture and integrating it, and a meticulous line-by-line process will only result in my dropping lines.
I am an inherently bad sales person, because it's actually painful to me to approach people, and my persuasive skills are probably measured in negative numbers.
But I can execute prodigies in software coding without even straining.
This idea that anybody can do anything (except CEOs, who therefore require special levels of compensation) really needs to die. We're not all interchangeable cogs, no matter how hard you flog us.
Same here. Usually the coding mistakes occur in the easiest code, and are usually the easiest to detect and fix. The hard and undetected bugs are the ones that are the result of multiple pieces of code interacting in unexpected ways, easy, medium or hard at the individual code chunk level doesn't really matter.
The other source I have found is leaving unspecified paths open to users. You think that you don't have to prevent a user from doing something because it should work. It is actually more effort to prevent that use case. Then, you get bit in the ass because the user expects a different behavior, its not tested very well since it is unspecified, and often no one even really made a conscious choice to "add" the behavior which has effectively become a feature which now needs to be supported.