Slashdot Mirror
Researchers Test Developer Biometrics To Predict Buggy Code
rjmarvin writes: Microsoft Research is testing a new method for predicting errors and bugs while developers write code: biometrics. By measuring a developer's eye movements, physical and mental characteristics as they code, the researchers tracked alertness and stress levels to predict the difficulty of a given task with respect to the coder's abilities. In a paper entitled "Using Psycho-Physiological Measures to Assess Task Difficulty in Software Development," the researchers summarized how they strapped an eye tracker, an electrodermal sensor and an EEG sensor to 15 developers as they programmed for various tasks. Biometrics predicted task difficulty for a new developer 64.99% of the time. For a subsequent tasks with the same developer, the researchers found biometrics to be 84.38% accurate. They suggest using the information to mark places in code that developers find particularly difficult, and then reviewing or refactoring those sections later.
Every time I hear about a terrifyingly invasive means of "improving performance" its targeted at developers. Is it just selection bias, or does the world actually hate us?
the researchers summarized how they strapped an eye tracker, an electrodermal sensor and an EEG sensor
I'm sure being hooked up to something similar to a polygraph doesn't make a developer more stressful at all. Was the fact that they had all this equiptment hooked up to them a factor in their statistics?
When your developers cringe at a project, when they encounter a subroutine or callback that literally makes them groan, you've found exactly what needs to be refactored. if you find a python wrapper around a godforsaken class, or find explitives cursing a dead gods name in a forgotten universe, thats the code that needs your attention. Project managers, section leaders, whoever has direct line-of-sight communication with the dev pit needs to pay more attention.
the problem is 'refactoring' is a lie. as a DevOps (christ i hate that fucking word) engineer, I've been faced with rotting festering codebases for years in my career on a daily basis. the issue is business priorities interfering with good coding practices. I and 2 junior devs might want to go rip up a few thousand lines of horror-code to make everyone more productive, but we get denied. why?:
1. downtime is unacceptable for this application. this code controls so much, does so many things, and is so obscure (say it with me, payments processing subsystem) that to do ANYTHING to it is literally worse than pistol whipping the CEO's daughter.
2. New New NEW! we need to get in those swim lanes and stand up in those scrums nice and straight so we can deliver optimum ROI to our dear customers! who cares if the system crashes 5 times a month because this module is satans petrified asscrack, google just launched their new $app so our new $cloud_app_pro needs to go live NOW!.
3. we had the resources, but uber elite coders in our ranks were ganked to other projects months ago. they havent seen the code in 3 months, and we're sure they'll be along to help us again once they put in their 2 weeks and show up in flip flops for the knowledge transfer.
4. you were ganked from the refactor project and are now plugging away at an irritating new web 9.0 cash money matic piece of code that marketing wont stop skullfucking and your boss cant deliver fast enough. Catch this rabbit though and you'll be able to sit down and think through...wait....what was the refactoring project about again? oh christ is that CVS?
what this technology will get used for
efficiency sampling in your dev groups. eye tracking and biometrics will now subtly be included in SCRUM/ITIL/six sigma/devops/management wankfest.
Good people go to bed earlier.
They tested 16 developers and gave statistics with four significant figures. I think you would need to test at least 100,000,000 developers to get such precise measurements. Who do they think they are? Dr. Spock on Star Trek?
We don't see the world as it is, we see it as we are.
-- Anais Nin
They tested 16 developers and gave statistics with four significant figures. I think you would need to test at least 100,000,000 developers to get such precise measurements. Who do they think they are? Dr. Spock on Star Trek?
Naw, they just used a really accurate ruler, made each measurement 10 times and averaged their results...
You make an excellent point. There is no indication in the fine article about how accurate their results could be statistically, and given their really small sample size it doesn't seem likely 4 significant digits is justified.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
well duh, that vein on my forehead starts to throb and my eye starts to twitch when I read dumb code.
oh wait, that was my commit
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Example?
That's where we're at?
I see this garbage all the time... and I never understand it. Growing up my father ran a factory and was damned good at it. His people showed up on time, did great work, had low scrap and were very productive. How did he manage this amazing feat? Minders that followed you everywhere in the factory? Discreet blood samples taken hourly? No...
They had stats. That's it. You produce X parts per week. Go way above that get a bonus, go way bellow that, you get fired. If everyone is getting bonuses the stats would rise... if everyone was getting low stats they'd first check for procedural problems that might be hindering peoples work and baring that they'd lower their expectations. It worked marvelously well... people would think of ways to go faster and bring them up... because it meant a bonus until everyone else caught on. Anything that made production harder was immediately reported because people wanted their bonus.
Damn near every successful factory in the country works this way. Do the same thing for code. What my father always said was "They could spend 7hrs in the bathroom every day... I don't care... if they can come out at hit 1000% efficiency that last hour to make up the difference it's fine with me. But they better keep in mind their peers are going to eventually figure out how they pulled it off and change the curve."
1) Arrogance. You know that average developers have a hard time with some kinds of code, but you're a superprogrammer, and you don't have those problems. If someone decides later that there's something wrong with your code, well, they should've gotten their requirements straightened out before they told you to go and build it. The only time you lose your cool is when you have to deal with idiot managers, analysts, or users.
1) Complacency. You've been pounding on this code forever, and you just don't care any more. Yeah, there'll be bugs, people will yell, they'll get fixed. That's just the way development goes. Why get worked up about it?
I think it the extra bogus precision has something to do with the conversion between Imperial and Metric developers.
(Oh, and something something dark side.)
They can understand how a toilet is cleaned, how a sale is made, how a 1099 is filled out, how a fire drill works, how a sandwich is put together, how oil is changed, etc... but Coding might as well be a dark art.
Disclaimer: I am in hardware myself and may completely miss the point here. However, our software/firmware folks do agile programming involving dividing programming problems into pieces which are assigned to programmers, followed-up on large whiteboards and being daily discussed in "scrum meetings" etc. (I may be confusing some concepts here but that is of less importance). The point being that your statement, that programming is some sort of unique dark-art-which-cannot-be-measured-by-managers, appears untrue to me and, honestly, rather pedantic. What these guys are doing is quite measurable. Maybe not by a silly measure like "lines of code", but by the measure of number of problems being solved, having a complexity that apparently everyone of them agrees on.
Indeed, the CEO doesn't know the exact details of how this works, but neither does he personally count the number of cleaned toilets.