Programmers for Scientific Research?

An AC submits: "We have recently had a lot of trouble trying to hire competent programmers for a research project. We are not programmers, so it is somewhat difficult to evaluate the competence of CS graduates. However, it seems that many of them are quite business oriented and that is not what we are looking for. On the other hand, the ones with adequate math skills etc. are often interested only in 'pure' CS. We would desperately need some kind of 'all-around lab hackers' capable of scientific problem solving, and confortable working with all sorts of software and hardware, but have not been able to find such persons. Does the Slashdot audience have any suggestions of how to attract the hackers having 'by programmers for scientists' - attitude?" [Update: 03/24 09:24 AM by michael : Note that although my email address is the only one on this submission, I'm not the one looking for programmers - I work for slashdot (although I used to program for a DOE laboratory, and that's why I found this question interesting and posted it). I've received a bunch of misdirected emails from people who thought this was interesting work, so if the submitter is reading this story, he/she might want to put contact information in a comment below.]

Comments

A few blunt comments from an old geek.

[bfwebster]

(1) Your quest is constrained in large part by the fact that only a small fraction of the population has any actual talent in software development and engineering.

(2) Even many of those individuals with talent have insufficient knowledge of (and/or, apparently, desire to learn about) the art and science of software engineering and so persist in making the same stupid mistakes that have been well-documented for 30+ years.

(3) As a result, anyone who has had to recruit software developers can tell you how much muck you have to sift through to find the gems.

(4) I can't speak for the relevance of most CS departments; I know that my undergraduate CS program (BSCS, BYU, 1978) helped me tremendously when I went out into the real world. But that may have been an anomoly; I had some brilliant teachers with real-world experience (one had worked at Bell Labs; another went on to co-author and co-found Word Perfect).

(5) After some years in the workforce, many of those with talent and skills find they can double or triple their salary by becoming a consultant. This leads to a talent-flight from organizations.

In short, you're trying to find someone with talent, training, inclination to your topics and circumstances, and a lack of awareness of how much s/he could be making elsewhere. :-)

Best of luck. ..bruce..

Classic problem -- you are not alone

[martyb]

There is not much detail in your problem statement, so there's much guesswork going on here, so please bear with me.

A key factor, I suspect, is your statement: "We are not programmers, so it is somewhat difficult to evaluate the competence of CS graduates."

The Problem: It strikes me you are running into the classic problem of not understanding what programming can do easily and what is difficult. More than likely, you've been sold a story from some instrument vendor that the XYZZY 2000 can do lots of wonderful things, and you really need it for what you are trying to do, and the best part is that it has a port that allows data aquisition and control from a computer!

Right. (Sarcasm intended.)

Here's a guess at what your situation is:

You've got a feel for the conceptual side of what you want to accomplish in the big picture, and the programming aspect is some vague, nebulous, "black box" where some programmer is supposed to come in and make it all work? But, things take MUCH longer than expected, and does not do what you want it to, and when the programmers try to explain what their difficulties are, it's so arcane (to you) that it sounds like they are just trying to evade their assigned responsibilities.

I've seen this KIND of problem countless times in my 25 years of experience programming, designing, and testing.

Suggestions:

• Learn Programming! A couple courses in Algorithms, Data Structures, and Numerical Computing, I think, would give YOU tremendous insight into the very real challenges of programming. Then, you can BEGIN to fairly assess and understand what you are trying to do.
• Hire a Seasoned Programming Manager It's one thing to hack together a couple of small little programs to do some ad hoc tasks. It's quite another thing to develop code that has solid error checking, has consistent user interface design, and that is also maintainable, reusable, and extensible. Hire someone who knows the difference and who can act as an interface between what you [think you] WANTt and the realities of what you really NEED.
• Design, Design, Design! Planning is Key! Step-wise refinement is Your Friend. This is a classic problem I've encountered where the thought is "We need this ASAP! We don't have time to plan it all out... can't you just do blah, blah, blah for now and we'll get to the rest, later?" Build a SOLID foundation in your design, and CLEARLY IDENTIFY the difficulties. If things are well-understood, the programming is effortless. Continual change requests and edge-case surprises will cost you far more time and energy in the long run than it would take to try and do it right the first time. (This IS difficult; human nature seems to want to look at the things we can do, and to avoid the things that are problematic.) In short, if it's hard to design, it's gonna be hard to code until you DO design it. Recognize and understand that truth and you'll have come a long ways!
• Hold Design and Code Reviews! Get involved, be open-minded, learn! Granted, the coding stuff may look like so much gibberish to you at first. I imagine calculus or particle physics did once, too. If you can learn THAT, you can learn some of this, too, if you want to and understand the need for it. Reviews provide a forum for a "meeting of the minds", an opportunity to openly share what challenges have been faced and the efforts employed to manage them. The abstract becomes concrete and there is a gradual growth of mutual understanding of the scientific challenges (so the developers better understand WHAT you are trying to accomplish - what is important and what is not) as well as an understanding of the programming challenges (NP-complete problems, inconsistent data formats, dirty/inconsistent data, bugs in the instrument's programming interface, etc.) Again, a seasoned programming manager should understand these issues and can help guide things along (but, from experience, DO NOT have the programming manager attend the code reviews! Really closes down communications when the guy/gal who has responsibility for hiring, firing, reviews, and promotions is sitting across the table -- problems just go unmentioned for fear of reprisal!)

Lastly, it has been my experience that there is a lot of truth in the adages: "The longest distance between two points is a shortcut." and "Good. Fast. Cheap. Pick any two, but you cannot have all three at once." (e.g. if you need it Right Now, and it has to be Bulletproof, it's going to cost you a lot.) I truly wish you well on your efforts and hope that something I've shared here from my own experience has been able to help.