Overcoming Intuition In Programming (amasad.me)

An anonymous reader writes: Amjad Masad, a programmer who works at Facebook, has put up a post about intuition and programming. It's based on a series of experiments (PDF) into how the presentation of a problem affects the learning involved in solving it. Researchers found that if they made a test deliberately hard to understand, those taking the test would exhibit greater understanding after solving it than those who were presented with a more intuitive wording of the same problem. Masad discusses how the research applies to software engineering: "Programming is an intellectually challenging task, but luckily we invent tools to make it manageable. I find that up to a certain point, the intuitive and easy properties of a given language, framework, or library might start to have negative effects.

From personal experience and from mentoring beginners I noticed that when using tools that allow us to reason within our intuition, anytime we're faced with some difficulty we feel that we've done something wrong. And although we might have the necessary skills to overcome the difficulty, we often start questioning and revising our work." He concludes, "Code reuse, libraries, sharing, and open-source are very important to software engineering, but we should be careful to not enable the belief that programming should be as easy as gluing things together."

Comments

Too Late

[chill]

"Code reuse, libraries, sharing, and open-source are very important to software engineering, but we should be careful to not enable the belief that programming should be as easy as gluing things together."

That is the management view of programming and a major corporate goal. This way it reduces the skills needed to complete the task, and hence you can pay less for the less skilled laborers.

Why do you think the average salary of a Windows Admin is lower than that of a Unix/Linux Admin? Because Microsoft pushed the "we've made it simple, just push the button" marketing drek and aimed it squarely at the management crowd -- who bought it hook, line and sinker.

"They made it easy, so I shouldn't have to pay you as much because anyone can do it. I'll just hire some kid with the latest MS cert..."

Re:Too Late

[gstoddart]

Right up until you meet the limits of what a low skilled person can do.

And then you can quickly get into uncharted territory which requires much more understanding.

Sure, that low skilled person can do "some", "many", or possibly even "most" of the tasks. And they can also drive you off a cliff through lack of understanding.

Anybody who has ever dealt with outsourced admins who can follow a script can tell you that when those people go outside of the script they will become utterly useless, if not outright dangerous.

And at that point, you're deeply screwed if there's not someone around who actually understands the rest of the stuff. It's not pretty to watch some noob with a little knowledge completely screw up a corporate environment.

Re:Too Late

[Anonymous Coward]

Why do you think the average salary of a Windows Admin is lower than that of a Unix/Linux Admin? Because Microsoft pushed the "we've made it simple, just push the button" marketing drek and aimed it squarely at the management crowd -- who bought it hook, line and sinker.

"They made it easy, so I shouldn't have to pay you as much because anyone can do it. I'll just hire some kid with the latest MS cert..."

It's more complicated than that. Salaries are lower because there's a herd of "kids with the latest MS Cert" that also bought into the "it's easy" line of thinking. It's not, but it sure looks that way. Managers are always looking to save a dollar of course, and many of them don't have any ability to distinguish talent from dreck. Even after they hire the dreck, they might be forced to put up with it as the "new normal".

It stems from a mentality of measuring salaries, but not measuring consequences of fucking up. Let's say John knows what the hell he's doing, but is paid $120,000 a year and rarely screws things up. John managed to increase productivity and save $50,000 last year through some innovative thinking. Joel is paid $50,000, but is constantly fucking shit up. Just last week Joel took down the network through one of his changes, and cost the company $20,000 in lost productivity. Over the past year Joel has cost the company approximately $200,000 in lost productivity due to his own point/click mentality and a management team that encourages this rather than learning and risk management.

So John costs $70,000 a year, and Joel costs $250,000 if you add up all the costs and savings from each. Yet if you only look at salaries, it looks like John should be replaced with 2 Joels, at a cost savings of $20,000!

So part of the problem is not measuring employees true worth.

This message brought to you by..

[DarkOx]

Researchers found that if they made a test deliberately hard to understand, those taking the test would exhibit greater understanding after solving it than those who were presented with a more intuitive wording of the same problem.

Paid for by the society of project managers and business software analysts.

Specialization

[Tony Isaac]

Code reuse, libraries, sharing, and open-source are very important to software engineering, but we should be careful to not enable the belief that programming should be as easy as gluing things together

This is the wrong conclusion. Most of the time, we should give preference to tools and practices that have already become best practices, without necessarily questioning each one every time. Of course, there is room for challenging best practices and libraries, but that's for people who are interested more in the best practices and libraries, than for the people trying to use them to create something useful to end users.

You don't need to know how to repair a car to drive one. The guy who repairs your car doesn't need to know how to build a motor or a transmission, only how to install them. The guy who assembles the motor doesn't need to know the finer points of metallurgy. The guy who refines the metals doesn't need to know the finer points of mining. Each of these stages of production can have their own issues that need to be resolved, but the guy driving the car needs to worry only about staying safe on the road and reaching his destination.

Programming should be the same way. I shouldn't have to debug the IP stack to make my program connect to the Internet, nor should I have to reinvent build systems to produce a product.

Specialize!

Re:Glueing things together is how I teach OO desig

[Austerity Empowers]

C weilding academic to me

Say what? I don't know too many (computer science) academics who'd touch C with a ten foot pole. Those of us who use C do it for real work (and like it/won't get off it).

If I want to live in an ivory tower I'd either a) use the latest theory compliant language/compiler du jour or b) refuse to code at all since that is an implementation issue, and instead pontificate on what is missing from latest language.

Re:Glueing things together is how I teach OO desig

[Rei]

Careful, C++ has its own share of unintuitive weirdness. Take the following example:

std::map<int, int> map;
map.emplace(1, 1);
auto iter = map.begin();
std::cout << iter->first << ", " << iter->second << std::endl;
map.emplace(0, 0);
std::cout << iter->first << ", " << iter->second << std::endl;

This prints out:

1, 1
1, 1

Nothing unusual there, right? std::map iterators are non-invalidating and you're not touching "iter", so it should be (and is) remaining the same. But what if we use reverse iterators (and correspondingly switch the side of the iterator we do the insert on)?

std::map<int, int> map;
map.emplace(1, 1);
auto reverse_iter = map.rbegin();
std::cout << reverse_iter->first << ", " << reverse_iter->second << std::endl;
map.emplace(2, 2);
std::cout << reverse_iter->first << ", " << reverse_iter->second << std::endl;

1, 1
2, 2

Despite sounding like the same sort of thing ("iterators"), forward and reverse operators have some very different properties in a key area. A non-invalidating forward iterator will always remain pointing to the exact same element regardless of what happens with other parts of the container. This does not apply to reverse iterators, as they are implemented in a rather unintuitive way - they actually point to the element after the element that they pretend to point to, and so changes to other elements can change what they appear to point to.

There's a lot of things like this in C++ that can slip past a person for years before it actually bites them. Don't get me wrong, I love C++ and think C is a rather dangerous language (from a memory safety standpoint) that requires that its authors reinvent the wheel over and over again. But C++ does have some weirdness in places that can pose hazards. For example, from a more beginner-perspective, what percentage of users have at one point been frustrated by trying to understand why a pointer in one of their classes is getting freed unexpectedly, due to not realizing the dangers of the implicit copy constructor/assignment operator when it comes to pointers? I bet that's bit almost everyone at some point in their career. Sure, you can "reason out" that that would happen, but most people learn it by being bitten once or twice.

Re:Glueing things together is how I teach OO desig

[Anonymous Coward]

Having just finished a stint in academia, this. This 1000 times over. I actually worked along side someone who didn't understand why python wasn't used for firmware programming going on about how using it would fix all sorts of issues. Trying to make him understand the problems with resource limitations was completely useless. He just couldn't grasp that infinity was just a concept that didn't actually exist.

Re:Oh you mean you want unintuitive code

[mwvdlee]

Yeah, only people who speak perfect english can ever be credible.
Fuck Einstein, Curie and all those stupid Greek philosophers.

Re:Glueing things together is how I teach OO desig

[toddestan]

Even if you only use (say) namespaces or a modest amount of ad-hoc overloading, C++ is a better C.

That's fine for your own projects, but the problem comes in with dealing with other's code. Everyone seems to have their own subset of C++ they like to use. The problem is that everyone's subset is a bit different. So you have to be at least aware of everything in C++, just so you know it when you run across it. (once again, this doesn't apply for your own projects)