Overcoming Intuition In Programming (amasad.me)

An anonymous reader writes: Amjad Masad, a programmer who works at Facebook, has put up a post about intuition and programming. It's based on a series of experiments (PDF) into how the presentation of a problem affects the learning involved in solving it. Researchers found that if they made a test deliberately hard to understand, those taking the test would exhibit greater understanding after solving it than those who were presented with a more intuitive wording of the same problem. Masad discusses how the research applies to software engineering: "Programming is an intellectually challenging task, but luckily we invent tools to make it manageable. I find that up to a certain point, the intuitive and easy properties of a given language, framework, or library might start to have negative effects.

From personal experience and from mentoring beginners I noticed that when using tools that allow us to reason within our intuition, anytime we're faced with some difficulty we feel that we've done something wrong. And although we might have the necessary skills to overcome the difficulty, we often start questioning and revising our work." He concludes, "Code reuse, libraries, sharing, and open-source are very important to software engineering, but we should be careful to not enable the belief that programming should be as easy as gluing things together."

Comments

Too Late

[chill]

"Code reuse, libraries, sharing, and open-source are very important to software engineering, but we should be careful to not enable the belief that programming should be as easy as gluing things together."

That is the management view of programming and a major corporate goal. This way it reduces the skills needed to complete the task, and hence you can pay less for the less skilled laborers.

Why do you think the average salary of a Windows Admin is lower than that of a Unix/Linux Admin? Because Microsoft pushed the "we've made it simple, just push the button" marketing drek and aimed it squarely at the management crowd -- who bought it hook, line and sinker.

"They made it easy, so I shouldn't have to pay you as much because anyone can do it. I'll just hire some kid with the latest MS cert..."

Re:Too Late

[DigiShaman]

Which is funny, because in the world of Windows Administration, either you're a complete idiot, or an experienced Windows sage that's very good at cleaning up other peoples mess, and running the network properly. There is no middle ground unless you've gained the experience under the wing of such a sage.

Re:Too Late

[gstoddart]

Right up until you meet the limits of what a low skilled person can do.

And then you can quickly get into uncharted territory which requires much more understanding.

Sure, that low skilled person can do "some", "many", or possibly even "most" of the tasks. And they can also drive you off a cliff through lack of understanding.

Anybody who has ever dealt with outsourced admins who can follow a script can tell you that when those people go outside of the script they will become utterly useless, if not outright dangerous.

And at that point, you're deeply screwed if there's not someone around who actually understands the rest of the stuff. It's not pretty to watch some noob with a little knowledge completely screw up a corporate environment.

Re:Too Late

[Anonymous Coward]

Why do you think the average salary of a Windows Admin is lower than that of a Unix/Linux Admin? Because Microsoft pushed the "we've made it simple, just push the button" marketing drek and aimed it squarely at the management crowd -- who bought it hook, line and sinker.

"They made it easy, so I shouldn't have to pay you as much because anyone can do it. I'll just hire some kid with the latest MS cert..."

It's more complicated than that. Salaries are lower because there's a herd of "kids with the latest MS Cert" that also bought into the "it's easy" line of thinking. It's not, but it sure looks that way. Managers are always looking to save a dollar of course, and many of them don't have any ability to distinguish talent from dreck. Even after they hire the dreck, they might be forced to put up with it as the "new normal".

It stems from a mentality of measuring salaries, but not measuring consequences of fucking up. Let's say John knows what the hell he's doing, but is paid $120,000 a year and rarely screws things up. John managed to increase productivity and save $50,000 last year through some innovative thinking. Joel is paid $50,000, but is constantly fucking shit up. Just last week Joel took down the network through one of his changes, and cost the company $20,000 in lost productivity. Over the past year Joel has cost the company approximately $200,000 in lost productivity due to his own point/click mentality and a management team that encourages this rather than learning and risk management.

So John costs $70,000 a year, and Joel costs $250,000 if you add up all the costs and savings from each. Yet if you only look at salaries, it looks like John should be replaced with 2 Joels, at a cost savings of $20,000!

So part of the problem is not measuring employees true worth.

Re:Too Late

[yagu]

There should be a moderation category "Scary".

Re:Glueing things together is how I teach OO desig

[Rei]

Careful, C++ has its own share of unintuitive weirdness. Take the following example:

std::map<int, int> map;
map.emplace(1, 1);
auto iter = map.begin();
std::cout << iter->first << ", " << iter->second << std::endl;
map.emplace(0, 0);
std::cout << iter->first << ", " << iter->second << std::endl;

This prints out:

1, 1
1, 1

Nothing unusual there, right? std::map iterators are non-invalidating and you're not touching "iter", so it should be (and is) remaining the same. But what if we use reverse iterators (and correspondingly switch the side of the iterator we do the insert on)?

std::map<int, int> map;
map.emplace(1, 1);
auto reverse_iter = map.rbegin();
std::cout << reverse_iter->first << ", " << reverse_iter->second << std::endl;
map.emplace(2, 2);
std::cout << reverse_iter->first << ", " << reverse_iter->second << std::endl;

1, 1
2, 2

Despite sounding like the same sort of thing ("iterators"), forward and reverse operators have some very different properties in a key area. A non-invalidating forward iterator will always remain pointing to the exact same element regardless of what happens with other parts of the container. This does not apply to reverse iterators, as they are implemented in a rather unintuitive way - they actually point to the element after the element that they pretend to point to, and so changes to other elements can change what they appear to point to.

There's a lot of things like this in C++ that can slip past a person for years before it actually bites them. Don't get me wrong, I love C++ and think C is a rather dangerous language (from a memory safety standpoint) that requires that its authors reinvent the wheel over and over again. But C++ does have some weirdness in places that can pose hazards. For example, from a more beginner-perspective, what percentage of users have at one point been frustrated by trying to understand why a pointer in one of their classes is getting freed unexpectedly, due to not realizing the dangers of the implicit copy constructor/assignment operator when it comes to pointers? I bet that's bit almost everyone at some point in their career. Sure, you can "reason out" that that would happen, but most people learn it by being bitten once or twice.

Re:Odd title

[gstoddart]

Yeah, I always found coding (and especially debugging) required a level of intuition ... precisely because it was more than just gluing pieces together.

I understand you don't want to rely too much on intuition, because it's hard to sound like anything other than voodoo, but sometimes the voodoo is still a real thing.

I worked with someone years ago who liked to go on about how everything should be abstracted and pretty/elegant according to whatever was popular that month. He read the books and magazines incessantly, and wouldn't shut up about them.

The problem is he often wrote shit code he couldn't maintain or debug because he'd abstracted things so much it was impossible for him to follow his own code, or know where to look when things went wrong. A small enhancement request left him squealing how the code wasn't designed to do that and he'd have to rebuild it. Meanwhile the rest of us went "so, all of that is in here, and if I just nudge this a little it's all done".

I'm sure he got better over time, but for someone who was so loudly a proponent of the latest language theories and methodologies, he never seemed to understand how his neat intellectual model in no way translated into maintainable, readable, or sometimes even useful code. But his insistence on following all of these things usually had the result of him making absolutely terrible design choices.

These frameworks and methodologies sound awesome on paper, but you can still use them to write complete garbage code which is brittle, inflexible, and often completely wrong for what you're trying to use it for.

Whereas the guys who learned to program and debug without the syntactic sugar and frameworks to build upon, those guys tended to have a bigger picture view of the pieces. Which means you can zero in on where you think it likely went sideways instead of staring blankly wondering why your monument to methodology is now a teetering mess you have no idea where to begin with when there's a problem.