(Over-)Measuring the Working Man

HughPickens.com writes: Tyler Cowen writes in MIT Technology Review that the improved measurement of worker performance through information technology is beginning to allow employers to measure value fairly precisely and as we get better at measuring who produces what, the pay gap between those who make more and those who make less grows. Insofar as workers type at a computer, everything they do is logged, recorded, and measured. Surveillance of workers continues to increase, and statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace. Consider journalism. In the "good old days," no one knew how many people were reading an article, or an individual columnist. Today a digital media company knows exactly how many people are reading which articles for how long, and also whether they click through to other links. The result is that many journalists turn out to be not so valuable at all. Their wages fall or they lose their jobs, while the superstar journalists attract more Web traffic and become their own global brands.

According to Cowen, the upside is that measuring value tends to boost productivity, as has been the case since the very beginning of management science. We're simply able to do it much better now, and so employers can assign the most productive workers to the most suitable tasks. The downsides are several. Individuals don't in fact enjoy being evaluated all the time, especially when the results are not always stellar: for most people, one piece of negative feedback outweighs five pieces of positive feedback.

Comments

Clickbait wins

[king neckbeard]

This metric would seem to be encouraging authors that write the clickbaitiest titles. Sounds wonderful.

Re:Clickbait wins

[nbauman]

This metric would seem to be encouraging authors that write the clickbaitiest titles. Sounds wonderful.

Cowen's assumption is that the things that are easy to measure are also meaningful and contribute to the organizations' success. In journalism, that's not usually true.

I once worked for a weekly newspaper in Brooklyn. The publisher had more (family) money than brains. The paper was doing badly and he was getting desperate.

One week, we ran a British-style tabloid headline, "4-year-old girl raped, murdered." It had the highest newsstand circulation of any issue.

This was followed by a wave of angry subscription cancellations. One subscriber wrote, "I don't want this garbage coming into my house."

The paper continued on its downward trend and finally shut down.

The moral: following a meaningless metric can give you a boost in the short term, and harm the enterprise in the long term.

Like VW's emission test results....

Re:Clickbait wins

[gweihir]

Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

Re:Clickbait wins

[tlhIngan]

Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

The problem with metrics is, well, you get what you measure.

And for a lot of metrics, what you measure is not what you're wanting to measure, because what you want to measure is vague and not easily measureable. E.g., productivity is not something you can measure. You can't take two people and easily say that one is more productive than the other.

And there are other metrics you CAN measure directly, but the cause and effect are so far removed from the people that it's impossible to actually determine who contributed. E.g., savings. Sure you can say the company saved $100,000 this quarter, but which employee did the most?

And even then, there are hidden consequences.

As I said above, you get what you measure. If you measure savings, you will get savings. But you may also have to deal with the consequences of it - perhaps regular maintenance becomes irregular and you're setting yourself up for a catastrophic failure in a couple of years. But since you'll have 8 quarters of meeting the metric, it doesn't matter if the failure costs more than what was saved. Or you'll find people will struggle with obsolete equipment way beyond its replacement date to make the numbers come in lower.

Likewise, if you try to measure productivity, people will make the measurement. If you measure keystrokes, yes, you'll find keystrokes increase. But that might mean "space" and "backspace" are the most popular keys.

If you measure lines of code, you'll find a lot of copy-pasta coding.

And so on - you will get what you measure. You will not get what you don't measure. And the weaker the relationship between what you REALLY want to measure and the proxy you're using, the more useless the measure is.

Data? Statistics?

[war4peace]

Now wait a second. Management, generally speaking, doesn't understand data. They like to look at a nice Powerpoint slide with people ranked from best to worst. this means that all measures taken relative to workers' productivity get aggregated/coalesced into one, which will most likely be skewed because the aggregation algorithm would never take all variables into consideration.
Management won't look at 15 metrics per worker and try to understand the data; they want one value and that's it. Not good.

Consider journalism. You have Worker A and Worker B. You give A an assignment about work effectiveness evolution through history, and B gets to write about Kim Kardashian's choice of panties colors. A writes a 5000 word article, well documented, with references and shit, with a serious title. B writes a 300 word article filled with generic panties pictures with a clickbait title. A gets 300 views, B gets 5 million views. The algorithm generates a value based on efficiency and ROI, and A gets a score of 5/100 while B gets a score of 100/100. You look at the values, fire A and promote B.

Now you know why most articles out there are about Kim Kardashian and panties. Incidentally, you know why automatically measuring productivity can go tits up very quickly.

Nothing to see here...

[kschendel]

Sounds like a regurgitation of Taylorism and time-and-motion studies, for the digital worker. Will people never learn? You get what you measure, and if you aren't extremely careful, you'll cause dysfunction because the measurement goal isn't aligned with the organizational goal. (It always looks like it should be, but it rarely is.)

In any case there's nothing new here, just another well-meaning nitwit looking for the magic bullet.

Activity or productivity

[DarkOx]

statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace.

This is only true if you know what to measure. Otherwise you are measuring activity. For example one programmer may type out lots of quick lines to empirically discover the format of a string a library returns for a given inputs, another might go directly to the documentation. One will press more keys, but which is more productive? I don't think you can always expect the correct answer if the statistic you use is average key presses per hour.

If someones job is to paint unpainted widgets in bin A and paint them and put them in bin B, that we can pretty accurately measure their productivity by determine how many widgets are in bin B each day and comparing them with others who do the same work, or can we? What about the defect rate? Measuring is hard, knowing what to measure is harder.

How do measure the productivity of a corporate staff attorney? What about route / switch admin? Is one who puts in more change requests more productive or does that just mean (s)he fails to plan ahead?

Be careful what you measure you will probably get favorable results, but its the side effects that will hurt you.

Re:Activity or productivity

[nbauman]

You have to know your product and what makes for a quality product and tailor your production process to that.

The cult of management says that a good manager can manage anything -- and doesn't even have to understand the product. So the top manager of a potato chip company can move in and run a computer chip company.

Or the top manager of a computer company can reform the education system.

I know what's not going to be measured

[Coisiche]

A similar assessment of CEOs and other board positions.

Flawed Assumption

[oh_my_080980980]

"But there’s another fundamental driver of income inequality: the improved measurement of worker performance. As we get better at measuring who produces what, the pay gap between those who make more and those who make less grows."

This assumes people who produce more get paid more. This has not been the case for the past 30 years. That's why there is an income inequality. Wages have not kept pace with Productivity. So the issue has nothing to do with measuring productivity. This has to do with companies keeping worker pay low in order to increase dividends and CEO pay, which is a consequence of lower rates. If companies had to reinvest in the company, worker pay would keep pace with productivity.

Slashdot's own karma system

[m00sh]

Slashdot moderation system used to measure us as a total of karma over all posts to measure the contribution to Slashdot.

Slashdot had to stop using those because of karma whores.

Even meaningless numbers are a strong motivators to cheat a system. You have to be very careful about what you do. Improving those metrics will triumph over quality and ethics.

Metrics are rarely accurate

[nehumanuscrede]

Case in point.

I can spend the time to manually type in a few hundred lines of configuration data for a Router / Switch for every device I manage.

or

I can spend the time to build an app or program that will effectively build the same configuration for me, guaranteed error free. I need
only change the unique data for the site which can be done prior to the program launching.

So, if my employer is tracking how much I type or how many windows I click on in a day*, which of the two above scenarios is the more
efficient methods of getting a job done ? Because I didn't sit for three days straight and manually configure these systems, then I'm not
as productive ?

Heh. I'll say again, metrics are rarely accurate enough to base decisions on by themselves.

*Which is really easy to fudge with a simple script or program.

Re:Who came up with that bullshit line?

[tbannist]

Not to mention the relentless gaming of any measurement system by all parties that erodes whatever value it might have.

I have an illustrative anecdote:

A company that I used to worked for, decided to have a bug fixing contest. They decided that they would pay a bonus to their software developers for every bug they fixed so they could lower the defect rate on their software. At first the project seemed to be a roaring success, the number of fixed bugs climbed quickly, however, the budget for the bonuses ran out only a few weeks after the contest started. An examination of the payouts quickly raised suspicion among some of the managers running it. The numbers showed that some of the testers were finding more than 10 times the number of bugs that they used to find, while others were finding the exact same number. It didn't make sense because they weren't paying any bonuses to the testers. A short investigation revealed that some of the developers were deliberately including bugs in the code before they released their work to testing, some went so far as to tell their selected tester what and where the bug was, and then splitting the bug bounty with the tester who sent the bug back to them to fix. Of course, the developers and testers who were caught collaborating were all fired. However, the fake bug fixing displaced real testing work, and fewer real bugs were fixed during the contest, and the company had to recruit new people to replace the people they fired, so the defect rate went up because normal testing was displaced, some of the deliberate bugs actually made it through testing, and the new developers and new testers who replaced the people fired were not as familiar with the product and more problems slipped through while they were settling in to their new duties.

The moral, is that when money is involved it will not take long for people to figure out how to game the system, and quite possibly achieve the exact opposite of what they were supposed to being doing.