(Over-)Measuring the Working Man

HughPickens.com writes: Tyler Cowen writes in MIT Technology Review that the improved measurement of worker performance through information technology is beginning to allow employers to measure value fairly precisely and as we get better at measuring who produces what, the pay gap between those who make more and those who make less grows. Insofar as workers type at a computer, everything they do is logged, recorded, and measured. Surveillance of workers continues to increase, and statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace. Consider journalism. In the "good old days," no one knew how many people were reading an article, or an individual columnist. Today a digital media company knows exactly how many people are reading which articles for how long, and also whether they click through to other links. The result is that many journalists turn out to be not so valuable at all. Their wages fall or they lose their jobs, while the superstar journalists attract more Web traffic and become their own global brands.

According to Cowen, the upside is that measuring value tends to boost productivity, as has been the case since the very beginning of management science. We're simply able to do it much better now, and so employers can assign the most productive workers to the most suitable tasks. The downsides are several. Individuals don't in fact enjoy being evaluated all the time, especially when the results are not always stellar: for most people, one piece of negative feedback outweighs five pieces of positive feedback.

Comments

Clickbait wins

[king neckbeard]

This metric would seem to be encouraging authors that write the clickbaitiest titles. Sounds wonderful.

Re:Clickbait wins

[gweihir]

Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

Re:Clickbait wins

[tlhIngan]

Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

The problem with metrics is, well, you get what you measure.

And for a lot of metrics, what you measure is not what you're wanting to measure, because what you want to measure is vague and not easily measureable. E.g., productivity is not something you can measure. You can't take two people and easily say that one is more productive than the other.

And there are other metrics you CAN measure directly, but the cause and effect are so far removed from the people that it's impossible to actually determine who contributed. E.g., savings. Sure you can say the company saved $100,000 this quarter, but which employee did the most?

And even then, there are hidden consequences.

As I said above, you get what you measure. If you measure savings, you will get savings. But you may also have to deal with the consequences of it - perhaps regular maintenance becomes irregular and you're setting yourself up for a catastrophic failure in a couple of years. But since you'll have 8 quarters of meeting the metric, it doesn't matter if the failure costs more than what was saved. Or you'll find people will struggle with obsolete equipment way beyond its replacement date to make the numbers come in lower.

Likewise, if you try to measure productivity, people will make the measurement. If you measure keystrokes, yes, you'll find keystrokes increase. But that might mean "space" and "backspace" are the most popular keys.

If you measure lines of code, you'll find a lot of copy-pasta coding.

And so on - you will get what you measure. You will not get what you don't measure. And the weaker the relationship between what you REALLY want to measure and the proxy you're using, the more useless the measure is.

Data? Statistics?

[war4peace]

Now wait a second. Management, generally speaking, doesn't understand data. They like to look at a nice Powerpoint slide with people ranked from best to worst. this means that all measures taken relative to workers' productivity get aggregated/coalesced into one, which will most likely be skewed because the aggregation algorithm would never take all variables into consideration.
Management won't look at 15 metrics per worker and try to understand the data; they want one value and that's it. Not good.

Consider journalism. You have Worker A and Worker B. You give A an assignment about work effectiveness evolution through history, and B gets to write about Kim Kardashian's choice of panties colors. A writes a 5000 word article, well documented, with references and shit, with a serious title. B writes a 300 word article filled with generic panties pictures with a clickbait title. A gets 300 views, B gets 5 million views. The algorithm generates a value based on efficiency and ROI, and A gets a score of 5/100 while B gets a score of 100/100. You look at the values, fire A and promote B.

Now you know why most articles out there are about Kim Kardashian and panties. Incidentally, you know why automatically measuring productivity can go tits up very quickly.

Re:Data? Statistics?

[Intrepid imaginaut]

That's a bit simplistic. If the target audience is statistical management types those 300 views could translate to more value for the publication than an article about panties, but it depends on the publication, which is why you don't usually see in-depth scholalry research in the red tops. The red tops and clickbaiters also make much more money than most academic publications but that's their entire purpose. Once the metrics are calibrated correctly quality can indeed be measured and used to improve a publication's offerings.

Nothing to see here...

[kschendel]

Sounds like a regurgitation of Taylorism and time-and-motion studies, for the digital worker. Will people never learn? You get what you measure, and if you aren't extremely careful, you'll cause dysfunction because the measurement goal isn't aligned with the organizational goal. (It always looks like it should be, but it rarely is.)

In any case there's nothing new here, just another well-meaning nitwit looking for the magic bullet.

Activity or productivity

[DarkOx]

statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace.

This is only true if you know what to measure. Otherwise you are measuring activity. For example one programmer may type out lots of quick lines to empirically discover the format of a string a library returns for a given inputs, another might go directly to the documentation. One will press more keys, but which is more productive? I don't think you can always expect the correct answer if the statistic you use is average key presses per hour.

If someones job is to paint unpainted widgets in bin A and paint them and put them in bin B, that we can pretty accurately measure their productivity by determine how many widgets are in bin B each day and comparing them with others who do the same work, or can we? What about the defect rate? Measuring is hard, knowing what to measure is harder.

How do measure the productivity of a corporate staff attorney? What about route / switch admin? Is one who puts in more change requests more productive or does that just mean (s)he fails to plan ahead?

Be careful what you measure you will probably get favorable results, but its the side effects that will hurt you.

Re:Activity or productivity

[Bengie]

That's a step in the right direction, but it still doesn't measure quality very much. Design of the code is also very important. How easy is the code to maintain, extend, refactor, re-use? Does it interact in negative ways with other code? What happens when you push it to its limits? What scaling does it have? What are its failure cases and how does it handle them? How intuitive is the code? Is it written in a way that makes it difficult to use it incorrectly while being simple to use correctly or in new ways that could also be correct?

Re:Activity or productivity

[RatherBeAnonymous]

It also doesn't capture the difficulty of the work. Some people gravitate towards tough or time consuming problems while others will tick off large quantities of easy tasks.

Who came up with that bullshit line?

[damn_registrars]

the pay gap between those who make more and those who make less grows

That line clearly comes from someone supporting the insane wages of top executives in this country under the illusion that they are actually productive. They rarely make anything in terms of actual productivity - they are paid a lot primarily in reward for being well-connected.

I know what's not going to be measured

[Coisiche]

A similar assessment of CEOs and other board positions.

Flawed Assumption

[oh_my_080980980]

"But there’s another fundamental driver of income inequality: the improved measurement of worker performance. As we get better at measuring who produces what, the pay gap between those who make more and those who make less grows."

This assumes people who produce more get paid more. This has not been the case for the past 30 years. That's why there is an income inequality. Wages have not kept pace with Productivity. So the issue has nothing to do with measuring productivity. This has to do with companies keeping worker pay low in order to increase dividends and CEO pay, which is a consequence of lower rates. If companies had to reinvest in the company, worker pay would keep pace with productivity.

Slashdot's own karma system

[m00sh]

Slashdot moderation system used to measure us as a total of karma over all posts to measure the contribution to Slashdot.

Slashdot had to stop using those because of karma whores.

Even meaningless numbers are a strong motivators to cheat a system. You have to be very careful about what you do. Improving those metrics will triumph over quality and ethics.

Metrics are rarely accurate

[nehumanuscrede]

Case in point.

I can spend the time to manually type in a few hundred lines of configuration data for a Router / Switch for every device I manage.

or

I can spend the time to build an app or program that will effectively build the same configuration for me, guaranteed error free. I need
only change the unique data for the site which can be done prior to the program launching.

So, if my employer is tracking how much I type or how many windows I click on in a day*, which of the two above scenarios is the more
efficient methods of getting a job done ? Because I didn't sit for three days straight and manually configure these systems, then I'm not
as productive ?

Heh. I'll say again, metrics are rarely accurate enough to base decisions on by themselves.

*Which is really easy to fudge with a simple script or program.

Re:This sounds like a problem only for slackers.

[gweihir]

99.9999999% of organizations will choose to measure something more concrete in hopes that it will be a reasonable proxy for productivity. It won't be.

And that is the real problem. There is no known way to overcome it. But MBA bean-counters that lack any understanding of reality will go for metrics every time, regardless of how misleading they are, because these cretins are unable to do anything else.

Race to the bottom

[jenningsthecat]

FTA: "The result is that many journalists turn out to be not so valuable at all. Their wages fall or they lose their jobs, while the superstar journalists attract more Web traffic and become their own global brands."

So at least in journalism, only the most popular will have a subsidized voice, and the rest will have to pay their own way if they want to share their insights. Since when is popularity the ultimate measure of value in a society, especially when it comes to news? Sometimes people really need to hear the stuff that's scary, uncomfortable, guilt-inducing, etc., even though it's not popular. If we continue down this road then I hope everyone enjoys having Fox-style reporting as the only available news source.

Yes, I realize I've used a 'reductio ad absurdum' argument, but I don't think I've gone very far into absurdity here. It strikes me that in a lot of ways this kind of 'metric' is merely measuring quantity when its purveyors seem to think that it measures quality. Maybe that's because quantity is so much easier to measure. But like a drunk searching for his keys under a streetlight because 'the light's better there', it's probably counterproductive.