Slashdot Mirror


(Over-)Measuring the Working Man

HughPickens.com writes: Tyler Cowen writes in MIT Technology Review that the improved measurement of worker performance through information technology is beginning to allow employers to measure value fairly precisely and as we get better at measuring who produces what, the pay gap between those who make more and those who make less grows. Insofar as workers type at a computer, everything they do is logged, recorded, and measured. Surveillance of workers continues to increase, and statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace. Consider journalism. In the "good old days," no one knew how many people were reading an article, or an individual columnist. Today a digital media company knows exactly how many people are reading which articles for how long, and also whether they click through to other links. The result is that many journalists turn out to be not so valuable at all. Their wages fall or they lose their jobs, while the superstar journalists attract more Web traffic and become their own global brands.

According to Cowen, the upside is that measuring value tends to boost productivity, as has been the case since the very beginning of management science. We're simply able to do it much better now, and so employers can assign the most productive workers to the most suitable tasks. The downsides are several. Individuals don't in fact enjoy being evaluated all the time, especially when the results are not always stellar: for most people, one piece of negative feedback outweighs five pieces of positive feedback.

30 of 165 comments (clear)

  1. Clickbait wins by king+neckbeard · · Score: 4, Insightful

    This metric would seem to be encouraging authors that write the clickbaitiest titles. Sounds wonderful.

    --
    This is my signature. There are many like it, but this one is mine.
    1. Re:Clickbait wins by nbauman · · Score: 4, Interesting

      This metric would seem to be encouraging authors that write the clickbaitiest titles. Sounds wonderful.

      Cowen's assumption is that the things that are easy to measure are also meaningful and contribute to the organizations' success. In journalism, that's not usually true.

      I once worked for a weekly newspaper in Brooklyn. The publisher had more (family) money than brains. The paper was doing badly and he was getting desperate.

      One week, we ran a British-style tabloid headline, "4-year-old girl raped, murdered." It had the highest newsstand circulation of any issue.

      This was followed by a wave of angry subscription cancellations. One subscriber wrote, "I don't want this garbage coming into my house."

      The paper continued on its downward trend and finally shut down.

      The moral: following a meaningless metric can give you a boost in the short term, and harm the enterprise in the long term.

      Like VW's emission test results....

    2. Re:Clickbait wins by gweihir · · Score: 4, Insightful

      Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    3. Re:Clickbait wins by tlhIngan · · Score: 4, Insightful

      Indeed. Metrics are very tricky. Applied wrongly (the standard case) they harm you a lot. Also, in many cases, "productivity" is not what it seems to be, unless you are a mindless factory worker that only produces more or fewer of a specific part and quality is not a concern. For example, I have made brief remarks in meetings that ended up preventing massive losses to customers. How do you measure that? Right, you cannot.

      The problem with metrics is, well, you get what you measure.

      And for a lot of metrics, what you measure is not what you're wanting to measure, because what you want to measure is vague and not easily measureable. E.g., productivity is not something you can measure. You can't take two people and easily say that one is more productive than the other.

      And there are other metrics you CAN measure directly, but the cause and effect are so far removed from the people that it's impossible to actually determine who contributed. E.g., savings. Sure you can say the company saved $100,000 this quarter, but which employee did the most?

      And even then, there are hidden consequences.

      As I said above, you get what you measure. If you measure savings, you will get savings. But you may also have to deal with the consequences of it - perhaps regular maintenance becomes irregular and you're setting yourself up for a catastrophic failure in a couple of years. But since you'll have 8 quarters of meeting the metric, it doesn't matter if the failure costs more than what was saved. Or you'll find people will struggle with obsolete equipment way beyond its replacement date to make the numbers come in lower.

      Likewise, if you try to measure productivity, people will make the measurement. If you measure keystrokes, yes, you'll find keystrokes increase. But that might mean "space" and "backspace" are the most popular keys.

      If you measure lines of code, you'll find a lot of copy-pasta coding.

      And so on - you will get what you measure. You will not get what you don't measure. And the weaker the relationship between what you REALLY want to measure and the proxy you're using, the more useless the measure is.

  2. Data? Statistics? by war4peace · · Score: 4, Insightful

    Now wait a second. Management, generally speaking, doesn't understand data. They like to look at a nice Powerpoint slide with people ranked from best to worst. this means that all measures taken relative to workers' productivity get aggregated/coalesced into one, which will most likely be skewed because the aggregation algorithm would never take all variables into consideration.
    Management won't look at 15 metrics per worker and try to understand the data; they want one value and that's it. Not good.

    Consider journalism. You have Worker A and Worker B. You give A an assignment about work effectiveness evolution through history, and B gets to write about Kim Kardashian's choice of panties colors. A writes a 5000 word article, well documented, with references and shit, with a serious title. B writes a 300 word article filled with generic panties pictures with a clickbait title. A gets 300 views, B gets 5 million views. The algorithm generates a value based on efficiency and ROI, and A gets a score of 5/100 while B gets a score of 100/100. You look at the values, fire A and promote B.

    Now you know why most articles out there are about Kim Kardashian and panties. Incidentally, you know why automatically measuring productivity can go tits up very quickly.

    --
    ...gis sdrawkcab (usually not responding to ACs; don't bother posting as AC)
    1. Re:Data? Statistics? by Moloth · · Score: 2

      I agree with you. From a business point of view Worker B had 1000 moron viewers who clicked on the bait then the ads and generated more money than Worker A's smarter viewers and only clicked on 10 ads. Worker B made more money with less time so he is promoted and this is why we see click bait content everywhere simply because it is more profitable. I like most others here find it awful but this is what the net has become (internet 2 - the search for more money). At least we can recognise click bait and avoid it.

      Incidentally, you know why automatically measuring productivity can go tits up very quickly.

      From smarter viewers point of view, not from a profit / moron perspective. So is there any incentive for good, well researched articles? The only things I can think of is pay walls or the hobbyist who doesn't have a pure profit motive. For ad based revenue you probably want the morons eyeballs.

  3. Nothing to see here... by kschendel · · Score: 4, Insightful

    Sounds like a regurgitation of Taylorism and time-and-motion studies, for the digital worker. Will people never learn? You get what you measure, and if you aren't extremely careful, you'll cause dysfunction because the measurement goal isn't aligned with the organizational goal. (It always looks like it should be, but it rarely is.)

    In any case there's nothing new here, just another well-meaning nitwit looking for the magic bullet.

  4. Activity or productivity by DarkOx · · Score: 5, Insightful

    statistical analysis of large data sets makes it increasingly easy to evaluate individual productivity, even if the employer has a fairly noisy data set about what is going on in the workplace.

    This is only true if you know what to measure. Otherwise you are measuring activity. For example one programmer may type out lots of quick lines to empirically discover the format of a string a library returns for a given inputs, another might go directly to the documentation. One will press more keys, but which is more productive? I don't think you can always expect the correct answer if the statistic you use is average key presses per hour.

    If someones job is to paint unpainted widgets in bin A and paint them and put them in bin B, that we can pretty accurately measure their productivity by determine how many widgets are in bin B each day and comparing them with others who do the same work, or can we? What about the defect rate? Measuring is hard, knowing what to measure is harder.

    How do measure the productivity of a corporate staff attorney? What about route / switch admin? Is one who puts in more change requests more productive or does that just mean (s)he fails to plan ahead?

    Be careful what you measure you will probably get favorable results, but its the side effects that will hurt you.

    --
    Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
    1. Re:Activity or productivity by bigpat · · Score: 2

      If someones job is to paint unpainted widgets in bin A and paint them and put them in bin B, that we can pretty accurately measure their productivity by determine how many widgets are in bin B each day and comparing them with others who do the same work, or can we? What about the defect rate?

      I've seen this work out (or not) first hand. You can't just pay (or evaluate performance) on one metric. But usually you can figure out some formula that can measure performance. Lines of code is never the metric in software (actually it is the opposite as software that has more lines per unit of functionality is slower). That is like paying your house builder based on the number of nails they hammer or the amount of wood they use. Neither is the number of unique graphics generated in graphic design a good metric, since generating more of anything of lower quality is going to make for dissatisfied customers as you are forcing them to sift through all the junk. For anything the trick is making sure there are checks in place to ensure quality without increasing cost substantially. You have to know your product and what makes for a quality product and tailor your production process to that. Doesn't mean it can't be done, but micromanaging merely one part of the entire process while neglecting other parts usually leads to poor outcomes.

    2. Re:Activity or productivity by nbauman · · Score: 4, Funny

      You have to know your product and what makes for a quality product and tailor your production process to that.

      The cult of management says that a good manager can manage anything -- and doesn't even have to understand the product. So the top manager of a potato chip company can move in and run a computer chip company.

      Or the top manager of a computer company can reform the education system.

    3. Re:Activity or productivity by Bengie · · Score: 3, Insightful

      That's a step in the right direction, but it still doesn't measure quality very much. Design of the code is also very important. How easy is the code to maintain, extend, refactor, re-use? Does it interact in negative ways with other code? What happens when you push it to its limits? What scaling does it have? What are its failure cases and how does it handle them? How intuitive is the code? Is it written in a way that makes it difficult to use it incorrectly while being simple to use correctly or in new ways that could also be correct?

    4. Re:Activity or productivity by RatherBeAnonymous · · Score: 3, Insightful

      It also doesn't capture the difficulty of the work. Some people gravitate towards tough or time consuming problems while others will tick off large quantities of easy tasks.

  5. Who came up with that bullshit line? by damn_registrars · · Score: 3, Insightful

    the pay gap between those who make more and those who make less grows

    That line clearly comes from someone supporting the insane wages of top executives in this country under the illusion that they are actually productive. They rarely make anything in terms of actual productivity - they are paid a lot primarily in reward for being well-connected.

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
    1. Re:Who came up with that bullshit line? by tbannist · · Score: 4, Interesting

      Not to mention the relentless gaming of any measurement system by all parties that erodes whatever value it might have.

      I have an illustrative anecdote:

      A company that I used to worked for, decided to have a bug fixing contest. They decided that they would pay a bonus to their software developers for every bug they fixed so they could lower the defect rate on their software. At first the project seemed to be a roaring success, the number of fixed bugs climbed quickly, however, the budget for the bonuses ran out only a few weeks after the contest started. An examination of the payouts quickly raised suspicion among some of the managers running it. The numbers showed that some of the testers were finding more than 10 times the number of bugs that they used to find, while others were finding the exact same number. It didn't make sense because they weren't paying any bonuses to the testers. A short investigation revealed that some of the developers were deliberately including bugs in the code before they released their work to testing, some went so far as to tell their selected tester what and where the bug was, and then splitting the bug bounty with the tester who sent the bug back to them to fix. Of course, the developers and testers who were caught collaborating were all fired. However, the fake bug fixing displaced real testing work, and fewer real bugs were fixed during the contest, and the company had to recruit new people to replace the people they fired, so the defect rate went up because normal testing was displaced, some of the deliberate bugs actually made it through testing, and the new developers and new testers who replaced the people fired were not as familiar with the product and more problems slipped through while they were settling in to their new duties.

      The moral, is that when money is involved it will not take long for people to figure out how to game the system, and quite possibly achieve the exact opposite of what they were supposed to being doing.

      --
      Fanatically anti-fanatical
  6. I know what's not going to be measured by Coisiche · · Score: 4, Insightful

    A similar assessment of CEOs and other board positions.

  7. Can they differentiate ... by PPH · · Score: 3, Funny

    ... between my typing in the IDE and the Slashdot comments section? Because I may very well be the most productive worker our company has.

    --
    Have gnu, will travel.
  8. Flawed Assumption by oh_my_080980980 · · Score: 4, Insightful

    "But there’s another fundamental driver of income inequality: the improved measurement of worker performance. As we get better at measuring who produces what, the pay gap between those who make more and those who make less grows."

    This assumes people who produce more get paid more. This has not been the case for the past 30 years. That's why there is an income inequality. Wages have not kept pace with Productivity. So the issue has nothing to do with measuring productivity. This has to do with companies keeping worker pay low in order to increase dividends and CEO pay, which is a consequence of lower rates. If companies had to reinvest in the company, worker pay would keep pace with productivity.

  9. Slashdot's own karma system by m00sh · · Score: 4, Insightful

    Slashdot moderation system used to measure us as a total of karma over all posts to measure the contribution to Slashdot.

    Slashdot had to stop using those because of karma whores.

    Even meaningless numbers are a strong motivators to cheat a system. You have to be very careful about what you do. Improving those metrics will triumph over quality and ethics.

  10. Re:Glorious! by kilfarsnar · · Score: 2

    Instead of using technology to reduce working hours and "trickling down" benefits to everyone, we use it for things like this!

    Indeed. And with a 65% workforce participation rate. Just one more indication (as if we needed another) that the current system serves owners, not labor.

    --
    "What the American public doesn't know is what makes them the American public." -Ray Zalinsky (Tommy Boy)
  11. Metrics are rarely accurate by nehumanuscrede · · Score: 4, Insightful

    Case in point.

    I can spend the time to manually type in a few hundred lines of configuration data for a Router / Switch for every device I manage.

    or

    I can spend the time to build an app or program that will effectively build the same configuration for me, guaranteed error free. I need
    only change the unique data for the site which can be done prior to the program launching.

    So, if my employer is tracking how much I type or how many windows I click on in a day*, which of the two above scenarios is the more
    efficient methods of getting a job done ? Because I didn't sit for three days straight and manually configure these systems, then I'm not
    as productive ?

    Heh. I'll say again, metrics are rarely accurate enough to base decisions on by themselves.

    *Which is really easy to fudge with a simple script or program.

  12. Re:Holistic synergy by gweihir · · Score: 2

    And that is the other thing the metrics-fanatics overlook consistently: Most metrics are easy to spoof. Quite often it does not even require smart people to do it. At the same time, application of metrics significantly lowers loyalty, as people rightfully perceive themselves to be seen as "just a number". So unless the metrics bring massive increases in productivity, using them is already a losers game.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  13. Re:This sounds like a problem only for slackers. by gweihir · · Score: 3, Insightful

    99.9999999% of organizations will choose to measure something more concrete in hopes that it will be a reasonable proxy for productivity. It won't be.

    And that is the real problem. There is no known way to overcome it. But MBA bean-counters that lack any understanding of reality will go for metrics every time, regardless of how misleading they are, because these cretins are unable to do anything else.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  14. Re:Every monitoring system will be gamed! by tehcyder · · Score: 2

    Become invaluable

    No one is invaluable in a company, unless it's a one-man entrepreneurial band type of set up.

    --
    To have a right to do a thing is not at all the same as to be right in doing it
  15. Re:False assumption by PopeRatzo · · Score: 2

    Agreed, but they're using the word "value" purely in the financial sense, not as a measure of your worth as a human being or anything.

    Of course they are. It's how you use language as a tool of control. Notice also that the same word is used for "loyalty" to a company and loyalty to a family or creed.

    We have been conditioned to see ourselves in terms of our value to the ownership class.

    --
    You are welcome on my lawn.
  16. Race to the bottom by jenningsthecat · · Score: 3, Insightful

    FTA: "The result is that many journalists turn out to be not so valuable at all. Their wages fall or they lose their jobs, while the superstar journalists attract more Web traffic and become their own global brands."

    So at least in journalism, only the most popular will have a subsidized voice, and the rest will have to pay their own way if they want to share their insights. Since when is popularity the ultimate measure of value in a society, especially when it comes to news? Sometimes people really need to hear the stuff that's scary, uncomfortable, guilt-inducing, etc., even though it's not popular. If we continue down this road then I hope everyone enjoys having Fox-style reporting as the only available news source.

    Yes, I realize I've used a 'reductio ad absurdum' argument, but I don't think I've gone very far into absurdity here. It strikes me that in a lot of ways this kind of 'metric' is merely measuring quantity when its purveyors seem to think that it measures quality. Maybe that's because quantity is so much easier to measure. But like a drunk searching for his keys under a streetlight because 'the light's better there', it's probably counterproductive.

    --
    'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
  17. In the field of Statistics... by frank249 · · Score: 2

    ... the N's do justify the means.

    --

    Today's vices may be tomorrow's virtues.

  18. Physical labor & metrics by Rinikusu · · Score: 2

    Unlike many of my office co-workers, I've done manual/physical labor in warehouses and what not. When I hear about office productivity and reporting, I often wonder just exactly what it is they're measuring because for software development, none of the metrics actually seem to apply. With physical labor, I never felt like I had a moment to breathe. Twenty eight boxes per minute was the standard, the goal, and the basis for all future performance metric evaluations (including raises and bonuses). Clock in. Start. Twenty eight boxes per minute. Small boxes. Big boxes. Heavy boxes. Broken boxes. reach, grab, place. reach, grab, place. No moments to think about life, what I want for dinner, what my professor talked about in my day classes. Box. Box. Box. Box. Box. Box. Twenty eight times per minute (my rate was around 35-40, though. I miss being young, but I don't miss wasting that youth on boxes). As I moved into the "white collar" world, the standards seemed to change. "File X papers/hour. Answer X phone calls per hour." Same sort of goals. I knew exactly where I stood at all times in the grand queue of things.

    As a developer.. It's a completely different world. Lines of Code? Bullshit, everyone knows how to game that. Milestones? Same deal. Half the time we're completing projects in 1/8th the allocated time and browsing the web the rest, the other half we're scrambling because they only allocated 1/8th the time it actually requires to get the job done (Try working for the .gov as a software developer.. Estimates aren't really an art as much as a "what can I say to make the guy in charge of my contract happy?", apparently). Etc. Productivity as a measurement in software development is increasingly idiotic. Whenever I hear "agile" and "scrum" I hear "We're trying to make objective measurements on something that really has no objective measurements because we have to check this box right here that says I have to have objective proof I'm actually working and doing my job!" Bleh.

    --
    If you were me, you'd be good lookin'. - six string samurai
  19. Baloney! by al0ha · · Score: 2

    "Today a digital media company knows exactly how many people are reading which articles for how long.."

    Baloney - the may know roughly, with many factors contributing to X percentage of error, including bots, people that land on a page then are distracted to do something else, thus never actually read the article though they are parked on it, then click away after a few minutes, and myriad other reasons I don't have time to type or can't even conceive of at this moment.

    --
    Did you ever wake up in the morning, with a Zombie Woof behind your eyes? -- FZ
  20. Re:False assumption by PopeRatzo · · Score: 2

    Most of us have been conditioned to understand that the same word can have different meanings in different contexts.

    If that were true, then phrases such as "Pro-Life" and "Freedom of Choice" wouldn't be so effective on the masses.

    No, highly charged words cross a threshold that most people don't even realize exists. Marketing relies on this phenomenon. I would dare say that even people who expect it, like you and me, are susceptible. In fact, our willingness to believe that we are impervious to this effect might even make us more susceptible.

    --
    You are welcome on my lawn.
  21. Re:That has always been the case by ultranova · · Score: 2

    The internet hasn't changed anything there.

    Sure it has: I can get my fuzz and wuz for free there. So if all you have to offer is scandals, propaganda and yesterday's news, you have nothing worth paying for.

    --

    Forget magic. Any technology distinguishable from divine power is insufficiently advanced.