A New Corporate AI Can Read Your Emails - and Your Mind (fortune.com)

"Okay, as of last night, who were the people who were most disgruntled...? Show me the top 10." An anonymous Slashdot reader shares their report on a fascinating Fortune magazine article: "One company says it can spot 'insider threats' before they happen -- by reading all your workers' email." Working with a former CIA consultant, Stroz Friedberg developed a software that "combs through an organization's emails and text messages -- millions a day, the company says -- looking for high usage of words and phrases that language psychologists associate with certain mental states and personality profiles...

"Many companies already have the ability to run keyword searches of employees' emails, looking for worrisome words and phrases like 'embezzle' and 'I loathe this job'. But the Stroz Friedberg software, called Scout, aspires to go a giant step further, detecting indirectly, through unconscious syntactic and grammatical clues, workers' anger, financial or personal stress, and other tip-offs that an employee might be about to lose it... It uses an algorithm based on linguistic tells found to connote feelings of victimization, anger, and blame."
The article reports that 27% of cyber-attacks "come from within," according to a study of 562 organizations that was partly conducted by the U.S. Secret Service, with 43% of the surveyed companies reporting an "insider attack" within the last year.

Comments

it also works in reverse

[epine]

No intelligence here.

The mindlessness of this technology is it's number one selling point.

As rumour goes around (you're soaking in it), dutiful employees will onboard yet another reason to paint within arbitrary and demeaning corporate lines like good passionless drones (have I flunked the test?)

Here is a rather chilling passage from The Gulag Archipelago by Aleksandr Solzhenitsyn.

At the conclusion of the conference, a tribute to Comrade Stalin was called for. Of course, everyone stood up (just as everyone had leaped to his feet during the conference at every mention of his name). ... For three minutes, four minutes, five minutes, the stormy applause, rising to an ovation, continued. But palms were getting sore and raised arms were already aching. And the older people were panting from exhaustion. It was becoming insufferably silly even to those who really adored Stalin.

However, who would dare to be the first to stop? ... After all, NKVD men were standing in the hall applauding and watching to see who would quit first! And in the obscure, small hall, unknown to the leader, the applause went on — six, seven, eight minutes! They were done for! Their goose was cooked! They couldn't stop now till they collapsed with heart attacks! At the rear of the hall, which was crowded, they could of course cheat a bit, clap less frequently, less vigorously, not so eagerly — but up there with the presidium where everyone could see them?

The director of the local paper factory, an independent and strong-minded man, stood with the presidium. Aware of all the falsity and all the impossibility of the situation, he still kept on applauding! Nine minutes! Ten! In anguish he watched the secretary of the District Party Committee, but the latter dared not stop. Insanity! To the last man! With make-believe enthusiasm on their faces, looking at each other with faint hope, the district leaders were just going to go on and on applauding till they fell where they stood, till they were carried out of the hall on stretchers! And even then those who were left would not falter ...

Then, after eleven minutes, the director of the paper factory assumed a businesslike expression and sat down in his seat. And, oh, a miracle took place! Where had the universal, uninhibited, indescribable enthusiasm gone? To a man, everyone else stopped dead and sat down. They had been saved!

The squirrel had been smart enough to jump off his revolving wheel. That, however, was how they discovered who the independent people were. And that was how they went about eliminating them. That same night the factory director was arrested. They easily pasted ten years on him on the pretext of something quite different. But after he had signed Form 206, the final document of the interrogation, his interrogator reminded him:

"Don't ever be the first to stop applauding."

Once known, becomes completely useless

[gweihir]

People will just use other channels for that type of communication. And there is also a serious risk: Many people will not communicate needed information for fear to be caught by this. In the ultimate consequence this can do much more damage than it helps.

Legal dept will advise them differently

[nehumanuscrede]

This stuff falls into pseudo-science much like a polygraph does. The first time they fire someone based on what amounts to ' digital profiling ' it will likely be quite a costly mistake.

Besides, there is nothing in my contract that states I have to like my job. I just have to do it.

I would think that if folks were not afraid of the fallout, any given company would find that a rather significant percentage of their workforce thinks less than positive thoughts about their job in general.

Re:Legal dept will advise them differently

[NormalVisual]

This stuff falls into pseudo-science much like a polygraph does.

Agree 100%. I was tasked to implement a sentiment scoring feature like this in a corporate monitoring/surveillance product a couple of years back, intended to evaluate IMs, emails, etc. for potential leaks/corporate espionage/other bad stuff. I told them that we didn't have the time or in-house expertise to do it ourselves, and every single one of the commercially available libraries that I evaluated turned out to be snake oil.

Re:Not necessarily bad

[Dogtanian]

It could be an interesting idea in linguistics and data mining to identify potential workplace threats and troubled workers.

Being an "interesting" idea from an intellectual point of view says absolutely *nothing* about whether it's a good idea or not.

There shouldn't be an expectation of privacy in workplace emails. If you want that, use a private account to discuss things.

Okay; the fact you're expressing that pat response here suggests that you don't understand (or weren't paying attention to) the difference between this and the typical (straightforward) "employers are reading my workplace email" thread. I actually wonder whether you even got the point of the story at all.

This isn't spying on people directly expressing hostile or subversive thoughts against the company, this is using it on (potentially) superficially work-related and neutral email content to determine the underlying psychological attitude of the employee.

Given that the employee is probably *required* to use email in this manner as part of their job, and given that this isn't something they're likely to be consciously doing (else they'd avoid doing it, duh) it's not as if they have a choice in the matter.

Whether this is good or bad comes down to how you react to an alert.

The issue here- and the reason most people quite rightly expressed the (supposedly) "kneejerk" reaction you dismiss- is that they already know based on past experience how large corporations or similar entities- i.e. the people likely to be buying this technology- will probably use this sort of power.

For genuinely troubled employees, however, this might actually be useful if it leads to a confidential meeting with a third party or ombudsman who tries to help the employee.

Yeah, because large US-style corporations are well-known for protecting employees with problems and won't simply use this as an early warning on someone they can get rid of before they become a problem. Or might not have, but why take the chance?

I saw the example in the story. A nice, touchy-feely way to justify an intrusive technology, but let's get real here.

If it's used to actually help troubled employers who might not reach out for help on their own, it could actually help people while protecting the company. If used properly, it's a good thing.

The question is, how likely to you think it is to be used "properly" in your sense of the word?

Your problem is that you seem to view the technology in a purely abstract sense- i.e. one that could theoretically be used for good or bad. Well, theoretically it could be, yes.

However, your so-called "tinfoil hat crowd" knows damn well that such technologies don't exist in isolation, know what type of people it's been designed for, and the type of people and organisations it's likely to be sold to. Based on past experience, it's not unreasonable to draw such conclusions on how it's likely to be used.

So, you can keep expressing your (repeated) dismissal of its critics as "paranoid delusional", but that doesn't make your counter-argument any stronger.

Clippy says...

[RuffMasterD]

Hi! It looks like you are angry. Perhaps you are in a fragile state of mind and would like to kill everybody. Would you like some assistance with that? Oh, by the way, I read that your colleagues hate you, your boss is about to fire you, your girlfriend is sleeping with your best friend, and your parents never wanted you.