Slashdot Mirror

Once the majority of people realize that all their behavior is turned into these scores, and that these scores have increasing influence over their lives, you will start to see serious chilling effects.

Heck, we are already seeing those.

In the long run this could lead to social cooling, where society becomes more rigid, less able to change.

TFS describes a bog-standard fraud detection system that nearly every financial institution in the world uses.
These are the systems that drive those texts you get from your bank asking to confirm a transaction, or or trigger security questions when you login from a new computer, or disable your account due to suspicious activity.

In regards to chilling effects and whatever social cooling is, quite obviously, people still spout garbage day-in day-out online, with no remorse, no concerns whatsoever, with no end in sight. What are you smoking?

Once the majority of people realize that all their behavior is turned into these scores, and that these scores have increasing influence over their lives, you will start to see serious chilling effects.

Heck, we are already seeing those.

In the long run this could lead to social cooling, where society becomes more rigid, less able to change.

Social cooling.

People are starting to realize that this 'digital reputation' could limit their opportunities.

Look: Remember the very first porn sites way back in the 90s? We clicked to see titties and then a question popped up: "Are you over 18 or over?"

- Yes
- No

We clicked, "No," and went to Disney.

Having IQs higher than asphalt, we made another run at it and gave the answer that fit our needs.

Few on /. are unaware of behavioural modification techniques.

People are slowly learning to shape their narrative on social media to avoid pain and gain favour.

Give it time. Over the years people will start to understand how the data driven business model really works. That profiling is not just about personalised ads, but equally about handling you as a risk, which often means denying you opportunities such as jobs or cheap insurance. The real businessmodel of these companies is the continuous background check.

In a few years the 'data is the new oil' narative will backfire on Silicon Valley, as the 'data as a pollutant' metaphor will become all to apt. This comparison will then lead us to ask: what is the data version of global warming?

It's Social Cooling.

Western companies are just as eager to build these things, and sell them for 'safety' or 'self improvement'.

Social Cooling is becoming a real possibility.

Social Cooling is a word we can use to describe these large scale chilling effects: https://www.socialcooling.com

That's not really true unfortunately.

Cambridge Analytica was a relatively small databroker which focussed on deriving psychological profiles from your data through 'inference'. Larger databrokers have 8000 scores for sale. All of them feed your mundane data (social media scraped, smart home data, etc) to algorithms that then compare your data to that of other people they know more about, and if your data exhibits similar traits they will infer that you also fit in certain categories.

This talk explains things:

Social Cooling - big data's unintended side effect

But I also recommend checking out the work of:

- "Weapons of Math Destruction" by Cathy o Neill (book)

- "Black Box Society" by Frank Pasquale (book)

For me it's the Social Cooling I worry about. When people feel watched and judged "at scale", then large scale chilling effects might occur. This is China's goal. But in the west we should want to avoid these effects, as they hinder our ability to sustain a vibrant democracy. See:
Social Cooling website

We really need to start looking at the long term effects of these systems om societies. After all, here in the west the market is slowly piecing together a similar infrastructure (databrokers and their ilk).

While China covets these chilling effects, here in the west we might best frame when as an unwelcome side-effect, e.g. Social Cooling. https://www.socialcooling.com/

In 1995, French philosopher Giles Deleuze, building on the work of Foucault, perfectly explained what is going on here in his 3-page text "Postscript on societies of control". We are moving from societies of discipline to societies of control, he explains.

Discipline
- A system punishes people once they break the rules (law), but not before.
- Transparent and accountable, at least in current western societies.
- Ultimately builds on the governments monopoly of power. You play by the rules because the government has tanks.
- Expensive.

Control
- Permanent measuring and nudging, whether you are guilty or innocent.
- Increasingly hidden in datacenters and proprietary algorithms.
- Weaponizes social control by making social interactions measurable (social media), and thus designable. You play by the rules because you want to stay included in society.
- Cheap, as you are basically crowdsourcing control to the people, who themselves apply the pressure to each other.

All societies have both systems. But the social control system used to be informal and difficult to 'design' (although the Stasi already had a working beta version). That has changed with the rise of the internet, which has allowed us to cheaply measure and record everything. Couple that with the rise of psychological knowledge (nudging, etc), and you have a pretty interesting substrate.

The Chinese seem to have read Foucault and Deleuze's work better than we in the west did. At least they know that they're building..

Here in the west this could be a useful narrative to steer clear of this possible future:
https://www.socialcooling.com/

Deleuze's text:
https://cidadeinseguranca.file...

This is going to speed up social cooling..

https://www.socialcooling.com/

There's another serious unintended consequence that I'm more worried about: Social Cooling. It describes how the reputation scores that databrokers like Equifax make and sell are increasingly influencing your job opportunities and other aspects of your life. As people become aware of this reputation economy they start to self-censor and avoid risk in order to have good scores.

https://www.socialcooling.com/

It will even get some attention in a public hearing on 'horizontal privacy' by the Dutch Government this thursday.