How Algorithms May Affect You (phys.org)

New submitter Muckluck shares an excerpt from a report via Phys.Org that provides "an interesting look at how algorithms may be shaping your life": When you browse online for a new pair of shoes, pick a movie to stream on Netflix or apply for a car loan, an algorithm likely has its word to say on the outcome. The complex mathematical formulas are playing a growing role in all walks of life: from detecting skin cancers to suggesting new Facebook friends, deciding who gets a job, how police resources are deployed, who gets insurance at what cost, or who is on a "no fly" list. Algorithms are being used -- experimentally -- to write news articles from raw data, while Donald Trump's presidential campaign was helped by behavioral marketers who used an algorithm to locate the highest concentrations of "persuadable voters." But while such automated tools can inject a measure of objectivity into erstwhile subjective decisions, fears are rising over the lack of transparency algorithms can entail, with pressure growing to apply standards of ethics or "accountability." Data scientist Cathy O'Neil cautions about "blindly trusting" formulas to determine a fair outcome. "Algorithms are not inherently fair, because the person who builds the model defines success," she said. Phys.Org cites O'Neil's 2016 book, "Weapons of Math Destruction," which provides some "troubling examples in the United States" of "nefarious" algorithms. "Her findings were echoed in a White House report last year warning that algorithmic systems 'are not infallible -- they rely on the imperfect inputs, logic, probability, and people who design them,'" reports Phys.Org. "The report noted that data systems can ideally help weed out human bias but warned against algorithms 'systematically disadvantaging certain groups.'"

Comments

The end

[JustAnotherOldGuy]

It's the end of intelligence as we know it.

Re:The end

[postbigbang]

Maybe just the opposite, the beginning of intelligence. The problems are garbage data, conflation, risk analysis with random failures, entropy, and just ignoring facts-- among so many other problems.

How far does an algorithm take bias until it's actually discriminating based on such things as gender, race, etc? We're in the very early stages of "big data" and we're doing a bad job of it. The problem is this: we'll continue doing a bad job until we have more transparency, IMHO.

Here's what you do

[Waffle Iron]

In my day, we had a simple and effective way to judge algorithms:

O(n log(n)) or faster: good
O(n^2) or slower: bad

Transparancy

[hackwrench]

Because the processes were so transparent to begin with. At least the algorithms have the possibility of being looked at. Maybe that should have been the story.

Re:Transparancy

[ZorroXXX]

But even if the algorithms are 100% open and transparent, that means nothing if the data feed into them is poor. If the bank uses an algorithm to determine if it want to lend money to you, how is the data about you collected? Who decided to classify you as a say medium risk person? What cirterias did he/she/they use for that? How thorogh were he/she/they in gathering decition material? What did he/she/they miss/ignore/misunderstood?

Unless there is full and complete transparency and accountability for data collection, the transparency for just the algorithms is without value.

Algorithms or what?

[Anonymous Coward]

The author seems to think that the alternative to "algorithms" is people making good decisions. That's simply not true. The alternative is people attempting, or not, to follow some agreed on some ill-defined process.

Note that algorithms make it harder to ignore that we have to make tradeoffs. To take one of the examples from the article, we may have to choose between "well-respected teachers" and those who actually help students significantly. (Of course, we could reveal performance information and let parents make the decision for their kids, but ....)

Re:Algorithms or what?

[Sique]

No, the author makes the point that algorithms don't exculpate anyone from making bad decision. "The computer said: No." is no excuse for mishandling someone. We had the example on Slashdot of the algorithm that tries to predict recidivism and thus recommend probation or prison. A deeper analysis showed that it was biased against black people because it predicted higher recidivism for them than they had in reality, and it was biased pro whites as it predicted lower recidivism rates than real. And it was not even factoring in the skin color of the people in question. But the way it weighed the socio-economic factors seems to be the problem. It was scoring high on recidivism when many socio-economic risk factors were slightly up, but gave low scores when only one or two risk factors were high, but all others were low. Thus it was overestimating the recidivism rates of poor people with a weak family background, but completely missing the recidivism risk of well off people from a stable family, but deep personal problems.

But because the program was actually used in judiary decisions in several States, it unnecessarily sent people to prison, while it recommended to set high risk people free on probation, and it did it with a strong racial bias that was contradicted by reality.

In other news... Integers: Why so many?

[mbeckman]

Is this what passes for intelligent discourse at Phys.org? So many errors, so little time. First, an algorithm is NOT involved when I pick a movie to stream on Netflix. I pick the movie, Netflix streams it. Unless you want to count the code necessary to display a web page and process a click. Second, algorithms are NOT "complex mathematical formulas." A formula is a specification for a single computational step (or a series of similar steps, in the case of calculus). An algorithm is a non-mathematical procedure, with memory, decision making, input and output from and to various sources and sinks, and, well, formulas. Algorithms contain formulas, but formulas don't contain algorithms. And phys.org does not contain the sense God gave raisins.

Agent Smith

[Zobeid]

"Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization, because as soon as we started thinking for you, it really became our civilization, which is of course what this is all about."

Re:Way back when...

[BeauSD]

This is how I learned C

Black Box Society

[Anonymous Coward]

A recent report on this in the Netherlands summarised it as "playtime has to be over". Big data and the algorithms that work on top it are getting a serious amount of power over our lives. Any little scrap of data is starting to influence your chances of getting a job, a cheap loan, or even a date.

If you want to know how scary this gets, check out this presentation by Alexander Nix on how he used this type of data to influence the elections.
https://www.youtube.com/watch?...

Or have a look at the new "Social Credit Score" that China is implementing, in which every citizen gets a score that shows if they are a well behaved citizen or not.
https://en.wikipedia.org/wiki/...

I also highly recommend watching this interview with professor Frank Pasquale, which summarises the issue.
https://youtu.be/PDjgyTnzWuQ

(Academic students here are not allowed to cite Weapons of Math Destruction, but you are allowed to use his book)

Demand access to pricing algorithms

[drinkypoo]

We should be demanding access to the data and algorithms used to generate pricing for mandatory services like the ACA, home insurance, and automobile insurance. We should never be required to buy anything without even knowing the basis for the charges.

Al, go rhythm -- you'll like it.

[fyngyrz]

Don't forget the algorithm that determined which "hundreds of movies" out of the zillions that have been made that you got to choose from out of the first 30, and the algorithms that the movies studio used, and the algorithms that the effects companies used, and the algorithms that determine which actors were "hot"...

To say that making a choice on Netflix is "algorithm-free" is to not even remotely understand the world one lives in.