How Algorithms May Affect You (phys.org)

New submitter Muckluck shares an excerpt from a report via Phys.Org that provides "an interesting look at how algorithms may be shaping your life": When you browse online for a new pair of shoes, pick a movie to stream on Netflix or apply for a car loan, an algorithm likely has its word to say on the outcome. The complex mathematical formulas are playing a growing role in all walks of life: from detecting skin cancers to suggesting new Facebook friends, deciding who gets a job, how police resources are deployed, who gets insurance at what cost, or who is on a "no fly" list. Algorithms are being used -- experimentally -- to write news articles from raw data, while Donald Trump's presidential campaign was helped by behavioral marketers who used an algorithm to locate the highest concentrations of "persuadable voters." But while such automated tools can inject a measure of objectivity into erstwhile subjective decisions, fears are rising over the lack of transparency algorithms can entail, with pressure growing to apply standards of ethics or "accountability." Data scientist Cathy O'Neil cautions about "blindly trusting" formulas to determine a fair outcome. "Algorithms are not inherently fair, because the person who builds the model defines success," she said. Phys.Org cites O'Neil's 2016 book, "Weapons of Math Destruction," which provides some "troubling examples in the United States" of "nefarious" algorithms. "Her findings were echoed in a White House report last year warning that algorithmic systems 'are not infallible -- they rely on the imperfect inputs, logic, probability, and people who design them,'" reports Phys.Org. "The report noted that data systems can ideally help weed out human bias but warned against algorithms 'systematically disadvantaging certain groups.'"

Comments

The end

[JustAnotherOldGuy]

It's the end of intelligence as we know it.

Re:The end

[postbigbang]

Maybe just the opposite, the beginning of intelligence. The problems are garbage data, conflation, risk analysis with random failures, entropy, and just ignoring facts-- among so many other problems.

How far does an algorithm take bias until it's actually discriminating based on such things as gender, race, etc? We're in the very early stages of "big data" and we're doing a bad job of it. The problem is this: we'll continue doing a bad job until we have more transparency, IMHO.

Re:The end

[Anonymous Coward]

And the beginning of a vastly superior form of intelligence.

Re:The end

[Archtech]

Er, not really. As long as the "intelligence" takes the form of algorithms, that means human beings are devising sets of rules for computers to follow. That is not very intelligent - or, at least, the intelligence involved is indirect, remote and attenuated. The people who specify the software's behaviour must communicate what they want clearly, unambiguously, completely and consistently to the programmers, who then have to do the same thing in their code. Finally, the computer does whatever the original specifiers could think of in response to events that they were able to conceive of. A physical analogy would be trying to tie your shoelaces using a pair of 30-foot-long tweezers - only much worse.

The very essence of real intelligence is the ability to recognise patterns immediately and respond to them in creatively flexible - if not always entirely new - ways. The art of making neural networks and the like, which are able to work that way, is in its infancy.

And even when those systems become "production strength", we will face their biggest problem: non-transparency. How far can you trust a superhuman intelligence that not only doesn't explain to you the reasons for its decisions, but is fundamentally unable to do so?

For details, see James Hogan's SF novel "The Two Faces of Tomorrow" (you can skip the fictional part for our purposes here, and just read the lectures on AI). By the way, Hogan was a computer engineer.

Re: The end

[Anonymous Coward]

Stupid bitch blames algorithms for hiliary's lies and defeat. This shit is beyond fucking retarded.

Re: The end

[Anonymous Coward]

Various machine learning applications in banking have to be intentionally optimized away from accurate prediction -- even when protected classifications like race and national origin are omitted from their inputs, there is enough correlated information that an accurate predictor of things like likelihood of repaying a loan will run afoul of federal anti-discrimination law.

Re:The end

[Anonymous Coward]

The problems are garbage data, conflation, risk analysis with random failures, entropy, and just ignoring facts-- among so many other problems.

That is where your Facebook and other "Social data" go: They decide whether you are going to have credit or not. If you don't get credit you don't get rich. If you are not rich you don't get credit. Think of an echo chamber on steroids.

Captcha: privacy

Re: The end

[Anonymous Coward]

You are leaving out the money part. Credit is not rich. Its debt. You want to be rich, have money as savings. You want to be wealthy, have money as assets/investments. Most don't need credit except for a car (still poor), a home (very common), school loans (credit may affect rates).

Poor/stupid people use credit (cards/loans), for things they should not. Living beyond thier ability to pay and stay in the black. This is the poor persons mentality.

Here's what you do

[Waffle Iron]

In my day, we had a simple and effective way to judge algorithms:

O(n log(n)) or faster: good
O(n^2) or slower: bad

Re:Here's what you do

[Austerity Empowers]

In my day we wrote algorithms down on paper, and labelled the steps with numbers or letters!

1. Take a slice of bread out of the bag.
2. Place it in toaster
3. Set to light brown
4. Push down lever

We judged algorithms by taste.

Re:Here's what you do

[Anonymous Coward]

Why are you toasting your bread bag? Though perhaps you should plug in the toaster and check the circuit breaker and electric bill first. Like so many programs that assume a GUI exists and don't fall back to text UIs when a graphical display isn't available. Programming robust programs is difficult.

You are making too much use of this resource

[Anonymous Coward]

Enter this captcha so we know you're not a robot. Then go away for awhile.

Transparancy

[hackwrench]

Because the processes were so transparent to begin with. At least the algorithms have the possibility of being looked at. Maybe that should have been the story.

Re:Transparancy

[Anonymous Coward]

First of all, the problem is these systems increasingly take away power from the judicial and other democratic systems, which actually were somewhat transparent. You can see the law that affects you if you want, it's public. With algorithms, you can't. An example are the 'risk of recidivism scores' that increasingly influence judges' sentencing. Those, it turned out, disproportionally called black people risky.
https://www.propublica.org/ser...

Secondly, those new algorithms are rarely transparant, for a number of reasons. There is an incredibly faith in 'neutral' algorithms and 'objective' data, and even if you break through that resistance, you still find that the algorithms are often carefully guarded business secrets. Do you know how the Facebook algorithm that shows you your news works? Can you hold its designers accountable?

The whole thing, if anything, just adds a new layer of obfuscation that is actively abused. See the Volkswagen Diesel scandal for example.

Seriously, read Weapons of Math Destruction, it's awesome. Or check out Frank Pasquale's book:
https://youtu.be/PDjgyTnzWuQ

Re:Transparancy

[ZorroXXX]

But even if the algorithms are 100% open and transparent, that means nothing if the data feed into them is poor. If the bank uses an algorithm to determine if it want to lend money to you, how is the data about you collected? Who decided to classify you as a say medium risk person? What cirterias did he/she/they use for that? How thorogh were he/she/they in gathering decition material? What did he/she/they miss/ignore/misunderstood?

Unless there is full and complete transparency and accountability for data collection, the transparency for just the algorithms is without value.

Re:Transparancy

[jenningsthecat]

... At least the algorithms have the possibility of being looked at. Maybe that should have been the story.

Algorithms that determine what goods and services a person has access to, the quality of said goods and services, and how much he or she pays for them, need more than the "possibility" of being looked at. They need legislation that requires them to be publicly released in their entirety before being put into use, and every modification, bug-fix, etc. needs to be published as soon as, or before, implementation takes place. And there need to be really onerous and consistently enforced penalties for not releasing them.

Algorithms have too much power over too many aspects of our lives for us to allow them to be another opaque, inaccessible tool employed by tyrants and by the wealthy to use and control us plebeian masses. An FOSS-like model needs to be mandatory for those algorithms that can and will shape our lives and, often, our deaths. Practically speaking, that means that ALL algorithms used by business and government should pass into the public domain immediately upon deployment.

Re:Transparancy

[hackwrench]

The Volkswagon Diesel scandal was because the decisions Volkswagon executives made weren't transparent, not algorithms. Despite laws being readable, when it comes to enforcing and interpreting them, you have to rely on nontransparent judges and juries. People aren't transparent and it has always gone back to people, no matter how much you want to throw shade on the new stuff, or claim that this or that aspect of the old stuff somehow made things transparent.

Who cares about this

[Anonymous Coward]

THEres a goddamn Russian spy ship off our coast and they have goddam nukes

Re: Who cares about this

[Kkloe]

At least you know you have it there, would be more of problem if you didn't know

Re: Who cares about this

[PopeRatzo]

Unless they don't care if you know, in which case that may be even worse. Now that they've taken 1600 Pennsylvania Avenue, who's going to stop them?

Re: Who cares about this

[Anonymous Coward]

No, the real russian agents are living in 430 South Capitol St. SE, Washington, D.C., 20003

Computer says no...

[Anonymous Coward]

...no you can't do what you want.

But who can I talk to?

No one, the computer has made up its mind.

Re:Computer says no...

[Anonymous Coward]

...no you can't do what you want.

But who can I talk to?

No one, the computer has made up its mind.

When the power to the computer is disconnected, there is no computer - it's a paperweight.

The reader is left to draw his or her own conclusions regarding the usefulness of the preceding info.

Re:Computer says no...

[Anonymous Coward]

You don't have the power to disconnect the computer, criminal terrorist scum.

Computer says my ass is covered. Fuck you, I got mine.

Certain groups?

[AHuxley]

What groups?
Some federal database might buy a state database and find lots of illegal migrants getting free city or state services?

That a person is a religious covert? Does their faith or cult have issues? A person buying products or searching for topics that get reported and tracked?
A person looking to travel? Most of the US online tracking is looking for any trace of radicalization and mobilization. Is a person of interest looking up interesting things?
Get some US gov/mil work? Need a polygraph? Expect to be tracked online for a while before the polygraph to see if you are doing any online or book reading to evade the polygraph test.
If your searching for words that the US gov has an interest in tracking expect to be tracked. Brands sell their user data in bulk to anyone. Governments can buy sorted data from the private sector on any topic.

Re: Certain groups?

[Kkloe]

All that a government might want to do something about in the future, don't just think about the us where the gov already have their people's asses

Re:Certain groups?

[Anonymous Coward]

Polygraph does not work.

>The consensus view among scientists is that polygraph testing has no scientific basis?
>The FBI considered the creator of the lie detector test to be a phony and a crackpot?
>The man who started the CIA's polygraph program thought that plants can read human thoughts?
>The foremost polygraph advocate in academia was discredited by a federal judge?
>A prominent past-president of the American Polygraph Association is a phony Ph.D., and this premier polygraph organization doesn't consider it an ethics problem?
>The longest polygraph school produces newly minted polygraphers in just 14 weeks -- less than half the time it takes to graduate from a typical barber college?
>The National Center for Credibility Assessment (the erstwhile DoD Polygraph Institute) suppressed a study suggesting that innocent blacks are more likely to fail the polygraph than innocent whites?
>The researcher who developed the U.S. Government's polygraph Test for Espionage and Sabotage "thought the whole security screening program should be shut down?"
>The National Academy of Sciences concluded that "[polygraph testing's] accuracy in distinguishing actual or potential security violators from innocent test takers is insufficient to justify reliance on its use in employee security screening in federal agencies?"
>Spies Ignatz Theodor Griebl, Karel Frantisek Koecher, Jiri Pasovsky, Larry Wu-tai Chin, Aldrich Hazen Ames, Nicolás Sirgado, Ana Belen Montes, and Leandro Aragoncillo all passed the polygraph?
>One of the most prolific serial killers in U.S. history passed the polygraph and killed again?
>Al-Qaeda and Iraqi insurgents know full well that the lie detector is bogus?
>You don't have to be a psychopath, go to spy school, or somehow believe your own lies to fool the polygraph? (We'll reveal how it's done.)

Algorithms or what?

[Anonymous Coward]

The author seems to think that the alternative to "algorithms" is people making good decisions. That's simply not true. The alternative is people attempting, or not, to follow some agreed on some ill-defined process.

Note that algorithms make it harder to ignore that we have to make tradeoffs. To take one of the examples from the article, we may have to choose between "well-respected teachers" and those who actually help students significantly. (Of course, we could reveal performance information and let parents make the decision for their kids, but ....)

Re:Algorithms or what?

[Sique]

No, the author makes the point that algorithms don't exculpate anyone from making bad decision. "The computer said: No." is no excuse for mishandling someone. We had the example on Slashdot of the algorithm that tries to predict recidivism and thus recommend probation or prison. A deeper analysis showed that it was biased against black people because it predicted higher recidivism for them than they had in reality, and it was biased pro whites as it predicted lower recidivism rates than real. And it was not even factoring in the skin color of the people in question. But the way it weighed the socio-economic factors seems to be the problem. It was scoring high on recidivism when many socio-economic risk factors were slightly up, but gave low scores when only one or two risk factors were high, but all others were low. Thus it was overestimating the recidivism rates of poor people with a weak family background, but completely missing the recidivism risk of well off people from a stable family, but deep personal problems.

But because the program was actually used in judiary decisions in several States, it unnecessarily sent people to prison, while it recommended to set high risk people free on probation, and it did it with a strong racial bias that was contradicted by reality.

Re:Algorithms or what?

[avandesande]

Culpable or not people find it easy to offload their morals when there is a system to blame. Take Nazi Germany as example...

Re:Algorithms or what?

[Sique]

Yes, it's Milgram all over again. The seemingly knowledgable expert is replaced by the seemingly neutral algorithm, and thus people blindly follow their directions.

"persuadable voters"

[CaptainDork]

"undereducated voters"

Re:"persuadable voters"

[Ihlosi]

It is a mistake to assume that anyone who is susceptible to rhetoric is 'uneducated'. Rhetoric and persuasion exploit the time gap which it takes for knowledge and reason to kick in, for one thing.

Re:"persuadable voters"

[Anonymous Coward]

It is a mistake to assume that anyone who is susceptible to rhetoric is 'uneducated'. Rhetoric and persuasion exploit the time gap which it takes for knowledge and reason to kick in, for one thing.

As proved by the highly educated Germans choosing to follow A. Hitler.

Re:"persuadable voters"

[CaptainDork]

... mistake to assume that anyone who is susceptible to rhetoric is 'uneducated' ...

Think that over a bit:

In the 2016 election, a wide gap in presidential preferences emerged between those with and without a college degree. College graduates backed Clinton by a 9-point margin (52%-43%), while those without a college degree backed Trump 52%-44%.

Re:"persuadable voters"

[Catiline]

I read the phrase "think that over" as meaning you disagree with the quoted text. However, your link does not contradicting that statement, unless you also believe Clinton employed less rhetoric during the campaign. Wikipedia defines rhetoric as "the art of discourse, wherein a writer or speaker strives to inform, persuade or motivate particular audiences in specific situations". Even if we use the less formal definition ("excessively flowery or emotional, often meaningless, speech") can you truly say Clinton employed fewer such devices?

I find such assertions ridiculous; the politician's primary job is to be persuasive, and the most effective tools to do so are the subtle, often trite, phrases which hook into their audience's existing biases. Someone arguing that a slogan, whether "Stronger Together" or MAGA, is not an attempt to persuade tells me more about their bias than it does the politician using it.

Re:"persuadable voters"

[Anonymous Coward]

Think that over a bit:

Perhaps you should think it over yourself, but I'd be surprised if you've ever done ANY thinking for yourself.

You're assuming (because you're an idiot) that EVERYONE that voted for Trump HAD TO BE PERSUADED and everyone that voted for Hillary naturally made the choice themselves. All the newspapers, Hollywood icons, etc. that were screaming for months how Trump was Hitler reincarnated and somesuch was just... you know... NO PERSUASION THERE. Posting bullshit infographs that mean nothing? NO PERSUASION THERE. You decided YOURSELF to vote for Hillary Clinton (who personifies literally EVERYTHING you hated about George W Bush)) and not vote for Trump because he was homophobic (despite all evidence to the contrary) and racist (despite all evidence to the contrary) and evil.

You're stumbling around like a zombie.

The origin of the word ...

[CaptainDork]

Al Gore rhythm

Thank you BeauHD for more proof

[the_Bionic_lemming]

I'm hoping everyone on Slashdot posting here understands what an algorithm actually is.

My guess is they don't, because BeauHD obviously doesn't in his rush to slam trumpet.

https://en.wikipedia.org/wiki/...

To put in layman terms - everything automated takes parameters and creates a result. It's because people make a sorta flowchart that draws on information and slam bam Kazam - outputs a result.

Which is now evil because of Trump.

Whipslash, please, I'm begging.

Tell him to stop shifting the site to SJW and EnviroNazi!

Computer says

[rsilvergun]

no

Thank you

[Anonymous Coward]

Author, for referring to them as algorithms and not 'OMG AI!!'. No forward progress can be made ethically, practically, or scientifically with such ridiculous, pubescent misnomers.

Re: Thank you

[Anonymous Coward]

insurance claim adjudication is quite automated. all the BS about "death panels", that stuff is built into the insurance platforms already.

car claim? theres vendors who do the adjudication on different aspects of your claims on behalf of your insurance co. theres companies that have a high rate of denying your claims. so naturally theres people you can call that will help advocate on your behalf to get what you deserve, because they have special knowledge those companies dont want you to know...

Health insurance is big into this... from "is that ICD10 code covered by your plan today" to "does it jibe with the other ones on the claim" etc., have you medically received too many xrays (not just cost limits)...theres vendors who manage authorizations for those too.

there is at least one company that tracks lifetime medical xray exposures (on medical and financial bases), and provides authorizations or denials back based on that...

insurance claims all at least initially all evaluated and determined by algorithms...

already sneaking in will be even more external monitoring of subscribers - FitBit data, household IoT appliances, agreeing to sign up for "wellness" programs at work in exchange for a few bucks here and there, etc.
Go to chiropractor on your health plans' bill? you'll get asked aggressively if said claims are related to a car or work place accident...

Soon to come, "excercise with Kim Kardhassian"! apps on your Vizio tvs, where Kim 'talks' with you through your workouts! (but 24x7 internet connectopn required of course...)

"Winston, are you ok? you havent done your excercises today! we're worried about you!"

again, it'll seem voluntary at first. but soon enough it will not really be: want a tax refund? only if you signed up to the wellness apps.
want your SNAP check this month? Medicaid benes? SSA? Medicare?

slowly and surely...

It is easily slipped in with ads like "you're fit and healthy. We'll help you stay that way so you dont have to help pay for those that are too lazy, er, keep our rates low for you...(you know who we're talking about!" as Walmart shopper, homeless, Sec 8 housing, old people, etc... pics flash by on the ads...)

we just cannot seem to resist these appeals and not participate or get a couple of bucks... money is money, right? your friends tell you...

Just look at MetroMile car insurance ads on TV...

source: I work at health insurance co with its multi+faceted claims platform and apps. This exposure also made me notice more details about a PIP car insurance claim I had last year too and how it managed my medical claims...

In other news... Integers: Why so many?

[mbeckman]

Is this what passes for intelligent discourse at Phys.org? So many errors, so little time. First, an algorithm is NOT involved when I pick a movie to stream on Netflix. I pick the movie, Netflix streams it. Unless you want to count the code necessary to display a web page and process a click. Second, algorithms are NOT "complex mathematical formulas." A formula is a specification for a single computational step (or a series of similar steps, in the case of calculus). An algorithm is a non-mathematical procedure, with memory, decision making, input and output from and to various sources and sinks, and, well, formulas. Algorithms contain formulas, but formulas don't contain algorithms. And phys.org does not contain the sense God gave raisins.

Re: In other news... Integers: Why so many?

[Anonymous Coward]

The algorithms control what Netflix presents as options to you to watch. It prioritized search results. And it changes out trailers and descriptions while you're browsing.

Re: In other news... Integers: Why so many?

[Anonymous Coward]

Thankfully there are alternatives to Netflix. You heard about them a few days ago right here. The Pirate Bay, Primewire, Movie4k, TorrentProject and TorrentButler.

Re: In other news... Integers: Why so many?

[Anonymous Coward]

ironic if they were really being run by the stidios or Netflix themselves...

Re:In other news... Integers: Why so many?

[Anonymous Coward]

I wouldn't go as far as calling algorithms "non-matematical procedures". If you take a linear algebra course for example, you're bound to come across lots of algorithms (e.g. the Gram-Schmidt algorithm) where every step of the procedure is an equation, and it would be appropriate to describe the entire algorithm as a complex equation. Similarly, lots of the algorithms used to "pick a movie to stream" are in fact just complex statistical equations. Even in the case of more classic compsci algorithms, such as sorting algorithms, there are subfields of mathematics that study precisely these algorithms.

Re:In other news... Integers: Why so many?

[Anonymous Coward]

A deterministic algorithm is a computable function and therefore a math formula.

Re:In other news... Integers: Why so many?

[Anonymous Coward]

Do you really think an algorithm is not involved in picking a movie from Netflix? There are hundreds of movies available, so unless you looked at all titles before you chose, it was an algorithm that decided which ones to show you to choose from.

In other words, say there are 300 movies you could have streamed on Netflix, but you only looked at 30 titles before choosing one to watch. The algorithm that selected those 30 is the one involved in this scenario.

dom

Re:In other news... Integers: Why so many?

[dwpro]

Not to mention the algorithms compiling/compressing/encoding/drm'ing/encrypting/routing/load balancing/estimating/throttling. And that's just the NSA.

Re:In other news... Integers: Why so many?

[Anonymous Coward]

Algorithms are mathematics.

https://en.wikipedia.org/wiki/Turing_machine

Way back when...

[SuperKendall]

In my day, we had a simple and effective way to judge algorithms:

Well in MY day we didn't have the luxury of ignoring the 'C' so readily.

Re:Way back when...

[BeauSD]

This is how I learned C

Re:Way back when...

[Pseudonym]

We still don't.

algorithm says you're unemployable

[Anonymous Coward]

Your resume just doesn't have the keywords we're looking for. Fuck off and die.

Agent Smith

[Zobeid]

"Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization, because as soon as we started thinking for you, it really became our civilization, which is of course what this is all about."

Re:Agent Smith

[Anonymous Coward]

Sadly the last years of the 20th century were the peak of our civilization, for the years which followed brought us into the 21st century of the War On Terror, the Great Recession, and President Trump.

Re:Agent Smith

[geekmux]

Sadly the last years of the 20th century were the peak of our civilization, for the years which followed brought us into the 21st century of the War On Terror, the Great Recession, and President Trump.

I more thought of the last years of the 20th century as peak corruption. You know, back when you could sell vaporware for millions, resulting in the dot bomb crash.

Or maybe peak corruption was in 2008 when the deregulated banking industry started fucking about, resulting in one of the worst financial crashes in history.

And then we come to today, where companies can file for multi-billion-dollar IPOs after demonstrating a unique ability to lose hundreds of millions per year, and may never sustain profitability, resulting in...

Seems the only constant we've demonstrated is our ability to accept corruption as a legitimized component of capitalism, resulting in a predictable Rise and Fall.

Re:Agent Smith

[Anonymous Coward]

Sadly the last years of the 20th century were the peak of our civilization, for the years which followed brought us into the 21st century of the War On Terror, the Great Recession, and President Trump.

You forgot Big Data. There used to be a benevolent web search called Google back in the end of 20th century. It is no more.

Re:Agent Smith

[david_thornley]

I generally think of corruption as the state where people in positions of authority make decisions based on what they personally get from involved people rather than according to how their authority should be used. For example, if a police officer writes speeding tickets based on whether the driver hands over $100 or not is corrupt. By this criterion, I'm not sure your examples qualify.

Selling vaporware or stock typically involves unforced agreement on both sides. It's stupid to invest much in vaporware, and to get involved in some IPOs (I was amazed that Facebook stock sold at its IPO price), but bad judgment isn't corruption, and I'm not sure fraud should be either. I don't know about all the incentives involved in the banking crisis. Loan origination companies issued NINJA mortgages because they knew they could sell them for a profit. It all wound up in a horrifyingly expensive house of cards, but I don't know how much actual corruption was involved.

Algorithms are just more weapons

[doug141]

"A federal appeals court decisively struck down North Carolina’s voter identification law on Friday, saying its provisions deliberately “target African-Americans with almost surgical precision” in an effort to depress black turnout at the polls."-July 29, 2016
https://www.nytimes.com/2016/0...

Re:Algorithms are just more weapons

[Pinky's Brain]

One of 14 unequal outcomes that will make you say "fuck having law and order and shit".

The next step

[Anonymous Coward]

... who is on a "no fly" list ...

So, this is the next step in push-button policing: No investigation of the unwanted behaviour, no questioning of the dissident, probably, not even a confirmation of the report; the data and a secret algorithm decide someone is too dangerous to walk on a plane but can drive a vehicle without causing harm. Given that people have landed on the no-fly list for having a name that 'sounds' like a terrorist, it's not much of a failure in national security.

The real danger, like with arrest records and police records, is that someone will decide it's a good reason to deny someone on the list, access to modern services. We've seen it already with US politicians demanding the "terrorists" on the no-fly be denied the ability to buy more guns. Their demands didn't (and couldn't) take guns away from those dissidents, just provide punishment for being a "terrorist".

Even your coding success depend on Google's result

[Anonymous Coward]

For example, someone who has been searching internet as a systems admin, will have pre-trained their version of Google to efficiently answer questions in one domain, which the generic search might not provide. So, one resolves bugs faster, and the other loses the job.. But with Google's choice at hand, Google could just tune the level of its usefuleness for different goals on a per-person basis... Give some users more useful version of Google than others, etc. Would having the Jeff Dean's version of Google search make you a better programmer, or an ice hockey professional player's version of Google?

Black Box Society

[Anonymous Coward]

A recent report on this in the Netherlands summarised it as "playtime has to be over". Big data and the algorithms that work on top it are getting a serious amount of power over our lives. Any little scrap of data is starting to influence your chances of getting a job, a cheap loan, or even a date.

If you want to know how scary this gets, check out this presentation by Alexander Nix on how he used this type of data to influence the elections.
https://www.youtube.com/watch?...

Or have a look at the new "Social Credit Score" that China is implementing, in which every citizen gets a score that shows if they are a well behaved citizen or not.
https://en.wikipedia.org/wiki/...

I also highly recommend watching this interview with professor Frank Pasquale, which summarises the issue.
https://youtu.be/PDjgyTnzWuQ

(Academic students here are not allowed to cite Weapons of Math Destruction, but you are allowed to use his book)

Demand access to pricing algorithms

[drinkypoo]

We should be demanding access to the data and algorithms used to generate pricing for mandatory services like the ACA, home insurance, and automobile insurance. We should never be required to buy anything without even knowing the basis for the charges.

Re:Demand access to pricing algorithms

[geekmux]

We should be demanding access to the data and algorithms used to generate pricing for mandatory services like the ACA, home insurance, and automobile insurance. We should never be required to buy anything without even knowing the basis for the charges.

When greed and corruption pervert capitalism, one doesn't have to look far to understand pricing.

Let's also not pretend insurance is a business concept that has ever struggled to survive. They collect a few billion, and then hire an army of lobbyists to ensure their flavor greed is mandatory.

How this corrupt process works isn't some kind of mystery to solve.

models may re-enforce themselves too

[tommeke100]

Take the deployment of Police resources. They try to predict in what area crimes are more likely to happen at a time in point. Area has higher crime rate, police forces are deployed in the area. Result: even more crimes recorded in that area.
Model says people are not the right candidate to get the job. Result: you don't get the job and other people who look good to the model get the job. Again, more data that "proves" those people should not get that job and the other people do.

Re:models may re-enforce themselves too

[Pinky's Brain]

Few people count the results of random searches in the crime rate. Crime rate depends on crime reported, not recorded.

Re:models may re-enforce themselves too

[david_thornley]

Before models, managers would interview potential new hires, and would generally hire or not hire based on the impression they got of the candidate. That's not necessarily better than a faulty algorithm. The algorithm can be examined and changed.

It's a long winded warning of...

[azemon]

... garbage in, garbage out.

Re:It's a long winded warning of...

[Anonymous Coward]

That's essentially true. Also if you read that book, there are some government solutions proposed. In other words a power grab with more tax money spent and wealth distributed to cronies of the guy in charge.

But that's business as usual for the US government. Garbage in, garbage out.

Sjw's

[argStyopa]

The fact is that SJWs cannot seem to comprehend that inequality in result isn't itself proof of some bias, PARTICULARLY if the bias-factor isn't even part of the algorithm.

Further, the fear is that simple objective analysis will occur without human intervention, and thus lack someone to call racist, sexist etc (in essence, so they're pre-labeling the author of algorithms as racist, sexist etc.).

For example
Your algorithm shows that people below a certain income level fail to repay loans at the normal rate, so it calculates the interest rate upcharge needed to offset the lower rates of repayment. Said upcharge seems to be applied more frequently to minority borrowers = "racism" (even though it's based entirely on income, not skin color)

en francais

[holophrastic]

The first movie I ever watched on netflix was Inside Out. At the end, Netflix's first recommendation was that if I liked Inside Out, I should watched Inside Out in french.

Great algorithm there. Oh the complexity. What's next? The spanish version?

Probably the worst suggestion any person could have ever made to anyone outside of a french class.

The algorithm must have been so happy. Think about it. It found a movie, where every word spoken is totally different, but there's a 100% match on the title! Woohoo! A perfect match! What a perfect recommendation!

Years of netflix recommendation engine contests. Well done.

Foolishness

[Anonymous Coward]

Thrre will be no point in the future at ehich we just get to turn off our brains. Relying on software to this extent is the height of stupidity, regardless of its capabilities. It isn't 'biased' - it's effing software. We have become so unfathomably lazy and stupid it boggles the mind. Don't want software fucking things up? Then don't implement it where it shouldn't be implemented, it's that simple, as it will likely never reach a state of 'good enough', let alone 'perfect' for these types of situation. This is all greed and insanity on overdrive, and nobody in the U.S. gives a shit so long as they can include a selfie. We are doomed, perhaps, but it's because we are fucking morons, not because we have technology.

Not so

[fyngyrz]

Er, not really. As long as the "intelligence" takes the form of algorithms, that means human beings are devising sets of rules for computers to follow.

No, it doesn't mean that at all. You're not considering that machines can write algorithms. And they certainly can. Genetic software (which we can very accurately describe as an implementation of "nature's algorithm") has been doing that for decades now, and the deep learning mechanisms we're just beginning to explore now could be leveraged in similar ways, perhaps already are.

And that's without any real advances towards actual AI. With such advances... who knows where algorithms might go. Or come from.

Al, go rhythm -- you'll like it.

[fyngyrz]

Don't forget the algorithm that determined which "hundreds of movies" out of the zillions that have been made that you got to choose from out of the first 30, and the algorithms that the movies studio used, and the algorithms that the effects companies used, and the algorithms that determine which actors were "hot"...

To say that making a choice on Netflix is "algorithm-free" is to not even remotely understand the world one lives in.

Re:Al, go rhythm -- you'll like it.

[Anonymous Coward]

To say that making a choice on Netflix is "algorithm-free" is to not even remotely understand the world one lives in.

And this is Slashdot. Imagine how clueless Joe sixpack is...

Meta Thinkers

[zifn4b]

Doesn't work on us. We can think at the same level as the creators of said algorithms. It's not a "War of Math Destruction" it's a "War of Meta Thinking". It is akin to playing a game of chess and trying to guess based on previous experience from playing your opponent what you think they will do and make a move to counter it. But if you're opponent is thinking in the same manner and is thinking you might think in this way and anticipates you arriving at that conclusion, he/she can counter your counter.

This is why society especially in America seems to have a preference for blind obedience. They are "easier" to deal with (read: easier to exploit) than those who are more aware of these types of games.

Re:Meta Thinkers

[Anonymous Coward]

I simply spent two years building up an immunity to iocaine powder. -PCP

 

a "certainty" code smell

[epine]

You're not considering that machines can write algorithms. And they certainly can.

I've spent my entire life trying not to be this dim. Yes, very clever work there treading on the narrow definition—while engaged in 100% baby flush.

The ridiculousness of this is apparent to any thinking person in less time than it takes to type "Wittgenstein".

Because some human process defined the solution gradient that the "genetic" software optimized over—ad infinite turtle—in an act of algorithmic emancipation now glibly lumped under the verb "write" by the baby-impervious membrane all-too-tragically-often comporting itself as "logic".

some linguistic navel gazing

[epine]

After pressing "submit", in a split-second second evaluation, I noticed that that sentence I wrote does not quite work.

Problematic:

Because some human process defined the solution gradient that the "genetic" software optimized over—ad infinite turtle—in an act of algorithmic emancipation now glibly lumped under the verb "write" by the baby-impervious membrane all-too-tragically-often comporting itself as "logic".

Less problematic:

Because some human process defined the solution gradient—ad infinite turtle—that the "genetic" software optimized over, in an act of algorithmic emancipation now glibly lumped under the verb "write" by the baby-impervious membrane all-too-tragically-often comporting itself as "logic".

Cognitively, this is a turtle too soon for full effect.

Even less problematic, but horrifying:

Because some human process defined the solution gradient that the "genetic" software optimized over—ad infinite turtle—optimized over in an act of algorithmic emancipation now glibly lumped under the verb "write" by the baby-impervious membrane all-too-tragically-often comporting itself as "logic".

Horrifying because I've always regarded this kind of sub-phrase repetition as the hallmark of hack speechwriters. One can partially excuse this by placing a semicolon in front of the repeated phrase, but here the semicolon is incompatible with the closing mdash.

Language is a complex solution gradient, one that humans have yet to successfully express. Sad. All those unemployed genetic algorithms, awaiting human clue.

Re:Even your coding success depend on Google's res

[Anonymous Coward]

We don't like this guys worldview, let his search results get lower quality results. We can counter this belief system by giving only results that are contrary to it.

algoritms shape attitude

[Anonymous Coward]

Facebook has been proving this for years.

whatever does not profit or align with FB's board, does not "trend".

Outrage is driven by algorithm on every social media platform.

And now the algorithms push "fear" to the level where it manifests as real world violence.

Credit ratings often self-full-filling prophecies

[knorthern knight]

> You are leaving out the money part. Credit is not rich. Its debt. You
> want to be rich, have money as savings. You want to be wealthy, have
> money as assets/investments. Most don't need credit except for a car
> (still poor), a home (very common), school loans (credit may affect rates).

That's how it used to be. Nowadays, credit ratings are part of the hiring process. If you have a bad credit score, you can't get a promotion, or in some cases even a job. So you have no income and default on loans. And the credit rating algorithm shows up as being "successful".

Or at best, you're stuck in a lower income job. Either way, you end up poorer because of credit ratings, even if you have no intention of applying for a loan.