Algorithm Predicts US Supreme Court Decisions 70% of Time

stephendavion writes A legal scholar says he and colleagues have developed an algorithm that can predict, with 70 percent accuracy, whether the US Supreme Court will uphold or reverse the lower-court decision before it. "Using only data available prior to the date of decision, our model correctly identifies 69.7 percent of the Court's overall affirm and reverse decisions and correctly forecasts 70.9% of the votes of individual justices across 7,700 cases and more than 68,000 justice votes," Josh Blackman, a South Texas College of Law scholar, wrote on his blog Tuesday.

Comments

biased algorith

[Dthief]

I (read: anyone) can make an algorithm that fits any previous data (even only using data that precedes the "prediction")......testing future predictability is the only way this means anything.

Re:biased algorith

[Anonymous Coward]

Yes, and then when the algorithm doesn't work you finetune it a bit and test again and suddenly you end up with an algorithm that has been trained on all data without actually training it against all data.

One should be very skeptical against future predicting algorithms. Until they have been released in the wild for a while without the developer tampering with it it is pretty safe to guess that it more or less is another version of the Turk, even if its inventor doesn't realize it.

The same principle can be applied to market research or climate studies. If the algorithm used is tampered with to produce more accurate results one can assume that it is useless.

Useless

[Jiro]

According to http://www.scotusblog.com/stat... the Supreme Court recently affirmed 27% of lower court decisions and reversed 73%. This means that if you guess that the Supreme Court reverses the lower court every time, you'll be 73% accurate. 70% accuracy is ridiculously low if you can get 73% accuracy *without* taking into consideration the records of each justice or any other kind of details.

Re:is it better than random?

[mrvan]

That is correct, but not what the GP meant. If you can model the distribution (e.g. you 'know' that B is 90%) then you can weigh your random guessing such that it is correct in >50% of the cases, even without looking at the case itself (it is still 'random' in that sense)

Extreme case: I can predict whether someone has Ebola without even looking at them with >99.99% accuracy by just guessing "no" every time, since the prevalence of Ebola is >.001%.

Suppose the supreme court has 70% chance of overturning (e.g. because they choose to hear cases that have 'merit'), then an algorithm that guesses 'overturn' 100% will have a 70% accuracy. A random guess that follows the marginal of the target distribution (e.g. guess 70% overturn) also scores >50% (58% to be precise).

Re:Replace them

[Impy the Impiuos Imp]

Lawyers: We want people to carry their rights with them, even when operating as a group of people Congress defined as a "corporation" because Congress cannot force them to give up their First Amendment rights.

Scotus (in the voice of Nomad): Logic correct. Opposing lawyers are in error. Must sterilize.