Slashdot Mirror
Algorithm Predicts US Supreme Court Decisions 70% of Time
stephendavion writes A legal scholar says he and colleagues have developed an algorithm that can predict, with 70 percent accuracy, whether the US Supreme Court will uphold or reverse the lower-court decision before it. "Using only data available prior to the date of decision, our model correctly identifies 69.7 percent of the Court's overall affirm and reverse decisions and correctly forecasts 70.9% of the votes of individual justices across 7,700 cases and more than 68,000 justice votes," Josh Blackman, a South Texas College of Law scholar, wrote on his blog Tuesday.
I (read: anyone) can make an algorithm that fits any previous data (even only using data that precedes the "prediction")......testing future predictability is the only way this means anything.
www.RacquetUp.org - Helping Detroit Youth
Maybe it'll cut down the number of frivolty in lawsuits once businesses realize their cases are futile.
Or not.
They beat my 50% accurate algorithm...
Just identify the wrong decision and they are bond to pick it ;-)
If the decisions have 50/50 distribution, then a random guess is right 50% of the time. For any other distribution it's more than that. Soooo 70% is at best a little bit better than random guess, at worst equal to it.
So does that make them 70% full of shit, or 30%?
Gay Nigger gaydar detects niggers with 1000% accuracy.
People have been predicting outcomes for years. There was a story a couple of months back about something similar. And here's a link to a group that stated 75% success predicting the outcome prior to oral arguments, back in 2004 http://www.jstor.org/discover/10.2307/4099370?uid=3738032&uid=2&uid=4&sid=21104566455723. I can't comment on the relative academic merits of either though.
--- To save space, would readers please insert their own witty comment -here-
It would be useful to know how many of the court's decisions are affirm vs reverse. If 70% are affirm, it's not impressive to be able to correctly predict 70% of decisions since you can just always guess on "affirm". Of course, if it is equally distributed (50% affirm, 50% reverse), getting 70% correct shows the algorithm has some prediction power (assuming it is trained on a different dataset than is used for evaluating it). But it is impossible to determine if this is the case, based on the information in the article.
According to http://www.scotusblog.com/stat... the Supreme Court recently affirmed 27% of lower court decisions and reversed 73%. This means that if you guess that the Supreme Court reverses the lower court every time, you'll be 73% accurate. 70% accuracy is ridiculously low if you can get 73% accuracy *without* taking into consideration the records of each justice or any other kind of details.
if defendant.bank_balance > plaintiff.bank_balance
winner = defendant
else
winner = plaintiff
I'd guess about 90% accurate.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Just write an algorithm that determines which decision will leave the American peoples' ass stretched open the widest, when they're bent over and fucked.
I wish they could replace the SCOTUS with computers.
Lawyers, "We want corporations to have the same rights as people."
SCOTUS (in the voice of Nomad from ST:TOS), "Non sequitur. Case dismissed."
So what they've done is create a system marginally less accurate than "return 'reversed'".
Install software in the helmet, Set the judges loose on the city....
I AM THE LAW!
Do not look at laser with remaining good eye.
A more useful algorithm would be one that evaluates if the decision is free from bias and not influenced by money.
Of course, finding the test data for it would be a bit of a problem.
The article talks about predicting decisions going back to 1953. It also says it's easy to come up with good predictors for specific time ranges. Your rejection algorithm works well for the last year or so, but the article you cite is based on the last years statistics only. The actual article talks about using a whole pile of inputs and learning a good predictor. It sounds like it would have easily learned your strategy, though the article isn't clear. Apparently the algorithm is doing just about as well as humans trying to predict the decision, where the best humans have just a small amount better track record.
A 70% prediction rate is not impressive. In the UK, where the weather seems pretty unpredictable, "it will be pretty much the same as yesterday" is right about 70% of the time. Weather forecasting and track individual storms, but It took a long time and a lot of research for the weather forecast success flat rate to get any better than this. The model may be important: the success rate probably isn't.
If we outlawed private campaign funding it would be correct 100% of the time.
It's within 1 standard deviation of being no better than a coin toss.
Here's that algorithm:
#DEFINE FREE_FROM_BIAS_AND_FINANCIAL_INFLUENCE 0
#DEFINE FREE_FROM_BIAS_BUT_INFLUENCED_BY_MONEY 1
#DEFINE FREE_FROM_FINANCIAL_INFLUENCE_BUT_POLITICALLY_BIASED 2
#DEFINE POLITICALLY_BIASED_AND_INFLUENCED_BY_MONEY 3
int algorithm() {
return POLITICALLY_BIASED_AND_INFLUENCED_BY_MONEY;
}
I wouldn't be surprised if the primary predictive trait used is simply to check the biases of each judge and then assume they will vote along those biases. Assuming conservative judges will vote conservative and liberal judges will vote liberal should give you a pretty good score right off the bat.
-- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
Tyranny equals enough money has been thrown into the court room.
The US Constitution is only about 4 pages, 4400 words (and the bulk of that is structural & procedural minutiae about the US government).
The role of the USSC is simply resolving if a law does or does not conform to the US Constitution.
Given those relatively limited boundaries, it shouldn't be that complex of an issue to predict algorithmically the results of a given judicial ruling, one would think. (The devil's in the details about parsing meaning and context.)
Of course, I believe phrases like "the right to keep and bear arms shall not be infringed" are indisputably clear, and I'm astonished that people can find convoluted ways to try to tear it apart syntactically.and semantically.
-Styopa
Correct. I did my thesis work in this area (predicting court outcomes). If you can't beat simply predicting reverse every time, you have nothing.
From a common sense perspective. Consider the effort the court goes through in selecting a case and all the cases that don't get selected. Consider why cases go before the supreme court. The ones that the court hears are naturally going to be those that someone things are important enough to reverse.
I mean when will this ever end?
Name: Mr. Anon E Mouse; SSN: 555-55-5555
Per their own data:
They reviewed 7700 cases.
The court reversed 5077 of those cases.
So the court reverses 66% of cases it sees. Which makes sense, that's what the court does.
So I can get damn close to their results with my model which is: "The court will reverse 100% of the time"
I don't see their model in there, and I don't really care to look that hard. But they said they used the same data previous models did. Most of that data are things like:
Which court heard the origional case?
Was the decision liberal or conservative?
etc...
It seems to be more of a case of, the court is overturning politically motivated decisions made by lower courts. i.e. a Liberal leaning decision out of California is likely to be overturned... or a conservative leaning decision out of texas. But the reverse, a conservative leaning ruling out of California is not. So if a court rules against it's nature it's more likely to stand when heard by SCOTUS, which makes intuitive sense.
I have a better algorithm... written in one line of perl:
Accuracy: 73%
Source http://www.scotusblog.com/stat...
-- I was raised on the command line, bitch
If Obama wants it the GOP will say No.
You can only predict things that have not yet happened. I'll be much more impressed If they publish their predictions to future decisions and these turn out to be 70% correct.
Just follow 3 simple rules; they vote in favor of corporations, vote in favor of the gov't, and vote against the individual. You will be able to predict the court more often than not.
putting the 'B' in LGBTQ+
It's slightly more accurate than a coin flip! Nice! Something tells me even a Fox News anchor would perform better.
Shouldn't this algorithm predict 100% of past court decisions? What we are really interested in is whether it can predict the future - let it run for a few court terms and THEN announce something to the media.
Despite all the (partially true) snark. Isn't this a good thing? Shouldn't the highest court of the land be producing rulings that are predictably consistent with previous rulings? Unless a case is truly novel, past performance should be a good predictor of future performance here, since case law is cumulative.
It should be trivially easy to predict the outcome of SCOTUS cases, as most constitutional issues are black or white.
The constitution does a couple of things. First, it lists a bunch of things the Government CAN'T DO, period, no matter how much it wants. Second, it says that it can only do what the people give it permission to do, and nothing else.
Too often I hear people talking about what the framers "might have meant" or worse, what their "intent might have been." Folks, these are not at issue, because what they meant, and what they intended, are clearly spelled out in their writings of the time. Between the Federalist Papers, the DoI, the Constitution, and the multitude of books these people authored, it is all spelled out in excruciating detail, and by design, without room for creative interpretation.
... it works everytime! https://www.youtube.com/watch?v=pjvQFtlNQ-M
Let the judicial betting begin!
What One has is a more accurate model, which is how science works: One creates a model, tests it, adjusts to improve its accuracy, lather, rinse, repeat. It's worked pretty well for centuries.
... You ignore reality, sure.
is still withing chance because the error bars on the are huge.
The Kruger Dunning explains most post on
... Americans often like to blame Others for problems: Judges, Politicians, Muslims, "illegal Aliens", pro-Lifers, Gays, city Residents, the Poor, the Unemployed, the business Owners, the Religious, the Old, the Young, etc., etc., etc. So, given the large american presence on /., if You find analysis as intelligent as Your's obviously is, it will be a rare gem to be treasured, indeed.
70% isn't very hard to guess. Just basic guessing should get you 50% ave. Then you take political votes, liberal judges are a lock in how they vote even if it goes against the constitution (Obamacare) while republican judges will generally follow what the law says.
With that in mind, 70% is not a good %. It should be 100% possible to predict outcome before the final vote. Read the law and constitution and use it to determine the outcome. You will just get a few wrong with some judges not following the constitution/law.
I'd actually have expected more. I mean, let's be reasonable here:
A 50% accuracy can be achieved by the average unbiased coin. Now throw in the rulings that are easy to foresee because any other decision would be politically suicidal and you should easily arrive at more than 70%.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
...Is that in most cases the decision to 'give cert' for Supreme Court review of given lower-court cases vests with the Justices' law clerks. This may be as weird as that decision to give the Librarian of Congress veto power over unlocking our cellphones, but that's the way it is. How accurate can any model be at delving into the minds of law clerks?
I could flip a coin and be 50% accurate...
I would hope that accuracies far above 50% can be obtained. A random supreme court isn't something anyone wants.
The Supreme Court is dominated by a bunch of fanatic right wing corporate toadies.
So, the decision comes down in favor of corporations (on economic issues) or social conservatives (on social issues).
The Constitution has nothing to do with it.
The Supreme Court is the ultimate cheerleader for our fascist state.
I don't read your sig. Why are you reading mine?
It should be that most legal practitioners can predict Supreme Court rulings, since courts follow the law and past decisions. The Supreme Court, and all courts, could be replaced with an algorithm that correctly applies the laws to facts (facts as found, normally by juries).
The 70+% reversal rate reflects the role of the Supreme Court: to correct the mistakes of lower courts in interpreting the law, most often in the interaction and ambiguity of the law. There's no reason to accept a case unless there's good reason to reverse it.
Finally, the reasoning of the court matters as much as the ruling; by hypothesis, the problem with the case (and others like it), is mis-application of the law, and the reasoning is intended to correct that, so courts can be consistent and people know what the law is so they can follow it.
So applying a scientific idea of "prediction" here is wrong. These are not physical phenomena or stochastic or random processes where a "prediction" is based on "discovering" a "model" of what is "really happening" like science. The rulings are intended to produce clarity. They are bound up mainly in the ambiguity and conflicts of law (as codified) and its application, so the "model" disclosed in each ruling is really an anti-model: tweaks applied to a not-yet-working semantic system to make it more... predictable.
True, the rulings are text and you can analyze them. But I think unless the model is essentially about the application of the law in question, it's probably not illuminating.
I know, can we fire them all and just buy 1 copy of the program. Then, when someone wants an appeal, we could just run the program and have a nice easy answer.
69.7% of the time the answer is "yes"?