Slashdot Mirror
Algorithm Predicts US Supreme Court Decisions 70% of Time
stephendavion writes A legal scholar says he and colleagues have developed an algorithm that can predict, with 70 percent accuracy, whether the US Supreme Court will uphold or reverse the lower-court decision before it. "Using only data available prior to the date of decision, our model correctly identifies 69.7 percent of the Court's overall affirm and reverse decisions and correctly forecasts 70.9% of the votes of individual justices across 7,700 cases and more than 68,000 justice votes," Josh Blackman, a South Texas College of Law scholar, wrote on his blog Tuesday.
I (read: anyone) can make an algorithm that fits any previous data (even only using data that precedes the "prediction")......testing future predictability is the only way this means anything.
www.RacquetUp.org - Helping Detroit Youth
Maybe it'll cut down the number of frivolty in lawsuits once businesses realize their cases are futile.
Or not.
They beat my 50% accurate algorithm...
Just identify the wrong decision and they are bond to pick it ;-)
If the decisions have 50/50 distribution, then a random guess is right 50% of the time. For any other distribution it's more than that. Soooo 70% is at best a little bit better than random guess, at worst equal to it.
So does that make them 70% full of shit, or 30%?
People have been predicting outcomes for years. There was a story a couple of months back about something similar. And here's a link to a group that stated 75% success predicting the outcome prior to oral arguments, back in 2004 http://www.jstor.org/discover/10.2307/4099370?uid=3738032&uid=2&uid=4&sid=21104566455723. I can't comment on the relative academic merits of either though.
--- To save space, would readers please insert their own witty comment -here-
It would be useful to know how many of the court's decisions are affirm vs reverse. If 70% are affirm, it's not impressive to be able to correctly predict 70% of decisions since you can just always guess on "affirm". Of course, if it is equally distributed (50% affirm, 50% reverse), getting 70% correct shows the algorithm has some prediction power (assuming it is trained on a different dataset than is used for evaluating it). But it is impossible to determine if this is the case, based on the information in the article.
According to http://www.scotusblog.com/stat... the Supreme Court recently affirmed 27% of lower court decisions and reversed 73%. This means that if you guess that the Supreme Court reverses the lower court every time, you'll be 73% accurate. 70% accuracy is ridiculously low if you can get 73% accuracy *without* taking into consideration the records of each justice or any other kind of details.
if defendant.bank_balance > plaintiff.bank_balance
winner = defendant
else
winner = plaintiff
I'd guess about 90% accurate.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Just write an algorithm that determines which decision will leave the American peoples' ass stretched open the widest, when they're bent over and fucked.
So what they've done is create a system marginally less accurate than "return 'reversed'".
Install software in the helmet, Set the judges loose on the city....
I AM THE LAW!
Do not look at laser with remaining good eye.
The article talks about predicting decisions going back to 1953. It also says it's easy to come up with good predictors for specific time ranges. Your rejection algorithm works well for the last year or so, but the article you cite is based on the last years statistics only. The actual article talks about using a whole pile of inputs and learning a good predictor. It sounds like it would have easily learned your strategy, though the article isn't clear. Apparently the algorithm is doing just about as well as humans trying to predict the decision, where the best humans have just a small amount better track record.
A 70% prediction rate is not impressive. In the UK, where the weather seems pretty unpredictable, "it will be pretty much the same as yesterday" is right about 70% of the time. Weather forecasting and track individual storms, but It took a long time and a lot of research for the weather forecast success flat rate to get any better than this. The model may be important: the success rate probably isn't.
Lawyers: "We want wealth to equal power."
SCOTUS: "Done. You owe me an upgrade."
Well, there are some things. You can say, for instance, that a court's decision should not depend on the gender of the judge, or the time of day, or on whether the appellant is of the same race as the judge. If you check and find out that these irrelevant things do matter, then you can say that there is injustice even without having any opinion on which way any particular case should go.
But there is very little research of this sort. At the end of the day, the court system is more about convincing people that justice has been served, than about actually serving justice.
If we outlawed private campaign funding it would be correct 100% of the time.
It's within 1 standard deviation of being no better than a coin toss.
Here's that algorithm:
#DEFINE FREE_FROM_BIAS_AND_FINANCIAL_INFLUENCE 0
#DEFINE FREE_FROM_BIAS_BUT_INFLUENCED_BY_MONEY 1
#DEFINE FREE_FROM_FINANCIAL_INFLUENCE_BUT_POLITICALLY_BIASED 2
#DEFINE POLITICALLY_BIASED_AND_INFLUENCED_BY_MONEY 3
int algorithm() {
return POLITICALLY_BIASED_AND_INFLUENCED_BY_MONEY;
}
I wouldn't be surprised if the primary predictive trait used is simply to check the biases of each judge and then assume they will vote along those biases. Assuming conservative judges will vote conservative and liberal judges will vote liberal should give you a pretty good score right off the bat.
-- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
Tyranny equals enough money has been thrown into the court room.
The US Constitution is only about 4 pages, 4400 words (and the bulk of that is structural & procedural minutiae about the US government).
The role of the USSC is simply resolving if a law does or does not conform to the US Constitution.
Given those relatively limited boundaries, it shouldn't be that complex of an issue to predict algorithmically the results of a given judicial ruling, one would think. (The devil's in the details about parsing meaning and context.)
Of course, I believe phrases like "the right to keep and bear arms shall not be infringed" are indisputably clear, and I'm astonished that people can find convoluted ways to try to tear it apart syntactically.and semantically.
-Styopa
Correct. I did my thesis work in this area (predicting court outcomes). If you can't beat simply predicting reverse every time, you have nothing.
From a common sense perspective. Consider the effort the court goes through in selecting a case and all the cases that don't get selected. Consider why cases go before the supreme court. The ones that the court hears are naturally going to be those that someone things are important enough to reverse.
I mean when will this ever end?
Name: Mr. Anon E Mouse; SSN: 555-55-5555
Per their own data:
They reviewed 7700 cases.
The court reversed 5077 of those cases.
So the court reverses 66% of cases it sees. Which makes sense, that's what the court does.
So I can get damn close to their results with my model which is: "The court will reverse 100% of the time"
I don't see their model in there, and I don't really care to look that hard. But they said they used the same data previous models did. Most of that data are things like:
Which court heard the origional case?
Was the decision liberal or conservative?
etc...
It seems to be more of a case of, the court is overturning politically motivated decisions made by lower courts. i.e. a Liberal leaning decision out of California is likely to be overturned... or a conservative leaning decision out of texas. But the reverse, a conservative leaning ruling out of California is not. So if a court rules against it's nature it's more likely to stand when heard by SCOTUS, which makes intuitive sense.
I have a better algorithm... written in one line of perl:
Accuracy: 73%
Source http://www.scotusblog.com/stat...
-- I was raised on the command line, bitch
If Obama wants it the GOP will say No.
You can only predict things that have not yet happened. I'll be much more impressed If they publish their predictions to future decisions and these turn out to be 70% correct.
It's slightly more accurate than a coin flip! Nice! Something tells me even a Fox News anchor would perform better.
Shouldn't this algorithm predict 100% of past court decisions? What we are really interested in is whether it can predict the future - let it run for a few court terms and THEN announce something to the media.
Despite all the (partially true) snark. Isn't this a good thing? Shouldn't the highest court of the land be producing rulings that are predictably consistent with previous rulings? Unless a case is truly novel, past performance should be a good predictor of future performance here, since case law is cumulative.
It should be trivially easy to predict the outcome of SCOTUS cases, as most constitutional issues are black or white.
The constitution does a couple of things. First, it lists a bunch of things the Government CAN'T DO, period, no matter how much it wants. Second, it says that it can only do what the people give it permission to do, and nothing else.
Too often I hear people talking about what the framers "might have meant" or worse, what their "intent might have been." Folks, these are not at issue, because what they meant, and what they intended, are clearly spelled out in their writings of the time. Between the Federalist Papers, the DoI, the Constitution, and the multitude of books these people authored, it is all spelled out in excruciating detail, and by design, without room for creative interpretation.
... it works everytime! https://www.youtube.com/watch?v=pjvQFtlNQ-M
Let the judicial betting begin!
Lawyers: We want people to carry their rights with them, even when operating as a group of people Congress defined as a "corporation" because Congress cannot force them to give up their First Amendment rights.
Scotus (in the voice of Nomad): Logic correct. Opposing lawyers are in error. Must sterilize.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
What One has is a more accurate model, which is how science works: One creates a model, tests it, adjusts to improve its accuracy, lather, rinse, repeat. It's worked pretty well for centuries.
... You ignore reality, sure.
is still withing chance because the error bars on the are huge.
The Kruger Dunning explains most post on
... Americans often like to blame Others for problems: Judges, Politicians, Muslims, "illegal Aliens", pro-Lifers, Gays, city Residents, the Poor, the Unemployed, the business Owners, the Religious, the Old, the Young, etc., etc., etc. So, given the large american presence on /., if You find analysis as intelligent as Your's obviously is, it will be a rare gem to be treasured, indeed.
70% isn't very hard to guess. Just basic guessing should get you 50% ave. Then you take political votes, liberal judges are a lock in how they vote even if it goes against the constitution (Obamacare) while republican judges will generally follow what the law says.
With that in mind, 70% is not a good %. It should be 100% possible to predict outcome before the final vote. Read the law and constitution and use it to determine the outcome. You will just get a few wrong with some judges not following the constitution/law.
I'd actually have expected more. I mean, let's be reasonable here:
A 50% accuracy can be achieved by the average unbiased coin. Now throw in the rulings that are easy to foresee because any other decision would be politically suicidal and you should easily arrive at more than 70%.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
...Is that in most cases the decision to 'give cert' for Supreme Court review of given lower-court cases vests with the Justices' law clerks. This may be as weird as that decision to give the Librarian of Congress veto power over unlocking our cellphones, but that's the way it is. How accurate can any model be at delving into the minds of law clerks?
I could flip a coin and be 50% accurate...
I would hope that accuracies far above 50% can be obtained. A random supreme court isn't something anyone wants.
The Supreme Court is dominated by a bunch of fanatic right wing corporate toadies.
So, the decision comes down in favor of corporations (on economic issues) or social conservatives (on social issues).
The Constitution has nothing to do with it.
The Supreme Court is the ultimate cheerleader for our fascist state.
I don't read your sig. Why are you reading mine?
More like: They already do. Let each advocate and vote as his conscience dictates,. Their votes will align with the values of any corporation they might create.
Corporate personhood gives the owners a double vote.
Pardon me, but what I'd read was corporate personhood originated with the state legislatures (specifically the State of Delaware), not Congress.
Not at all. The entire concept of a corporation is that you can invest without operating. The corporation acts on its own and independent of it's owners if they chose not to directly participate. This is important if you ever wanted to make something happen but didn't have the skills or just wanted to invest finds for other purposes.
Now corporate person hood, which is not the same as corporations are people, has been upheld by the supreme court since 1819 with Trustees of Dartmouth College v. Woodward. But it is not a double vote. If anything, it is having speech and the freedom to say things that contradict itself but not vote more than once. No corporation has the ability to vote in government elections that I know of.
It should be that most legal practitioners can predict Supreme Court rulings, since courts follow the law and past decisions. The Supreme Court, and all courts, could be replaced with an algorithm that correctly applies the laws to facts (facts as found, normally by juries).
The 70+% reversal rate reflects the role of the Supreme Court: to correct the mistakes of lower courts in interpreting the law, most often in the interaction and ambiguity of the law. There's no reason to accept a case unless there's good reason to reverse it.
Finally, the reasoning of the court matters as much as the ruling; by hypothesis, the problem with the case (and others like it), is mis-application of the law, and the reasoning is intended to correct that, so courts can be consistent and people know what the law is so they can follow it.
So applying a scientific idea of "prediction" here is wrong. These are not physical phenomena or stochastic or random processes where a "prediction" is based on "discovering" a "model" of what is "really happening" like science. The rulings are intended to produce clarity. They are bound up mainly in the ambiguity and conflicts of law (as codified) and its application, so the "model" disclosed in each ruling is really an anti-model: tweaks applied to a not-yet-working semantic system to make it more... predictable.
True, the rulings are text and you can analyze them. But I think unless the model is essentially about the application of the law in question, it's probably not illuminating.
Sure, and absent the whole corporations = people with free speech, they can remain hands off if they like. No change there.
Corporations don't actually get a vote, but they do spend bazillions on lobbiests and campaign funds. Instead, how about they tell their owners (stockholders) what will be good for the company (and presumably the investment) and if those stockholders care to lobby or donate, they are free to do so.
Nowhere in there are the owners being denied the ability to carry their rights with them (as the post I replied to claimed (or more properly, stated that others claim).
It does not make any sense at all to set up something where investors and owners can be hands off and then require them to be hands on in order for the corp to fulfill its fidiciary duties/responsability.
Maybe you are not aware, but nost of the colonies that became states were set up by companies. They were chartered and run by them. The founders knew exactly what they were and capable of in ways that make today's companies look like kids selling cookies. And yet they purposely gave them autonomy and a responsability to act in the interest of the company. You seem to be the outlier here.
The corporation would in no way be impared from fulfilling it's fidiciary duties. In the same way that making theft and fraud illegal doesn't prevent a corporation from fulfilling it's fidiciary duties.
As for the rest, you are referring to crown charters that were granted by the king of England when the states were still colonies and so subject to English law and the king. I cannot imagine why you might think that would have any bearing on U.S. law. Further, I can't imagine why you would think the founding fathers were on-board with that given the Revolutionary War.
Once the Constitution was signed (marking the beginning of the U.S. government as we know it), the federal government granted a very few charters to great controversy. After that, the whole mess was left to the states.
I know, can we fire them all and just buy 1 copy of the program. Then, when someone wants an appeal, we could just run the program and have a nice easy answer.
69.7% of the time the answer is "yes"?
I never said the corporation would in no way be impared from fulfilling it's fiduciary duties. I said it makes no sense to force the hands off benefit of a corporation aside in order to maintain it.
Or in other words, you could no longer purchase stock X and have it in your retirement only knowing you have stock X as an investment. You could no longer purchase a mutual fund for the same without knowing all the ins and outs of each and every company and listening to their pitch on what might harm them and adjust your speech accordingly. It simply makes no sense at all to deny corporations the ability to advocate for themselves via speech. The vast majority of people would either be flying blind because they have occupations and obligations other than watching stock and laws/politics or only invest in a small number of companies.
It sounds like you like to run around the track twice quibbling about all the trees and flowers in some attempt to complain that the stadium seats are not comfortable. I completely understand why you cannot understand. It's importance is because the same people who revolted from the king, who were subject to those crown charters and their independent actions, are also the same people who made the US law. If they felt the same as you, they would have specifically prohibited it. But guess what, they didn't believe the same as you and they didn't prohibit it- even after revolting against England and setting up their own country twice.
I do not understand why you would even point this out. Of course the federal government made very few charters and left the rest to the states, it is by design of the US constitution. But lets not pretend that the states knew nothing about the crown charters or the companies operating under them so they were clueless in their law too. The fact of the matter is that they knew this full well and likely were created as colonies who grew to nation states under them. Like I said, the founders knew and didn't care about what you do. They made corporations so they had the ability to operate independent of a single owners or groups of owners and they did this knowingly of all the ills of corporations too.
Or in other words, you could no longer purchase stock X and have it in your retirement only knowing you have stock X as an investment. You could no longer purchase a mutual fund for the same without knowing all the ins and outs of each and every company and listening to their pitch on what might harm them and adjust your speech accordingly
Good. Absentee owners are a problem we would be better off without. If you can't be bothered to read a newsletter every 2 years and understand the issues, you should probably abstain from voting anyway. Really though, there are only two likely states. In one, you already agree with what teh corporation wants and so you vote accordingly. In the other you believe differently and it is best if the corporation you own part of doesn't act against you.
You chose to refer to the authority of the founding fathers, and claim they appointed corporations to run the states. I pointed out that it was bollocks.
The states did grant charters, but they included revocation clauses with real teeth and mandated that the corporation operate only within it's charter AND for the public good (or have it revoked). This was quite different from the crown charters.
Poppycock. The entire concept of a corporation is to separate investment from the operations of the business. But the silliness of your statement is full of ill places conception. The exact same can be said about people who do not even know who the current vice president are but were convinced to register and vote. No one things they should abstain from voting- at least no one who is taken seriously.
Wrong. The state of convincing you otherwise exists and should exist. It is best if the corporation can make a case and you can act accordingly to your own convictions. This idea that corporates can spend money on speech and you all the sudden have no leg to stand on or somehow magically lose all your objections is nothing but a myth.
I did no such thing. I even spelled it out in the last post in which I said those founders were subjected to those corporations and those same founders made laws and did not take steps to limit corporations for those purposes. Please stop arguing what was never said and pay attention.
Of course. But what they did not do was limit their rights of speech, their ability to enter into contracts separate from their owners or achieve levels of debt and default separate from their owners who took no part in operating the corporation. In short, the states, who many were created as colonies subject to the corporations, create a system that clearly generates autonomy of the corporations giving them a status as a separate entity with the ability to act as people from the start. You are essentially saying "ahh,, but they dotted their I's differently so the otherwise identical system is completely different" which isn't the case at all.
They had the ability to enter into a contract and otherwise conduct business as a legal fiction. They had no Constitutional rights until a few blunders by the Supreme Court granted them, but that was years later.
lol.. No, they always had constitutional rights and the supreme court said congress cannot take them away. The court started affirming these rights as early as 1819 when all the founding fathers were still around and the constitution and the role of government was well understood.
Congress shall make no law means what it means. It does not mean congress can make a law in certain times or that congress can make a law if something is a certain way, it means congress can make no law prohibiting the freedom of speech. It is clear and always has been clear- even when you attempt to shoehorn it into something else.