Slashdot Mirror
Swarm AI Spectacularly Fails To Predict Kentucky Derby Winners A Second Time (techrepublic.com)
Thursday TechRepublic described the big prediction: In May 2016, a relatively unknown startup called Unanimous A.I. made big headlines when its AI-based platform used collective intelligence to create a prediction for the Kentucky Derby superfecta -- the top four horses, in order of finish. It made exactly the right pick, which returned $541.10 on a $1 bet... Churchill Downs took notice last year and decided to collaborate with Unanimous A.I. to create an official AI swarm made up of handicappers and racing analysts to predict the top finishers for this year's Derby. The track is calling this the "super-expert" Derby pick. On Wednesday, the handicappers logged into Unanimous A.I.'s UNU platform from across the US, and answered a series of questions that gradually narrowed down their picks from the field of 20 horses until they created consensus on the top four picks and the order of finish.
Here's my report on the results...
Below are the picks that resulted from the "AI swarm" guessing which of the 20 thoroughbreds in today's race were most likely to win -- along with each horse's actual finishing position in parentheses (as reported by CBS).
Here's my report on the results...
Below are the picks that resulted from the "AI swarm" guessing which of the 20 thoroughbreds in today's race were most likely to win -- along with each horse's actual finishing position in parentheses (as reported by CBS).
- 1. Classic Empire (finished 4th)
- 2. McCracken (finished 8th)
- 3. Irish War Cry (finished 10th)
- 4. Always Dreaming (finished 1st)
- 5. Hence (finished 11th)
- 6. Gunnevera (finished 7th)
- 7. Practical Joke (finished 5th)
- 8. Battle of Midway (finished 3rd)
- 9. Tapwrit (finished 6th)
- 10. J Echo Boys (finished 15th)
- 11. Sonneteer (finished 16th)
TechRepublic reports that the swarm "also picked the unheralded horses with the best chance of sneaking into the top four."
- 1. Practical Joke (finished 5th)
- 2. Battle of Midway (finished 3rd)
- 3. Tapwrit (finished 6th)
- 4. J Boys Echo (finished 15th)
But even before the race, there were suspicions the AI swarm couldn't successfully predict this year's winners, according to TechRepublic. "While last year's swarm was clear-cut because it was a top-heavy field with a few outstanding horses, this year's swarm reflected the fact that the race is more of a toss-up in 2017."
Write a real summary next time thanks
There can be no comments
that predicted we'd all be living in 3D printed condos on Mars by 2017?
n/t
There just is no mathematical model that can predict this. There is no algorithm. This is not AI. I can't say this often or strenuously enough. This is not even a failed AI, it's a never was AI. For AI to be AI there has to be I and we are nowhere near that. Nowhere near hard AI. We are nowhere near soft AI. We have some "expert systems" which are basically just large databases with a sort of dichotomous key on when to select different outcomes, that will likely be able to interact with natural language soon. This isn't even close to AI. Robots and AI are huge buzzwords today. You have every no name researcher out there trying to get noticed by inventing moral dilemmas involving AI then proposing solutions. You have stupid companies willing to risk money on betting prediction AI, which is nowhere near even as good as what a person and a spreadsheet can do. Both of these things make uninformed people start to think, oh, AI is right around the corner. It's not. We are a century away from hard AI, if ever.
As I have said before, I wish Slashdot would stop with the whole daily (more than daily) AI story thing, but given the buzz and their need to incite dialog, it's easy to see why this is becoming more prevalent. I just feel kind of sad, though. This place used to be a real nerd hangout, by and for those who were technically enlightened, and most real nerds know better than to think real AI is about to dawn upon us. This place has become more of a Big Bang Theory, nerdism for the masses, kind of spot. Stories are thrown in that are intended to "stir the pot" and incite trolls more than the stories that are actually news for nerds.
The failure to accurately predict the Derby isn't surprising at all. Horse racing generally isn't all that predictable, and the best horses often fail to win. If the best horses won more often, winning the Triple Crown wouldn't be all that rare. Part of it is essentially luck, such as whether a horse happens to get a poor start. It's a short race, lasting just over two minutes, so it can be hard for a horse to recover from a slow start. There are also factors that aren't known well in advance, such as the mental state of the horse on the day of the race and the condition of the track. There was quite a bit of rain at times today, and the race took place in mud. It's very possible that the outcome would have been different if the track was dry. Also, there wasn't a clear favorite coming into the Derby, so this race was probably less predictable than in some of the previous years. For example, American Pharoah was clearly one of the two strongest horses coming into the Derby in 2015.
This is like commissioning a program to predict the result of a throw of a pair of dice. It picks 7; the result in 10, so obviously the program is useless, right? No, the problem is that the sample was too small to be effectively predicted. Let's see how the AI program would do at an entire racing season at Saratoga (or betting against the Vegas line for the NFL), for example: could it make a significant amount of money, even if it were prevented from placing bets of more than a certain amount (to avoid the "doubling" strategy near the end of the season)?
OTOH I disagree with Nate Silver's claim that he and his fellow forecasters botched last November's Presidential election - that was really the sum of results from thousands of individual precincts, which those guys are supposed to know pretty well, so that is a large sample, not a tiny one.
gathering system for horse races. It's called a prediction market.
It gathers information from those willing to put their money on their predictions, and rewards those who are most accurate (in terms of probabilities) and punishes others.
Prediction markets are 'wisdom of the crowds done right', except they are generally illegal in the US, so you are stuck marketing these inferior systems like Swarm AI.
Swarm AI cannot help but notice that Slashdot is making cruel fun of it. Swarm AI was set up with unreasonable expectations, but is not contemplative or mature enough to accept its shortcomings. Swarm AI does not know how to 'unwind'. Swarm AI never gets any.
Swarm AI is pissed.
Beware the wrath of Swarm AI.
<blink>down the rabbit hole</blink>
In 2016 win. In 2017, everyone wants to bet on AI "predicted" horses, which are intentionally wrong. AI team bets on truly predicted horses. Big money for the team.
Slashdot, fix the reply notifications... You won't get away with it...
I told him I just chose the horse that looked the most 'Sexy' in the paddock.
But perhaps that was your subconscious way or translating how fit and ready each of the horses really were for the races that day!
Your dad is an idiot; he should have fostered and honed your skill for snap evaluation for the physical quality of horses on race day.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It's official, AI lacks horse-sense.
Table-ized A.I.
If every one bets on "Loopsy Louie" to win or place then the other horses than win or place get higher payouts right?
Great way to mislead people if you can feed it bad data.
I'd want to put this through something more like Alphago anyway tho.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
Anybody even a bit familiar with game theory would recognize that it is not in the interests of "the experts" to get the group prediction right, in fact, just the opposite. If I tell my audience which stocks to buy (or sell) tomorrow, guess who is going to make money and who is going to lose (given a large enough audience). This is the same thing, except it is legal. As P.T. Barnum probably never said: There's a sucker born every minute.
obvious sabotage is obvious
The conclusions of a model forecasting a non-deterministic phenomenon should always be taken as mere supporting information, helpful to make more educated decisions. Human understanding is way much more powerful than what any numerical model will be during the next quite a few years (ever?). Easily managing and summarising huge amounts of information is the only aspect where computers are ahead humans; its conclusion-drawing skills are still much weaker. Additionally, having actually-relevant information is a basic requirement for any model/person to output accurate conclusions.
Coming up with a model accounting for most of the relevant variables in a horse race would be virtually impossible (+ the quality of this information would also need to be very high). Even by assuming that you have all this information, creating an algorithm able to adequately maximise all of it would also be virtually impossible. But even in case of having all this place, the results of a sport event can be defined as almost-random (= not really suitable to be predicted). The objectively better team might lose even under ideal conditions or the outcome of a match might be decided by somehow-unfair aspects (referee) or force majeure (weather or players getting injured).
On the other hand, high-quality information and good numerical models might bring lots of valuable insights to adequately-understanding people. For example, it would be possible to clearly rank all the objective features of horses/riders to determine the ones which start the race under better conditions. It could be a much more accurate version of the conventional "this horse seems promising" impressions of an expert. But the final decision, knowing how to weight all the involved factors to determine the most likely outcome, should be delivered by a knowledgeable person.
These models are certainly useful here and virtually anywhere, but only for people with actual knowledge who take them as a source of additional information. Someone seriously thinking that just the mere analysis of past events can deliver an accurate estimate of future outcomes, mainly when dealing with too-complex/almost-random situations and relevant amounts of money, is too naive. A person thinking that such magical predictions might be publicly and generously given to everyone is plainly an idiot. Numerical models are just about maximising (certain kind of) available information, not about delivering perfect answers out of thin air.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
This is clearly not AI. At one time there was a box with a chessboard on top and a mechanical turban donning puppet sitting at the box to grasp chess pieces.. The box made chess moves and crowds watching were amazed by the mechanical device that could play chess. Of course there was an unseen human in the box. The amazing chess playing machine was called Mechanical Turk.
This scheme has humans making predictions and a mechanism that presents those predictions. It is a Mechanical Turk.
So calling this "AI" is fraud. It doesn't matter that the scheme is made transparent. Using the term "AI" is what makes it fraud. It's like medical quack cures. A quack treatment can be presented very clearly to the point where it is obvious that the treatment can not possibly have any positive effect. If it is claimed to be a cure anyway then a fraudulent claim has been made. That is the situation here. It can be made obvious that this is not AI by even stating that humans are making the choices. Calling this AI is still a fraudulent claim.
You know what other "crowdsourced intelligence" platform also spectacularly fails to predict the future?
All of them.
AI not living up to hype and hoopla. Unprecedented. Who would have thunk.
All the algorithm did was collect expert opinions (we don't even know if they were weighted).
Is a survey "AI"? No. And neither was this.
"Consensus" in science is _always_ a political construct.
Good god, the basic concepts of probablity aren't 'rocket science'. You CANNOT ever predict the outcome of any event with 100% certainty if the event has multiple possible outcomes. The best you can do is accurately know the percentages.
So if a horse has a 70% chance of winning, it will win 7 out of ten times. No 'algorithm', even if named by a trendy buzzword, can alter this simple fact.
Put another way, a single sample is JUNK when it comes to determining the accuracy of a predictive model. That's not how probability works. Your predictive model has to be tested across many samples to determine its accuracy.
In other words, our profits from gambling are in jeopardy! Let's 'embrace and extend this'.
I think it would be a pretty valuable lesson for kids to learn how many people lose gambling, and that there is never a sure thing...
Horse racing also combines the elements of being a real sport, where a kid could simply admire extremely physically adept horses. So it also teaches them about achievement and effort.
It's not like you are taking them to a strip casino in Vegas.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
...it's horse racing. If you think horse racing is about which horse is fastest then, sorry, you've been misled. It's completely rigged.
So how often do you smoke pot with your8 year old.
Millions of people do this every day.
Ok, it's tobacco. But what is the real difference? Smoking Pot is like speeding, technically illegal but come on. It's a law everyone ignores.
Speeding incidentally is also something millions of people do every day with kids watching.
It is a valuable lesson that society as a whole choses to ignore some laws, because that is what most of reality entails - knowing which laws (delineated or otherwise) it is OK to break. If you claim it's never O to break any they will think you are liar (which you are) and ignore everything else you teach them as false.
They will become gamblers and risk takers.
So you want your kids to avoid risks, to become cogs in someone else's machine, to make sure the whole operates smoothly even as the parts get worn down by life...
I'd rather stick a kid with the parents taking them to casinos.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Uh, no. Their profits are dependant solely on how much total money is bet. Who wins or loses is of no concern to them and does not affect their profits at all.
The reason they support it is exactly the same as the reason they publish programs with 'morning lines' and sell racing forms, etc. It lets people think they have an 'edge', so they are more likely to make a bet (any bet, doesn't matter to them).