Swarm AI Spectacularly Fails To Predict Kentucky Derby Winners A Second Time (techrepublic.com)

Thursday TechRepublic described the big prediction: In May 2016, a relatively unknown startup called Unanimous A.I. made big headlines when its AI-based platform used collective intelligence to create a prediction for the Kentucky Derby superfecta -- the top four horses, in order of finish. It made exactly the right pick, which returned $541.10 on a $1 bet... Churchill Downs took notice last year and decided to collaborate with Unanimous A.I. to create an official AI swarm made up of handicappers and racing analysts to predict the top finishers for this year's Derby. The track is calling this the "super-expert" Derby pick. On Wednesday, the handicappers logged into Unanimous A.I.'s UNU platform from across the US, and answered a series of questions that gradually narrowed down their picks from the field of 20 horses until they created consensus on the top four picks and the order of finish.
Here's my report on the results...
Below are the picks that resulted from the "AI swarm" guessing which of the 20 thoroughbreds in today's race were most likely to win -- along with each horse's actual finishing position in parentheses (as reported by CBS).

• 1. Classic Empire (finished 4th)
• 2. McCracken (finished 8th)
• 3. Irish War Cry (finished 10th)
• 4. Always Dreaming (finished 1st)
• 5. Hence (finished 11th)
• 6. Gunnevera (finished 7th)
• 7. Practical Joke (finished 5th)
• 8. Battle of Midway (finished 3rd)
• 9. Tapwrit (finished 6th)
• 10. J Echo Boys (finished 15th)
• 11. Sonneteer (finished 16th)

TechRepublic reports that the swarm "also picked the unheralded horses with the best chance of sneaking into the top four."

• 1. Practical Joke (finished 5th)
• 2. Battle of Midway (finished 3rd)
• 3. Tapwrit (finished 6th)
• 4. J Boys Echo (finished 15th)

But even before the race, there were suspicions the AI swarm couldn't successfully predict this year's winners, according to TechRepublic. "While last year's swarm was clear-cut because it was a top-heavy field with a few outstanding horses, this year's swarm reflected the fact that the race is more of a toss-up in 2017."

Comments

A sigh

[Excelcia]

There just is no mathematical model that can predict this. There is no algorithm. This is not AI. I can't say this often or strenuously enough. This is not even a failed AI, it's a never was AI. For AI to be AI there has to be I and we are nowhere near that. Nowhere near hard AI. We are nowhere near soft AI. We have some "expert systems" which are basically just large databases with a sort of dichotomous key on when to select different outcomes, that will likely be able to interact with natural language soon. This isn't even close to AI. Robots and AI are huge buzzwords today. You have every no name researcher out there trying to get noticed by inventing moral dilemmas involving AI then proposing solutions. You have stupid companies willing to risk money on betting prediction AI, which is nowhere near even as good as what a person and a spreadsheet can do. Both of these things make uninformed people start to think, oh, AI is right around the corner. It's not. We are a century away from hard AI, if ever.

As I have said before, I wish Slashdot would stop with the whole daily (more than daily) AI story thing, but given the buzz and their need to incite dialog, it's easy to see why this is becoming more prevalent. I just feel kind of sad, though. This place used to be a real nerd hangout, by and for those who were technically enlightened, and most real nerds know better than to think real AI is about to dawn upon us. This place has become more of a Big Bang Theory, nerdism for the masses, kind of spot. Stories are thrown in that are intended to "stir the pot" and incite trolls more than the stories that are actually news for nerds.

Re:A sigh

[PopeRatzo]

There just is no mathematical model that can predict this. There is no algorithm. This is not AI. I can't say this often or strenuously enough.

And there is most definitely no "collective intelligence".

All you have to do is read the news to see that.

Re:A sigh

[fluffernutter]

Why don't you tell us how you really feel?

Re:A sigh

[Hylandr]

There just is no mathematical model that can predict this. There is no algorithm. This is not AI. I can't say this often or strenuously enough. This is not even a failed AI, it's a never was AI. For AI to be AI there has to be I and we are nowhere near that. Nowhere near hard AI. We are nowhere near soft AI. We have some "expert systems" which are basically just large databases with a sort of dichotomous key on when to select different outcomes, that will likely be able to interact with natural language soon. This isn't even close to AI. Robots and AI are huge buzzwords today. You have every no name researcher out there trying to get noticed by inventing moral dilemmas involving AI then proposing solutions. You have stupid companies willing to risk money on betting prediction AI, which is nowhere near even as good as what a person and a spreadsheet can do. Both of these things make uninformed people start to think, oh, AI is right around the corner. It's not. We are a century away from hard AI, if ever.

I Agree with this sentiment wholeheartedly.

More to the subject of betting on horse races though, it doesn't matter how expert or intelligent a system is. If you aren't looking at the correct metrics for evaluating a Horse's ability to win the results will always be garbage.

My father used to be my beard when I was a child and I was given small allotment to bet and I usually did pretty good. When asked how I chose the horses my response ended up getting me banned from going ever again. I told him I just chose the horse that looked the most 'Sexy' in the paddock.

As noted about AI, there's no amount of intelligence that quantify what I feel is an instinctual observation.

Re:A sigh

[Anonymous Coward]

It depends on the quality of the inputs, but I would expect that the people making the predictions are knowledgeable about horse racing and the field. As I understand it, Swarm is ensemble mean of individual predictions. The idea is that there is generally some skill to each individual prediction, but the ensemble mean of the individual predictions should have more skill over the span of many races than the individual predictions. In principle, this isn't a bad idea. However, on any given race, the ensemble mean might fail miserably at making an accurate prediction.

As I understand it, the Swarm prediction was made well in advance of the Derby, which means that the prediction wouldn't have properly accounted for the weather conditions today. The race was run in sunny conditions, but there had been quite a bit of rain earlier in the day, and the track was muddy. That certainly has an impact on the outcome of the race, but that very important information wouldn't have been known with much certainty when the prediction was made. Horse racing generally isn't all that predictable to begin with. Also, there wasn't a clear favorite coming into this race, making it harder to predict. I don't think you can evaluate the quality of Swarm based on the skill at predicting one race.

Re:A sigh

[ArylAkamov]

My father used to be my beard when I was a child

Pics or it didn't happen.

Neat beard has multiple meanings though.

Re:A sigh

[g01d4]

Over 20 years ago I collected the handicapper picks from the LA Times over the course of at least one season from Santa Anita and Hollywood Park. Each handicapper had his own methodology and the key was that they were relatively independent. One went for long shots, one typically went with the favorite and one was in between. The plan was to train a neural network to tease out which handicapper worked best in a given race. I didn't quit my day job.

Re:A sigh

[Koby77]

It's an insidious form of gambling, such that it appears that one could make logical predictions based on starting positions, past performance, surface conditions, ect. Most race tracks even publish a booklet which lists the past outcomes of all the competitors in each of the races. I have concluded, however, that the outcomes are sufficiently random such that no amount of information can predict the winners. It's like attempting to pick the outcomes on a roulette wheel; you will lose money to the house.

Re:A sigh

[rtb61]

There is a mathematical model that you can guarantee one hundred percent you will win with horse racing. It is the one bookies use to calculate the payout on bets based on the amount bet on an particular wins or places, so that no matter who wins you collect more in bets than you pay out. You need to know nothing what so ever about horses to win, you just need to understand statistics and some psychology so you can scam the mug punters. Just like all other, so called gambling, the arse holes taking the bets cheat by stacking the betting odds in the favour so no matter what happens over time they win and the mug punters lose. This should be illegal, the only way you should be able to run a gambling concession is where the odds are equal and fair otherwise it is just corrupt cheating with paid off politicians.

Re:A sigh

[Mr D from 63]

Swarm AI should have been able to predict that it would get the race predictions wrong.

Re:A sigh

[MrL0G1C]

FML slashdot mod system broken, I have mod points, I tried to mod your comment as informative but system said 'already at limit' and then some idiot modded you as troll but now I can't mod because 'already modded' which clearly isn't true because you don't have an informative mods at the time of this post.

Re:A sigh

[retchdog]

the articles says they feed the algorithm with expert opinions (and presumably the experts' predictive accuracy on various races, and a whole lot of extra metadata). it basically combines the experts' prediction with weightings according to their consistency and correctness (and whatever other metrics) and produces a final ranking. iow, the "instinctual observation" is all in the backend.

no, it's not AI. it's just statistics, but that sounds boring so since AI is still impossible, they/we appropriated that word to mean "statistics that isn't boring or can at least be made to look impressive".

this is not as dishonest as it seems. "flashy demos made out of smoke and mirrors" is almost the definition of AI to date.

Re:A sigh

[jeremyp]

I certainly agree this is not AI.

Firstly, I think I'm pretty intelligent, at least compared to the average computer, and I doubt if I could predict the result of any horse race. Assessing probabilities of horse races is not AI.

Secondly, it's not AI if your algorithm involves "ask a lot of natural intelligences (aka people) what they think the answer is". That's no more AI than somebody hidden away answering questions over an IM link.

As fore the swarm idea, with horse racing, it already exists. The betting odds are a manifestation of the way thousands of people think the race will end. I wouldn't be surprised if they were quite accurate (over a large number of races) given there is a lot of money to be made or lost on getting them right.

Re:A sigh

[Derec01]

... You have stupid companies willing to risk money on betting prediction AI, which is nowhere near even as good as what a person and a spreadsheet can do. Both of these things make uninformed people start to think, oh, AI is right around the corner. It's not. We are a century away from hard AI, if ever.

I'm an ML researcher, so I'm totally with you on the overall sentiment. We don't even know the right questions to ask, let along solve, in hard AI. However, the notion that a prediction AI is nowhere near as good as what a person can do is now behind us for a lot of tasks and what's happening now is deployment. For instance, a number of trained systems are better than humans at diagnosing radiology results:
https://www.forbes.com/sites/p...
http://news.stanford.edu/2017/...

Strong AI is going to be garbage for a while, but whatever the name, machine learning doesn't have to be hard AI to change a lot our everyday lives.

Re:A sigh

[Hylandr]

feed the algorithm with expert opinions

All things being equal, if betting on horse racing were a science then it wouldn't be called gambling. There would also be no need to attempt to determine who would win.

I say load as many metrics as possible, and let the computer know who won, then help the computer figure out whats important to look at.

Re:A sigh

[retchdog]

doesn't work. overfitting.

Re:A sigh

[bws111]

What are you talking about? There is no 'statistics' or 'psychology' involved. There is no 'scam'. There is no 'stacked odds' or 'over time'.

What they do is quite simple and well-known. If you bet $1, they keep a percentage as their fee for running the service. This is called the takeout. The rest of the money goes into the pool for that type of bet. At the end of the race, the pool is split up among the winners.

You are betting against all the other gamblers. The bookie is not betting at all.

Re:A sigh

[syntotic]

Any true forecast will need more data than we can provide short of placing sensors and monitors on every horse and jockey. There is simply not enough information to predict consistently.

We already have an optimal swarm intelligence

[Procrasti]

gathering system for horse races. It's called a prediction market.

It gathers information from those willing to put their money on their predictions, and rewards those who are most accurate (in terms of probabilities) and punishes others.

Prediction markets are 'wisdom of the crowds done right', except they are generally illegal in the US, so you are stuck marketing these inferior systems like Swarm AI.

Re:We already have an optimal swarm intelligence

[EMN13]

Note that a prediction market is not particularly more likely to be accurate than any other machine learning technique. If there's been one thing that's been demonstrated time and time again over the years, it's that there are many techniques that can work, but that to get truly excellent results, appropriate data collection, selection, filtering etc. is critical. It's easy to get charmed by techniques that have a great story and convincing argument they'll work - but that doesn't mean they're the best.

Re:We already have an optimal swarm intelligence

[Procrasti]

> Note that a prediction market is not particularly more likely to be accurate than any other machine learning technique. If there's been one thing that's been demonstrated time and time again over the years, it's that there are many techniques that can work, but that to get truly excellent results, appropriate data collection, selection, filtering etc. is critical. It's easy to get charmed by techniques that have a great story and convincing argument they'll work - but that doesn't mean they're the best.

Firstly, a prediction market isn't a machine learning method (nor is Swarm AI)... but...

No, a prediction market is always likely to be more accurate than any given prediction technique because it is not a prediction technique in itself, but rather a meta-prediction system that optimally combines existing prediction methods and exponentially rewards the best ones.

For example, assume a new machine learning technique comes along that is consistently better able to predict the probabilities of which horses are likely to win than all the existing techniques when combined in existing prediction markets. That means it will produce probabilities that are more accurate than the market and can therefore take advantage of this difference to realise long term exponential profits from these markets. However, as it reuses those gains each time, it becomes a bigger player in those markets, which drives the markets towards its own predictions, quickly removing profit opportunities for itself and others. Pretty soon, the prediction markets start to reflect the predictions of the new machine learning technique over the older techniques.

So, because it is a meta-prediction technique that optimally combines the results of all participating prediction techniques, it is always going to be better than any given underlying prediction system, or at worst case be as good as the best underlying prediction method.

Re:We already have an optimal swarm intelligence

[EMN13]

What you call meta-prediction is really just a variation of https://en.wikipedia.org/wiki/... - a prediction market isn't radically different. Indeed even within one model, combining separate predictions is useful - low-level density estimation layers deep learing have some similarity.

The overlap between markets and ML (and e.g. evolution) is that they're both complex optimization problems, where finding the "true" solution is generally infeasible. There are various approaches to come up with a best guess, but they all need to make various assumptions about the problem space to work - for one, that the problem space is "smooth" in some sense (so by exploring the current solution and nearby choices, you have a chance of going in the right direction), and e.g. that there aren't too many local minima.

Not all problems are like that, and sometimes a problem takes some massaging until it is a candidate. And these techniques - including markets - do *not* generally converge to the global optimum, the converge to some local optimum. If you're lucky, or under some non-obvious preconditions, then it's global.

Re:We already have an optimal swarm intelligence

[Place a name here]

So, because it is a meta-prediction technique that optimally combines the results of all participating prediction techniques, it is always going to be better than any given underlying prediction system, or at worst case be as good as the best underlying prediction method.

No, because you're always going to take some hit determining what the optimal ensemble is (and even moreso if situations change or if there's noise). In exponential reweighting, the regret has an ln(N) term, which is what keeps you from just making an optimal AI by dumping an exponential number of algorithms into a virtual prediction market and getting a result "as good as the best underlying prediction method".

In practice, if you run the prediction market with real people, you're at risk of all the fun stuff that sometimes makes real markets behave poorly, such as bubbles. This happens because there's a feedback loop where people alter their predictions based on everybody else's predictions, as in a Keynesian beauty contest. In contrast, "the best underlying prediction method" would not have such problems because there would be no market for it to be influenced by.

Re:We already have an optimal swarm intelligence

[Procrasti]

I'm pretty sure you don't get what a prediction market is. It is not machine learning, and it is not a learning method.

It is not a prediction method in itself. It allows disparate prediction methods to compete with each other with real world resources, ie, money.

It makes no assumptions about smoothness. The only assumption it makes is that various outcomes will manifest with various probabilities. Something that does have a global optimum.

An ensemble is not a meta-prediction method, it's simply another prediction method. A prediction market is not in itself a prediction method, it's completely brainless and useless except for the way it incentivises and rewards correct predictions from other prediction methods.

Again, no matter what prediction method you use, including ensembles, a prediction market will exponentially reward the most correct one over time. So it is always going to reflect the results of all the best prediction methods available. Ensembles do not have this property.

(Note: Am familiar with ML, and earned a living from exploiting inefficiencies in horse racing prediction markets for nearly decade).

Re:We already have an optimal swarm intelligence

[Procrasti]

I was talking about real prediction markets, not as use in training NNs with virtual prediction markets. I can't think of any situation where this would even make sense... but maybe?

And, while I agree with you that prediction markets have the faults you state, they still turn out to be the best method we have available of collating the wisdom of crowds. Empirical data from horse racing results show that horses do in fact win at the rate predicted by the markets to a very high degree of accuracy.

Finally, if you produced a method that was more accurate than the market, which did not suffer from the feedback loop you suggest (or any other problem), then the market would significantly and exponentially reward this method until the market reflected the predictions of your method... meaning the market would then indeed reflect the best underlying prediction method, as I originally stated.

If anything Swarm AI is a prediction market, but with equal waiting for every "expert, and no reward feedback mechanism to promote the accurate and remove the inaccurate players.

Re:We already have an optimal swarm intelligence

[Procrasti]

Not really... because you can't know the odds you are getting when you place the bet.

Imagine we are betting on the outcome of sum of two dice rolls, where we know the most likely outcome is seven, but this still only happens 1 in 6 times. So, if you are getting better odds than 1 in 6, you should bet on seven, but if you are getting less than 1 in 6, you should bet against it. On a prediction market, someone who does this (within constraints of the kelly criterion) will long term profit... but with pool betting, we can't tell the odds we will get until the pool is closed... meaning that the correct betting strategy isn't known until you can no longer use the information!

Prediction markets reward those who provide more correct information (than the market itself) to these markets. Pools reward those who are more correct than the average bettor, but we don't know what the average bettor will do until it is too late to make use of this information, meaning that people can't meaningfully provide predictive information to betting pools.

Prediction markets really are the optimal way to collate the wisdom of crowds, and, unlike the above posters suppose, it doesn't suffer from the "No Free Lunch Theorem" because it is not a learning algorithm or a prediction method in itself, but a meta-prediction method, ie an optimal way to combine other prediction methods and exponentially reward the most correct ones.

Re:We already have an optimal swarm intelligence

[Procrasti]

Another way to pput it is that prediction markets don't suffer from the "No Free Lunch Theorem" because they are not prediction methods themselves, but a way of combining prediction methods.

Ensemble learning is STILL a prediction method, and so still suffers from the NFL theorem.

Re:We already have an optimal swarm intelligence

[EMN13]

A prediction market is a a prediction method that runs on a bunch of humans, not a computer. It must obey the same convergence laws as all other such processes, from other markets, to evolution, to human learning, and indeed machine learning.

It definitely does need to make assumptions about smoothness to be able to find even a local optimum - in the infinite space of possible prediction methods the market is exploring, if the optimum method is almost identical to a bunch of other terrible methods, then the market isn't likely to find that solution. If, by contrast, the nearby methods show promise but arent' *quite* as good, then normal market action works: people will see the winners getting rich and try to beat them at their own game by trying variations. Some of those variations might be better than the orignal; etc. This process is obviously highly complicated in practice; "nearby" isn't even a clearly defined term, and yet is still applies.

Philosophically, I doubt a prediction market will ever find a global optimum - not sure that it matters, however. There's no way we can tell, I suspect.

The point isn't that a market "is" a prediction method - sure from some perspective it is; yet there are lots of interesting differences too. The point is that it's risky to assume a prediction market will always beat any other prediction method; i.e. that it is an "optimal swarm intelligence". It's one possible and known effective way to leverage a diversity of other prediction methods and ongoing research into new ones. But it may not be optimal. We don't know that. In several ways, we *know* it's not optimal: not just are there known issues with markets in general, but more specifically, markets necessarily react only when some of its actors already have: markets are slow. It can be worth a lot of money to be faster, and other methods that don't rely on actors to interpret information for them can have an edge there.

Re:We already have an optimal swarm intelligence

[Procrasti]

Here's what you are not getting, it ISN'T a prediction method.

If all you have is stupid predictors acting on the market, all you get is stupid results (no better than chance). Ensemble learning (from your earlier example) does better than this (it can do better than its inputs, even with brain damaged sub agents).

Their advantage is in the way they exponentially reward better prediction agents over worse ones. They do so in such a way that, in the long term, the market reflects the consensus of the best ACTUAL predictors that use it.

Finally, you keep forgetting, that if you develop a MORE optimal prediction method, a prediction market will soon come to reflect the results of THAT method... so it is a meta-prediction method... it is then, by definition, at least as good as the best ACTUAL existing prediction method! Hayek famously proved they result in optimal allocations, which is how economists say you cannot do better than them theoretically. So there really ARE reasons to believe that they are truly optimal.

> It can be worth a lot of money to be faster

Exactly, they reward *timely* information... that is exactly the property you *want* them to have. In any case, they do reward actors who remove inefficiencies. Actually, exploiting inefficiencies is the same as making them more efficient... These are all 'good' things.

I agree as a mapping from real world inputs to output probabilities is inherently as crazy as any other problem (probably much harder than simple image recognition, NLP and the usual ML problems, though DNNs are often used with much success in horse racing)... but it is clearly smooth and simple in the OUTPUT space. Each outcome has a theoretical actual probability (given all *possible* available information) and moving closer or further from this is easily and obviously smooth... even though we may not be able to know what that actual probability should be. In contrast, we generally don't train NN's with probabilities, we train them with certainties (we don't label images as 95% cat 5% dog, we say it IS a cat - ignoring 'dark knowledge' methods), but we then treat the results probabilistically.

So, because it ISN'T a prediction method, it doesn't suffer from the "No Free Lunch Theorem"... it isn't a learning method or a input-output prediction mapping at all! You'll (probably) never be able to use a prediction market to classify images as cats or dogs, only for true unknown probabilistic outcomes like for sports events, weather, elections, etc.

If you're in the US, you've probably never had a chance to use a prediction market, because they are generally illegal there. I have no idea why.

Re:We already have an optimal swarm intelligence

[Place a name here]

If you're in the US, you've probably never had a chance to use a prediction market, because they are generally illegal there. I have no idea why.

I imagine the reason they're illegal is that they can produce perverse incentives. Suppose, for instance, that there's a prediction market on when the next act of terrorism will occur in the US, as defined by some criterion. You predict tomorrow, then go (or get a dupe to go) snipe off some people and send a manifesto to the newspapers. Since the market considered tomorrow to be very unlikely, you get a lot of money doing this, and a strategy like that might seem rather tempting for someone who's poor or desperate enough.

I'd thus expect play money prediction markets to be more easily accepted, since there's no reason to murder for Shiny Points or whatever. But I wouldn't be surprised if the law just refers to prediction markets in general rather than real money prediction markets in particular.

Re:We already have an optimal swarm intelligence

[Place a name here]

I was talking about real prediction markets, not as use in training NNs with virtual prediction markets. I can't think of any situation where this would even make sense... but maybe?

I might have been a bit hasty in my informal impossibility argument, as it were, but my line of thought was like this:
- The claim is that a prediction market can act as well as the best participant of that market, i.e. have zero regret.
- This should hold irrelvant of what the inputs are, whether the inputs are predictions by people or by other simpler algorithms.
- If the claim were true, you could dump a lot (and I mean an extreme amount) of comparatively simple algorithms into a virtual prediction market, and by the claim, the market would act just as well as the very best algorithm of the lot, producing a super AI.
- Since we don't have such super AIs in the real world, the claim must be wrong (and it is, because of the log(N) term).

More generally, if prediction markets were to have zero regret compared to the best expert, every kind of ensemble method would be easy. Just dump the individual methods into a prediction market and you'd get at least as good performance as the best individual method. The prediction market algorithm is the same irrespective of whether its inputs are by people or by computer algorithms, after all.

If anything Swarm AI is a prediction market, but with equal waiting for every "expert, and no reward feedback mechanism to promote the accurate and remove the inaccurate players.

Perhaps surprisingly, it's often hard to do better than nonadaptive methods. See for instance the Variance method of An empirical comparison of algorithms for aggregating expert predictions. Then again, I have no idea what particular algorithm Swarm AI uses; it may be a bad nonadaptive method.

Re:We already have an optimal swarm intelligence

[Procrasti]

> Suppose, for instance, that there's a prediction market on when the next act of terrorism will occur in the US, as defined by some criterion

Yeah, in this and the keynesian beauty contest example the outcomes are dependent on the market itself... so, you can use them to create assassination markets or can you get weird feedback loops.

So, where the outcome is independent of the market itself they work. You're not going to change tomorrow's weather by betting on it, yet that would be useful for many people, but the pope might get shot if the odds go long enough!

I would think that it should be regulated to that extent... I think people know the difference... other than that, why are they illegal for?

Also, play money doesn't work... it''s not money.

Re:We already have an optimal swarm intelligence

[Procrasti]

> The claim is that a prediction market can act as well as the best participant of that market, i.e. have zero regret.

The market has no regret... some agents will, others won't.

> - If the claim were true, you could dump a lot (and I mean an extreme amount) of comparatively simple algorithms into a virtual prediction market, and by the claim, the market would act just as well as the very best algorithm of the lot, producing a super AI.

That doesn't follow... it doesn't do better than the best prediction on any given outcome... it's over time the best on average, You show me your exponential set of simple algorithms that contain at least one super AI, and the market outcome will be the super AI. It's just a matter of TPS for the market...

I think a virtual market type setup would, for example, find the long term optimal weighted sum of AI expert predictions...

I think why you wouldn't use a virtual market for training an ensemble, is that you have all the information already... you can condition an ensemble on the input... but in the real world we do not have that information, we just have our information, and markets create the incentive to provide that information, because we can profit from that it.

The real market is already a super intelligence... collating more information than any individual human (or not) expert can by being the weighted sum of expert human opinions already!

You have to show me your AI that can profit from what already exists first!

Funny thing is, they do exist and the best of them are thriving! Still I don't think we have any super AIs yet!

BTW: Swarm AI just seems to be an equal weighting of random 'experts'... and assigns it as win. That's what I'm saying, it can't possibly beat the market... unsurprising it failed to pick a 1 in 70k event... at least that's what the market thought, right? If anything it's a consensus creator...

See subject for more information.

Re:We already have an optimal swarm intelligence

[Place a name here]

Also, play money doesn't work... it''s not money.

Again perhaps surprisingly, play money vs real money doesn't seem to have much of an effect (See also this post).

If you consider the market to be a kind of weighted voting algorithm that exponentially amplifies the predictions of good experts (as you've said), then it doesn't greatly matter what the weights correspond to in the real world. All that matters is that good experts get their weights amplified, bad experts "go broke", that you have enough players to begin with to catch some good experts, and that the players can't do Sybil attacks to get around going broke.

As sites like StackOverflow show -- or MMOs for that matter -- there are plenty of people that are incentivized simply by getting a high score, even if that high score doesn't translate into real money. So while it may seem counterintuitive that play market should work, it's not that weird if you consider it from the point of view of an exponential weighting algorithm.

Re:We already have an optimal swarm intelligence

[Place a name here]

The market has no regret... some agents will, others won't.

That would be regret in the game theoretical sense. The regret of a strategy is the best payoff you could get minus the best you got; the "opportunity cost". There's an example here.

Since the market is based on individual predictors, the best it can do is somehow knowing the best predictor at each instant. That would correspond to zero regret. Any algorithm based on experts advice would have a regret greater than or equal to zero, even though it may not have an "emotional" regret.

Re:We already have an optimal swarm intelligence

[Procrasti]

I read a chunk off of your pdf... if you can't exclude users who were getting their information from actual exchanges to inform them of their play money bets, you shouldn't be surprised when they work... just bet the difference between the play money market prediction and the real market and you'll profit.

Useless internet points do provide utility even when they aren't money... but they don't provide utility like actual money.

Re:We already have an optimal swarm intelligence

[Procrasti]

I know what you mean by regret. The market doesn't have regret, the stupid actors do, and the smarter ones have joy!

The market is already the best predictor we have!

Knowing the best predictor at any instant would pick the one that said the winner will win... likely the best predictor at any instant is an actual idiot. It does 'learn' the long term best experts.

You had it with exponentially adjusted weighted sum of experts.

Swarm AI Is Losing Its Patience.

[TheRealHocusLocus]

Swarm AI cannot help but notice that Slashdot is making cruel fun of it. Swarm AI was set up with unreasonable expectations, but is not contemplative or mature enough to accept its shortcomings. Swarm AI does not know how to 'unwind'. Swarm AI never gets any.

Swarm AI is pissed.
Beware the wrath of Swarm AI.

Clever AI

[hcs_$reboot]

In 2016 win. In 2017, everyone wants to bet on AI "predicted" horses, which are intentionally wrong. AI team bets on truly predicted horses. Big money for the team.

Why is that not the best answer

[SuperKendall]

I told him I just chose the horse that looked the most 'Sexy' in the paddock.

But perhaps that was your subconscious way or translating how fit and ready each of the horses really were for the races that day!

Your dad is an idiot; he should have fostered and honed your skill for snap evaluation for the physical quality of horses on race day.

Re:Why is that not the best answer

[Hylandr]

Overall, as an adult with kids of my own, I have to question the moral quality of a man that takes a kid to the races to begin with.

It was a rough time anyways.

C'est la vie.

Re:Why is that not the best answer

[Kokuyo]

Why? Because morals?
I disagree wholeheartedly!
It is my job as a parent to prepare my kids for life in this world. Temptation is a constant part of that. Learning to indulge and then stop at the right moment can be an invaluable skill.
Also analyzing odds and trying to predict outcomes hones the mind.
Not to mention that gambling is fun.
And last but not least, abstinence has never solved any problem in this world ever, because abstinence is a pipe dream. It's like abandoning all research into power generation other than a perpetuum mobile.

Re: Why is that not the best answer

[Brockmire]

Found another horse fucker.

Re: Why is that not the best answer

[Hylandr]

I don't see this as a straw man argument, rather an example of real behavioral patterns.

I would not have bid anyone to go back to their prostitute, but we have all heard the phrase 'water seeks it's own level', or 'birds of a feather, flock together'. Participation in one of these activities is a reliable indicator of the quality of personal associations and generally serves as an accurate profile of the lawless demographic.

If you're not shy about hiding your vices, how much worse are your secrets?

Re:Clickbait

[arglebargle_xiv]

"Attempts to AI the Kentucky Derby are Sick and Depraved", Thompson, Hunter S.

Toldja!

[Tablizer]

It's official, AI lacks horse-sense.

Wrong results improve payouts right?

[Maxo-Texas]

If every one bets on "Loopsy Louie" to win or place then the other horses than win or place get higher payouts right?

Great way to mislead people if you can feed it bad data.

I'd want to put this through something more like Alphago anyway tho.

Games

[Anonymous Coward]

Anybody even a bit familiar with game theory would recognize that it is not in the interests of "the experts" to get the group prediction right, in fact, just the opposite. If I tell my audience which stocks to buy (or sell) tomorrow, guess who is going to make money and who is going to lose (given a large enough audience). This is the same thing, except it is legal. As P.T. Barnum probably never said: There's a sucker born every minute.

These models only provide additional info

[CustomSolvers2]

The conclusions of a model forecasting a non-deterministic phenomenon should always be taken as mere supporting information, helpful to make more educated decisions. Human understanding is way much more powerful than what any numerical model will be during the next quite a few years (ever?). Easily managing and summarising huge amounts of information is the only aspect where computers are ahead humans; its conclusion-drawing skills are still much weaker. Additionally, having actually-relevant information is a basic requirement for any model/person to output accurate conclusions.

Coming up with a model accounting for most of the relevant variables in a horse race would be virtually impossible (+ the quality of this information would also need to be very high). Even by assuming that you have all this information, creating an algorithm able to adequately maximise all of it would also be virtually impossible. But even in case of having all this place, the results of a sport event can be defined as almost-random (= not really suitable to be predicted). The objectively better team might lose even under ideal conditions or the outcome of a match might be decided by somehow-unfair aspects (referee) or force majeure (weather or players getting injured).

On the other hand, high-quality information and good numerical models might bring lots of valuable insights to adequately-understanding people. For example, it would be possible to clearly rank all the objective features of horses/riders to determine the ones which start the race under better conditions. It could be a much more accurate version of the conventional "this horse seems promising" impressions of an expert. But the final decision, knowing how to weight all the involved factors to determine the most likely outcome, should be delivered by a knowledgeable person.

These models are certainly useful here and virtually anywhere, but only for people with actual knowledge who take them as a source of additional information. Someone seriously thinking that just the mere analysis of past events can deliver an accurate estimate of future outcomes, mainly when dealing with too-complex/almost-random situations and relevant amounts of money, is too naive. A person thinking that such magical predictions might be publicly and generously given to everyone is plainly an idiot. Numerical models are just about maximising (certain kind of) available information, not about delivering perfect answers out of thin air.

What a surprise

[OneHundredAndTen]

AI not living up to hype and hoopla. Unprecedented. Who would have thunk.

A collection of opinions is not "AI"

[fygment]

All the algorithm did was collect expert opinions (we don't even know if they were weighted).
Is a survey "AI"? No. And neither was this.

'took notice last year and decided to collaborate'

[Fly Swatter]

In other words, our profits from gambling are in jeopardy! Let's 'embrace and extend this'.

Important lesson

[SuperKendall]

I think it would be a pretty valuable lesson for kids to learn how many people lose gambling, and that there is never a sure thing...

Horse racing also combines the elements of being a real sport, where a kid could simply admire extremely physically adept horses. So it also teaches them about achievement and effort.

It's not like you are taking them to a strip casino in Vegas.

All the time?

[SuperKendall]

So how often do you smoke pot with your8 year old.

Millions of people do this every day.

Ok, it's tobacco. But what is the real difference? Smoking Pot is like speeding, technically illegal but come on. It's a law everyone ignores.

Speeding incidentally is also something millions of people do every day with kids watching.

It is a valuable lesson that society as a whole choses to ignore some laws, because that is what most of reality entails - knowing which laws (delineated or otherwise) it is OK to break. If you claim it's never O to break any they will think you are liar (which you are) and ignore everything else you teach them as false.

They will become gamblers and risk takers.

So you want your kids to avoid risks, to become cogs in someone else's machine, to make sure the whole operates smoothly even as the parts get worn down by life...

I'd rather stick a kid with the parents taking them to casinos.

Re:'took notice last year and decided to collabora

[bws111]

Uh, no. Their profits are dependant solely on how much total money is bet. Who wins or loses is of no concern to them and does not affect their profits at all.

The reason they support it is exactly the same as the reason they publish programs with 'morning lines' and sell racing forms, etc. It lets people think they have an 'edge', so they are more likely to make a bet (any bet, doesn't matter to them).