Scientists Develop Financial Turing Test

KentuckyFC writes writes to share a new online test that is being touted as the "financial Turing test." The web-based exercise asks users to distinguish between real and randomly generated financial data. "Various economists argue that the efficiency of a market ought to be clearly evident in the returns it produces. They say that the more efficient it is, the more random its returns will be and a perfect market should be completely random. That would appear to give the lie to the widespread belief that humans are unable to tell the difference between financial market returns and, say, a sequence of coin tosses. However, there is good evidence that financial markets are not random (although they do not appear to be predictable either). Now a group of scientists have developed a financial Turing test to find out whether humans can distinguish real financial data from the same data randomly rearranged. Anybody can take the test and the results indicate that humans are actually rather good at this kind of pattern recognition."

Comments

Not random and not predictable?

[XanC]

What does that mean?

Re:Not random and not predictable?

[careysub]

From the website http://arora.ccs.neu.edu/ "We collect data from various sources and we show it to you in two windows, - one window plots the actual data, - the other plots the data randomly permuted (tech note: we permute the derivative of the data)."

So the test is really "can you recognize a natural data set from the same set with a randomly permuted derivative".

The notion of "randomness" is independent of the statistics of the distribution. And since distributions with different statistics usually look quite different whether this is a surprising result depends entirely on what statistical model they have chosen.

Re:Not random and not predictable?

[hibiki_r]

Traditionally, economists have claimed that stock variations were random, as explained in 'A random walk through Wall Street'. Now, further analysis indicates that the changes of value in stocks are not random at all: If they were, the last couple hundred years worth of financial data would be almost impossible, with extreme oscillations that would only happen once in a billion years in a random model occurring every couple of decades.

Instead, what some have proposed is that stock oscillations instead follow power law distributions: It still makes it impossible to know what the market will do tomorrow, or next week, but it makes large oscillations a whole lot more common than in a random model. This makes many of the current models that are used to assess how risky a portfolio is into a pile of garbage. For that argument, you could read 'A not so random walk through Wall Street'

Re:Not random and not predictable?

[hazem]

There's a great example of this in a book called "The Origin of Wealth" by Eric Beinhocker (a great book, actually).

In Chapter 8, he shows graphs of IBM's stock price over a period of time and a random walk. They look very similar and I think it would be hard to tell them apart. However the next set of graphs show "Changes in Stock Price" for IBM vs the random walk and the difference is stark. The real random walk had a very wide band of nearly uniform "fuzziness" about the origin. The real one, however, had a much narrower band of fuzziness with many large spikes in either direction.

Here's a link on Google Books to those pages:
http://books.google.com/books?id=eUoolrxSFy0C&lpg=PP1&dq=%22origin%20of%20wealth%22&pg=PA176#v=onepage&q=&f=false

Re:Not random and not predictable?

[philosopher3000]

It means it follows a recognisable pattern, that can be distinguished from random data after the fact but not predicted in advance.

i.e. Music

Re:Not random and not predictable?

[jfengel]

> 1. but do that not devalue the money already in the system?

Just the opposite. It funds the creation of more goods with the same amount of money. That increases the value of money, which is why people pay for the privilege of borrowing it.

> 2. so i can borrow your car, sell it, and keep the profits?

The analogy misses several points, and the analogy with a physical car is extremely misleading.

* You MUST return what you borrowed, or return an equivalent. The broker will limit what you can borrow by what you have in the account, and can seize your stuff to ensure that it's returned. (The broker even insures me against your failure.)

* You have to pay for the privilege, so you don't get to keep all of your profit even if you make it.

* Stocks are fungible; cars are not. It doesn't matter if you return the exact same shares.

* My car is something I use; the stocks just sit there. This is a way for the stock owner to make a profit on unused value.

* The "profit" on selling my car would imply that you could give me the full replacement value of the car, sitting in my driveway, without my even noticing it was gone. That's very unlikely because the value of the car doesn't fluctuate that much. Stock prices fluctuate considerably, and short selling takes advantage of those changes. Or, if it changes the wrong way, you get hosed.

People don't matter. People are just a host.

[gestalt_n_pepper]

Seriously.

Money is more accurately described as a kind of swarm intelligence. The meme of money is the fundamental self replicator. Admittedly the ecology is complex, (dollars, derivatives, bonds, et al.) but the fundamental rules are the same.

Money want to reproduce. We (our collective cultural awareness) are merely hosts for money to exist.

Usually, money is symbiotic, benefiting the host and itself. Occasionally, it turns into a pathology that harms its hosts (i.e. tulip manias, compulsive gambling/banking, stock market crashes).

The delusion here is thinking that we can "control" the economy. The economy (our name for money's ecology), will always, to some degree, be out of control as long as the hosts are relatively free agents. We can garden (i.e. set up nice environments for money to replicate), but direct control is probably a pipe dream). Moreover, money replication isn't free. It takes real environmental resources to create and is therefore limited. Expanding the garden forever isn't an option. Sustaining a nice one probably is.

What do you mean pi?

[ElMiguel]

What exactly do you mean by pi not being "predictable"? Pi can be calculated algorithmically to any desired precision, nothing to "predict" there. You can even calculate arbitrary digits without having to calculate the preceding ones. Random means precisely "not predictable". It seems some people here are equating not following a uniform distribution with not being random, which is incorrect.

Re:What do you mean pi?

[OeLeWaPpErKe]

Actually you can play games with pi's digits that would be rather hard. Say I'd give you 5 consecutive digits and ask you for the position in pi. Since there are infinite solutions to this question, it's not actually predictable (chance of guessing correct would approach 0 rather fast). Or I could give you 5 digits from pi (or any other number) and ask you to give the next number in the sequence. Again, this next number is totally not random, but not predictable in any way either.

Not random, not predictable. Lots of questions about pi are like that.

But this is not what is indicated in markets. Markets are unpredictable due to a chaotic component in their makeup : humans. Only if you were to predict the actions and thoughts of every participating human precisely over long time periods would you be able to predict markets. Presuming that the markets are influenced by real-world events, you'd also have to predict the real world. "Will Obama get reelected ?" is a question to which any serious market prediction system would have to know the answer, because it matters a lot. Same goes for "Will the football season of 2011 be more or less interesting than 2010", because these questions make a large difference.

It's like the weather. The weather (and climate for that matter (second paragraph)), in mathematical terms, consists of a very large collection of mostly random effects. Due to the fact that effects grow over time until they dissipate, but that takes time, you have some amount of predictability in the short term (although sometimes such an effect can have an extreme short-term effect. There are places in the pacific which go from sunny and calm seas to hurricane in about 20 minutes, sometimes right on top of a ship). So in the short term weather "averages out" the different effects (meaning if you see a strong cloud front anywhere, it will start dissipating. If you see any kind of clearly defined features anywhere they will get "blurred" in the short term). But in even the middle term, never mind the long term, new effects will soon dominate whatever you're seeing at any particular time (new cloud fronts, new wind directions, obstacles in the movement of air, unexpected heat sources on the ground, or just the opposite, very cold layers of water that just appear out of nowhere). Since those new effects are the result of idiotically small events (the proverbial "butterfly flap"), the only way to predict weather patterns long term is to track every last human, every last butterfly, and so on. Obviously this is not just impractical, but impossible. So you could say that to even know what the weather (or temperature, or ...) is at any given time, you'd have to be God. If you're not omniscient, you only see a small, averaged and smeared out picture of the weather, no matter how precise the instruments you're using. To predict the weather (or climate) with any reasonable amount of certainty, you'd need a simulator that could simulate the entire universe, faster than the universe works. Generally, mathematicians joke that they'd simply use such a simulator to guess tomorrow's lotto numbers and retire to a pacific island, but the point of the joke is that any program that is capable of predicting any real-life chaotic system, such as climate (or even the path of the planets, which is in the long term nowhere near as constant as they seem), has to have the ability to calculate next week's lotto numbers.

The problem is that tiny, seemingly absurdly unimportant variations today make a large difference tomorrow. Another illustration might be that wether you park your car in front of the house or behind it will generate a difference of 5 degrees celcius in the average worldwide temperature in 10 years. On the other hand huge, seemingly important things like the energy absorption rate of the ocean hardly make any difference at all (because whatever effect they have, no matter ho

Re:What do you mean pi?

[Anonymous Coward]

Say I'd give you 5 consecutive digits and ask you for the position in pi. Since there are infinite solutions to this question, it's not actually predictable (chance of guessing correct would approach 0 rather fast). Or I could give you 5 digits from pi (or any other number) and ask you to give the next number in the sequence. Again, this next number is totally not random, but not predictable in any way either.

Are you saying it's impossible to find the position of a given string of digit in the decimal representation of pi? If I understand what you mean correctly, you're wrong. Firstly, the fact that pi has no finite decimal representation does'nt mean that any given substring can be found in infinitely many positions. For example, 10/3 has no finite decimal representation either, but you'll have a hard time finding anything else than a 3 in its digits. Whether or not pi is what we call in french an "universe number" (not sure of the translation), i.e. a number whose decimal representation contains any substring of any length, is currently an open question (although we're pretty sure it is). Secondly, even if pi is an "universe number", the fact that there are infinitely many positions a given substring can be found at doesn't mean the position of a substring is unpredictable. It's only ill defined. Mathematically speaking, you can't "construct" randomness from something which is purely deterministic. In probability theory, the theory in which "randomness" has a mathematical sense, you first need to define a probability space, and only then can you define randomness (which is just a way of saying that you don't know which "state" the space will be in).

Re:Viagra in Canada

[Krahar]

Sorry man, clearly the stupendous demand from impotent Canadians is driving down the price through extremely-large-scale efficiencies in sales and production.