Scientists Develop Financial Turing Test

KentuckyFC writes writes to share a new online test that is being touted as the "financial Turing test." The web-based exercise asks users to distinguish between real and randomly generated financial data. "Various economists argue that the efficiency of a market ought to be clearly evident in the returns it produces. They say that the more efficient it is, the more random its returns will be and a perfect market should be completely random. That would appear to give the lie to the widespread belief that humans are unable to tell the difference between financial market returns and, say, a sequence of coin tosses. However, there is good evidence that financial markets are not random (although they do not appear to be predictable either). Now a group of scientists have developed a financial Turing test to find out whether humans can distinguish real financial data from the same data randomly rearranged. Anybody can take the test and the results indicate that humans are actually rather good at this kind of pattern recognition."

Comments

Not random and not predictable?

[XanC]

What does that mean?

Re:Not random and not predictable?

[BradleyUffner]

What does that mean?

pi?

Re:Not random and not predictable?

[jackhererUK]

It means it follows a recognisable pattern, that can be distinguished from random data after the fact but not predicted in advance.

Re:Not random and not predictable?

[colonelquesadilla]

It's a chaotic system, but it has certain patterns that seem to repeat. The thing I noticed after looking at a few, is that the real ones are easily identifiable by the development of resistance and support levels, which technical traders use to find probable entry and exit points. Basically, the hypothesis is that a group X holds the stock, they tend to have some psychological barrier price in common at which they would sell, and another at which they would buy more, this selling and buying makes it difficult to break through those price points. When it approaches one of those points trading goes up, if something has changed to make the stock more attractive to another group, or to make it less attractive to the group of traders that tends to hold it, it will change hands, and the new investor group will have new barriers. So over any given time period you will notice a lot of closing stock prices at close to the same level, then a sudden jump, and new level it bounces between, etc.

Re:Not random and not predictable?

[$RANDOMLUSER]

Slashdotters!! If you had a goddam girlfriend, you'd know what "not random and not predictable" meant.

Re:Not random and not predictable?

[Anonymous Coward]

A system can be random but so complex that the only working model of it is the system itself. Such a system is not random but effectively not predictable either.

Re:Not random and not predictable?

[careysub]

From the website http://arora.ccs.neu.edu/ "We collect data from various sources and we show it to you in two windows, - one window plots the actual data, - the other plots the data randomly permuted (tech note: we permute the derivative of the data)."

So the test is really "can you recognize a natural data set from the same set with a randomly permuted derivative".

The notion of "randomness" is independent of the statistics of the distribution. And since distributions with different statistics usually look quite different whether this is a surprising result depends entirely on what statistical model they have chosen.

Re:Not random and not predictable?

[Anonymous Coward]

Totally random = high entropy.
Totally predictable = low entropy.

Neither (totally) random nor (totally) predictable means it's somewhere in the middle
of those two extremes, it has medium entropy, which makes it possible to recognize
by its entropy, once you've gathered enough data.

Re:Not random and not predictable?

[Archangel Michael]

Chaos Theory. Patterns in otherwise seemingly random outcomes. If you look at the details, for instance each snowflake, you'd come to the conclusion that each snowflake is unique (they are), however if you take a step back, you'll notice that the randomness of snowflakes becomes clear in that each snowflake conforms to a pattern that is apparent even as each snowflake is unique.

I know that this is a fairly poor explanation of chaos theory, so don't butcher me too much.

Re:Not random and not predictable?

[presidenteloco]

Kind of like?

You could not have predicted what I was going to type here, but you can understand it once I've typed it?

I have been accused of being random in the past.

Seriously though, to say whether the stock market is "random", you have to define the question more
precisely. If you mean: is the next "index value" that the market is going to generate random? Yes. And thus
is the sequence of those that it is going to generate "random". Yes. I am pretty sure that it is ok for the definition
of random for the pattern that a sequence of random (unpredictable) events to have low complexity statistics.

e.g. a random process can generate a uniform distribution of values

Re:Not random and not predictable?

[pclminion]

A chaotic system is one where arbitrarily small perturbations always lead to arbitrarily large divergence in phase space. What this means is that even though a system might be following a completely causal underlying law of behavior, it still cannot be predicted because it would require having infinitely accurate knowledge of the parameters.

Because measuring apparatus always involves noise, and noise is of some finite value, this means that the arbitrarily small (yet IMPORTANT) perturbations cannot be resolved against the noise background. This places a very limited time window on your ability to make predictions.

Basic examples of this are the Lorenz attractor, the chaotic pendulum, etc.

Re:Not random and not predictable?

[FlyingBishop]

If you had a decent girlfriend, you'd be in just the same boat.

Re:Not random and not predictable?

[hibiki_r]

Traditionally, economists have claimed that stock variations were random, as explained in 'A random walk through Wall Street'. Now, further analysis indicates that the changes of value in stocks are not random at all: If they were, the last couple hundred years worth of financial data would be almost impossible, with extreme oscillations that would only happen once in a billion years in a random model occurring every couple of decades.

Instead, what some have proposed is that stock oscillations instead follow power law distributions: It still makes it impossible to know what the market will do tomorrow, or next week, but it makes large oscillations a whole lot more common than in a random model. This makes many of the current models that are used to assess how risky a portfolio is into a pile of garbage. For that argument, you could read 'A not so random walk through Wall Street'

Re:Not random and not predictable?

[BitZtream]

It means that while its not based on pure randomness, the scope and breadth of the object in question are too large to simulate or emulate in a way that allows us to predict whats going to happen.

Its like a weather pattern.

Both are not random, and are entirely predictable. We are just unable to predict either because we don't have the processing power or the monitoring power to know enough about what we are trying to simulate to accurately predict the outcome. There are simply too many pieces to the puzzle for us to predict, but patterns are apparent and in hindsight, most of the things could have been seen if we had the right focal point and indicators to work with.

Re:Not random and not predictable?

[at_slashdot]

22222222222222222233333333333333333333333333333337777777777777777777777777

These numbers are not random, but they are not predictable (at least not by you).

Re:Not random and not predictable?

[samkass]

If you ask a human to write a series of 100 numbers, picking them "at random" between 1 and 10, you're going to get a list of numbers that has measurably different characteristics than purely random data. In particular, you'll tend to get much too few repeating sequences. It's still not predictable.

Re:Not random and not predictable?

[daremonai]

You wouldn't happen to be a member of the California legislature, would you?

Re:Not random and not predictable?

[Gerzel]

No random does not necessarily correspond to entropy, although it usually does.

Re:Not random and not predictable?

[mikael]

Fractal patterns- a line graph over a long period can be decomposed into a base pattern that is repeated at different amplitudes at different frequencies (hourly, daily, weekly, fortnightly, monthly, annual quarters, yearly). Such analysis can be used to synthesize patterns such as music instruments (audio grains) ocean waves, cloud patterns (Perlin textures), and terrain (fractal landscapes). For 2D and higher, these ratios can vary according to direction as well.

It might be that different traders have different buying/selling patterns, and that these are all superimposed on top of each other.

Re:Not random and not predictable?

[AndrewNeo]

I think you mean 0 through 9.

Re:Not random and not predictable?

[Mister Whirly]

Yep, in a statistics class I had once, half the class was assigned to make up "random number" lists, and the other half actually generated a random number list using actual randomly generated numbers (by using dice). It was very easy to tell which were made by people, becasue as you stated, there were not enough repeating sequences as in the truly random ones.

Re:Not random and not predictable?

[DerekLyons]

It's a chaotic system, but it has certain patterns that seem to repeat.

That seem to repeat more or less exactly, but actually repeat within an unpredictable range of values and on an unpredictable range of cycle timings.

Which is why there are as many technical trading systems as there are gambling systems, with roughly the same results.

Re:Not random and not predictable?

[Anonymous Coward]

Slashdotters!! If you had a goddam girlfriend, you'd know what "not random and not predictable" meant.

Wrong. I do have a girlfriend, and she is entirely predictable: I'm wrong & she's right.

Re:Not random and not predictable?

[hazem]

There's a great example of this in a book called "The Origin of Wealth" by Eric Beinhocker (a great book, actually).

In Chapter 8, he shows graphs of IBM's stock price over a period of time and a random walk. They look very similar and I think it would be hard to tell them apart. However the next set of graphs show "Changes in Stock Price" for IBM vs the random walk and the difference is stark. The real random walk had a very wide band of nearly uniform "fuzziness" about the origin. The real one, however, had a much narrower band of fuzziness with many large spikes in either direction.

Here's a link on Google Books to those pages:
http://books.google.com/books?id=eUoolrxSFy0C&lpg=PP1&dq=%22origin%20of%20wealth%22&pg=PA176#v=onepage&q=&f=false

Re:Not random and not predictable?

[AndersOSU]

it's more that they're trying to read tea leaves, and it's called technical analysis. Some of it makes sense (at least from a psychological/sociological perspective - and don't forget, the market is just people), like levels of resistance and support, while other is just shear lunacy, like the famous head-and-shoulders. However, because there are so many people looking at the same things, it becomes a self-fulfilling prophecy. A head-and-shoulders supposedly means the stock is going to drop. Mind you, there's not fundamental reason why this is so, but because everyone thinks that's what it means they sell their stock, and sure enough, the price drops.

Re:Not random and not predictable?

[mikael]

If it weren't for the fact that all the other inputs to the system were human, analysis might work, but as you say once you get everyone else looking to see what everyone else is doing and trying to analyze each other, it just ends up being a twitchy feedback system.

Re:Not random and not predictable?

[Anonymous Coward]

No, he meant between 1 and 10.

Re:Not random and not predictable?

[DriedClexler]

Create an encoding method for algorithms that outputs a graph. Randomly generate a short algorithm.

It's not random, because it has a low Kolmogorov complexity (shortest program that outputs the data). But it's not predictable either, because you don't know *which* simple program it is.

Re:Not random and not predictable?

[guruevi]

I never understood this trading thing. Basically, all you do is transfer money around and hope you get more by making the right choices. However by making money, you are causing somebody else to lose money. All-in-all the global system doesn't make or lose any money unless somebody adds more product thereby reducing the worth of the same product already in the hands of somebody else.

I also don't know how doing bad things here can cause the markets to crash. You either have stock or you don't, selling stuff you don't have should be impossible because the market should keep track of what each person has. If all-of-a-sudden there is a major increase in money going around for a certain product without anyone actually producing the product (a so-called bubble) it should be clearly visible.

Anyway, maybe I'm ignorant but for some reason, some people get rich off it.

Re:Not random and not predictable?

[u38cg]

Yes, you are pretty ignorant, I'm afraid. Don't be ashamed, you're in the larger group. Happily, a dose of economics would sort you out a treat. To sort out your central misunderstanding, neither the amount of wealth or the amount of things that you can buy or the amount of work there is to be done is fixed. They relate to each other in rather complex ways, but the upshot is that we can all become richer - and if you don't believe me, ask your great-great-grandfather, or a Chinese factory worker saving up her wages to pay for an education.

Re:Not random and not predictable?

[Prof.Phreak]

But chaotic systems are generally predictable short term... while markets aren't (otherwise everyone would make a ton of money trading within seconds of observations).

Re:Not random and not predictable?

[Anonymous Coward]

Not predictable in advance? Nonsense. Ask any good derivatives trader...

Re:Not random and not predictable?

[jfengel]

There are a couple of things you're missing:

1. There is money pumped into the system from outside. When you buy a stock, you're really investing in a company that makes things that (hopefully) people want. You may not be investing in the company directly, but by purchasing it from somebody who did, you're making the initial investment possible.

So it's not a zero-sum game. You're making money available to people who make stuff. If you think that's worthless, try starting a company some time without borrowing money somehow.

2. You can sell stuff you don't have by borrowing it from other people. There's nothing wrong with that: you're paying for the privilege.

You can also make a promise to sell it without even borrowing it first, assuming you can get your hands on it before the deal actually concludes. That kind of "naked short" is considerably more dangerous, because it's completely unconstrained, and that IS forbidden.

People do make all sorts of bad choices; some of them do play the stock market as if it were some kind of lottery. Those people lose money. Smart people, however, can generally make money by putting it where it's needed.

Re:Not random and not predictable?

[Anonymous Coward]

They repeat more or less exactly. What repeats is the behaviour of participants, not "patterns" or "prices". For those who are suitably tuned, it is possible to determine the intent of the relevant participants and make decisions based on that. It is therefore possible to accurately forecast most liquid derivatives most of the time.

The "technical trading systems" you speak of are nonsenses marketed to a gullible, greedy, and irresponsible public who use them as excuses to gamble. They have no value whatsoever in accurate predication of future price and liquidity conditions.

You refer to an "unpredictable range of values and on an unpredictable range of cycle timings". I can assure you that none of this is unpredictable, but constructed in such a way that very few people will ever understand it, and even fewer will be able to make consistently correct predictions.

Re:Not random and not predictable?

[Anonymous Coward]

it's more that they're trying to read tea leaves, and it's called technical analysis.

"Technical analysis" is utter nonsense marketed to a gullible, greedy, irresponsible public who use it as an excuse to gamble...

Some of it makes sense (at least from a psychological/sociological perspective - and don't forget, the market is just people), like levels of resistance and support, while other is just shear lunacy, like the famous head-and-shoulders.

Some of it appears to make sense. What if I told you that the concepts of resistance and support are sheer lunacy, and the head and shoulders (properly interpreted) "makes sense"?

A head-and-shoulders supposedly means the stock is going to drop. Mind you, there's not fundamental reason why this is so, but because everyone thinks that's what it means they sell their stock, and sure enough, the price drops.

Everyone? Who buys the stock that others sell, and why?

Re:Not random and not predictable?

[pclminion]

The rate at which the system diverges is characterized by a set of numbers called Lyapunov exponents -- if the stock market is indeed a chaotic process, its short-term unpredictability could mean that it has particularly large exponents. Of course, there's no proof that the stock market is a chaotic system, but chaos is a good example of something which is both "non-random" and "unpredictable."

Re:Not random and not predictable?

[jonadab]

> What does [not random and not predictable] mean?

It means that while some outcomes are more likely than others, the stock market still surprises even the experts sometimes.

Re:Not random and not predictable?

[jmalicki]

In other words, part of what determines the value is a component of random noise. In other words, the series is random. Not uniform, not brownian motion, etc., but a random process nonetheless.

Re:Not random and not predictable?

[colonelquesadilla]

What always struck me as odd about the "random walk" conjecture is that it seems that if stock prices were random walking they should in the short term follow some sort of normal distribution. They don't seem to. Some traders like to use bollinger bands, which place standard deviation lines above and below a moving average. That always seemed silly to me, since it is well known that stock prices are not even close to following a gaussian distribution. Furthermore, the whole idea seems to lose sight of the fact that these aren't numbers coming out of the ether, they are being produced by a large number of people, each behaving in different, but somewhat predictable ways. That makes for a very complex system sure, but not a random one.

Re:Not random and not predictable?

[colonelquesadilla]

5? I'm sure I can find a polynomial for which the next solution is 5. Does that count?

Re:Not random and not predictable?

[pclminion]

No. It is your attempt to measure the system which has a random component. The uncertainty is not in what the system is doing but in your measurements of its parameters. The system is in some well-defined state, but in order to predict it, you need infinitely accurate measurements, and you can't have those.

Of course, any system operating in the real world will be subject to random variations, but chaos is not limited to noisy systems. Even a simple set of mathematically perfect differential equations can exhibit chaos.

Re:Not random and not predictable?

[Anonymous Coward]

How is the amount of work that can be done not constrained? This is one of those Randian tenets I can't understand.

If we weren't constrained by the natural world, we would all be gods, right?

Re:Not random and not predictable?

[Anonymous Coward]

There's no difference, unless you're going to insist on equivocating between "numerals" and "numbers".

Re:Not random and not predictable?

[Anonymous Coward]

This trading thing....

"all you do is transfer money around and hope you get more by making the right choices"

I'll correct that for you.

"What we do is transfer wealth from those who do not intimately understand how markets function to those who have done the work and earned the right to understand, and who additionally behave correctly by obeying the strict protocols the market imposes."

Hope is irrelevant. A good trader does not hope to make the right choices - they make the right choices by default. This comes from years of experience and an understanding of how all of this is made to operate. As a consequence they absorb wealth from those who trade, but are less informed.

People trade for a variety of reasons. Some have to trade. Not all people who trade do so to realise a directional profit. There are producers, consumers, hedgers, speculators, arbitragers, etc. There are some who provide a service to those who need to trade, by taking the other side. The successful trader will provide a service to uninformed traders, by standing ready to become buyer to their seller and vice versa when they need to trade.

The truly expert traders choose to earn their living, if you wish to think in such terms, by becoming proficient in detecting where a market will move next, based on the motivations of certain market participants. Part of this is knowing where the uninformed traders are likely to err, and profiting from that information.

Futures are zero sum and no "new wealth" is created when a contract expires - shorts must provide delivery, longs take delivery, and everyone else has netted off the exposure and gone flat. No new goods are produced.

What it provides is a nearly insurmountable challenge which appeals to a certain character. Those with the intelligence, persistence, and discipline will eventually succeed. It is the most efficient way to accumulate wealth when practised correctly, and it yields a certain intellectual satisfaction.

It is the only profession where results can be achieved in minutes that might otherwise take days of road digging or sheep shearing or other.

Re:Not random and not predictable?

[antdude]

What is a girlfriend? :P

Re:Not random and not predictable?

[Anonymous Coward]

Basically, all you do is transfer money around and hope you get more by making the right choices. However by making money, you are causing somebody else to lose money. All-in-all the global system doesn't make or lose any money unless somebody adds more product thereby reducing the worth of the same product already in the hands of somebody else.

This is not very far from reality. Markets go up because companies create value (or are expected to create value), which is then redistributed in the market (through debt or equity). But if you're talking about professional traders, moving money around is not the way they make money. Basically, the sell financial products (~=investments) at a margin, and this is where they make money. Why a margin? Because they're supporting the risks of market moves, which can make them either earn or lose money. But (most) professional traders try to be as much 'market neutral' as they can, and neither win nor lose money from market moves. In fact, this is the biggest part of their job : hedging the risks.

selling stuff you don't have should be impossible because the market should keep track of what each person has

You're referring to short selling. Usually, you borrow the shares you want to sell from someone else, and then you sell them. Later on, you rebuy them on the market and
reimburse the guy you borrowed them from (with interest). So you're not "lying" to anyone, and clearing houses keep track of what each person has every day.

some people get rich off it

Some people also lose their shirt. So there is no contradiction here.

Re:Not random and not predictable?

[Anonymous Coward]

Randian theory presupposes the existence of god-like men, no?

Re:Not random and not predictable?

[hitmark]

atlas?

Re:Not random and not predictable?

[hitmark]

1. but do that not devalue the money already in the system?

2. so i can borrow your car, sell it, and keep the profits?

Re:Not random and not predictable?

[philosopher3000]

It means it follows a recognisable pattern, that can be distinguished from random data after the fact but not predicted in advance.

i.e. Music

Re:Not random and not predictable?

[hitmark]

i think that disconnect between wealth and money is what confuses most of us. This as its easy to do something like wealth == money. But money is just a gobetween for direct exchange of physical objects, or physical objects and performing a service.

so when the topic of wealth comes up, there is a cognitive disconnect between the economist and the joe on the street. Heck, economy may be the second most jargon laden area of study next to religion...

Re:Not random and not predictable?

[hitmark]

they do, but its not a human trader thats doing it. Several of the biggest financial houses have automated systems linked to the stock exchanges, performing transactions based on small changes in the market. Funny enough, they make use of clusters of the same hardware that power pc games.

Re:Not random and not predictable?

[Anonymous Coward]

"... one where arbitrarily small perturbations always lead to arbitrarily large divergence in phase space". You've only described a high gain system. High gain is indeed one characteristic of a chaotic system, but it has other important attributes as well. I also question your "always". Small perturbations at the input can also (unpredictably) result in small divergences at the output - so one might more accurately say that the input range is much smaller than the output range. More broadly, a chaotic system is one in which deterministic (and therefore predictable) inputs result in non-deterministic (and therefore unpredictable) outputs.

I would also add that even where the input perturbations can be detected above the noise floor you're no better off at prediction (even if you know the deterministic transfer function) as you never really know the starting point of the system. This is the basis of algorithmic pseudo-random number generation, although the algorithms, inputs and initial conditions of real world chaotic systems are astronomically more complex. This is why nobody has ever worked out the theoretical single winning shot in pool that pockets all the balls in the right order. In principle, if you were able to restart the system from exactly same initial conditions, the output would proceed identically, but you can't in the real world as you don't know exactly what those conditions were and they cannot be inferred in retrospect.

Re:Not random and not predictable?

[DavidShor]

But chaotic systems are generally predictable short term... while markets aren't (otherwise everyone would make a ton of money trading within seconds of observations).

In general, there are a ton of people who make a ton of money trading within seconds of observations. You hear hedge fund managers say things like "There's not anywhere near enough capital in the world to soak up all the arbitrage out there". There are all sorts of short-run patterns like momentum that are still out there, despite plenty of studies pointing them out. I actually worked in this sort of thing specifically in an internship last year, but I signed an NDA, so that's probably all I can say about that.

Less controversially, if you want to predict stock prices 3 minutes from now, current prices would do a pretty good job. There are even well developed public models that will give you a good idea of the error bound!

But in terms of this whole debate, I'd say stock prices are random. Not uniform, but models that look at asset prices as log-normal random walks do pretty well. The ones that use heavier tails and stochastic variance and all do even better. One can model chaotic systems with probabilistic tools pretty well too though, so I think the distinction is a bit pointless.

Re:Not random and not predictable?

[jfengel]

> 1. but do that not devalue the money already in the system?

Just the opposite. It funds the creation of more goods with the same amount of money. That increases the value of money, which is why people pay for the privilege of borrowing it.

> 2. so i can borrow your car, sell it, and keep the profits?

The analogy misses several points, and the analogy with a physical car is extremely misleading.

* You MUST return what you borrowed, or return an equivalent. The broker will limit what you can borrow by what you have in the account, and can seize your stuff to ensure that it's returned. (The broker even insures me against your failure.)

* You have to pay for the privilege, so you don't get to keep all of your profit even if you make it.

* Stocks are fungible; cars are not. It doesn't matter if you return the exact same shares.

* My car is something I use; the stocks just sit there. This is a way for the stock owner to make a profit on unused value.

* The "profit" on selling my car would imply that you could give me the full replacement value of the car, sitting in my driveway, without my even noticing it was gone. That's very unlikely because the value of the car doesn't fluctuate that much. Stock prices fluctuate considerably, and short selling takes advantage of those changes. Or, if it changes the wrong way, you get hosed.

Re:Not random and not predictable?

[DavidShor]

You're mangling things a bit. Here is a cleaned up version of what you were trying to say:

.

Initially, it was believed that asset prices followed a log-normal random walk. That is to say, if Y(t) is equal to price on day t, that Log(Y(t))=Log(Y(t-1))+e(t), where e(t) is a normally distributed error term.

Very quickly, people realized that this wasn't true, and that rare events happened more much often then the previous model predicted. Instead, many academics and investors found that using a model where the error term followed a distribution with heavier tails was more accurate. Usually, something like an Alpha-Stable distribution. (I'm ignoring stuff like stochastic volatility and drift...)

Unfortunately, the math on the initial model is much easier to deal with, because normal distributions have a lot of great properties, and so many people on wall-street used the initial model instead of the more complicated but accurate one. This caused problems.

But model is no more or less "random" then the other.

Re:Not random and not predictable?

[DavidShor]

"What always struck me as odd about the "random walk" conjecture is that it seems that if stock prices were random walking they should in the short term follow some sort of normal distribution. "

.

In the short term, stocks *do* move around following some sort of normal distribution (Not really normal, but fatten the tails a bit and you're good). Take a look the daily returns histogram at http://www.wolframalpha.com/input/?i=IBM .

" Some traders like to use bollinger bands, which place standard deviation lines above and below a moving average. That always seemed silly to me, since it is well known that stock prices are not even close to following a gaussian distribution."

Even if the distribution isn't normal, standard deviations are still useful. http://en.wikipedia.org/wiki/Chebyshev's_inequality is the simplest theorem, but there are more complicated ones like the Petunin inequality that give better bounds. The basic rule is as long as your distribution isn't too weird, the 2-3 75-99 rule works pretty well...

"Furthermore, the whole idea seems to lose sight of the fact that these aren't numbers coming out of the ether, they are being produced by a large number of people, each behaving in different, but somewhat predictable ways. That makes for a very complex system sure, but not a random one."

Eh, pseudo-random number generators are produced by a large number of circuits, behaving in different but almost perfectly predictable ways. That complex systems sometimes act indistinguishably from random ones is a neat result of ergodic theory...

Re:Not random and not predictable?

[hitmark]

1. how can it be the same amount when the amount was just increased from a outside source? or are we talking about different "amounts"?

2. so the system only really work when one is trading in non-physical "objects"?

Re:Not random and not predictable?

[jfengel]

1. I was referring to "value", the total amount of wealth, in terms of products made and services performed. That increases, and it's assigned a dollar figure, but that's a price tag, not an actual dollar. It's not increasing the number of actual dollars out there. There are more goods to buy with the same number of dollars.

(Fractional-reserve banking, on the other hand, genuinely does devalue the dollar. But that's a whole different story.)

2. You can actually do it with physical commodities as well. You can short-sell grain, pork bellies, or frozen concentrated orange juice. But what you're really shorting is the promises to deliver them, which is a lot easier than shipping around actual pork bellies. Those promises, unlike the pork bellies, are fungible.

Re:Not random and not predictable?

[Anonymous Coward]

Summation,

"I'm smarter than you so I deserve to have your money"

Look closely folks, this is the sort of parasitic mind that pervades the financial sector.

Exploit loopholes, bend laws, make new laws that keep the money flowing into his own pockets - whatever it takes, it's just him being smarter at the game.

Oh Java!

[BadAnalogyGuy]

Is the test where we have to decide whether to install Java?

Because I pass.

Re:Oh Java!

[Anonymous Coward]

I fail

Re:Oh Java!

[vtcodger]

What's a Java? Does it bite? Is it housebroken?

Here Is The Simple Form:

[WrongSizeGlass]

* Do you have any money?
- If 'No', please leave.
- If 'Yes', please give me your money.
* Did you give me all your money?
- If 'No', you pass. Please leave.
- If 'Yes', you are a fool. Please Leave.

Turing Test?

[Anonymous Coward]

Based on 2 previews, I failed. Does that mean I am not human?

Re:Turing Test?

[sinrakin]

I failed five times in a row, then I read the article. It says humans can "learn" to tell the difference between the series, not that they can tell immediately. However the quote says "It's not hard to see why. In feedback sessions, the players say that the real data was smoother than the randomised data or vice versa and that these patterns were easy to spot after a few goes". So it sounds like people actually don't know how they're recognizing the patterns. Actually, I'd bet you could construct data that would fool people if you superimposed a few random series with different periods, say: quarterly, weekly, daily, etc.

Economists ...

[Anonymous Coward]

Various economists argue that the efficiency of a market ought to be clearly evident in the returns it produces.

The market is only efficient within a narrow range of economic activity. When economic activity exceeds the top and bottom ranges you get bubbles and panics - inefficient markets. We see them all the time.

I really wish economists would stop assuming that for any given economic activity, the conditions and their subsequent results can be extrapolated across the board. That's why, whether it's the Chicago school or the Keynesians, they can point to data (a selected portion of economic activity) that supports their view, when in fact all schools of economics is correct in their little slice of economic activity and conditions.

Re:Economists ...

[geoffrobinson]

I tend towards the libertarian end of the spectrum, especially in economics. The past three or so years has made me think about economics a lot and my underlying assumptions.

My conclusion is that it is extremely hard to come up with a very good detailed economics theory because you can't really test all that well. And your data sample set is limited.

What you would really need is access to alternative realities. What would have happened if the Fed didn't reduce rates to really low levels 9 years ago? What would have happened if we let the banks go under instead of bailing them out? I have guesses but no firm idea. You can't rerun the Great Depression and fine-tune all the variables.

So when I see guys like Paul Krugman insist we need a bigger stimulus, the first thing I think is "this is completely unfalsifiable."

Re:Economists ...

[blahplusplus]

The real issue is that the price mechanism can't deal with the complexity of the real world, it's too easy to externalzie and hide other important real world data and costs that price mechanisms of market theoreticians can never take into account.

Efficient markets could work if everybody was god, but people are not 1) equally skilled 2) have equal time to analze information 3) do not have unbiased access to opportunities 4) Human psychology tends to treat those with much money as special or royalty, when most rich people have simply taken advantage of large numbers - they are not necessarily rich because they are better then competing products or better then everyone else for instance, rather lots of complex factors go into "financial success" often times it involves criminal behaviour (i.e. bail out of the banks, the stock market ponzi game, etc)

Re:Economists ...

[Maxo-Texas]

The market is also inefficient when any participant with more resources uses some of those resources to change the rules of the market to favor that participant.

For example, the same movie sells in china for 2.49 and in america for 19.99 (often with an english soundtrack for both). In an efficient market, the movie would be purchased there for 2.49 and sold here for 4.99 making a 100% profit for someone. But artificial rules restrict this.

For example, drugs which are out of patent are sold in India and China for 10 cents a pill but for 33 cents a pill in the united states (example- metformin) and those in patent are sold for about 10 to 50 cents a pill in india and china and for 5.00 a pill in the united states. (and apparently viagra is much cheaper in canada than the u.s.). In an efficient market those pills would be purchased, imported, and resold. But artificial laws prevent this rational activity.

For example, Microsoft absolutely slaughtered many competitors through illegal monopolistic behaviors (for which it lost many court cases years after the competitors were dead or crippled). In an efficient market, there would have been other options.

Re:Economists ...

[Qzukk]

the first thing I think is "this is completely unfalsifiable."

And now you understand why there's no Nobel prize for economists.

Re:Economists ...

[cynical kane]

That's not market efficiency. In your example, the moviemakers would respond by making movies 19.99 globally, with the market failure of the Chinese not being able to afford movies.

Price discrimination* is a key part of economic efficiency when a monopolistic competitor** has control over their market goods. If the competitor sets prices without discrimination, this causes inefficiency because buyers (the Chinese) and sellers (the moviemaker) never get to trade, and market efficiency is defined as maximizing trade within the market.

* The market kind, not the racist kind.
** A monopolistic competitor refers to a competitor that has control over a narrow niche in a wide market, and is not the same as a monopoly.

Re:Economists ...

[Maxo-Texas]

Okay... you are using some special meaning of an efficient market.

I'm talking plain english here.

If hot dogs are selling for $1 on this block and $15 three blocks away, an "english language" efficient market is going to close that gap.

Americans are basically being pumped dry of wealth at this point. You either sneak around the system (by taking a thermos with a hot dog into the theater) or just refuse to participate.

Re:Economists ...

[Anonymous Coward]

For example, Microsoft absolutely slaughtered many competitors through illegal monopolistic behaviors (for which it lost many court cases years after the competitors were dead or crippled). In an efficient market, there would have been other options.

That merely points out one of the other flaws in the idea of a "free" market. Nothing succeeds like Success. Sometimes known as "Nobody ever got fired for buying IBM^WMicrosoft".

In engineering terms, this is a positive feedback circuit, and uncontrolled positive feedback is the exact opposite of self-correction. Monopolies exist because people are too greedy to leave well enough alone even when the end result would be the same.

Re:Economists ...

[The Wild Norseman]

and apparently viagra is much cheaper in canada than the u.s.

And here I thought that America was full of cheap pricks...

Re:Economists ...

[ElAurian]

There is nothing wrong with price discrimination. All companies should be allowed to sell their products for whatever price they want, based on any criteria they want (except, arguably, race/sex/religion etc, because society doesn't like it).

However, using the legal system to enforce that price discrimination is a bad idea. Without the extensive legal barriers set up to "make prices fair", and with uniform labor/production/safety etc laws worldwide, the average incomes of people in the West and people in China (in this example) would equalise much more rapidly.

When corporations hijack governments to protect their own interests, it is bad for everyone.

Re:Economists ...

[Anonymous Coward]

Really? I know that a company can maximize profit by price discriminating, and selling at price specific to the individual, but is it really the case that the company would sell at $19.99 globally? I doubt that the US profit maximizing pricepoint and the world profit maximizing pricepoint are the same. If they could not discriminate by location or individual, the company would be best to sell at the world profit maximizing price, not the US profit maximizing price, no?

Re:Economists ...

[hitmark]

4) human psychology == calvinism?

Re:Economists ...

[hitmark]

plain english, and economic english are not the same. This allows the economists to hide what they are saying in jargon that will mean something different when read as plain english.

Re:Economists ...

[BeanThere]

There is nothing wrong with price discrimination.

Not only is there nothing wrong with price discrimination, but many business models rely on it for production to be viable at all. In the movie example, if movie producers could not get more than a few dollars per movie anywhere, many would not be able to cover the costs of movie production and would simply go out of business --- yay, then you have 'no movie at all' as the alternative. (Yes there are very profitable movies but many movies do little more than break even currently, and some run at a loss.) The same is true in software (e.g. expensive 'professional editions' vs inexpensive student or lower-end editions) and hardware, e.g. the classic example of CPU pricing which is so non-linear. It might look 'unfair' and you might think 'but gee, this price is artificial and the company could just be selling the same thing to me for less' but in reality the company can *not* just sell you the same thing for less because aggregate income would drop below overall cost of production if the more expensive markets weren't helping subsidise the lower-end market. It's not some 'evil' thing companies do, it's usually the choice of do that or have no product at all.

Re:Economists ...

[BeanThere]

That merely points out one of the other flaws in the idea of a "free" market.

Not many people claim that free markets always produce economically optimal results, but that isn't the idea. It's not a "flaw", it's just a side effect of how free markets work with human participants.

Nothing succeeds like Success.

Yes and no, maintaining success is not easy even when built on success; there are a bazillion counter-examples of highly successful and well-entrenched companies in some market, who practically overnight disappear into obscurity. Borland for example once totally dominated the development tools market, and in no time at all they vanished into obscurity.

Key is Jumps

[ObsessiveMathsFreak]

A friend of mine actually came up with this test a few months ago and sent an email around with 8 series to see if people could spot real data from randomised ones (Maybe it got chained on to a wider audience).

The key to the test is that random walks typically don't undergo large jumps or oscillations. In fact, they're generally quite a bit smoother than real data. I see that TFA comes to more or less the same conclusion(I think).

The moral if this story is that 99% of normal probability theory does not easily apply to financial time series data.

Re:Key is Jumps

[ottothecow]

The jumps in something like a stock price are mostly due to the fact that big chunks if information hit the market at once. If we have an efficient market (meaning the investors are well informed and value the company according to all currently available information), you are bound to see a big hit when a company has an earnings call that goes "hey guys...you know how we said we would earn 23 cents a share? yeah...well we lost 15".

These instances are decidedly not random but tied to the facts of the underlying business and since news is usually released in quarterly calls and SEC filings, there will have to be large one-time corrections rather than random-looking up and down movements over time that trend toward the final price.

Re:Key is Jumps

[Fnord666]

The moral if this story is that 99% of normal probability theory does not easily apply to financial time series data.

Just remember that in most economic theories, the real world often turns out to be a special case.

Re:Key is Jumps

[u38cg]

Would that be because time series analysis is a quite separate discipline to simple inference?

little in common with the Turing test

[snarkh]

The test is to distinguish computer-generated graphs from the actual stock prices.
It seems to have very little to do with the actual Turing test.

Re:little in common with the Turing test

[FlyingBishop]

Actually it sounds to me like a computer could be made to recognize this as well as a human pretty easily. It's just statistics. But I must be missing something, because an economist would certainly have as good an understanding of the Turing Test as I do of economics.

Re:little in common with the Turing test

[snarkh]

Yes, I don't see why a computer cannot be made to recognize this. It just shows that the current models do not model the stock movements very well.

Re:little in common with the Turing test

[JoelKatz]

Or, to put it another way, it just shows that they found a way to make random data that doesn't look very much like actual financial data.

not (ever) predictable = random

[presidenteloco]

"markets are not random (although they do not appear to be predictable either)"

Ummmm. Isn't one of the leading definitions of a "random" process that it is
a process which exhibits maximum complexity, and thus is not predictable
except by the execution of the identical process. ?

i.e. "inherently unpredictable by any algorithm simpler than the process itself" = "random"

Re:not (ever) predictable = random

[Digicaf]

It's been a while since I was in math, but I believe the key is that "unpredictable == random" only when "unpredictable" is completely unbound by statistical analysis. If "unpredictable" is bound by a statistical equation, then it is not random, even if it is only loosely bound.

E.g. if some function can predict the result of said system with an average rate of success that is greater than 0% but less than 100%, then the system can be said to be not random and not predictable.

Re:not (ever) predictable = random

[radtea]

Ummmm. Isn't one of the leading definitions of a "random" process that it is
a process which exhibits maximum complexity, and thus is not predictable
except by the execution of the identical process. ?

Yes it is. So what?

You've pointed out that randomness implies unpredictability.

You seem to be asking about how non-random data can be unpredictable, which is an unrelated question.

A => B does not mean !A => !B.

Everyone in Canada has decent health care. That is completely unrelated to the question of whether anyone in the United States or France or Australia (that is, people who are not in Canada) has decent health care.

Re:not (ever) predictable = random

[gmuslera]

What about not random, but the amount of variables involved is high and not all known or acknowledged by all the players?
The algorithm could be simpler than the process, but for running it you need information that some of the players won't disclose.

Anyway, some of the key elements could be related to complex enough system (i.e. weather, how Katrina changed markets? how predictable it was with i.e. 2 weeks in advance? o human behaviour unrelated to market, like 911)

Re:not (ever) predictable = random

[Obfuscant]

E.g. if some function can predict the result of said system with an average rate of success that is greater than 0% but less than 100%, then the system can be said to be not random and not predictable.

That has to be incorrect.

Look at coin flips. The function "X = heads" predicts the result of that system correctly on average 50% of the time. Using your definition, the flip of a coin is not random and not predictable, when the truth is it is random and not predictable.

Random statistics require an equal probability of each outcome. The digits of PI are, as a set, random, but quite predictable in sequence. The closing price of stock X will follow something close to a normal distribution, thus it cannot be random. Of all the prices that a stock COULD have, you are more likely to be correct if you pick one close to the previous price. I.e., the chance of being right is greater than 1 in [all possibilities].

Re:not (ever) predictable = random

[Obfuscant]

Everyone in Canada has decent health care.

Crap. I posted to this thread so I lost my ability to mod this as flamebait. Defining "decent" as "sometimes ok, sometimes ineffective, always delayed and rationed" is dishonest at best.

Re:not (ever) predictable = random

[SwordsmanLuke]

You're right, that's phrased poorly. What it *should* say is that "markets are not random although they do not appear to be entirely predictable either".

No one can 100% predict the movements of the market - but because it's not actually random, you can predict correctly better than half the time - which means you can make money.

Re:not (ever) predictable = random

[jonadab]

It depends on whose definition of "random" you use.

Mathematicians and computer scientists generally (and statisticians always) define random as "any given result is as likely as any other", which is not entirely the same thing as merely "unpredictable". For example, in cryptography, mere unpredictability is adequate for most purposes, but such series are not considered to be truly (mathematically) random, hence the term "pseudorandom".

Normal people on the street (i.e., laypersons when it comes to mathematics) generally define "random" in a way that is closer to what a statistician would call an _even_ distribution, but with additional constraints. For example, if a series of thirty integers between 1 and 10 starts with 1, it's not fully "random". If it ends with 10, it's not fully "random". If it contains an ascending sequence of three adjascent numbers anywhere within it, most people would say that it is "not very random".

Similarly, if the "random" option on a "show my photos" screensaver picks the same picture twice in five minutes when there are fifteen minutes' worth of photos, it's "not random enough", and the software should be programmed to avoid this so as to keep the selection "random". The same logic applies to "random" playlist generation. Mathematically speaking, constraining a sequence in these kinds of ways makes it _less_ random (since it makes some outcomes much more unlikely than others, if not outright impossible), but most people will perceive the result as _more_ random, because they're not using the mathematicians' definition of "random".

This is one reason why the option is often called "shuffle" instead of "random". Programmers are uncomfortable using the word "random" to describe the kind of randomness users want. Personally, I wrote my own playlist generation algorithm, and it is extensively non-random. It goes out of its way to switch between genres as often as possible (after every track isn't possible, because more than half of my collection is baroque; but it switches genres as often as it can), and to mix up each genre so that the next track from the same genre will be by a different artist or composer, and to mix up each artist or composer so that the next track by the same guy will be a different song, preferably from a different album. Additionally, it plays any given track significantly more often if I've given it a higher rating. I know that the algorithm is very deterministic and entirely non-random, because I wrote it. There isn't a single call to rand() in there anywhere. My family, however, would certainly describe the resulting playlist as "random", because the songs are all mixed up. It never plays the same kind of thing twice in a row, ever. In other words, it fits the layman's definition of "random".

So yeah, there are different ideas about what "random" means.

Personally, I have come to the conclusion that the word "random", without further qualification, is too ambiguous and unclear to be very useful, and so I have begun to make an effort to substitute other words and phrases whenever possible. If I mean that a given value was chosen for no particular reason other than that one needed to be chosen, I say "arbitrary". If I mean "cryptographically secure", I say that. (Of course, there are different levels of cryptographic security. But at least you know exactly what _kind_ of randomness I'm talking about.)

Re:not (ever) predictable = random

[Anonymous Coward]

Defining what appears to be your personal experiences as "the health care everyone in Canada has" is dishonest at best. "Sometimes ok, sometimes ineffective" -- I'm going to go ahead and say that's true but it's true regardless of the country. "Always delayed and rationed" is patently false. I've certainly heard anecdotes of it delayed and rationed. I've had great experiences with Canadian healthcare and poor experiences with US healthcare, and no personal experiences with any other kind.

Also, clearly his premise was a throwaway line, not the central thesis, and therefore questions of honesty are rather absurd.

Re:not (ever) predictable = random

[JoelKatz]

Actually, that's not even strictly necessary. For example, a coin flip is, in a sense, completely unpredictable. But in another sense, you can perfectly predict that it will either come up heads or tails, and it will not come up 3 or Fred.

To put it more precisely, you can perfectly predict that half the time a coin will come up heads and half the time it will come up tails. So if you frequently have opportunities where you'll make $50 if it comes up heads but lose $45 if it comes up tails, you can reliably make money over time betting on coin flips, even though the coin flips are entirely unpredictable.

It is in this sense that you need not predict the market to make money. You need only determine probabilities for various market outcomes and take only bets where you have a statistical chance of a net gain.

Not What It Appears To Be......

[Anonymous Coward]

I clicked the link. Instead of a "Financial Turing Test", it looks more like a "Slashdot My Website In Seconds Test".

Re:Not What It Appears To Be......

[AndrewNeo]

Well, you have to give them credit - they passed the test! They are most definitely Slashdotted.

That DOES NOT mean financial data is not random...

[viraltus]

It could mean that we humans are good distinguishing among different random patterns, but the fact we can do that says nothing about the randomnes of the series.

I already built this...

[jdigital]

Three years ago - see http://www.felixsalmon.com/000763.html

Oh welp. History repeats itself.

People don't matter. People are just a host.

[gestalt_n_pepper]

Seriously.

Money is more accurately described as a kind of swarm intelligence. The meme of money is the fundamental self replicator. Admittedly the ecology is complex, (dollars, derivatives, bonds, et al.) but the fundamental rules are the same.

Money want to reproduce. We (our collective cultural awareness) are merely hosts for money to exist.

Usually, money is symbiotic, benefiting the host and itself. Occasionally, it turns into a pathology that harms its hosts (i.e. tulip manias, compulsive gambling/banking, stock market crashes).

The delusion here is thinking that we can "control" the economy. The economy (our name for money's ecology), will always, to some degree, be out of control as long as the hosts are relatively free agents. We can garden (i.e. set up nice environments for money to replicate), but direct control is probably a pipe dream). Moreover, money replication isn't free. It takes real environmental resources to create and is therefore limited. Expanding the garden forever isn't an option. Sustaining a nice one probably is.

Re:People don't matter. People are just a host.

[HeckRuler]

Who let the biologist in here?
I thought we agreed that smoking jackets and top hats were required?
Jenkins! Chase off this riff raff at once!

Re:People don't matter. People are just a host.

[gestalt_n_pepper]

I'm a *psychologist* you insensitive clod (Well, I have the degree anyway - actually I develop software for a living)

Re:People don't matter. People are just a host.

[fridaynightsmoke]

Seriously.

Money is more accurately described as a kind of swarm intelligence. The meme of money is the fundamental self replicator. Admittedly the ecology is complex, (dollars, derivatives, bonds, et al.) but the fundamental rules are the same.

Money want to reproduce. We (our collective cultural awareness) are merely hosts for money to exist.

Usually, money is symbiotic, benefiting the host and itself. Occasionally, it turns into a pathology that harms its hosts (i.e. tulip manias, compulsive gambling/banking, stock market crashes).

The delusion here is thinking that we can "control" the economy. The economy (our name for money's ecology), will always, to some degree, be out of control as long as the hosts are relatively free agents. We can garden (i.e. set up nice environments for money to replicate), but direct control is probably a pipe dream). Moreover, money replication isn't free. It takes real environmental resources to create and is therefore limited. Expanding the garden forever isn't an option. Sustaining a nice one probably is.

It is possible for the economy to grow, even with hard-limited resources, for as long as technology continues to grow. New tech allows greater effeciencies to develop (think of computing power per kg of silicon, for example) which allows greater economic value to be derived from a given amount of resources. More value or use from a given input of natural resources or labour tends to decrease the 'real' price of those resources, because in a way they become less 'scarce'. A 'bigger economy' can mean 'cheaper stuff' as well as or instead of 'more stuff'.

Re:People don't matter. People are just a host.

[mejogid]

Your analogy does nothing but complicate financial markets. There's also no real consistency in your argument - you seem to be using vague and increasingly baseless environmental analogies that have no real connection to each other. Your last two sentences in particular also seem to have no real relevance to the earth as a system, economic or environmental, due to the input of the sun's energy and the potential for exploitation of resources within and external to the earth.

Re:People don't matter. People are just a host.

[gestalt_n_pepper]

Sadly, I can claim that it has as much predictive power as any other economic theory.

And of course it's just a theory - untested. It is, to some degree testable in silica, more so than many of the current models.

Re:People don't matter. People are just a host.

[FireIron]

Clearly, the money swarm has rejected me as a host carrier...

Re:People don't matter. People are just a host.

[gestalt_n_pepper]

I too seem to be born with a natural repellent.

Re:People don't matter. People are just a host.

[countertrolling]

As long as the roots are not severed, all is well. And all will be well in the garden.

In the garden.

Yes. In the garden, growth has it seasons. First comes spring and summer, but then we have fall and winter. And then we get spring and summer again.

Spring and summer.

Yes.

Then fall and winter.

Yes....

Life.. is a state of mind.

Re:People don't matter. People are just a host.

[hitmark]

why do i find that the only real measure of cost is the human work hours involved in the production of something?

and if thats true, and higher efficiency means augmenting or replacing those human work hours with machine work hours, where would the money come from to buy those finished products?

Viagra in Canada

[Comboman]

apparently viagra is much cheaper in canada than the u.s.

That's a function of supply and demand. We virile Canadian men don't need Viagra, so that drives down the price.

Re:Viagra in Canada

[sgtrock]

Funny, I always thought it had to do with the number of splinters you Canucks pick up from knotholes... I've always admired your high tolerance for pain, even if it is LaBatts induced. ;)

Re:Viagra in Canada

[DeadDecoy]

Or, because of the large demand, the marginal costs of producing extra pills approaches 0, thereby allowing them to sell pills at a cheaper price and maintain their profit margins.

Re:Viagra in Canada

[Krahar]

Sorry man, clearly the stupendous demand from impotent Canadians is driving down the price through extremely-large-scale efficiencies in sales and production.

Re:Viagra in Canada

[quarterbuck]

They are all manufactured in the same place, so efficiency would flow across the borders.Apparently impotency does not.
Almost half of Cialis sales are in USA and that does not include all the "discount viagra online" sales which are recorded in Canada, but are then shipped to US. http://www.insideindianabusiness.com/newsitem.asp?id=16591

Re:Viagra in Canada

[R3coiler]

Of the email advertisements I get for Viagra, those with Canadian URLs do seems to be a bit cheaper.

Re:Viagra in Canada

[Neoprofin]

Only half? The US has 10x the population of Canada, and is the third most populous nation in the world, and the only one of the three with something constituting disposable income.

Well, if you look at the graph on the front page

[SmallFurryCreature]

Well, if you look at the graph on the front page it becomes bloody obvious. The real data goes up and down, but does so over longer periods, it jitters but there is a direction to its movement overall. While the fake one goes up and down far to randomly, it simply does not look like a stock market result.

So yes, within this simple example, I could tell just because I know from years of news exposure what a financial graph tends to look like.

But to be honest, I am not sure it means anything. They could just have made the random sample to random. The real turing test is about actually putting some effort in the fake system to make people believe it is real. By this test your would have a turing test program that just reads random words from a dictionary and then ask people if they can spot the difference.

financial advice?

[bth]

IMHO, a more interesting financial turing test would be to distinguish between human and computer-generated financial advice.

Re:financial advice?

[Anonymous Coward]

I was terribly disappointed by the above article, as I was hoping it would provide a way to determine if Wall Street denizens were actually human, or some sort of highly evolved parasite.

Re:financial advice?

[hitmark]

there is a difference?

Poker

[The Abused Developer]

this is what it means; it means that all the crap on the Wall Street is nothing else but genuine *Poker*; and, as the Poker, the *Bank* drives the game and cheats as it wishes; you, the f&%^#$er on the main street just have to play for giving them the money. did someone expected something else?! not me,

Re:Poker

[colonelquesadilla]

You might mean blackjack. Poker is a zero sum game, some casinos take a rake or charge admission, sell drinks, or otherwise make their money, but the house doesn't generally even play poker, only provides a dealer and a table.

Much more interesting question

[Anonymous Coward]

A much more interesting question: Is the financial system turing capable? If yes, one could use the financial system to compute stuff.

The moral question is thus proposed;

[Maintenance Goof]

If I cannot predict a market, do I then, no longer have the right to kill the market? Does this mean that Wall Street is not only by results and facts, but by definition evil?

Re:The moral question is thus proposed;

[u38cg]

I don't understand your premise, argument, or concluding question. Help.

Re:The moral question is thus proposed;

[Maintenance Goof]

Oh, Goody. Thank you, Mad Gods of Slashdot, for granting me the opportunity to pedantically explain a weak joke. Alan Turing proposed a simple test to judge intelligence. If a device could fool someone into thinking it was intelligent, then it might be intelligent. Along with this test, was the ethical question of how we should treat an intelligent or potentially intelligent device. If a device is intelligent, is unplugging it morally questionable? If we grant that a market is intelligent, this may also imply that a market is capable of being evil. It also may imply that destroying a market is evil. Since markets tend to feed on and even on occasion destroy or assimilate other markets, a market being intelligent might well define a market as not only being evil in effect, but as being personally evil.

"Anybody can take the test"?!?

[Locke2005]

You mean, "anybody could take the test, before the server got slashdotted!"

What do you mean pi?

[ElMiguel]

What exactly do you mean by pi not being "predictable"? Pi can be calculated algorithmically to any desired precision, nothing to "predict" there. You can even calculate arbitrary digits without having to calculate the preceding ones. Random means precisely "not predictable". It seems some people here are equating not following a uniform distribution with not being random, which is incorrect.

Re:What do you mean pi?

[Mister Whirly]

I believe he may have been referring to the film Pi, and not the number.

Re:What do you mean pi?

[OeLeWaPpErKe]

Actually you can play games with pi's digits that would be rather hard. Say I'd give you 5 consecutive digits and ask you for the position in pi. Since there are infinite solutions to this question, it's not actually predictable (chance of guessing correct would approach 0 rather fast). Or I could give you 5 digits from pi (or any other number) and ask you to give the next number in the sequence. Again, this next number is totally not random, but not predictable in any way either.

Not random, not predictable. Lots of questions about pi are like that.

But this is not what is indicated in markets. Markets are unpredictable due to a chaotic component in their makeup : humans. Only if you were to predict the actions and thoughts of every participating human precisely over long time periods would you be able to predict markets. Presuming that the markets are influenced by real-world events, you'd also have to predict the real world. "Will Obama get reelected ?" is a question to which any serious market prediction system would have to know the answer, because it matters a lot. Same goes for "Will the football season of 2011 be more or less interesting than 2010", because these questions make a large difference.

It's like the weather. The weather (and climate for that matter (second paragraph)), in mathematical terms, consists of a very large collection of mostly random effects. Due to the fact that effects grow over time until they dissipate, but that takes time, you have some amount of predictability in the short term (although sometimes such an effect can have an extreme short-term effect. There are places in the pacific which go from sunny and calm seas to hurricane in about 20 minutes, sometimes right on top of a ship). So in the short term weather "averages out" the different effects (meaning if you see a strong cloud front anywhere, it will start dissipating. If you see any kind of clearly defined features anywhere they will get "blurred" in the short term). But in even the middle term, never mind the long term, new effects will soon dominate whatever you're seeing at any particular time (new cloud fronts, new wind directions, obstacles in the movement of air, unexpected heat sources on the ground, or just the opposite, very cold layers of water that just appear out of nowhere). Since those new effects are the result of idiotically small events (the proverbial "butterfly flap"), the only way to predict weather patterns long term is to track every last human, every last butterfly, and so on. Obviously this is not just impractical, but impossible. So you could say that to even know what the weather (or temperature, or ...) is at any given time, you'd have to be God. If you're not omniscient, you only see a small, averaged and smeared out picture of the weather, no matter how precise the instruments you're using. To predict the weather (or climate) with any reasonable amount of certainty, you'd need a simulator that could simulate the entire universe, faster than the universe works. Generally, mathematicians joke that they'd simply use such a simulator to guess tomorrow's lotto numbers and retire to a pacific island, but the point of the joke is that any program that is capable of predicting any real-life chaotic system, such as climate (or even the path of the planets, which is in the long term nowhere near as constant as they seem), has to have the ability to calculate next week's lotto numbers.

The problem is that tiny, seemingly absurdly unimportant variations today make a large difference tomorrow. Another illustration might be that wether you park your car in front of the house or behind it will generate a difference of 5 degrees celcius in the average worldwide temperature in 10 years. On the other hand huge, seemingly important things like the energy absorption rate of the ocean hardly make any difference at all (because whatever effect they have, no matter ho

Re:What do you mean pi?

[ElMiguel]

You're saying that because any given 5 consecutive digits appear infinite times among pi's expansion, pi is unpredictable? That is, multiple solutions = unpredictable? Sorry, but that doesn't make sense. By that definition, 1/9 = 1.11111... is unpredictable too, since if I give you the sequence "11111" and ask you for the position I took it from, you can't know the answer either! That question is equivalent to "guess the number I'm thinking of"; hardly a mathematical experiment.

My take is that predicting is nothing more that arriving at an answer by making calculations. Predictable = deterministic = non-random.

Re:What do you mean pi?

[BradleyUffner]

the only way to predict it is to calculate it from the beginning. You couldn't say, give me the next number following 5873465097. I could give you a run of any length from any point within the digit stream of pi and there is no way you could tell me the next number (short of me starting from the first digit).

Re:What do you mean pi?

[ElMiguel]

Follow the Wikipedia link in the comment you're responding to.

Re:What do you mean pi?

[BradleyUffner]

well holy crap. I had never heard that before. that's kinda awesome.

Re:What do you mean pi?

[Anonymous Coward]

Actually, there already is a definition for a chaotic system such as weather. Paraphrased, a dynamic system is chaotic when it is highly sensitive to initial conditions.

Re:What do you mean pi?

[Anonymous Coward]

Say I'd give you 5 consecutive digits and ask you for the position in pi. Since there are infinite solutions to this question, it's not actually predictable (chance of guessing correct would approach 0 rather fast). Or I could give you 5 digits from pi (or any other number) and ask you to give the next number in the sequence. Again, this next number is totally not random, but not predictable in any way either.

Are you saying it's impossible to find the position of a given string of digit in the decimal representation of pi? If I understand what you mean correctly, you're wrong. Firstly, the fact that pi has no finite decimal representation does'nt mean that any given substring can be found in infinitely many positions. For example, 10/3 has no finite decimal representation either, but you'll have a hard time finding anything else than a 3 in its digits. Whether or not pi is what we call in french an "universe number" (not sure of the translation), i.e. a number whose decimal representation contains any substring of any length, is currently an open question (although we're pretty sure it is). Secondly, even if pi is an "universe number", the fact that there are infinitely many positions a given substring can be found at doesn't mean the position of a substring is unpredictable. It's only ill defined. Mathematically speaking, you can't "construct" randomness from something which is purely deterministic. In probability theory, the theory in which "randomness" has a mathematical sense, you first need to define a probability space, and only then can you define randomness (which is just a way of saying that you don't know which "state" the space will be in).

Re:What do you mean pi?

[hazem]

I think what he means by "Pi is not predictable" is that not already knowing you're dealing with Pi, given a sequence of digits from the expansion of Pi, you could not easily predict the next digit.

For example, given the following: 7,8,4,8,8,9,1,0,1,5,9,8,6,0,3,0,9

without knowing that these are digits of pi in advance, you'd have trouble being able to predict that the next digit is 5.

Re:What do you mean pi?

[treeves]

but the same sequence of digits must occur somewhere in the decimal expansion of e, and other constants, but you couldn't tell if that sequence was "random" or not. In fact, pi and e should *also* contain 7,8,4,8,8,9,1,0,1,5,9,8,6,0,3,0,9,4 and 7,8,4,8,8,9,1,0,1,5,9,8,6,0,3,0,9,6 somewhere so it's not a matter of *predicting* the next number, is it?

Re:What do you mean pi?

[OeLeWaPpErKe]

Then why don't you answer the question. These are digits of pi ... which is the next digit ?

46457

You say it's predictable, great. Predict it.

I do think we all agree it's not random.

Re:What do you mean pi?

[OeLeWaPpErKe]

I'm saying it's impossible, given 5 digits of pi, to find the postition that particular sequence comes from.

I'm saying the question asked in the parent post cannot be correctly answered, even though it's a perfectly well defined sequence.

Re:What do you mean pi?

[ElMiguel]

The next digit is whichever you want. All of 464570, 464571, ..., 464579 are expected to appear infinite times in pi's decimal representation; each one of them is expected to appear once every million digits approximately.

Furthermore, "predictable" obviously doesn't mean "computable based on whichever information you choose to provide". You can compute the next digit of pi given its position, so it's predictable.

If you're not convinced, let me turn the table. Give me an example of something that you consider "predictable" and let's see how it's different from pi's decimal representation.

Re:What do you mean pi?

[Neoncow]

The next digit is 4.

http://www.google.ca/search?q=find+substring+in+digits+of+p

The string 46457 occurs at position 205,231 counting from the first digit after the decimal point. The 3. is not counted.

this query took 0.001749 seconds

The string and surrounding digits:

09722072920441600174 46457 44857898852191332549

this query took 0.001749 seconds

http://www.angio.net/pi/piquery

Re:What do you mean pi?

[Anonymous Coward]

Unless you specify where those digits lie, the question is ambiguous.

Really, you aren't anywhere near as clever as you think you are. Just fuck off.

Re:What do you mean pi?

[JoelKatz]

Even if you did know that these were digits of pi, you still couldn't know the next digit was 5. To know that the next digit was 5, you'd have to know that the sequence "7,8,4,8,8,9,1,0,1,5,9,8,6,0,3,0,9,3" doesn't appear anywhere in the digits of Pi (or else the next number could be 3), and there is no conceivable way you could know that.

Bad example

[Locke2005]

Generic metformin in is available in the US for $4 for 60 850mg tablets, or less that 7 cents per pill. You example is wrong, but your general point is not -- it is widely known that American pharmaceticals are sold for far less in other countries, even in Canada and Mexico. Basic economics say they should be more expensive elsewhere, due to transportation costs. And of course, the pharma companies due their best to make reimportation of drugs back into the US unlawful.

Re:Bad example

[Maxo-Texas]

Fair enough-- I pay $10 for 60 500mg tablets as a generic and miscalculated too (what is it? -- about 20 cents a pill I guess).

I'm not arguing it would be more expensive elsewhere, only that with such gross imbalances in price, in a rational market, people would arbitrage the price.
Our market is not rational

Re:Bad example

[hitmark]

individual humans may be rational. Herds are anything but...

ZOMG!

[coaxial]

Markets aren't rational?
People (especially people that think of themselves as being "smart") are prone to self-delusion?

But seriously, why would anyone hold on to the myth that markets are rational given the experimental findings of behavioral economics.

Oh that's right, it's the new religion to to keep the plebes down. Now excuse me, I have to cash my 30 million dollar bonus check for going bankrupt, because it's so hard to find such well qualified experts brain trust like me.

Re:ZOMG!

[myowntrueself]

But seriously, why would anyone hold on to the myth that markets are rational given the experimental findings of behavioral economics.

Why would anyone believe that human beings are rational agents given everyday experience?

Both internal and external experience should be enough to inform anyone who thinks about it that neither they nor the people around them are rational beings.

If one wished to drive the point home even more heavily, a visit to any number of internet chatrooms should present very clear evidence to wipe away any notion of humans as rational.

I think the only 'reason' anyone would hold the belief that humans are rational agents is, in the end, hubris.

Not predictable?

[Colin Smith]

Yeah, lots of noise at the daily level, but beyond that the signals emerge.

e.g.

ftse 100
http://uk.finance.yahoo.com/q/bc?s=^FTSE&t=my&l=on&z=m&q=l&c=

dow jones
http://uk.finance.yahoo.com/q/bc?s=^DJI&t=my&l=on&z=m&q=l&c=

The markets are powered primarily by inflation (forget CPI figures, they're heavily manipulated to look good, look at credit creation). August 15th 1971, the fundamental nature of money changed, debt became money, debt pays interest. Expansion in credit loaned into existence (by banks) is followed by collapse because of the interest. You should also take a look at interest rates over the period (couldn't find an online chart).

We've been living on bubbles for the last 30 years (there are many smaller credit bubbles in the charts before that), and will continue to do so until the money men lose their influence with the state... It's been 300 years in the UK so far.

Re:Not predictable?

[FooAtWFU]

Here's the Wall Street Journal correcting the Dow for inflation: Adjusted for Inflation, Dow's Gains Are Puny. Disheartening, but they note:

All of this might be enough to put investors off stocks entirely, until they consider the long-term alternatives. Measured over the 1978-2008 period, rather than over just one decade, stock performance in real-real terms actually is better than that of just about any other major investment class, Mr. Thornburg found: 4.5% a year. Stocks' ability to keep up with inflation over the very long haul may be their best selling point.

This makes sense, after all. If making boatloads of money on the stock market were so easy, you'd expect people to bid up the price of stocks until it wasn't much easier than making money investing anywhere else.

Re:Not predictable?

[Colin Smith]

Adjusted for Inflation, Dow's Gains Are Puny.

This makes sense. The increases in stocks (and commodities) are inflation so will neither massively outperform nor underperform it. Their apparent increases in value are the increases caused by the expansion of the money supply, and frankly the expansion of the money supply is predictable. Politicians, and bankers always do what's expedient for themselves.
 

Benford's Law

[Efreet]

I bet one of the main things they use in this is Benford's Law, which says that numbers beginning with small first digits are more likely to appear in logarithmically distributed data (like most financial data) than numbers with large first digits.

Random does not neet to be a random walk

[yooy]

Dude, it is the year 2010. Everybody knows by now that financial data does not follow a random walk (coin tossing). Stock market variations hat a "fat tail". Unfortunately it is hard to put thins into option pricing (problems with variance). This is actually a reason why far out of the money options are likely to be underpriced. I think Mandelbrot came up with this decades ago. Welcome to the real world. I heard more interesting things. Like the Peruvian who did not clone sheep but bacteria. Yea, that's right.

Re:Random does not neet to be a random walk

[RJHelms]

I think Mandelbrot came up with this decades ago.

Yeah, but most economists never read Mandelbrot. His economic research is very interesting but pretty much unheard of.

Re:Random does not neet to be a random walk

[u38cg]

Interesting, but not fundamental. The discovery of the nature of distributions in many financial transactions was insightful, but of limited practical application, as many quantities of importance in financial mathematics are normally distributed by the central limit theorem.

Re:Random does not neet to be a random walk

[NoOneInParticular]

Must be an economist speaking here. Two sentences, three errors.

Interesting, but not fundamental.

It is pretty fundamental that the nature of returns are emperically non-Gaussian. In a real science this observation should have killed all theory depending on assumptions of normality. Not so in economics. In economics Nobel prizes are awarded to guys that introduce models that critically depend on normality assumptions (Markowitz, Morton-Scholes, the GARCH guy).

The discovery of the nature of distributions in many financial transactions was insightful, but of limited practical application, ...

And introducing completely wrong models is practical? The problem with the assumption of normality is that it assumes that in the long run, the average (the return of an investment) depends on the most likely events. What Mandelbrot showed is that the return is dominated by the low probability events, the drops or rises in stock prices that are (according to Gaussian economists) too improbable to happen in the lifetime of the universe.

It is not that these Gaussian models are "wrong" in the usual sense, in that they are approximations. They are wrong because they make flawed assumptions. They give false security by giving the events that determine the outcome a zero probability of happening.

... as many quantities of importance in financial mathematics are normally distributed by the central limit theorem.

No, they're not. Really. That's what Mandelbrot showed. That is the fundamental bit. That economical theorists put fingers in their ears and have screamed "lalalala" for four decades does not make the Central Limit Theorem applicable to the economy.

Technically: the central limit theorem only applies when variance can be stably measured (is finite). When this is not the case, taking the limit (averaging) produces an stable-alpha-symmetric distribution (for which the Gaussian is just a special case). That is what's important. In financial math, you cannot simply assume that the Central Limit Theorem holds, you have to show that your data has finite variance first. If it has not, you will have to forego the power of Gaussian math (means, squared deviations and the like), and go with stable math (medians, sum of absolute deviations) instead. That's much harder.

Mandelbrot has emperically shown that the Central Limit Theorem is not applicable to cotton prices. After him many, many other scientists have shown that many financial instruments are non-Gaussian. This includes stock prices, futures, options, portfolios of these, etc, etc, etc. All this is ignored by Economical Theory that still is founded on the flawed assumptions of normality. It is like doing physics by building up on the four basic elements of Water, Fire, Earth and Wind. We know it's bogus, but it's a comforting story, so we keep it.

Completely random?

[RJHelms]

Various economists argue that the efficiency of a market ought to be clearly evident in the returns it produces. They say that the more efficient it is, the more random its returns will be and a perfect market should be completely random.

I'd really like to see a citation for this. I've studied a fair deal of economics in my day, and I don't remember anything even like a claim that a perfect market is completely random. Maybe I just studied the wrong (or maybe right, in this case) economics, but I can't think of any theoretical foundation for that.

If anyone can point me in the direction of real research on this, I'm very interested.

Re:Completely random?

[u38cg]

The point isn't that the market instruments (stocks) are unpredictable - the returns are simply proportional to their risk weighting and the risk-free rate. However, the returns of a shareholder in a perfectly efficient market are effectively random, since no-one can have better information or make better use of it than any other. Which raises the age-old question, is Warren Buffet actually a genius or is he just the one lucky guy who hit the extreme right of the long tail?

Re:Completely random?

[Anonymous Coward]

You know, there are a few people who have completely cracked the derivatives markets and get it right nearly all the time. It is possible to predict any instrument, sometimes months in advance. I have never met Warren Buffet but am quite certain his success is the product of market insight, hard work, and dedication - experience earned over decades.

Unfortunately one of the by products of success in this business is that the hoi polloi, who make judgements based on luck - see lotteries and other nonsenses - dismiss true success as the result of luck, rather than hard work, as a means of excusing their own mediocrity.

Such is the curse of modernity. You have revealed the depths of your own knowledge, in public, for all to see. Well done.

Re:Completely random?

[u38cg]

Yes, but given that there are large numbers of people involved in the derivatives markets, we would expect to see a small number of people to get it right all the time, not by expertise, but by dumb, dumb luck. How do we tell the difference? I would expect the stupid not to succeed, but is Buffet really substantially smarter than almost all other participants? I find it hard to believe.

By the way, it is possible to construct derivatives which have behaviour which cannot be empirically modelled, but can be controlled by the creator. Essentially, you can a href="http://www.cs.princeton.edu/~rongge/derivative.pdf">booby-trap derivatives. Chew that, Warren.

technical analysis

[Anonymous Coward]

looking for technical patterns, trends, and lines of support and resistance, i was much less accurate than i thought i would be. but the timer certainly didn't help. if i had more time to look at the charts, i think i would be better able to tell which chart ignored the "rules" of technical analysis more often. perhaps? it would be interesting to see how they generated their "random" charts. i'm starting to doubt my ability to read charts now.

Anonymous Coward

[Anonymous Coward]

I think that perhaps they need to develop something related to the Turing test for politicians' interviews. It sometimes really does just seem like they've memorized a list of vetted, safe responses from their staff and reply with whatever one has the most buzzwords in common with the interviewer's question. It often sounds a bit like a computer program looking up stock answers to things might respond...with a lack of finding anything in the database returning something like "I'll have to refer to my staff on that". I'm not that familiar with the theory behind the testing in detail but I wonder if there's a way to analyze conversations that have already taken place (e.g. news interviews) for a "Turingish" rating of sorts. Yeah, I said "Turingish"...

Economics ...p=r-c

[Anonymous Coward]

The most basic economic formula is profit equals revenue minus cost (P = R - C). Like you, many people ignore the cost, C, when analyzing the market.

China & USA have different costs, because they are separated by the Pacific Ocean. It costs something to ship a movie, e.g. DVD, from one place to the other. If these DVDs are made in China, then the Chinese locale has much lower costs. You can't just teleport those DVDs to North America for free and sell them.

Drugs, i.e. medicines, are highly regulated for efficacy & safety. The US FDA tests drugs and passes that cost onto the manufacturer, who passes it onto the patient (consumer). Once the Chinese & Indian consumers get the off-patent versions, years of data show the drugs to be safe, which lowers insurance costs to the manufacturer.

Viagra is a recreational drug, not a medically necessary one. If more people restricted their drug use to medical necessity, then drug price would go down. Demand for drugs & their prices are too high.

The crack-addict consumer is the cause of your INefficient market, not the suppliers.

Re:Economics ...p=r-c

[Maxo-Texas]

Do you think it would cost $17.00 to import dvd's into the U.S?

How many DVD's could one person buy for $2.99 and carry back and sell for 19.99 if there were not laws prohibiting them from doing so.

The difference in price is a result of laws which support an artificial market. The real cost to make the DVD is under $2.99 (most likely under a quarter). We pay more because the market isn't efficient because laws were passed to make it illegal to bypass artificial barriers.

The same is true for drugs. That .10 pill is sold for .10 at a profit in india and china. In all likelyhood, even if your importation theory is correct, it could be sold at a profit for much less than $1.00 in the U.S. The only reason it is $5.00 here is because artificial barriers block an efficient market.

Here's the problem. I don't mind shipping the jobs to india because they can get buy for $14,000 a year. I mind that I have to compete with them when they get medicine, dvd's, and many other products at the true price while I have to pay grossly inflated prices. It wouldn't matter if my income was stagnant if the price of goods was dropping by 10% per year. Which is how capitalism is *supposed* to work.

You know.. efficient market. If your prices are too high- someone will undercut them. Only now we have laws that block an efficient market. So I get to see jobs shipped overseas because I cost too much AND pay more for the same items.

I figure it lasts another 4-8 years personally. The end of this trend has already started.

Re:Economics ...p=r-c

[jonadab]

There's an additional complicating factor nobody has yet mentioned here: the difference between exchange rates and purchasing power parity. This is always a factor when you're comparing prices across international borders, and *particularly* between countries with drastically different economies, and in this case it's further complicated by the fact that the exchange rates for the Renminbi (i.e., the Chinese Yuan) do not float as naturally as those for most currencies, due to the Chinese government's as-yet incomplete adoption of free-market economic policies.

Exactly how much impact this has on relative prices of DVDs in each country could make an interesting paper, at the very least.

Re:Economics ...p=r-c

[vtcodger]

***Drugs, i.e. medicines, are highly regulated for efficacy & safety. The US FDA tests drugs and passes that cost onto the manufacturer, who passes it onto the patient (consumer). Once the Chinese & Indian consumers get the off-patent versions, years of data show the drugs to be safe, which lowers insurance costs to the manufacturer.***

Very plausible, but not actually correct I think. The FDA does not test drugs. The pharmaceutical companies test the drugs then try to persuade the FDA and it's overseas equivalents like the EMA (European Medicines Agency) to approve the drugs. Costs are higher in the US than elsewhere where national health programs are able to negotiate reduced costs whereas our enlightened conservative lawmakers who are dumber than a sack of rocks think that the free market will somehow reduce costs without a comparable negotiating organization to represent 300,000,000 American consumers.

As far as I can tell drug makers don't buy liability insurance. They self insure. But it probably doesn't much matter. If the drug maker screws up badly enough to end up in court, somebody will pay, and eventually that's probably the greatest available fool -- the American consumer.

Re:Economics ...p=r-c

[BeanThere]

Very plausible, but not actually correct I think. The FDA does not test drugs. The pharmaceutical companies test the drugs then try to persuade the FDA and it's overseas equivalents like the EMA (European Medicines Agency) to approve the drugs.

Technically yes, but the effect is the same, because the FDA requires the pharmaceutical companies to not just test the drugs, but do multiple exhaustive, comprehensive tests over several stages, that take years. And yes, that costs a lot and those costs are obviously ultimately paid by those who purchase the drugs.

p.s. if you have trouble reading it

[FooAtWFU]

If you ever get a link to the Wall Street Journal that you can't read, just plop the URL into Google and click on the link from there.

Thank you!

[XanC]

Very handy.

Financial markets are complex, not random

[Anonymous Coward]

This entire post (and the website discussed) are based on a faulty premise. Financial markets are not random systems. They are "complex systems". In neither case are they reliably predictable. For example, a person can recognize the difference between a snowflake and a random arrangement of ice crystals. That doesn't mean, however, that they can predict what shape the next snowflake will take.

nice

[Device666]

Why not run the generated numbers of that algo through the financial news and stock exchange servers. Maybe that would lead to better results than let Ben Bernanky run the dollar down by his frantic freaky evil moneyprinter.

Re:nice

[Anonymous Coward]

Run the dollar down? Where have you been since September 08?
Other countries are engaging in the same nonsense and are running their own currencies down faster than Bernanke can wreck the dollar. The obvious exception is the JPY, which is already wrecked.

Re: defining random as equally likely results

[presidenteloco]

Random defined as "any given result is as likely as any other".

If you are applying that to a sequence of values that has already been produced, that would be called a uniform distribution, not randomness.

If what you mean is "for the next value to be generated by this process: any given result is as likely as any other" then yes, that
being true would be equivalent to saying the process is random.

We have to consider the meaning of "equally likely" here. Equally likely as far as who or what is concerned?
Likelihood of an event occurring is not objective, even without taking into quantum physics into account.
It depends on how much information you have about the process that will generate the event.

If you are a maxwell's demon, and you
have been tracking each atom in the air, then you know exactly when the next atom is going to pass through the gate between two air filled chambers,
and which atom that will be. That event happening will not be random to you.
If you are anything else than that daemon (being the daemon really stands for being the process that actually is the atoms and their movement
and interactions) , i.e. if you are or have any predicting/modelling algorithm of the situation
that is less complex than the process that is moving the atoms, or even have an algorithm that is as complex as the real physical process,
but is not doing/modelling the exact same characteristics and changes as the real process is doing, then you will not be able to tell when
the next atom will go through the gate, nor which nearby atom will go through next. As far as you are concerned, any particular future time
is as likely as any other particular time, and any atom is as likely as any other to be next. You can tell me what the distribution of them
will be over a period of time, but you have zero information on the nature of that next single event that the one in question.

Re: defining random as equally likely results

[presidenteloco]

So a truly random process is one that is so complex that, without being the process itself, or an exact equivalent process that will behave precisely the same in all pertinent detail that governs its exact (not statistical) evolution) you cannot have any information that would allow you to guess with greater than chance probability the
value/characteristics of its next event, from among the possible values/configurations of the events generated by the process.

Lucky for us.

[Anonymous Coward]

""The problem is that tiny, seemingly absurdly unimportant variations today make a large difference tomorrow. Another illustration might be that wether you park your car in front of the house or behind it will generate a difference of 5 degrees Celsius in the average worldwide temperature in 10 years. On the other hand huge, seemingly important things like the energy absorption rate of the ocean hardly make any difference at all (because whatever effect they have, no matter how huge, it will get beaten by growing tiny effects). This is not a joke, or a "reduction ad absurdum" argument but the actual, literal truth (arbitrary small details have huge effects over time, while things that look very important and very big today hardly matter at all). This is a very difficult concept to grasp, and extremely unintuitive.""

Lucky for us I parked behind the house tonight. Now we don't have to worry about global warming anymore.

The precise definition of the mandelbrot set.

[jonaskoelker]

The quintessential chaotic system, mandelbrot's fractal, is just such an example. You take any imaginary number c = p(1).

The way I always heard it is you define f_c(z) = z^2 + c. Then you compose f_c with itself n times (for n = 0, 1, ...), and evaluate f_c^n at 0. So f_c^1(0) = f_c(0) = 0^2 + c = c.

Then you say p(n+1) = p(n)^2 + p(0), and you examine the value of p(infinity) for various values of c.

Eh, $\infty \not \in \mathbb{N}$. You examine the values of p(n) for all n.

This will draw the famous mandelbrot fractal.

To be exact, a point c is in the mandelbrot set if there exists a K such that for all natural numbers n the norm of f_c^n(0) is at most K.

It other words, by iterating (z -> z^2 + c) you never get further away from (0, 0) than a distance of K.

Side remark:

Say I'd give you 5 consecutive digits and ask you for the position in pi. Since there are infinite solutions to this question, it's not actually predictable

So you're saying I can't tell you what the answer is before... well, before what?

Also, since you're asking for the unique solution, your question is bogus. Why do you ask it?

Re:Well, if you look at the graph on the front pag

[DavidShor]

Yeah, it'd be interesting to see if people can distinguish stock data from something from a full stop CAPM-like model with all the modern bells and whistles put in.

Re: defining random as equally likely results

[jonadab]

> Random defined as "any given result is as likely as any other".
> If you are applying that to a sequence of values that has
> already been produced, that would be called a uniform
> distribution, not randomness.

On the contrary, uniformity is an additional constraint on the distribution that makes some sequences artificially more or less likely than others. If a given distribution is guaranteed to be uniform, then it is NOT random, in the statistical/mathematical sense. (However, a uniform distribution is likely to be described as "random" by normal people who have never had a statistics class.)

> Likelihood of an event occurring is not objective, even
> without taking into quantum physics into account.
> It depends on how much information you have about the
> process that will generate the event.

In a nutshell, yes. Well, mostly yes, for practical purposes.

Strictly speaking, if somebody has information that can be used to predict the outcome, then ipso facto the process is deterministic and thus by definition not random. Incidentally, some theologians (in particular, a pretty good percentage of Calvinists) hold that absolutely everything is 100% deterministic right down to the particle level, including human behavior and everything; if they're right, NOTHING is strictly random in the mathematical sense.

Which is why cryptographers talk about "cryptographically secure pseudorandom" number generation, which generally involves collecting entropy from some source that an attacker presumably doesn't have access to and can't predict, and using that to reseed a PRNG algorithm with sufficient frequency that no pattern could become detectable. This kind of number generation is NOT random in the mathematical sense, but it's good enough for many purposes because it's very difficult to predict without access to information that the attacker doesn't have. On the other hand, *you* do have the information, and the calculation could be duplicated, if you had any reason to do so, because it's totally deterministic. Because it's deterministic, cryptographers consistently avoid calling it "random". They use the word "pseudorandom" instead.

It's interesting that you brought up quantum physics, because particle-level events, such as radioactive decay, are widely considered to be the best available source of entropy, since even the top particle-physics experts haven't got the slightest idea how to predict them in detail, and so presumably neither does the guy trying to break into your server or read your encrypted email or whatever.

So yeah, if no outcome can be reliably *determined* to be more likely than any other outcome, given the information available to the relevant parties, then it just about may as well be random, for practical purposes. And that's actually fairly similar to what you said, more or less. (Except for that first part about "uniform". I disagree entirely with that part.)

Re: defining random as equally likely results

[presidenteloco]

Regardless of what else you may think of the book, there's
an interesting discussion of the relationship of randomness and predictability
and computational complexity in "A New Kind of Science" by Stephen Wolfram.