Slashdot Mirror
Mandelbrot Suggests A Hunt For Financial Patterns
Phoe6 writes "Wired is carrying an Open Letter of Benoît Mandelbrot, the father of the fractal, to the wizards of Wall Street, calling on them to recognize a pattern in the finantial and economic trends in the world. Mandelbrot says, If we can map the human genome, why can't we map how a man loses his livelihood? If millions can contribute a few cycles of their PCs to the search for a signal from outer space, why can't they join a coordinated search for patterns in financial markets?" I'd like to see a debate between Mandelbrot and Friedrich Hayek.
Are you familiar with Elliot cycles? Probably not. He came up with an idea like this around a decade ago. (Reading back issues of NewScientist can do wonders for you knowledge like this.) These ideas keep getting thrown around but never come to fruition because at their core they are inaccurate. As simple as that. Whether it's based on power laws, or assumptions about the nature of price spikes (up-up-down-up-up-down), trying to reduce markets to mathematical patterns invariably fails.
Exactly. The very nature of stock trading is that no money is created, it's only moved.
People that bought yahoo cheap and watched it become 100x as valuable and sold $100 of stock for $10000 didn't create money or value, all they did was tricked other people out of their money. That's why the stockmarket is a poor representation of economy today, the valuation of the stock isn't very closely tied to the long term dividends companies pay out.
In short, I agree with you. If everyone knew everything about the stock market then everyone would end up leaving the market because there'd be a distinct lack of rubes begging to give away their money for practically worthless pieces of stock paper which were about to take dives. (which is what happened to all the people that bought yahoo at the climax of its value)
This concept was explored in an Aussie movie of a couple of years back, called The Bank. A person previously wronged by a bank was employed to investigate stock market trends, using 'chaos theory' and 'fractal geometry'. Quite an interesting movie to watch.
I don't think he understands that financial and economic markets are linked to world events. Therefore, in order to accurately predict the movement of financial markets, you will need to be able to predict the future! Do you think that a computer could have predicted 9/11 or the Iraq war or elections of world leaders or the Microsoft settlement or a myriad of other news events which effect the direction of the markets? If anyone actually believes that we will be able to design a computer to do this, feel free to reply to me because I have a bridge to sell you!
(Before all you chart-heads jump on me, I do not think technical analysis or charting has any validity, so please do not waste any time trying to convince me otherwise, because it won't work!)
------
www.moneybythenumbers.com
You can't just map all the data in the stock market and look for a pattern. The amount of socio-financial data to pattern would be... incomprehensible. This is truly a "butterfly effect" situation. Millions of rumors swirl throughout the trading floor - some make it to us as news - and mapping all that data... it's not that it is impossible, but conceiving of that today is unlikely.
You'd almost need some kind of impartial Bayseian analysis to traffic through millions of petabytes of data. There are patterns surely, just like the weather. Someday, they will be just as apparent.
IMHO, it is not possible to predict the actions of millions of users (not to speak the 1000s of program trading systems) over a long period of time. However, given a sufficiently small window, it may be possible to predict the motion of a security with a better than random probability; and if you have a direct link to trading systems (i.e. low fees), you might be able eke out a meaningful return on investment.
As with most other things, you'd need a hefty investment to pull this off.
Mandelbrot is studying price fluctuations, whereas you are talking about equilibrium models. An equilibrium model can typically predict how much the value of a given stock will change a given period of time. It will not return a probability distribution, but rather an average value.
A model of the fluctuations can return a probability distrubution of outcomes. The variance of this distribution will not affect prices,
because investors are only interested in the mean value. Even if the variance is large there will always be an investor who is big enough to wish to take the risk.
Mandelbrot argues that models of fluctuations are important because they can predict the chances of total (economical) disaster. Largescale fluctuations in economy typically follow a power law distribution. This means that we can use the analysis of small fluctuations to predict the changes of large scale fluctuations. In other words we can predict the changes of an upcoming wallstreet panic.
That financial markets are stochastic is an assumption - with extreme levels of 'deterministic noise' it can appear stochastic - a small difference when specifying an equation to predict a future level can make a massive difference: this is chaotic. Stochastic modelling works OK because of the large levels of noise, most modern finance is built on the assumption errors are Gaussian, a framework formalised in the 60s but assumed even earlier. Sadly today anyone can trot out a Black Scholes thesis, support the group think in academia, and get a PhD from a decent institution (a disease in many disciplines other than economics/finance).
Think of financial markets like the weather - tomorrow is more likely to be the same as today, some minor variance, but saying it is going to be largely similar will mean you're right most of the time: this doesn't help pedict a storm. Likewise tomorrow the stockmarkets will likely be the same as today, but this doesn't help predict a crash. As weather the stock market model can be refined, but in the end it is chaotic and hugely deterministic (many agents looking at each other and others actions).
Many agents looking at each other's actions is important - the market does not exist without agents (buyers/sellers) - it is an endogenous process. A co-operative solution will not work - someone always has to do worse for anyone to do better than the market - the market and the economy is just the sum of the actions of participants, participants cannot move against the market as they, in sum, are the market. Calling on the market to recognise a pattern is folly - a small participant can take advantage of any pattern if discovered, but the market as a whole cannot because if they stop their present action to follow/takeadvantage of/neutralise the pattern they have stopped taking action that creates the pattern.
Patterns in financial markets have long been looked into, a good starting point on current thought, if interested, is technical analysis and elliott-wave theory
Can we, I rather doubt it... You only can predict some already occurred situations and patterns.
But that does not mean we already have encountered every possible (well probably there are infinite) situations to a crash.
The problem is, that many of these patterns only show one aspect of such a problem, the numbers, but forget about the root. Sociological factors. An overheated stock market does not necessarily need to lead to a total 1929 like crash, as does high stocks have to cause a crash (in most cases they do but not in all)
Sorry this is the search for the holy grail all over again. What will come out of it is simulation patterns for certain kind of situations which can give you a prewarning system. (But probably will be exploitet by a few to become richer and thus the patterns will be rendered useless again)
All I can see here is that in the end you will get as a result a probability based system again which even is questionable.
There is a lot of scientists who think that because they can create nuclear bombs or map the human genome they can forecast financial markets. They are all wrong, just like Mandelbrot
There is a nobel price winning theory in economics from Robert Lucas who says that because people react to forecasts, the forecasts become invalid. And that is where all these physisists get it wrong, they are not dealing with non-thinking dynamics.
I find all those people amusing, they never seem to make much money, exept by consulting. but ultimately this is a waste of brainpower.
You describe a model that is much more complicated than the models Mandelbrot are looking for. . There is no big science in these fluctuation models. The only trick is to fit all probability distributions to a power law.
Mandelbrot wasn't the first to model financial markets using fractals. Elliott was.
You'd get the name of God.
I've just read through the comments and it's hard to believe that no-one has mentioned Pi. The film is about a mathematician who is working on exactly this - spotting trends in the stock market by describing patterns in Chaos. It's a strange intense film and in addition to being pursued by shady stock market brokers, he's hounded by a jewish Kabbalah sect who believe he might be working out the true name of God.
Top of the list of 'Films to be Watched On Acid."
Aide-toi, le Ciel t'aidera - Jeanne D'Arc.
I gathered data on daily stock market returns on 5000 companies listed in Standard and Poor's listing of U.S. publicly traded companies.
I normalized then crunched the data through a fractal analysis tool that quantified the level of chaos (randomness) in the changes of each company's stock market value from one day to the next.
I understand the professors studied the data to determine any correlation between each company's chaos metric and the company's eventual bankruptcy.
Now, IANAM* and I have never read any of the resulting research papers, so I cannot tell you many details of the professors' findings.
However, I understand that the hypothesis was that changes in market value characterized by a high amount of chaos (randomness) would correlate to a robust, or healthy business model, just as the life sciences have found that a high degree of chaos correlates with a healthy system. As business managers' actions become constrained by the costs of bankruptcy, so the theory goes, the daily variation in stock market returns become less and less random.
I do recall that they did find a correlation between eventual bankruptcy and suppressed chaos. However, I seem to recall they also found a correlation between a successful turnaround and suppressed chaos.
I guess (IIRC) you could say that suppressed levels of chaos could be a predictor of future business distress, but not necessarily of future business failure.
A list of the research supported by the fractal study follows:
Lindsay, D.H. and A. Campbell. A Fractal Approach to Bankruptcy
Prediction. Business Research Yearbook: Global Business
Perspectives, (2), 1995, 13-17.
Lindsay, D.H. and A. Campbell. The Effect of Deregulation Upon
the Chaotic Properties of Stock Market Time Series Returns.
Business Research Yearbook: Global Business Perspectives, (3),
1996, 13-17.
Lindsay, D.H. and A. Campbell. A Chaos Approach to Bankruptcy
Prediction. Journal of Applied Business Research, (12)4, Fall,
1996, 1-9.
Lindsay, D.H. and Campbell, A. The Effect of Changes in
Proportional Institutional Ownership upon the Chaotic Properties
of Stock Market Time Series Returns. Business Research Yearbook:
Global Business Perspectives, (4), 1997, 267-271.
Lindsay, D.H. and Campbell, A. Beta and the Chaotic Properties of Time Series Returns. Business Research Yearbook: Global Business Perspective, (5), 1998, 7-11.
Lindsay, D.H. and Campbell, A. Public Pension Funds: The Effect of Negative Public Announcements on Chaotic Properties of Returns Business Research Yearbook: Global Business Perspective, (6), 1999,
Lindsay, D.H. and Campbell, A. Risk and Financial Distress: A New Approach, Business Research Yearbook: Global Business Perspective, (7), 2000,
*IANAM - I am not a mathematician. (I am an accountant. Accountants don't need math. We have tables.)
I'm time traveling, right now
This is called the efficient markets hypothesis. Not everyone agrees. The idea that every opportunity is fully exploited by an infinite # of eyeballs is empirically cute, but not descriptively accurate.
Will anyone ask here, what is a pattern? THe field of behavioral finance makes its objective to show how systematic flaws can be exploited due to the particpation of human investors that exhibit irrationality, overconfidence and inconsistency in the use of information sources, and those sources are weighted. Human participants give rise to patterns that have systemic effects. But to acknowledge and exploit this is to rebuff the fundamental philosophy of the SEC and modern economic theory...that somehow everyone pursuing competition by fullfilling inidividual incentives will have a group enhancing effect. It doesn't. And thats why the people who make money will always be acting on irrational or insider information.
But overall I agree that anything remotely patternistic to a human bean will immediately be cancelled out. The markets are at least that 'efficient'.
I remember when I was in grad school for fluid dynamics at the Von Karaman Institute there was a big fad on modelling turbulence using fractals. While it is true that turbulence is more accuratley described with fractal as opposed to Euclidian geometry, this doesn't necessarily mean that useful predictive model can be produced. The many attempts at modelling turbulence with fractals didn't really produce models more effective then the usual stochastic models that were used. There may have been more progress made in the past few years, but I'm not aware of any major breakthroughs. Granted turbulence is a different problem than the financial markets, but the deterministic chaotic behavior is the simular. Also if you could model the dynamics of the stock market, it is unlikely that I'd be willing to donate CPU cycles just so a few day traders can get rich. Mark
"Technical analysis of markets is a waste of time."
When you made this comment, were you aware that people apply technical analysis to problems other than forecasting market direction?
What Mandelbrot is suggesting is not the development of a predictive model for entry and exit of positions. Rather, he's suggesting a better model for evaluating the risk in a portfolio once the positions have already been established by whatever means the investor is using. Since risk in this context refers to the risk of unfavorable trade outcomes and trade outcome most definitely *is* a function of price, ((selling price - buying price) * shares transacted), technical methods are applicable.
"When a pattern is found, it is exploited by many, which changes whatever "meaning" the pattern had before."
Your second point is considerably closer to the mark. Any sort of pattern like "if the market closes on a high and it's Thursday..." is unlikely to be of any practical use. However, this doesn't mean all technical analysis is useless. Every trader has to decide whether to be long, short or out, when to get in, when to close the trade and how many shares to transact. Totally disregarding prices while making these decisions is reckless, at best. The alternative is to take price information into account when making these decisions which is, by definition, technical analysis.
As far as "exploited by many" is concerned, it can depend a lot on the payoff structure for the pattern. A pattern that wins every time will quickly be assimilated into common practice but a pattern that is psychologically difficult to trade will remain profitable for much longer. For example all of these patterns have the common trait of having been profitable for decades despite being public knowledge for most of that time. The reason why is that they result in winning trades only about a third of the time, making them emotionally stressful to use, keeping people away. Nevertheless, they make money because the average win is far larger than the average loss.
Besides all that, even if there were a holy grail of technical analysis, most people wouldn't use it. They'd insist that it was just an illusion, so the irony is that skepticism toward technical methods would prevent that holy grail from being crowded out, causing it to last longer than it would in a truly efficient market.
The state is the great fiction by which everyone tries to live at the expense of everybody else. ~F. Bastiat
Wallstreet is already doing this, and throwing some pretty powerful brain power and computer power at the problem. Just don't expect the players to share their information with us.
I have a friend who got his PhD in mathematics a few years back. His career options were: 1) Teach (there are far fewer teaching positions than there are candidates), 2) Work for the NSA, 3) or go work for a Wallstreet firm.
He lucked out and got a teaching position at a local city college.
It's not offtopic, dumbass. It's orthogonal.