Slashdot Mirror
Mandelbrot Suggests A Hunt For Financial Patterns
Phoe6 writes "Wired is carrying an Open Letter of Benoît Mandelbrot, the father of the fractal, to the wizards of Wall Street, calling on them to recognize a pattern in the finantial and economic trends in the world. Mandelbrot says, If we can map the human genome, why can't we map how a man loses his livelihood? If millions can contribute a few cycles of their PCs to the search for a signal from outer space, why can't they join a coordinated search for patterns in financial markets?" I'd like to see a debate between Mandelbrot and Friedrich Hayek.
using either linear algorithmic models or a parrallel neural network approach.
However while most people agree that past performance is indicative of the future nothing can predict what is going to happen. Things such as politics and current events have a huge impact but are not easily factored in to a computer program.
There are many sophisticated solutions to recognising and predicting complex patterns but with the stock market there is an element of trying to predict the lottery.
If the lottery is run properly then every draw should be completely random, any pattern detected in past draws should be about a useful as picking your numbers out of a hat.
As soon as a structure usable to make money has been found, so many people exploit it that it instantaneously disappear. This is especially true when such structures are explained to the public and not kept ultra-secret in some bank basement.
The problem with the market is that the knowledge humans have about it modifies it.
Let me recap Mandelbrot's point here.
In the April 2003 settlement of postbubble fraud charges, the biggest Wall Street firms agreed to cough up $432.5 million to fund "independent" research. Mandelbrot then makes the distinction between two kinds of research. One is the kind of research where analysts study a publicly traded company, and then give recommendations to buy, sell or hold. The other kind is fundamental economic research.
Mandelbrot then suggests that at least five percent of the settlement money be directed toward fundamental research. He does not say that we should look for a way to predict the markets with absolute certainty - that would be impossible, as many here have redundantly pointed out. (He would probably be insulted to know that so many here think that's what he advocated. He's not stupid, you know.)
He's talking about giving a boost to the kind of fundamental economic research that's already taking place. Stuff like risk management, for example. If you read the article, maybe you noticed that in the beginning he clearly gave examples of what's wrong with our present models in risk management.
See "Timescape", Benford. Also, time value of money, although the article neglects the inherent disutility of delayed gratification.
I think that the very act of prediction will change the outcome... basically making this impossible to practically achieve.
"The Psychohistorians", Asimov, from "Foundation", book one of the Foundation Trilogy. I'd agree with that offhand, but it would be nice to see a proof. (For a nasty view of why, read "In the Country of the Blind" by Flynn.)
//Information does not want to be free; it wants to breed.
Let's compare forecasting weather to forecasting the stock market. Based on statitical analysis we can predict what the weather is going to do for about 1 week based largely on massive amounts of historical data. The same thing happens in the stock market (albeit at a much much smaller time interval), these companies have sophisticated databases (www.kx.com optimized for time series analysis) with 50 years worth of stock data. Based on this they can "trend" the market for possibly minutes to hours. But having a minute's or hour's advantage in the stock market is enough of an advantage to make $.
C T= 105&STORY=/www/story/12-06-1999/0001089717/ /216.239.57.104/search?q=cache:fhBwDHEPW3EJ: www.kx.com/press_releases/TTWKXsystems.pdf+lehman+ terabyte&hl=en
e.g. Lehman brothers has 50 years of bond data, you can bet that this is NOT less than 2-3 terabytes of data:
http://www.prnewswire.com/cgi-bin/stories.pl?AC
http:
But just like weather, it is next to impossible to predict random events. Who saw 9/11 happening? How many people are going to die in the next Japanese earthquake? what industries will be affected by higher oil prices? etc etc... we are still at the whims of mother nature and random happenings that the market can not control. Think about it, these "random" events are quite expensive from a cost of life standpoint and an economical standpoint. That is why the US govt dumps money into trying to predict earthquakes, hurricanes, torandoes, flooding, etc etc, because in the end (as the bastion of capitalism) it allows the US to make more money by averting risk.
Other good books to read:
"The Intelligent Investor" Ben Graham (Buffet's tutor)
"Security Analysis: The Classic 1940 Edition" Benjamin Graham, David Dodd
Just like a previous post, once methods are introduced that can "make" money in the market, these methods slowly lose their impact:
from Graham "The moral seems to be that any approach to money making in the stock market which can be easily described and follwed by a lot of people is by its terms too simple and too easy to last."
PS: though I'm a kdb/k noob, I really like the language, it is like python on steroids.
PSS: contact me off list if you want a bit of hand holding to get kdb/k up and running on your machine (the download fits on a floppy)
goanuj1 at yahoo dot com
someone always has to do worse for anyone to do better than the market - the market and the economy is just the sum of the actions of participants, participants cannot move against the market as they, in sum, are the market.
Here you have quite adequately restated the problem; However, at the rsik of attempting to contribute to a topic growth weary of overcontemplation,
Proposed:
That the solution sought is not to permit the whole to move against the market - but to increase the effeciency of the market.
The Effeciency is the amount of value created by the act of corporation relative to the costs in overhead.
In theory, if we all buy into a well, we can dig deeper and get water which as individuals in out of reach.
The cost, is that our individual contributions may be wiped out by unforseen market forces.
For example, We invest in a mutual fund - the mutual fund invests in several drilling companies, and they and 12 other companies all reach water on the same day - we get our water, but not at the theoretical savings possible under an ideal transaction.
The difference in effeciency between an ideal market, and a real market is available as a net gain to all participants if it can be better attained.
I would suggest as a solution, a blend of Short term patents, a market for command economy directives, and a method of evaluating the relative worth of proprietary ideas.
Briefly as applied:
In 1980, about a hundred companies were working on hard disk technology. Some had good ideas, which they kept to themselves. they were all working on parallel projects, which as a result, muted the effeciency of the investment generally, and actually harmed the entire sector, because the result was less than optimal return on investment.
If companies with good ideas, were able to register them for short-term secret protection, and have them evaluated by trusted confidants elected by investors to look at all the competing ideas, and render an early "Command Market Directive" in other words - decide which company has the right to continue research.
In return, the companies with inferior technology can get early warning that they should spend their investors money on research in which they have a core competancy and protect the investors from blind parallel risk.
Companies would not be "forced" to heed the ruling of the common investors group, but investors would be warned, and this would have much the same effect.
Note however, that this solution has little to do with guessing the next days numbers, but rather by opening up the secret doors of publicly traded companies, and warning investors about high rates of overlapping aims.
We laid down too much dark fiber - for much the same reason, and those companies, and the wealth they represent were lost - not because of bad decisions, but because of current spending for access to an oversaturated future market.
The point again - is that the IS room to improve the market and it has little to do with understanding chaos, and more to do with understanding why large companies continue large investments in losing propositions.
AIK