Algorithmic Investors on Wallstreet

eldavojohn writes "Recently, setting up prediction markets that people play was the big thing to guess the future. But is there a chance that computers will replace investors? From the article: 'Quantitative investment managers use a model to identify sets of characteristics for their investments. Computing power is now relatively cheap. Obviously, computing power can access data almost instantaneously and simultaneously. Asset classes and financial instruments within those asset classes can then be screened and investments are selected. They reflect the manager's views.'"

Comments

Market News Writing Computers Also

[eldavojohn]

More and more I see computers being used to harvest and cultivate data for market analysts and investors. Even Thomson has built software to deliver market news. From that link:

Thomson has built some computer programs at $150k-$200k a pop to deliver automated articles on US market news. The programs can publish a news story on, say, company financials, within 0.3 seconds of their release to the NYSE or NASDAQ. This is purportedly helpful to hedge traders and others of their ilk.

$150-$200k? Looks like there might be some profit in artificial intelligence afterall. Although I wonder if this would even be considered AI?

Re:Market News Writing Computers Also

[Stellian]

More and more I see computers being used to harvest and cultivate data for market analysts and investors. Even Thomson has built software to deliver market news.

Folks who suggest replacing human investors with computer algorithms don't understand the basic workings of the stock market. You cannot predict the next value of a stock simply using past and current information from within the stock exchange. You cannot find a `pattern` in the stock price no matter how much computing power you use: there is no pattern, except for the well know economic cycles that influence all stocks. Besides, even if an algorithm could be devised, it would be useful only if it could be kept secret, otherwise using it on a large scale will deny any speculative gain.
The price of a stock is determined by external factors, and the key into being a good investor is access to information: who sued who, what is the union planing, what product is the competition developing etc. So to replace humans with algorithms, you must make them as intelligent as humans in the basic task of finding and understanding information. AI is ages away from this stage, and when/if we will finally have such powerful AI, the stock exchange will be our last concern.

Re:Market News Writing Computers Also

[EastCoastSurfer]

Actually a lot of patterns are showing themselves in the extreme short term (think seconds here). There are so many automated trader/AI types of software exploiting already these patterns, as soon as one is found it doesn't last long since others jump in. I don't have a link handy, but I read a good article awhile back about econophysicists looking for and finding short term patterns in the market.

I also know of a company nearby doing exactly that and doing well and have an acquaintance who retired at the age 35 or so after running his companies dept. who found (using algorithms) and exploited these patterns that don't exist.

Re:Market News Writing Computers Also

[mikecheng]

The company was started by one of the guys who mathmatically beat roulette

This is equivalent to saying that he "mathematically beat the tossing of a coin" i.e. the statement makes no sense.

Re:Market News Writing Computers Also

[HUADPE]

No, they managed to measure the velocity, of the ball, the wheel, and the other factors involved in roulette, and then quickly and accurately compute the path the ball would take. Once you release a coin in a flip, if monitored carefully and calculated correctly, you certianly can predict how it will land.

Is it difficult? Yes. That's why it's impressive. It is not impossible though.

Re:Market News Writing Computers Also

[k2enemy]

Mainly because the people who do this are are super secret. They don't want anyone to know how or what they are doing because the field is so competitive. It's the equivalent of an algorithm arms race.

They are not as secretive about their methods as you might imagine.

As noted by earlier posters computers are not used to "pick stocks", but to construct portfolios with desirable characteristics, find arbitrage opportunities, etc. I can give a little insight into the first. I'll gloss over a lot and use language somewhat loosely, so please don't jump if you know your finance :)

There is a tradeoff in the market between risk and return. You can construct a portfolio with a very high expected return, but it will involve a lot of risk. Alternatively, you could have portfolio with very little risk, but low expected returns. The trick is to get the highest expected return with the lowest expected risk. Here is where mathematical models run on a computer can help. The most famous and the one everybody knows about is the CAPM (capital asset pricing model). There is a lot of debate in academia over this model, but it is still useful in practical ways.

Last year I attended a lecture and had a discussion with Bob Litterman, the director of quantitative resources at Goldman Sachs. He oversees several billion dollars worth of investments and does so quite successfully. One thing he stressed was that all of the tools they use are publicly available in the form of academic literature that their competition tends to ignore. For example, they use a modified CAPM that allows an investor to incorporate their "views" about certain stocks or sectors into the portfolio problem (this is the somewhat famous Black-Litterman model). Generating these views is still a human endeavor, but then the computers generate the portfolios that accurately represent these views and that have high expected returns with low risk.

Nothing new here

[zero_offset]

Big deal, computer models have influenced trading for decades. And not only would it be "irresponsible" to fully automate trading (as the article states), it would also be "illegal". Computer-driven market analysis and prediction is a huge industry -- the big firms spend vast amounts of money on it. I'm not seeing what's newsworthy here, for slash or for El Reg.

Re:Nothing new here

[CastrTroy]

Yes, but if you could mostly automate it, you could do the trading a lot cheaper. Instead of paying highly qualified people hundreds of thousands of dollars per year, you could hire someone for $10 an hour to click on a sell/don't sell dialog box on a computer all day. The computer would be the one making the decisions, but the person would be giving the final order, making it not completely automated. Of course, the person would only ever click on sell, and the computer would only ever present an option which was a good idea to sell. However, the person would just be there to be the human loop in the process, and to ensure that there wasn't something extremely fishy going on with the trading.

Late reporting

[Shoten]

This is nothing new, and it's not even something that's restricted to the world of money managers. It's being used by individual investors now, and has been for years; it's called "technical investing". The definitions of combinations of factors (market cap, financials, etc.) are called 'screens', and are a common source of discussion on forums like those found on The Motley Fool. There's software for sale, priced for individual investors, and there are websites that will even allow you to save your screens to use periodically, looking for new possible stocks to buy into (or to check and be sure that your existing portfolio matches the parameters you want).

Replace investors?

[Aladrin]

Wow, great summary... The computers wouldn't be replacing investors, but 'investment advisors'... That's a whole different rung on the ladder. If they replaced the investors, there'd be no money and the stock market would die.

As for replacing the advisors... Even the article tells you that isn't going to happen. "They reflect the manager's views." Oh... So if there's no manager, there's no view... and the computer does nothing. So you can't drop the advisor.

This is simply another tool. It's not going to change much. My father will still complain bitterly when his portfolio loses money, and complain a little less when he's almost back to where he's started... again. And again.

The fact is... If everyone made money, the stock market would be an impossible thing. Some people will lose while some will gain. No magic piece of software is going to change that.

Re:Replace investors?

[vialation]

This would true except for the fact that our market is NOT a 'zero-sum' system. Just because someone makes money does not mean someone else has to lose money. Investors are investing in these companies for their own benefit, but what does the company do with this money? They use it to expand their business, fund things, etcetera. This creates wealth -- when a company produces a product, that is wealth, in terms of stock. Nobody lost money in the production of that product. In fact, others probably gained wealth as well, as that company may have bought materials and parts from other companies, with them making a profit off of selling their created wealth. Just because there is a static amount of physical cash in this system does not mean that the amount of wealth is static.

Re:Replace investors?

[Eivind]

The fact is... If everyone made money, the stock market would be an impossible thing. Some people will lose while some will gain.

That's a pretty fundamental misunderstanding.

If that was true -- if the stock-market was a zero-sum game where the only way to win was to have someone else lose the same amount, then there'd be no point in playing it. Your average return would be zero.

Luckily that is not the case for investments. It *is* the case for speculation (for example day-trading) but that's something else.

When you buy $1000 worth of oh, say, Arendal Fossekompani. You are buying a certain small part of a company. The company, as most companies, try to turn a profit. On the average, they manage that. Some companies make a loss and (if they stay like that) eventually go bankrupt. But the sum total of the profits (or losses) of all companies is hugely positive.

Now, your $1000 part of the company made say $100 of profits this year. They can do two things with this money. Either they divide their profit up and give it to the owners (that's the ones holding the stock), in which case you'd get $100 cash as dividend.

Or they can invest the money, for example use this years profit to improve the powerplant so that it'll produce more power next year. In this case you still own a certain part of the company, but it's a larger, more valuable company. (your piece should now be worth on the order of $1100, but market-forces can change this in either direction) The stock market is not a zero-sum game with no profits. It's a game where the profits are, over time, equal to the average profit of the companies you invest in.

Re:Replace investors?

[siliconwafer]

Investing in the entire market, since 1927, has returned on average of about 10% per year. That's a handsome return! This is the beauty of index investing, which you described. The layperson can easily purchase index funds and sit back and relax. This is a great alternative to the higher cost managed funds which mostly do not beat the market. With an indexed approach it is impossible to underperform the market, because you're buying the market.

Re:Replace investors?

[qbwiz]

Actually, it does help the company that releases the stock. IPOs can raise a lot of money for a company, which they can use to make more money. When the company becomes large and profitable, you get dividends, which benefit the person owning the stock. It works for both people.

Re:Replace investors?

[servognome]

Explain to me how any country, company or individual that grows slower than the global average is not a loser in this game.

Because they are still growing, the losers are those who grow slower than inflation.

Re:Replace investors?

[inKubus]

Yeah, it's smart to be in the market because you are getting the money of all the people who aren't on the market but spend money at these publicly owned businesses.

But the real growth of the economy is all based on debt, the government decides how much to lend out and that decides how big companies get and how much money there is to pay workers and stuff.

Of course, if you get too much money out there relative to other countries (like now, where the government has been printing $500B per year of new money in the form of debt), your currency falls and your money is worth less and less. You can only push so far before people begin to lose confidence. Having a good leader can really help out the confidence, both in and out of the country. But the economy is showing that it could take the huge infusion of money pretty well and we're sending it to China and they're still taking it so what do we care?

Now, what's going to happen when we move to one world currency? Well, it's going to be a long process and there's going to be a big war and eventually we'll all settle down and we can all move a bit slower. But as long as there are more people than there is available water, food, energy, etc. it will not be possible. I think we will probably hit that population number soon. I like to follow the progress of the "Euro" because that was a major major change for the world that we haven't even felt yet. In the West we have most of S. America tied to the dollar now. Since Europe is now the biggest economy in the world, once they get the kinks straightened out, free up trade between the member states and drop tax rates, they(you) are going to be a force to reckon with.

The only country with anything to fear right now is Russia; they have a huge land mass with huge resources and on the outside you have overcrowded China and Japan in the East, and Europe on the West. But I like the Russians, I think they are some of the most brilliant people. I think they will be our friends.

This is trading not investing.

[Anonymous Coward]

More importantly, the models provide insight into market inefficiencies to be applied rapidly across asset classes and the vast number of financial instruments within those asset classes.

What they're talking about is arbitrage and trading, not investing. Their trades are designed to be in the short-term. Sometimes, very short-term - within a second.

Reducing inefficiency is the key

[Registered Coward v2]

The key comment was:

More importantly, the models provide insight into market inefficiencies to be applied rapidly across asset classes and the vast number of financial instruments within those asset classes. Whole markets can be analysed daily for buy and sell indications at an individual instrument level. This enables portfolios to contain a larger number of instruments and reduce risk through greater diversification of the portfolio.

As inefficiencies are identified (such as when the return / risk ratio is not correct) provides an opportunity to increase returns by taking advantage of them. Of course, as more people use models the inefficiencies will be corrected quicker, leaving less opportunities to exploit. In effect, the market fixes itself. This, of course, is nothing new - markets adjust to new technologies all the time and eventually the opportunities they offered disappear; for example when the telegraph first came out no doubt someone discovered they could buy an item at one place for less then the same item where they were and arbitrage the prices - but as more people started doing that the spread disappeared.

LTCM anyone?

[dtd201]

The use of computers models to predict what to buy has been around for some time. The absolute belief in these models caused Long Term Capital Management to go under in 1998 ( see When Genius failed ). I also highly recommend reading Fooled by randomness

the stock market collapse in 1987

[Anonymous Coward]

back in 1987 when automated selling by computers was blamed for making the collapse worse

The most popular explanation for the 1987 crash was selling by program traders. Program trading is the use of computers to engage in arbitrage and portfolio insurance strategies. Through the 1970s and early 1980s, computers were becoming more important on Wall Street. They allowed instantaneous execution of orders to buy or sell large batches of stocks and futures. After the crash, many blamed program trading strategies for blindly selling stocks as markets fell, exacerbating the decline. Some economists theorized the speculative boom leading up to October was caused by program trading, while others argued that the crash was a return to normalcy. Either way, program trading ended up taking the majority of the blame in the public eye for the 1987 stock market crash.

http://en.wikipedia.org/wiki/Black_Monday_(1987)

Computer trading ... bah!

[Anonymous Coward]

The efficient market hypothesis states that the price of a stock reflects all that is known about a stock. It is therefore impossible to outperform the market on a long term basis. http://en.wikipedia.org/wiki/Efficient_market

Warren Buffet has outperformed the market over many years. http://en.wikipedia.org/wiki/Warren_Buffett

Buffet understands the economy and invests accordingly. The computer programs only understand the market. In other words they can't really respond faster than the rest of the market. Buffet can be years ahead of the market.

Re:Not illegal

[zero_offset]

I suppose it's a question of semantics. Fully automated trading *is* illegal. Automated trade execution requires a person in the loop (setting thresholds for example) and is highly regulated. I actually know a lot about this, I was writing market-timing fraud detection software for a living as recently as last year.

As for the question of "Why?", the answer is on the page you linked. Black Tuesday, for example.

Dangerous idea

[Opportunist]

Let's be honest here, the "human factor" was what made some huge enterprises possible. Because some humans believed in something the "numbers" alone didn't predict. Do you think Google would have found an investor in its early days?

Investment by numbers is by definition a rather conservative way to invest. In other words, put your money where there already is money. Risk investment is usually something done by visionaries, not by bean counters. And yes, 9 out of 10 times the idea bombs. But the one that works pays hundredfold.

(Good data && Good Algo)

[140Mandak262Jamuna]

Yes, cost of computing is falling like a rock. Yes, tons and tons of data are available and increasing more accessible. What XML tags and electronic submission requirements by SEC. So there is big money in "programmed trades"?

When everyone is crunching numbers on their head, the computing might give an advantage. If everyone is computing with compters, the advantage goes to the one with better algorithm. And the better "algorithm" might actually turn out to be thinking and looking at the global picture instead of madly computing.

When huge trades are decided by these algorithms it is almost like a huge herd of milling cattle. When the stampede, just get out of the way. But if someone could trigger a stampede and send it over the cliff, there will be rich pickings later. And in free markets, if there is a way for someone to make money, someone will.

I think algorithmic investing is the new name for the old "programmed trades" and it might actually make the thinking and studying fundamentals investors richer.

The code is

if( InputDataIsGood() && AlgorithmIsGood() && AlgorithmIsBetterThanOthers()){

Output(profit);

}else{

Output(loss);

}

Re:Not illegal

[El Torico]

Black Tuesday was October 29th, 1929. You must be thinking of another "black" day.

This makes the rounds every now and then...

[StressGuy]

A friend of mine tried writing his own auto-investment code to see if he could actually make a living at it. His plan was to dedicate a computer to doing nothing but scanning the market and making investments for him. Well, he's not doing that today, and he's probably one of the most intellegent people I know. I also rememeber a concern that, if everybody used automated investing, the market would become highly sensitive to change (or...unstable) as you'd run into situation where most, if not all, of the algorythms out there would react the same way to certain market changes.

Still, it's intriguing isn't it? I mean, one of the things I use computer programming for is to learn how things work. I look at it this way; a computer is rock-stupid, but it does exactly what you tell it to do. So, if I could write a code to do market analysis, I'd be learning the intricacies of how to do it along the way. Sure, most invesment sites have tools for you, but there is value in learning the underlying mechanisms.

Seems the best approach would be to write such a program to simply do the analysis, then you make the final commit to buy or sell. You'll have a good idea how to interpret what you get back because you told it what to do in the first place and you should be able to spot errors/weaknesses in your strategy. It could be downright symbiotic.

Price, Pattern, and Profit

[G4from128k]

First, there are patterns. You are right that those patterns have a limited capacity to absorb trading and that anyone who finds a pattern would do well to keep it a secret. The EMH (Efficient Markets Hypothesis) is best lampooned by the following old joke. Two economists are walking down the sidewalk when one of them spots a $100 bill in the grass. The first economist starts to pick up the $100 when the second economist tells him, "Don't bother, if it were a real $100 bill, someone would have already taken it." Moreover, EMH makes predictions about the statistical distribution of price movements and the volume of trading that are empirically false.

Second, the price dynamics are not entirely caused by exogenous factors. Investors, speculators, the media, and government officials do watch the prices. People make buy and sell judgments without any fundamental basis such a stock being "expensive" just because the stock is $300/share (never mind understanding the relationships between price per share and capitalization). Humans also have instinctual beliefs in patterns such as trends or momentum that are self-fulfilling. If enough traders believe in trends or momentum, they will trade in a way that creates trends.

The profitable patterns do exist (and so do a large number of profitless pseudopatterns). People with a very sound understanding of both market psychology and statistics can and do succeed.

Re:Price, Pattern, and Profit

[cartman]

I've never believed in the EMH, but I'm going to try to defend it anyway.

The EMH (Efficient Markets Hypothesis) is best lampooned by the following old joke. Two economists are walking down the sidewalk when one of them spots a $100 bill in the grass. The first economist starts to pick up the $100 when the second economist tells him, "Don't bother, if it were a real $100 bill, someone would have already taken it."

I've heard that joke many times, and it always seems to me like a false analogy. The EMH doesn't deny the possibility of luck; it denies the possibility of systematically beating a competitive market. The patch of grass is not a competitive market, and finding $100 there by luck is not beating it systematically. In other words, there are fundamental differences between equity markets and patches of grass. For example, shares of the grass are not being bought and sold, therefore information about the existence of the $100 is not being incorporated into prices, which is a fundamental assumption underlying the EMH.

A better analogy would be the following. Assume the existence of a patch of grass upon which a given amount of money falls according to some pattern. Assume also that there is a mature, well-developed industry to predict when the money falls. Assume also that the industry is competitive; ie, when one person takes the money from the grass, it's no longer there for another to take. Assume also that there is some monetary cost to visit the patch of grass and determine if there's any money there. Given all those assumptions, at some point, the grass would cease yielding abnormal returns--in other words, the cost of visiting the grass would equal the average amount found there, given the best available algorithm for determining how much money will be there.

Second, the price dynamics are not entirely caused by exogenous factors. Investors, speculators, the media, and government officials do watch the prices. People make buy and sell judgments without any fundamental basis such a stock being "expensive" just because the stock is $300/share (never mind understanding the relationships between price per share and capitalization). Humans also have instinctual beliefs in patterns such as trends or momentum that are self-fulfilling. If enough traders believe in trends or momentum, they will trade in a way that creates trends.

The EMH people would probably respond as follows. Granted, humans believe in patterns which can become self-fulfilling prophecies. Thus, they create a pattern. However other, more sophisticated traders are also aware of the pattern ("momentum") and will place trades that destroy the pattern. For example, if I (as an investor) recognize momentum then it would benefit me to buy shares at the beginning of momentum and sell short at the end, before the bubble bursts. If I do this profitably, then I (and other, similar investors) will control an increasing share of the money being invested, and "momentum" will no longer occur. Note that this pattern-destroying mechanism can occur with any pattern that could be recognized, including self-fulfilling prophecies of naive investors, and including momentum.

...Nevertheless, EMH aside, there are trends which can be identified. One example is the NASDAQ from 1997-2000, which is a particularly striking incidence of momentum. That trend persisted even though there was frank discussion by experts months beforehand that the NASDAQ was certainly in a tremendous bubble. The fact that momentum persisted for years despite publically available pronouncements by all experts that there was momentum, is difficult to reconcile with the EMH, since the EMH asserts that any such trend would automatically disappear.

I believe there's a fatal flaw with the EMH. I believe the EMH rests upon a number of assumptions, one of which is false. But this post is already long enough...

Comments from a friend in the business

[espressojim]

I have a friend who worked in the hedge fund game for a number of years. He's a brilliant mathematician, and worked on the models they used to inform their trading. The group he worked with was quite successful, and make a heck of a lot of money.

One of his most interesting comments: "The model can inform your decisions, but you have to know when to NOT trust the model." Another of his comments on a completely different talk: "Mathematical models are never perfect, but they can be useful."

The trading system can be modeled, but you can never capture all the true complexity of the real world. If you leave the model to do it's thing, if I know how it's going to act, I can game the system. If the world changes in a way that the model builders did not predict, then the system will also act inappropriately.

I can't imagine ever getting rid of all the traders out there, though I imagine expert systems will become more 'expert' as time goes on.