SEC Blames Computer Algorithm For 'Flash Crash'

Lucas123 writes "The US Securities and Exchange Commission and the Commodity Futures Trading Commission today issued an 87-page report (PDF) on the results of a months-long investigation into the May 6 'flash crash' that sent the Dow tumbling almost 1,000 points in a half hour. The Commissions are holding a single trading firm's automated trade execution platform responsible for the crash, saying it dumped 75,000 sell orders into the Chicago Mercantile Exchange over a period of minutes causing an already volatile market to come crashing down. The SEC has already enacted some quick rules to pause trading if a stock price should rise or fall by 10% in a five minute period, but the regulators said they expect the results of the investigation to prompt additional rules limiting the functions of automated computer trading systems."

Comments

A time out is the right solution.

[LostCluster]

Here's the way this went down. Because this malfunction dumped market (not limited) sell orders without matching buys, they quickly soaked up all of the buy bids that were on the market, leaving only outrageously low buy bids that usually don't see the light of day at the top of the pile. Those got filled too, and suddenly you've got everything trading at 90ish% off what it was a moment before. CNBC and other instant media realizes that something's amiss... Jim Cramer happened to be making his regular afternoon visit to the daytime programming and shouted out a pretend limit buy order for the stock he was scheduled to say was overvalued... he then "sold" that order a few moments later to show there was instant profits to be made by somebody. This selloff was nonsense, and the market quickly recovered to where it was before minus some losses for the fact that some of the investing public was losing faith in the system.

Now, since this was a malfunction, the people who lost 90% instantly and the people in the other side of those trades who made 80% did so by foul play. The flash crash trades were busted (market regulators ordered them undone) and the world went on like this never happened.

There used to be rules that if there was nonsense at the NYSE, the specialist on the floor would ask questions and stop processing trades. If there was no news to make a fundamental change in the stock and there were suddenly sellers but no legit-priced buyers... just shout out that this was going on and some buyers would be sure to show up.

But now, with many electronic places competing with the NYSE, an NYSE-only stop to computers damage that needs to be routed around, and the crash continued at these exchanges. So, the SEC at its level over all of these systems is establishing rules under which every exchange has to stop processing trades in the affected issues until there's enough time for the news of the event has spread and everybody's had a chance to react.

Market rules are based on trying to give everybody involved a fair chance to trade. Trading on information you have that isn't public yet is not allowed. Martha Stewart went to jail because she had money in IMClone, and was called before the news was out by somebody telling her an FDA trial had a failed result. She sold immediately, and then we she realized she had fouled, faked her phone records about the call. Gotta play fair... they are watching.

Re:A time out is the right solution.

[LostCluster]

Not quite. A stock's quote price is the last price at which a bidder's offer matched a seller's asking price. A "level 2" quote has two parts, the highest bid price that hasn't been matched up yet, and the lowest asking price that hasn't matched up yet. The true value is somewhere in between these two, but nobody knows where until somebody steps in between the high bid or low offer or somebody moves their price to get a deal. Any computer programmer should check that there's matching orders on the other side of their trade before placing a large order. A program that doesn't will execute just fine, but crash the economic system.

Re:A time out is the right solution.

[LostCluster]

Put in a market buy order with no limit, and you open yourself to limitless losses if there's nobody willing to sell at the moment. You'll match the first offer to sell no matter how high it is.

Put in a market sell order with no limit, and if there's no covering buy order you just put in a request to give it away for a penny a share.

Limit orders rule.

Re:A time out is the right solution.

[Maxo-Texas]

But if a large organization wanted to sell stock to itself at increasingly higher or lower prices there isn't anything you can do to stop it. It's illegal as hell but hard to prove.

the only thing that makes prices rational is a fluid market.
A low volume market produces irrational prices and makes it easy to move prices around inside the limits of rational prices.

Put it this way...

Millions of baby boomers are locked in on a large chunk of their retirement money at 14,000 dow.

As they get older that price they are willing to accept to cash out is degrading (a lot of boomers would cash out immediately if the market got above 13,000 now).

As long as the price doesn't get too high or too low, the boomers are paralyzed and the market is not fluid.

In 2012 to 2016, that price will degrade more. I think we have a decade of overhead pressure from boomers cashing out. At some point, the price won't matter- they'll *have* to cash out to pay bills or go back to work (oh yea, you can't really find work if you are in your 60's these days- I mean 50's.)

Re:A time out is the right solution.

[LostCluster]

It's perfectly legal for a business to say "We think we're worth 5% more a share, and we're willing to pay that price to anybody willing to sell." It's called a buyback, and as Cramer calls the "Sir Mix A Lot Corollary" he says "I like big buybacks and I cannot lie."

Re:A time out is the right solution.

[LostCluster]

It wasn't the phone company record that the call took place, it was her company record about what was discussed during the calls.

Her phone records that she presented investigators showed she had a pre-existing stop-loss order to sell the stock if it traded below $60, but others testified that was a lie fabricated after the illegal info was given to her. So, there's your obstruction. Yep, they couldn't prove the crime but could prove the cover-up.

Fire up the Wayback Machine, we're headed to March 4, 2004. Fox News, Forbes, USA Today

Re:Regulatory Agencies Don't...

[LostCluster]

"Prevent" is such a strong word. They're good at keeping bad things from happening, just not perfect.

So...

[rantomaniac]

Remind me, why do we have such a fragile system at the very core of modern civilisation?

Re:So...

[mrlibertarian]

Fragile? The system may have "broke" in a flash, but it also fixed itself in a flash. The only people who were hurt were those who sold because everyone else was selling (stop loss orders).

This entire issue boils down to a particular group of people whining about a single firm's stupid computer algorithm, because that algorithm broke the stupid computer algorithms that group relies on (i.e. stop loss orders). For value investors, this whole thing is just a bunch of noise. Civilization rolls on.

Ouch

[citking]

Hope the person(s) who wrote that algorithm aren't writing nuclear reactor code. I'll admit though that I'm a bad programmer too. Back when I did write code I used such gems as DIM TotalSales AS INTEGER. That didn't work so well.

Re:Ouch

[geekoid]

You didn't have to tell us you where a bad programmer, the VB code was a big enough clue~

Re:Was Windows to blame? Was Unix? Was Java?

[AnonymousClown]

It was a Solaris backend using a database on Linux that had a Java front end on a Windows PC. The trader monitoring the system was watching porn his Macbook Pro and didn't notice when things went kaflooey.

Re:Regulatory Agencies Don't...

[BoRegardless]

I agree that "prevent" could be considered wrong.

What I detect is that the smartest & most motivated people do NOT inhabit the regulatory agencies and indeed probably do not inhabit the exchanges themselves.

The various brokers hire people at much higher salaries &/or bonuses and pay them VERY well to find the tactics, some would say loopholes, to allow quick profits each day. That in itself is not what the original intent of share ownership markets were about.

I wonder when the word "Day Trader" was invented, but it certainly was quite awhile back but it didn't include the ability to do tens of thousands of trades out of one broker in a matter of seconds, and it certainly wasn't considered why we needed a share exchange in the first place. Exchanges were to allow companies to raise funds and to promote their value based on earnings and assets over time and allow a company to achieve an immortal status that an individual person could not achieve.

I think governments as the regulatory overseer are flawed, but then recognize the brokers are also very self-interested, so the whole mess needs more transparency.

That sort of transparency has been achieved with the likes of Linux.

I wonder if open sourcing the rules of the share markets could achieve the results where everyone knows the rules of the game & small individual investors have the same info that the large brokers do?

The worst thing in the world for a share market is to eliminate the small investor leaving only the whales to thrash about.

It is a big problem to solve and the self-interest of the big brokers cause all sorts of broken arms in WDC, if I guess right (meaning $s passed behind between arms).

Transparency is the only solution I see.

Because it works?

[Sycraft-fu]

Seriously. I think most people will admit it isn't perfect, and it looks like they are trying to improve the system as a result of this. However fundamentally, it works. It helps money move around more, so that businesses can get financing, individuals can invest and so on.

The reason why you find that prosperous nations have things like a stock market and other capitalist features is because they work. Doesn't mean you ignore them or let them run totally wild, but fundamentally they get the job done, where as a command and control economy does not. While it may add instability it also adds flexibility and that is important.

Re:Because it works?

[jd]

I'm not sure that the proposed solutions will fix the problem. I'd much rather a degradation in response times as a function of orders (so the more orders there are, the slower the system gets) rather than a temporary hold on that stock. Temporary holds assume that software won't do what it has always done in the past - try again until it gets through. If you flood the system with retries from enough computers, the results won't change. It will merely have short gaps in it. If you have gradual degradation, then flooding will slow things way down until the flood stops. The negative feedback loop will guarantee that a crash becomes impossible.

In fact, that is something the market could do with more of - negative feedback loops. It should be possible to prevent market bubbles as well as market bursts, as a bubble is just a positive feedback loop in the opposite direction.

Re:Because it works?

[houghi]

Just because something works does not mean it is a good idea. Ponzi schemes work.

Re:Because it works?

[gethoht]

Exchanges make boatloads of money off of High Frequency Traders(HFT). While their algorithm mows through a ton of investors stops, causing thousands of people to lose money, their algorithm gets the benefit of the doubt as any trade after a certain percentage swing gets nullified by the exchanges. In short, they get play by different rules then other investors. The easiest way to stop all this HFT flash crash shenanigans is to declare all trades of a freaked out algo valid. Then the people responsible for that algo lose tons of cash, as they rightfully should. That loss should incentivize banks and their programmers from writing shitty programs that freak out the markets. As it stands right now the banks and their algos have a win win situation. They get to make millions when their algos work but when their algo's freak out, the exchange gets to declare the trades invalid. Make the trades the algo makes completely valid and I guarantee you won't see algos freaking out as often.

Re:Because it works?

[Sycraft-fu]

I mean it works in terms of making an economy work better. It provides a good way for people to invest and borrow money, which is important. A fundamental principle of money is that it has to move around to do anyone any good. It is just a construct to facilitate trade. Trading goods and services is what actually makes an economy worthwhile. Money is just a construct to facilitate that. Well that means money is only good if people spend it, if it moves around. The stock market is something that helps that happen.

If you've a better idea, then it would be great to hear it. All I'd caution you on is to do some research, because as with many things in life there are economic solutions that are clear, simple, and dead wrong. A command economy would be one of those. People say "Why not just have the government control it all? It is resilient but can change quickly." Ya well, command economies have been tried numerous times and failed miserably because they don't deal with human nature well.

As it stands, an open investment market is one of those things that helps economic efficiency. You will notice the US is not unique in it.

Re:Was Windows to blame? Was Unix? Was Java?

[Mikkeles]

So; it would have been fine had they used *BSD ;^)

Valuation is an art

[sjbe]

One of the thing that was made clear to me over the last few years was that the price of stock is whatever the last person bid for it.

The price of ANYTHING is the price of the last accepted bid. Always has been, always will be.

It isn't based on the book value of the company.

Not directly, no. Really stock prices are usually based on a collective opinion of the future profit making prospects of the company. Sometimes though they are based on things that have little or even nothing whatsoever to do with profits. (Exhibit A is the dot com bubble in the late 1990s) The stock market is really not much different than any other form of betting and it only secondarily has anything to do with the actual finances of the company.

Value is a subjective thing. I'm an accountant in my day job and I'll be the first to tell you that valuation is probably more of an art than a science. Opinion plays a huge role because the same thing can be worth very different amounts to different people.

Insightful my ass

[SmallFurryCreature]

Except that stock speculation has NOTHING to do with investment anymore. Wall street does NOT invest, it speculates. It is gambling on the minute by minute perceived loss and gains in the world with a hefty amount of making events happen.

Take the recent event of a speculator simply buying up chocolate to drive up the price. What has that got to do with investment or making money go around? Nothing at all.

You have the idea that the stock market is still the old idea of buying a share in a ships voyage when this was first made official in Holland centuries ago.

Yes, if you buy shares in a company hoping to get dividend from it in the future, then you are investing. When you are shorting stocks on the difference in value over a period of minutes, that is NOT investment.

Stop pretending that it is.

The CME Group has issued somewhat of a rebuttal.

[Anonymous Coward]

According to the CME Group:

"The 75,000 contracts represented 1.3% of the total E-Mini volume of 5.7 million contracts on May 6 and less than 9% of the volume during the time period in which the orders were executed. The prevailing market sentiment was evident well before these orders were placed, and the orders, as well as the manner in which they were entered, were both legitimate and consistent with market practices. These hedging orders were entered in relatively small quantities and in a manner designed to dynamically adapt to market liquidity by participating in a target percentage of 9% of the volume executed in the market. As a result of the significant volumes traded in the market, the hedge was completed in approximately twenty minutes, with more than half of the participant's volume executed as the market rallied – not as the market declined. Additionally, the aggregate size of this participant's orders was not known to other market participants.

Additionally, the most precipitous period of market decline in the E-Mini S&P 500 futures on May 6 occurred during the 3½ minute period immediately preceding the market bottom that was established at 13:45:28. During that period, the participant hedging its portfolio represented less than 5% of the total volume of sales in the market."

http://cmegroup.mediaroom.com/index.php?s=43&item=3068&pagetemplate=article

The SEC/CFTC report is typical of something that we tend to see come out of government agencies (low quality analysis). Also, they didn't make any meaningful recommendations. It seems like that just tried to rush something out as quickly as possible to say, "Everyting is fine. Retail investors can hazard their capital again. We caught the evil, responsible financial firm and will sort them properly."

Re:Why not just simply ban the practice?

[rritterson]

As a buy-and-hold investor, why do you care whether high-frequency trading exists at all? The flash crash was largely erased shortly thereafter, so it wasn't like it artificially destroyed your wealth. As a person who believes that a core value of our moral system should be those things that do not impinge on the rights of others should be allowed (with notable and obvious exception), I find banning high value trading simply because we are afraid the market will do strange things is silly.

When it comes down to it, the flash crash was a boon for the buy and hold investor, since you got an opportunity to buy things at great prices. And, when it comes times to sell, having a bazillion automated trades in the system ensures your trade will get lost in the liquidity, practically guaranteeing a fair price. Wipe out market liquidity and you are suddenly at the mercy of whoever happens to want to buy that day.

Don't Get Out Much?

[mpapet]

What I detect is that the smartest & most motivated people do NOT inhabit the regulatory agencies

That is by design. The agency wasn't ever going to go away, but their efficacy sure did in the holy pursuit of unfettered Capitalism. What has that gotten the majority of Americans who believed in the wisdom and efficacy of deregulation?

-Banking system on national life support.
-Consumers with no confidence in many forms of economic activity.
-A series of economic bubbles

It never works out and yet voters are more than willing to get screwed again under the new mantra of "fiscal austerity." That's more pocket picking for the recovering Capitalists living in your parent's basement.

No bugs, Nothing went wrong

[Animats]

I just finished reading through the whole report. It's fascinating, if you're into this.

First, none of this involved a "bug" . All systems involved functioned as designed.

What's going on here is a logical consequence of the way the markets are set up. The Chicago Mercantile Exchange ("CME", the futures market, which started by trading grain) has a tradeable commodity called the "E-mini", which is a derivative security based on the S&P 500 stocks. Anyone can buy or sell contracts in E-minis, and can also buy or sell the underlying stocks. This generates a frantic amount of short-term trading from market players trying to profit from the differences between the two, which keeps the price of the E-mini close to the prices of the S&P 500 stocks.

None of this is productive activity, of course.

There's a consolidated feed from all markets that everybody gets. It has a few seconds of lag. To obtain an advantage in fast trading, some of the players buy direct exchange feeds with an average of 8ms (yes, 8 milliseconds) of lag.

What started the crash was that a fundamentals trader (one who actually pays attention to the companies involved) was selling $4 billion in stocks. Ordinarily, this isn't a big deal. They had a program throttling their rate of sale to 9% of market volume in the last minute, to avoid depressing the market. That's normal. So far, so good.

However, in response to this sale, the "high-frequency traders" started frantically trading back and forth to balance their portfolios. Their net effect didn't move prices much, but it pushed volume up. So the big seller started selling faster.

This generated enough volatility that some market players started dropping out, decreasing liquidity. That generated market imbalances which other traders started to exploit. Then, because of all this frantic trading, the consolidated market feed and the millisecond feed differed enough that some trading firms had data quality alarms and dropped out of trading. Since traders who are "market makers" are required to maintain buy and sell bids in the market, they defaulted to their default bids - buy at $0.01, sell at $100,000. Some trades actually took place at those prices. 895 shares of Apple stock were sold at $100,000. The price of Accenture fell from $30 to $0.01 in seven seconds, then recovered within the next minute.

Then "At 2:45:48, trading on E-Mini was paused for five seconds when the CME Stop Logic Functionality was triggered in order to prevent a cascade of further price declines". Yes, a 5-second automatic trading halt. That was enough to start to stabilize the E-mini contract trading on the CME. But by then, the E-mini was enough out of sync with the underlying stocks (mostly on the NYSE) that trading on the NYSE started to move stocks there to resync with the E-mini.

The NYSE still has a trading floor, which slows it down. This didn't help. But that's another story.

Nothing failed. Nobody did anything wrong. The original seller's strategy for unloading $4 billion in stock was reasonable. This is all a consequence of normal market operation. The report concludes that speeding up the consolidated market feed to get the 5-second lag (which was more than fast enough before program trading) down should be done. That's it.

Whether or not society should support an "efficient market" system to this extent is an question one is not supposed to ask.

Re:No bugs, Nothing went wrong

[7-Vodka]

A program that runs as designed but produces a undesirable result has a "design flaw" which is a class of "bug". Such things need to be "fixed".

What program are you talking about? If you read the parent post you will notice that no single program had an undesirable effect. The market as a whole entity was what failed, the failure was not in any individual program.

If you want to know what I think, the market didn't fail. What was a failure was the response of freezing markets and reversing trades.

Those trades were made by consenting adults in good faith or the trading systems that they entered into willingly. Money was lost and money was made. Fuck anyone who got burned, that's a normal day in trading. Or it should be, but unfortunately if a enough big players lose enough money they reverse the trades for them.

Let's get few facts straight

[alexmin]

Here are few important facts:
1. Waddell & Reed is the company whose aggressive selling triggered drop in S&P 500 futures price. The company is not HFT shop but rather long-term investment hedge fund. More here: http://www.bloomberg.com/news/2010-09-30/waddell-reed-e-mini-trades-are-said-to-have-helped-trigger-may-6-crash.html

2. According to SEC report, HFT traders played their intended role: smooth out short-term price volatility. However, due to enormous size of Waddell & Reed selloff (about $4 billion dollars in 75000 futures contracts during 20min.) they can do only that much. W&R just cut right through the order book on CME.

3. Slowing down the trading on NYSE did not help but rather hurt by locking up liquidity. Shitty NYSE Arca systems that handle ETFs overloaded and further exacerbated the problems.

4. At the end of day market returned to pre-crash levels. Long term investors were not hurt, W&R payed between 100 and 200 millions for their mistake.

5. Overall, market worked as expected.

Re:No bugs, Nothing went wrong(Except Corruption)

[Required Snark]

The key line is:

Whether or not society should support an "efficient market" system to this extent is an question one is not supposed to ask.

The algorithm didn't fail, Wall Street as an institution has failed. The simplistic view of why capitalism works is that individuals and institutions making informed decisions results in good allocation of resources. The profitable thrive and the unprofitable die, and on the whole society benefits.

None of the preconditions for capitalism exist in the current setup. The big entrenched special interests change the nature of the system so that they take profit and are shielded from risk. The technical term for this is "moral hazard". The TARP bailout is the perfect example of this. All the big Wall St. firms made huge amounts of money by playing insider games with mortgage back securities (MBS) and collateralized debt obligations (CDO), and then when their gambling resulted in failure, the were bailed out to the tune of ONE HALF TRILLION DOLLARS, and the government is left with the bad assets. And the people who caused the mess are still in charge and got to keep all the money they stole during the bubble, as well as the money they got from the Treasury. Does the phrase "moral hazard" seem sufficient to describe this behavior, or would "rape, pillage and burn" seem more appropriate?

Programmed high frequency trading (along with hedge funds) are another mechanism for taking wealth from the system that breaks the capitalistic model. The claim is the it "increases fluidity" and therefore make the market "more efficient". The plain English translation of "more efficient" is theft, and "increases fluidity" is like saying "magic pixie dust".

The real world value of a company cannot change at millisecond resolution. The only things of economic value that change that fast are electronic abstractions of money. Therefore, high frequency trading is completely disconnected from real world value, so no capitalism is involved. The system is built so that insiders can become personally wealthy because they are the insiders, not because they do an efficient (good) job of allocating resources and benefiting society.

This is identical to the MBS/CDO monstrosity, in that there is no clear real world description of how value is created. For MBS/CDOs there was a lot of math that no one making decisions really understood, but somehow mortgages from buyers who were previously unqualified could become AAA securities. For flash trading there is "fluidity" and algorithms that traders don't understand. It is the same kind of scam.

As long as the stock market allows high frequency trading it will be intrinsically unstable, because this kind of trading is about manipulating the abstract system, not about real world issues. No set of rules will change things, because computationally based trading is about taking advantage of rules to get advantage via manipulation.

The only kind of rule changes that will help are things like increasing the cost of individual trades or keeping electronic traders from placing trades that they cannot or do not intend to make. (Trading algorithm determine price points by placing lots of orders and seeing which ones get responses.) Or electronic traders must be forced to honor trades or hold assets that they are trading, so they are exposed to the market risks of the underlying securities. Right now there is no cost for these traders for any manipulative practices, which effectively decouples risk from reward. All these kinds proposals move this kind of trading back towards actual capitalism.

It will be very hard to get meaningful changes to high frequency trading because the powerful and personally corrupt Wall St. insiders don't want a capitalistic system, they want their guaranteed profits. It is much closer to a Mafia style protection racket then a system that enables real business activity.