Slashdot Mirror
Flash Crash Analysis of May 6 Stock Market Plunge
Jamie found an interesting site that has many charts and graphs about the strange May 6 stock market plunge and rebound. There's a lot of information to consume over there, but it does a pretty good job of showing high-frequency trading is getting to be a real problem.
and stay there because that's about what it's worth. The only reason it's up above 10,000 is because it's being propped up with funny money. Just a year ago it was at 6800...these are the same companies. Does anyone really believe that all of the companies listed are collectively worth 1.5 times more?
I knew the Flash plugin was unreliable, but I had no idea it was so bad that it was affecting the stock market.
.. about Flash crashing, I mean ...
Look at average frequency of trading for the two years leading up to the Flash Crash, and set that has the upper limit.
Not to be a conspiracy theorist, but I work with a bunch of math PhDs who specialize in stochastic processes. Two of them used to work in the financial sector before the crash. Everyone around here including me has come to the conclusion that someone planned a really big "oops" to make his friends very rich and get a few kickbacks. Sell it short, baby!
The problem with this is that since it has happened once, it *is* going to happen again in a slightly different way. Software glitches, fat fingering the keyboard, etc. are convenient excuses.
Old way: 10,000 trades a day, every few months or years the market dips for a few months and rebounds, every several years the market enters a deep recession for years.
New way: 10,000 trades a second, every few weeks or months the market dips for a few minutes and then rebounds, every several years the market enters a deep recession for years.
The artifacts of trading become more temporally frequent and more temporally limited; the artifacts of real economy (growth-recession cycle) don't change.
Support my political activism on Patreon.
Those which were at a 60% discount or greater, yes. And it's serious bullshit.
After all, I am strangely colored.
High Frequency Trading is _beneficial_ to the public markets at large, and why powerful interests keep blaming and attacking electronic trading as the root of all financial evils that befall us: http://www.tradersmagazine.com/news/high-frequency-trading-benefits-105365-1.html?zkPrintable=true
Unfortunately, the majority seem to be believing Rupert Murdock's Wall Street Journal and similar mouthpieces spouting all the "Electronic Trading must be taxed/stopped/restricted, it is destabilizing markets" rhetoric.
As mentioned in above link and In case you did not hear about the New York Stock Exchange specialists charged with fraud, an event referenced in the above link - it's pretty amazing: Richard Ney wrote a best selling book in 1970 ("The Wall St Jungle", interview NY Magazine 1970) with a few follow up books that all called out the NYSE Specialist families for fraud, explaining exactly how they defraud the public. At the time The Wall Street Journal boycotted anyone selling the best seller and Ney was not permitted as a guest on The Tonight Show - very unusual at the time for someone with such a long run best seller/controversial book - his message had touched a raw nerve. In response, the establishment had Ney widely counter-attacked, labeled a conspiracy theorist nut at every opportunity - comments like "what would an actor know of the stock market" were common and can be heard even today.
To prove Ney's wild eyed grand conspiracy theory right - The Department of Justice finally got around to charging the NYSE specialists for the exact fraud that Ney described - 33 year's after he wrote about the crime! In 2003 the Specialist firms quickly got their get out of jail free cards for a tiny fraction of what they had actually defrauded over the years. The story does not end there however... news came out shortly after that the NYSE was at long last going to move to an all-electronic exchange - and that the Specialists firms charged with defrauding the public were the very same that had been blocking the move due to their 30% NYSE stake. Everyone in the know + those that read Ney's book knew all too well of the massive fraud going on in full public view for at least 33 years (more like 212+ years), but it was not until these Specialist criminals blocked other powerful interests that the illegal behavior was actually pursued by the DOJ.
If ever there was an example of the lack of credibility for the DOJ, this is it. 33+ years of massive fraud in full public view, but the DOJ did not get around to prosecuting until it was ordered to - until it was necessary to coerce the Specialist family firms into letting the NYSE go electronic. Nothing to do with justice, or protecting the innocent being defrauded to the tune of billions of dollars over the decades. As an added insult, the DOJ let the criminals off the hook with a paltry fine. But then there is no surprise there, as Richard Ney said it best: "Regrettably, the arrangements that exist to preserve the traditions and legalize the frauds of the security industry are inseparable from the general organization of a society controlled by the financial establishment, a society whose laws and principal customs have been contrived to serve the special interests of the financial community,"
Voting Red or Blue will not change this arrangement of US society and it's laws - merely reinforce it.
...right here. One commenter had some interesting things to say about "quote stuffing":
So it might not be the big advantage that Nanex sees it as.
The Army reading list
The real problem isn't trading frequency. It's the basis for the market. Since most stocks have stopped paying significant dividends, there's very little basis for any one stock's value. The stock market today is solely based on betting whether an individual can accurately predict what the rest of the market will do in the future. Stock prices don't have anything to do, whatsoever, with the inherent financial value of the underlying company.
If I'm going to gamble, I'm going to Vegas. At least there, you get free drinks as you piss your money away.
I don't respond to AC's.
I have always thought that if you buy stocks that pay dividends you should have to keep them for a minimum of a year, and if you buy stocks that don't pay you should have to keep them for a minimum of 24 hours before being able to sell them. If you short stocks the sale should take 24 hours to take affect.
I mean really, what could happen in 24 hours that would honestly affect the value of a publicly traded company? Even the oil spill has taken weeks and months to lower BP's value, and really in the long run it is probably time to buy BP stock.
those with the screamiest servers the shortest fibre optic hop away from wall street get to play this game, no one else. it dedemocratizes the market. the ideal of a marketplace is that it is a meeting place of equals. if the guy with the most expensive servers and programmers money can buy is the only one who can profit though, the marketplace is now simply an oligopoly of the rich, not a place where the common investor can make his or her mark
of course, the market has never been a meeting place of equals, it has always been abused by the largest players in the marketplace. however the idea is to minimize this abuse, not excuse or accept it
what the market needs is a "tick", a "heartbeat": all trades, no matter from whom, must be made in the same 1 second or three second batch cycle. no one should be allowed to exceed this frequency. problem solved
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
The problem wasn't the NYSE. The problem was that when the NYSE decided to execute trading delays, the other markets replied "the computer sez no" and kept on trading at high speed. Because there weren't enough buyers at the time to satisfy all the selling, all the market sellers saw their prices plummet because the computers were programmed to find a buyer no matter what the price, as long as the transaction would clear.
So... what's the problem? If you picked up Accenture at a penny a share you should be fuckin' lucky. I wouldn't shiv that stock off to a homeless man for a nickel.
Oops... I farted.
Sell! Sell! Sell!!
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
This is the actual analysis:
May 6
John (one of the wall street support technicians) was playin flash games on his workstation.
The stock market is stored under "market2010.swf"
He forgot to move it into the right folder N:\smarkets\closed\market2010q2.swf usually when this happens it's not so bad because
Unfortunately, the network admin did a flash game sweep and deleted *.swf as a routine cleanup
What happened was, the market2010.swf was deleted by the routine scan, so now they having to recreate all the data from memory. Stockmen are now trying to guesstimate how many shares they own.
When they say fat fingered, they aren't joking. N:\smarket has TONS of files and it takes AGES to load a 1000 files in icon view on Windows 98!
Slashdot needs Geekcode | Can anyone recommend any good SCIFI? My tastes: Foundation, Startide Rising, CITY, Ringworld,
From a few pages into the write-up (http://www.nanex.net/20100506/FlashCrashAnalysis_Part4-1.html):
Definition of a DDOS (from http://searchsecurity.techtarget.com/sDefinition/0,,sid14_gci557336,00.html):
Quote stuffing looks like a DDOS to me, and should automatically be illegal. Of course, there are several technical differences that any lawyer could point out,thus making quote stuffing legal, so I'd recommend outlawing it just to be sure. Not often I get to say, in all seriousness, "There ought to be a law." {Most situations do not require new laws, only the proper application of existing laws.}
I hope this comment is well received... I could have moderated instead!
Persecutors will be violated!
I thought you said smell ...
Instead of putting in fixes at the exchange level, put something in at the SEC regulation level so it applies to all US exchanges. And yes that'll stabilize foreign exchanges too. Think about supply and demand and what sellers do when prices drop in market A and don't drop (or don't drop as far) in market B.
First option: bunch trades by time. Define a market tick, say 2 seconds. All trades that come in in a given tick get bundled together and executed as if they'd arrived in a random order at the end of the tick. The exchange is allowed to use any method to randomize and order the trades, the only rules are that the method can't be based directly or indirectly on the original arrival sequence or the original arrival time and the method can't give preference to any particular trader or type of trader. The bunching should have no effect on people who trade on timescales more than about 2x the tick, but makes trading on timescales less than the tick infeasible because the market simply won't execute your trade any faster than the tick.
Second option: random delays. Define a market tick, say 2 seconds. All trades, as they arrive, have a random delay between 0 and the tick length calculated (same rules as option 1) and have their execution delayed by that much. You're guaranteed to have your trade executed within 1 tick of it's arrival, but you can't know when within that 1 tick it'll actually be executed. Again the delay should have no effect on people trading on timescales larger than about 2x the tick, but trading on timescales less than the tick becomes infeasible.
That should smooth out the noise caused by high-frequency trading without seriously impacting things for anybody who's not trading on sub-second intervals. And it avoids the whole quagmire of trying to ban every different way of doing high-frequency trading and seeing the HFTs try to find loopholes and methods you haven't banned yet by simply setting a time resolution for the exchanges below which everything's just random noise.
the 'flash crash' had impact far beyond just the NYSE. Crude oil dropped $3/bbl in a very short period of time and rebounded. With commodities markets as leveraged as they are it would have been quite easy to rake in massive amounts of money if you knew the incident was coming
All this HFT stuff is zero-sum, if someone makes $10 on HFT, someone else loses $10.
HFT is a market parasite at this point and, IMO, ALL quotes should have a randomly induced delay between 0 and 1 second (with the delay being DIFFERENT to different participants), to eliminate the advantage of high frequency trading.
Test your net with Netalyzr
Either you trust them and you play their game, or you don't and you find some other way to invest your money. It would seem that generally the clientele are pretty pleased with their results.
The reason the stock market goes up is because more and more money goes into it. The reason more and more money goes into it is because governments around the world give preferential tax treatment to 'investments' in pension plans and the like, so people keep putting money in there in the hope that they'll get more back that way.
So, as usual, the root cause of the problem is the government funneling money into the markets through artificial incentives. Eventually people will start to realise it's a scam and stop throwing money away so that bankers can buy their third Porsche.
HFT 99% of the time is actually a healthy process - it allows relative mispricing to be quickly corrected and gives investors a chance to trade at prices which are fair whenever they come in to buy or sell.
HFT firms don't care if the P&G share is fundamentally worth 60, 5 or 100$: they are only concerned about the relative value of the stock versus other financial instruments. If for example you know that product X has a robust 1:1 correlation with product Y (for example cash vs. future), and one goes up while the other goes down for no apparent reason, the HFT guys step in and immediately buy X and sell Y until they are realigned. This will usually be for small volumes because such discrepancies don't last for long and are due to people who just aggressively buy or sell one of the products.
Simply put, HF trading firms inject liquidity into the market, they allow thousands of investors to be able to buy and sell literally any financial instrument with a relatively predictable cost (the bid/ask spread).
The real danger IMHO are the "high volume" traders, whether they are hedge funds who take directional bets or some large bank with dubious moral values. These guys will look for markets where they can push the prices up or down using sheer volume. I see this quite often on stocks which are prone to takeover rumours, "large buyers" or "large sellers" suddenly step in and start buying or selling anything they can get their hands on - all market makers panic and think that there is either something they don't know or someone who has insider knowledge. Once things settle, they calmly sell away their new position to other investors (the trend following sheep) who step in to trade on the unknown rumour.
The guys have some skin in the game. From their website: Nanex is the creator and developer of NxCore, a streaming whole market datafeed.
Their recommendations are moot too:
> 1. Quote and trade data must be time stamped by the exchanges at the time it is generated. This will ensure delays can be detected by everyone.
This data is already timestamped by all US exchanges. Data delays are mainly due to line delay and application delays that can be measured quite acccurately. Not many people do it properly though. Many does not look at it at all.
For individual long-term investor (5+ years) it does not matter at all anyway.
>2. Quote-stuffing should be banned.
Exchanges have already requirements for designated liquidity providers to quote at NBBO for X% of time (where generally > 90%). Presumably these providers receive some benefits for doing so (lower rates etc.) Do you think it is reasonable to force other people to buy and sell at a certain price? What if retail investor wants to sell his 100 IBM shares at $110 when market is $100?
> 3. Add a simple 50 millisecond quote expiration rule: a quote must remain active until it is executed or 50ms elapses.
Why exclude Alaska/Hawaiii? What if US citizen wants to trade from Baghdad Green zone and it takes 500ms to get signal there via sattelite link? Should we change thange the rules to accomodate that? Or should we allow people to compete by improving their systems?
The flash crash (and high-frequency trading in general) is really only symptomatic of a deeper underlying problem - the modern stock market has no fundamental reason to exist. When the concept of stocks originated, it was a way to own part of a company. Companies paid dividends, and so if they did well then they sent you (the investor) a check in the mail. In that way, stocks could be thought of as investments where the payoff was receiving profits in the mail.
But the stock market has changed into something different and really bizarre. Everybody knows that a company's profits and stability are what drive its stock prices. But why is that? Although a few stocks pay dividends, these days most don't. And to the common investor who holds a small percentage of overall shares, I don't think there's any easy way to _force_ dividends out of a company you own stock in. In a theoretical sense I could buy up enough shares of the company to force them to pay me dividends, but that's not something the average investor can realistically achieve.
That means that the the only payoff possible from my stocks is the money I could make by selling them. This is really strange if you think about it. If I'm never going to see significant dividends from Google, so their financial success or failure should have no underlying reason to affect stock prices. If the ONLY thing stocks are good for is selling them to somebody else, then they have no intrinsic underlying value. They don't pay dividends. I can't take them over to Google HQ and say "here's my share of the company, I'd like to take this office furniture now."
It's like the stock market has changed from a way to invest in companies and share their profits to some strange cult where everybody's drinking the kool-aid and the only people winning are brokerages. People put their retirements, their life savings, into something that has no intrinsic value whatsoever. It seems like the market is essentially dependent on an having ever-increasing influx of new buyers, like a sort of giant distributed pyramid scheme. To me, it's not a question of _if_ the stock market will collapse completely but more a question of _when_. Nothing logically inconsistent can endure forever.
---- I'll take you in a Hunt deathmatch any day.
FTFA:
"Add a simple 50 millisecond quote expiration rule: a quote must remain active until it is executed or 50ms elapses. If the quote is part of the NBBO, it may be improved (higher bid or lower offer price) at any time without waiting for the expiration period. "
Um, 50ms is not a humanly realistic decision span. It takes longer than that to recognize the color of the arrow pointing in the direction your stock price is going.
HFT is the epitome of arbitrage, and competed directly with human (or flesh-and-bones) trading. They should play by common rules that are at least realistic for both. This means that quote expirations should be measured in full SECONDS, not milliseconds.
This will, of course, destroy the obvious and worst advantage HFT has, that is the millisecond response to arbitrage opportunities. But it will allow at least the dedicated human trader an opportunity to participate in a market they are now just being beaten to by a machine.
We wouldn't allow a professional baseball team to use a mechanical pitching machine instead of a real-life pitcher. Dialing the speed up to 150MPH wouldn't enhance the game, and would instead overwhelm human batters with little hope of success no matter their skills. the game is INTENDED to be played between humans, as a test of skill and determination.
Stock trading should, in my humble and entirely uneducated opinion, be a game played among relative equals. HFT is breaking this in a way that is not useful to the market, does not add value to the capital stocks being traded nor the companies and shareholders represented, and defeats even the most concerted efforts by human traders to participate on even a marginally equal footing. No amount of analysis or even reaction by a real life trader can survive head-to-head with an HFT system.
If the purpose of the Stock Market is to offer opportunities to profit from purely technical conditions, even to allow profit from malfunctions, then HFT fits right in. But if the Stock Market is intended to provide opportunity to raise capital, develop value, and gain profit for those with good decision-making skills and insight, then HFT is an 'unfair' advantage to machines at the expense of all other players.
It is also, clearly, dangerous, and can cause significant disruption as well as loss to other players, in circumstances that are not related to actual market or economic conditions. A 'simple' delay of a few milliseconds in system response can result in the feedback loop observed in this case, and that is not useful to the markets. Quote stuffing is pure fraud. In fact, much HFT activity borders on fraud, as it is intended to deceive other players. Yes, it is. Looking for the arbitrage opportunity on a millisecond scale is nothing but machines battling machines. The only realist hope is to catch one in a moment (a definitely short moment) when something is not working right, and score.
Yes, HFT programs work to catch opportunities, and generally do with no great risk to the overall market. But in my naive opinion, this is not really useful to the market, nor the national economy as a whole.
Perhaps I should be asking the other question - does the Stock Market now serve the economy in a beneficial way, or is it now the province of software and arbitrage, primarily serving those who seek to take advantage of even momentary mismatches of pricing? And should it be permitted to continue down this road to the point where it is no longer feasible for a human trader to participate in short-term trading?
My investments are in mutual funds that exercise restraint and hold stocks for longer periods. If I were buying single issues, I would be planning on holding them for years in most cases. And I would feel genuinely cheated if I happened to choose to sell a stock at the same time as the market suffered a purely technological dip, and I was faced with a 30% drop in value for no reason other than the NYSE was queueing quotes for 20ms longer than normal.
Clearly, I have a naive view of the market. I think it should serve some purpose other than it seems to now. I know.
deleting the extra space after periods so i can stay relevant, yeah.
I'd never heard of Zero Hedge. Well it appears the guy who run it (at least according to Wikipedia) is banned from working at hedge fund companies for insider trading. That leads me to question the neutrality of his position. Seems like he's the kind of guy who wants any unfair advantage and as such may argue to keep those in place.
Not to cast scorn on what was obviously a very scary thing for many a day-trader, but for a long term institutional investor, this was barely a blip on the radar. REAL crashes happen for a reason, and you can almost always see them coming, at least in hindsight. Looking back at 2008 and 1929, there's a very clear pattern that indicates exactly what is wrong, and how the likely result came to be. They still can't even figure out what happened this time, and a day later, it's hard to tell why it really matters anyway.
In a REAL crash, there will be an underlying reason. Prior to the crash, a lot of smart people, investing over the long term, will recognize the signs that the stock market is about to become a bad investment, and will either stop investing in it, or it appears to be cost-effective, sell off some of their more risky holdings, and re-invest in something more recession proof. This is what starts the decline (which happens over a period of months). Then suddenly, one day, when everyone really gets all panicy, it'll start dropping more rapidly, the media gets in on the fun, etc.
The May 6 crash, on the other hand, had less to support it. Sure, there was that whole mess in Greece, but they're hardly holding up all the world markets anyway. One possibility is that someone purchased a huge amount of stock at significantly below market price. Since the ticker prices we are all intimately familiar with are little more than the last price something sold at, if enough consecutive trades go through at that same price, it will have the effect of suddenly dropping the market price for everyone.
For your normal investor who ISN'T hitting reload every 10 seconds, they aren't going to notice this. However, the computers will. And everyone who has put in automatic sell orders will get triggered, and those shares will also sell automatically at whatever cost someone will pay for them. Seeing a sudden sell-off, any automatic purchasing might also be temporarily put on hold, increasing the downward trend.
Now, some savvy investors, both long-term and short-term, realizing there is no sensible reason for the selloff, will see this as a rare buying opportunity, and will rapidly start purchasing shares in huge quantities. This will quickly cause the market to correct itself back upwards, until it gets close to where it was supposed to be. The problem solved itself. The REAL problem was investors who put in automatic sell orders, as if the stock in a stable company is going to plummet so rapidly that they won't have a chance to escape before the company goes bankrupt. Check some of the stock graphs for Enron and Worldcom. Those companies both went to pot practically overnight, but even then, you had months to watch the stock price fall, and could sell off at any time you wanted. Automatic sell-offs are just asking for trouble, and are just an excuse to not pay attention to your own money.
-Restil
Play with my webcams and lights here
The "analysis" fails to account for what was happening in the currency markets - specifically the USD/JPY market that day. The events in the Forex market preceeded the equities market all day - specifically there was a huge drop in the US dollar (vs Japanese Yen) ten whole minutes BEFORE the S&P plunged. Looking only at the stock market will never let you understand what happened - it was a crisis of confidence not in equities, but rather in the US dollar as a whole.
Seven puppies were harmed during the making of this post.
Stocks are for dividends. They're not baseball cards, for chrissake.
That's how it should be, but can never be as long as dividends are taxed more than capital gains. If I'm holding stock long term, I pay a lot less to the government if the company retains earnings and aims for growth than if they pay those earnings out as dividends.
Ever wonder why the market rewards growth over all sanity, and punishes reliable stable earnings? Tax law.
Socialism: a lie told by totalitarians and believed by fools.
There's a real easy solution to get rid of flash trading and excessive speculation and get the stock market back to investing...a sales tax on stock sales. I see no reason stocks should be exempt from other "products".
All those big houses use computerized trading to game the system, remember this story?
http://www.guardian.co.uk/business/2009/jul/06/golman-sachs-computer-codes-stolen
Then some fed prosecutor (I forget who know) let it out that this was bad because "in the wrong hands" the code could "manipulate the markets".
Well, in ANY hands that means it could and was designed to
"manipulate the markets". These too big to fail places get a license to steal, to skim off billions, and when that isn't enough, they still get bailed out, loot gets stolen from everyone else and handed to them. Then they can take that loot and buy bonds and government paper of assorted kinds, get even more money, put everyone else into "debt" to them.
And having their boys in and out of the Fed and Treasury and Sec..naw, that isn't a conflict of interest...