Slashdot Mirror

http://www.nanex.net/flashcrash/ongoingresearch.html
http://www.bloomberg.com/news/2013-11-26/speed-traders-meet-nightmare-on-elm-street-with-nanex.html

Nanex is the one creating this data. They're a small boutique market data firm out of the Northern Chicago suburbs. Their data showed that someone was leaking the Fed data a few seconds early.

Just think of the expenses that could be involved if one of these programs screws up.

This has already happened. The screw-ups in question lost over $400,000,000 in half an hour.

now I wouldn't have a problem with it if it was accessible to everyone, for example if anyone could buy machine time from vm's that were all given the information at the same time(artificially arranged, wouldn't work otherwise!) at the stock exchange.

Sure you would. You might not think so, but suppose that the exchange set up a perfectly equitable system in which identically configured were made available to every firm and provided with identical market feeds all perfectly synchronized so that no single trading VM has any advantage over any other.

I would give that system about three hours of run time before you discover that:

• - Your competitor has just acquired seventeen different vms through deals with other firms and is using them to dominate the market, pushing out firms using only one or two vms,
• - Your market feed is being saturated with bogus bids and offers which reduce your ability to see real market data, but somehow one of your competitors is able to filter it all out and trade ahead of you,
• - Other VMs on the same host as yours suddenly start running at 100% of their CPU and I/O capacity, causing slowdowns for you, and
• - A mysterious and completely untraceable hardware fault causes all vms except for the first seven to experience periods of unexplained latency at key times during the day.

Just look at the tricks that players in the game are already using, and ask yourself how changing the rules is going to stop them.

At 596 miles, the speed of light is indeed 3.2ms. Add in switching delays, etc. and you get closer to 5ms, and that's assuming fiber.

The 7 ms number is an estimate by Nanex, who are pretty knowledgable about the communication networks between the financial markets on the eastern seaboard of the US. See e.g. this, which also highlights that this sort of thing is all-too-common in today's markets.

http://www.nanex.net/aqck2/4437.html

http://www.nanex.net/aqck2/4436.html

http://www.nanex.net/aqck2/4437.html

http://www.nanex.net/aqck2/4436.html

From: http://www.cnbc.com/id/101056168

"Inside a room on the top floor of the William McChesney Martin, Jr. building, Fed officials instructed reporters not to send information about that decision to the outside world before precisely 2 p.m. as measured by the national atomic clock in Colorado.

The doors were locked at 1:45 p.m., and Fed staffers handed out copies of the statement at 1:50 p.m., allowing reporters a few minutes to digest the complicated document before reporting on its contents. At 1:58 p.m. television reporters were escorted out of the room to a balcony where cameras had been prepositioned. The Fed's security rules dictated that television reporters were not allowed to speak before precisely 2 p.m. Print reporters were told they were allowed to open a phone line to their editors at headquarters offices a few moments in advance of the hour, but not allowed to interact with people on the other end of the line until exactly two p.m."

So many hacked communications channels are still possible from this. The print writers can signal the editors when making phone calls before 2pm, without talking to them. For example, the editor can instruct the reporter to call them on landline if it's a sell, or his mobile number if it's a buy. The TV reporter can wear a jacket if it's a sell, or remove it if it's a buy, so someone across the building can monitor the balcony for pre-release signals... etc.

Also, from the http://www.nanex.net/aqck2/4436.html:
"It wasn't just gold. It was everything that traded. In fact, the 1/100th of a second after 2pm was the most active 10 milliseconds in the history of the U.S. Stock an Futures markets."

This was a major, major hack, and they waited as late as they could wait, without signaling their competitors.

http://www.nanex.net/aqck2/4436.html

The link you gave provided little insight into its cause. Do you have any more information about it?

The way I see it it's about the structure of the trades. From their text above the plots:

The price dropped from $796 to $775 in about 3/4 of a second, then rebounded to $793 a second later. The drop invovled 307 trades and 57,255 shares from 10 exchanges + dark pools. During the drop, there were 5 orders placed for every trade executed (meaning 4 orders placed/canceled for every trade).

Having about 1.5k quotes posted in 0.75s (with 80% being canceled) surely shows that HFT algos were active and yet did not absorb 57k shares of GOOG (that's about 2.5% of the daily volume at the time). Which is not a thinly traded stock by any measure. It's not a conclusive proof of HFT causing a drop of 2.5% in less than a second though, but I don't think you can easily get such a proof without knowing who traded what. There is only so much one can glean from trade data at this level after all.

Due to some time constraints I'll beg to be excused if, in lieu of the rest of the comment I was going to post, I'll give you this example (originally from here but the harvard link seems unavailable) of a HFT technique that, shall we say, 'plays' liquidity to increase volatility. The problem, as I see it, is how to discourage people who have the means to use them in such a fashion. Bear in mind that HFT requires quite an investment in infrastructure and software, so voluntarily refraining from taking advantage of that infrastructure in ways that exploit its advantage at the expense of slower traders is not a believable option. However, I'll also freely state that I have little faith in simple-sounding solutions to complex problems, which is what most pro- and anti-HFT rhetoric brings. Fortunately, it is not my job to find the proper solutions :-)

Nanex presented a video that shows 0.5 seconds worth of High Frequency Trading for the stock Johnson and Johnson. Pretty revealing.

There is no evidence that HFT causes spikes and crashes. Actual evidence says the opposite: by increasing liquidity, HFT reduces volatility.

Actual evidence from the guys who monitor these kind of things says nothing of the sort. I wish people would stop spouting this line about HFT and liquidity. Here is a relatively recent GOOG flash crash (April 22 2013) likely due to HFT (based on timespan and number of trades, number of orders placed per trade and number of exchanges involved).

"Nobody lost anything except the bot herders"

False. Anyone who has a stop loss got creamed, unless the trades get backed out (which I don't think happened).

Another interesting observation is we've been told that HFT ensures liquidity. No, that is also false: http://www.nanex.net/aqck2/4176.html since the HFT algos apparently stop trading in times of trouble. So, tell me again how the liquidity is supposed to work?

This garbage needs to stop. What's wrong with forcing a trade to remain valid for 5-10 seconds (some threshold in which a human can respond to) before it can be cancelled. I trade, and I have to say I support a transaction tax as well on all *placed* trades. So, everyone pays a toll to place a trade. We could use the money to try to convince SEC to do their damn job.

While it may increase the liquidity of the market, it doesn't necessarily promote good ideas.

Society benefits when good ideas are promoted through shareholder investments. The stock market did just great for 100 years without algorithmic trading, and the intro to algorithmic trading has caused several large instabilities in just the past two years:

http://en.wikipedia.org/wiki/2010_Flash_Crash

http://en.wikipedia.org/wiki/Knight_capital

This amazing graphic illustrates the effect of this type of trading on the volumes, and ...I believe that instabilities will only get worse with this type of activity.

http://www.nanex.net/aqck/2804.HTML

As the underlying article states the growth of "Quote SPAM" is really the problem. Most of this is "testing the waters" where a quote is tossed out to see the response and immediately cancelled with no resulting trade.

Fundamentally, it is simply gambling.

I'm above 40. What does HFT stand for?

High Frequency Trading. (And apparently, a development and deployment process so agile that nobody thought to see which system it was being deployed on :)

Why would you have to retest "everything" if youre only modifying the permadeath function?

That depends. Do you want your company featured on TheDailyWTF.com or not? It's full of stories about PMs and business owners who insist that changes be made without adequate testing, and none of them end well.

When you start working with big projects, large teams, and looming deadlines, you realize that nothing is ever that simple.

If you spend an afternoon writing a program that makes the screen flash and types "Hello, World!" over and over again until someone kills it, then yes all you need to do is comment out a line or two and everything will work perfectly.

When you are dealing with hundreds of thousands of lines of code, interdependent modules and inheritance diagrams that resemble buckyballs, all written by dozens of people over several years time, things get complicated.

If you honestly think that you can make changes to a product without needing to test them then you may have a future in high-frequency trading, but I advise you to be a little more careful in truly competitive markets like gaming.

I've read about 12 studies now [...] they all conclude the HFT is overall beneficial to the market as a whole.

That's funny. I've read studies which say otherwise, and I can even cite them.

How High Frequency Trading Harms Even Long Term Investors
- Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes. This alone should tell you something is rotten at the core.
- Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
- Quote spreads are much wider and less stable during market open, which causes many micro flash crashes in individual stocks.
- Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
- Mis-allocation of resources, both human and technological.
- If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
- HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
- During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

But I'm sure these are all good things, and help keep the markets healthy. I must just misunderstand how awesome they are.

So it is possible to create a large volume of "trades" without actually ever buying or selling anything?

They're not _trades_, they're _bids_. What these algos are doing is putting out bid and offer messages and then following them up with cancel messages a few milliseconds later. Here is the Nanex data which should have been linked in the article -- You can see the bids coming up from below and the offers down from above, all cancelled before anyone has time to react to them. If you do the math, even travelling in a straight line at the speed of light many HFT bids and offers don't have time to even make it from New York to Los Angeles before being cancelled. There is no intention of ever completing a trade at all.

It's all being done for the purpose of screwing with the market, flooding the market feeds of smaller players or just messing with other algo traders who may react to the pattern of bids and offers with their own orders. Most sane people agree that it's a bad thing, but it's difficult to decide exactly what can be done to fix it.

Anyone from Knight Capital want to comment?

If you want the real story try Stucchio's blog series beginning at: http://www.chrisstucchio.com/blog/2012/hft_apology.html

My pitiful TLDR summary of Stucchio's work combined with some other observations

Stucchio presents a fairy tale, picture perfect world that doesn't exist. Stucchio forgot to mention Brutus' algo which comes along and offers 20.01 for a stock, but cancels it 1 milliseconds later, then offers 20.00 for 1 ms, then 19.99, then 5,000 random prices between 18.00 and 20.00 for the next 800 milliseconds. Sometimes at the top of the book, sometimes not. All to confuse order routers or destroy latency tables in Thor like programs.

When Brutus sees an order coming in on Exchange A, his algo quickly backs off on the other exchanges because it knows that over time, it results in slightly higher profits.

Brutus also knows that exchange B will delay by X microseconds when the total # of messages in symbols A-C exceed 20,000. So if a delay is needed in ABC, just jam the other stocks in that group with useless quotes

I could go on all night.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

From a recent post I made

How High Frequency Trading Harms Even Long Term Investors

• 1. Investors are warned against using market orders and stops because HFT can and will suddenly withdraw their quotes . This alone should tell you something is rotten at the core.
• 2. Some universities, such as Georgetown, can no longer afford to buy TAQ market data for their professors or students to analyze. This will lead to less academic oversight, guidance and involvement, as well as students who are less prepared for careers on Wall Street. Data has become prohibitively expensive because of all the excessive quotes generated by HFT.
• 3. Quote spreads are much wider and less stable during market open , which causes many micro flash crashes in individual stocks.
• 4. Misleading price quotes interferes with price discovery, one of the core functions of a stock exchange.
• 5. Mis-allocation of resources, both human and technological.
• 6. If left unchecked by regulators, traders who want to process quotes, will soon need super-computers, 10 gigabit connections and their own engineering staff to have the same basic level of trading information they needed in 2006.
• 7. HFT generated so much Quote Spam in the flash crash, that it took 5 months for the SEC to assemble the data.
• 8. During the flash crash, excessive quotes from HFTs overloaded quote data feeds, causing severe delays: stock quotes from some exchanges were behind over 30 seconds during the height of the flash crash.

Only if you are in it for day-trading profits. And if you are, well, you deserve to be beaten senseless by some HFT algorithm.

If you're a long term investor, with a time horizon of many decades, this doesn't matter. For example, I have a stock I bought 15 years ago. It has gone up by around 3X in that time. HFT makes no difference to me when I've held a stock over many years or decades. The exact microsecond it sells doesn't matter to me after a period of decades.

Let me ask you something. Would you recommend homeowners ditch their fire or title insurance then? Because you have about the same chance of getting burned when the time comes when you need to sell that stock, as you have of a major fire. Might as well save on the premiums.

Seriously. I've talked to quite a few people who have been caught up in an "event" where they had to sell (for tax reasons, for financial reasons, etc.) and they just happened to pick the wrong moment of time to trade. And there have been thousands of such "bad times" so far this year.

...automated trading algorithms losing Knight Trading $440 million two months ago [slashdot.org]? Tell me, all those protection measures and penalties, did they protect the company running the automated trading software

I know you're asking a rhetorical question; but in this case, no, probably not. Read these excellent analyses by the inimitable Nanex:

Knightmare on Wall Street
The Knightmare Explained
(and see the other links from there).

In short, it appears that Knight had a high-speed, automated order generator which they used internally to test their algorithms -- and they accidentally deployed it into Production! As long as their own algo was taking the other side of the trade, that's ok; it's all a wash. But as soon as someone else starts hitting their orders...

Unfortunately for our discussion here, it's not clear from the Nanex articles whether Knight found and killed the test order generator themselves, or whether the exchange's "circuit breakers" pulled the plug on them (which is why I said "probably not" earlier).

...or the parties who engaged with trading with the automated trading software?

The parties who engaged in trading with them did very well indeed, thank you. Where do you think those $440 million went?

...automated trading algorithms losing Knight Trading $440 million two months ago [slashdot.org]? Tell me, all those protection measures and penalties, did they protect the company running the automated trading software

I know you're asking a rhetorical question; but in this case, no, probably not. Read these excellent analyses by the inimitable Nanex:

Knightmare on Wall Street
The Knightmare Explained
(and see the other links from there).

In short, it appears that Knight had a high-speed, automated order generator which they used internally to test their algorithms -- and they accidentally deployed it into Production! As long as their own algo was taking the other side of the trade, that's ok; it's all a wash. But as soon as someone else starts hitting their orders...

Unfortunately for our discussion here, it's not clear from the Nanex articles whether Knight found and killed the test order generator themselves, or whether the exchange's "circuit breakers" pulled the plug on them (which is why I said "probably not" earlier).

...or the parties who engaged with trading with the automated trading software?

The parties who engaged in trading with them did very well indeed, thank you. Where do you think those $440 million went?

...automated trading algorithms losing Knight Trading $440 million two months ago [slashdot.org]? Tell me, all those protection measures and penalties, did they protect the company running the automated trading software

I know you're asking a rhetorical question; but in this case, no, probably not. Read these excellent analyses by the inimitable Nanex:

Knightmare on Wall Street
The Knightmare Explained
(and see the other links from there).

In short, it appears that Knight had a high-speed, automated order generator which they used internally to test their algorithms -- and they accidentally deployed it into Production! As long as their own algo was taking the other side of the trade, that's ok; it's all a wash. But as soon as someone else starts hitting their orders...

Unfortunately for our discussion here, it's not clear from the Nanex articles whether Knight found and killed the test order generator themselves, or whether the exchange's "circuit breakers" pulled the plug on them (which is why I said "probably not" earlier).

...or the parties who engaged with trading with the automated trading software?

The parties who engaged in trading with them did very well indeed, thank you. Where do you think those $440 million went?

The Fool has a great article on this. You simply can't compete. http://www.fool.com/investing/general/2012/08/10/the-terrifying-graphic-that-shows-stock-trading-r.aspx

Wrong. Read the whole article you linked to... this chart shows high-frequency quoting, not trading. There's also have a link to the people who created the chart, which is worth a read.

Actually it has very little to do with fees. HFT results in a very high number of orders, but very few actual trades. Recent research by Nanex actually shows that there are less trades today than there were a few years ago. Fees are based on trades, not orders. So your entire thesis is wrong. But hey, don't let a complete misunderstanding of the problem get in the way of pontificating at length about it.

The fundamental issue with HFT is that we've taken strategies that were designed for human traders, and allowed them to be used by computers with nanosecond response times. The strategies used to be good ones, designed to advantage the broker who had a presence on the trading floor. Do they make sense now? Probably not. These strategies have the emergent feature of encouraging faster and faster trades. Even before computers took over we saw this in the mobs that used to form on trading floors, and the increased load on the floor brokers. Back then, it didn't matter too much, because the fundamental limit was the reaction time of a human being.

Now, it's not. These systems are designed to be too fast for a human to monitor, so, more and more, humans are being taken out of the loop. This leads to the insanity at Knight. All their many, many previous deployments had gone off without a hitch, why would this one be any different?

Disclaimer: I work at a stock exchange

Someone pointed me toward this article on another /. thread about the incident. Apparently, their testing program got packaged in with the code when it was deployed, and it's that test program that wreaked havoc.

The Knightmare Explained from Nanex Research: http://www.nanex.net/aqck2/3525.html

I can just see the the sysadmin "ohshitohshitohshit, unplug it unplug it now!"

And that's what happened, Three different times. Take a look at this graph of trading volume on NYSE (Taken from Nanex's detailed analysis of market activity during the Knightmare on Wall Street). After the market opened at 9:30 the trading system immediately started going berserk. Eyebrows were no doubt raised and meetings quickly called, but it wasn't until 9:48 that the decision to shut down the entire system was made. There's a gap at 9:49 when Knight goes offline and then back up at 9:50 when their insane trading pattern resumes. At 9:52 the "ohshitohshitohshit!" faction regains control and shuts everything down again, but when it comes back at 9:54 nothing has changed. There's a brief pause where everybody heads back to their desks to get their emergency trousers and then the test system is finally killed, staked and put out in the sunlight by 10:00.

So the lessons here are these:

1. Don't schedule code promotions for after last call.
2. Just because your monitoring and tracking system doesn't report something doesn't mean that it isn't happening.
3. If you work in Capital Markets, always keep an emergency kit including clean pants in your desk. You know you're going to need them, it's just a question of when.

Wow, I'm having a hard time either a) explaining the issue or b) getting people to take my reports seriously.

The issue (for me) is that orders are getting placed with no expectation of them getting filled. Is this so hard to understand?

From a comment at http://www.ritholtz.com/blog/2012/08/nanex-knightmare/:

"The point is to submit a large number of false trades that you have no intention of making until other traders start to notice them, then take advantage of their reaction. It works, because too many other actors in the market do not have enough time or information to recognize the pattern of bids and counterattack. More simply, it works by making markets less efficient to introduce an arbitrage opportunity.

Right now only a handful of specialized firms are seriously trying to exploit this, but as more money is made, I expect a lot more players to buy in. Eventually, there will be enough high speed action to make it a less effective strategy, but in the interim I’m expecting a lot more meltdowns."

If you quibble, please read http://www.nanex.net/aqck/2977.html and refute their argument.

It is surprising that the best article on KCG's trades was not shared yet. Check out http://www.nanex.net/aqck2/3522.html for a fascinating look at the trades. Chart 3 shows a millisecond interval chart with around 40 trades happening in under a second, each buying at the offer and immediately selling at the bid. As Nanex put it "you now have a system that's very efficient at burning money".

What's wrong with HFT? Well, apparently, HFT traders taking out the stream of pricing data available to non-HFT individuals by spamming the market with order cancellations and then using the fact that, because their expensive premium pricing data streams weren't affected, they had prices that were several hours more up to date than everyone else to make bank. Amongst other things.

http://www.nanex.net/aqck/3099.html

Nasdaq was totally F*d when Facebook opened, and yes it is thought that tons of HFTs pouring in caused this. What effects not being able to sell or buy correctly on opening day had on the stock are unknowable. It might have tanked faster, or it may have went up, but when the market acts spooky traders get spooked.

Now, personally, I do believe the facebook stock was a turd ready to tank and any position long held long enough would lose money, but there were serious execution problems on opening that could have very well caused his losses.

And we do have an idea of what's actually going on. Here's a detailed example of the recent Facebook IPO problems: http://www.nanex.net/aqck/3099.html.

Oh I read the comment. I just felt you framed it in far too broad a time. By hand, I can reverse a trade in a minute, or even 2 or 3. That isn't HFT. HFT is more like 30,000 trades in a minute. There are instances when HFT has gone so fast that they have executed trades on quotes that didn't exist until the future: http://www.nanex.net/Research/fantaseconds/fantaseconds.html

It is well known that the algos read news themselves, without human intervention.

However, the main argument in support of HFT is not "synchronization", but rather an increase in liquidity. Pro-HFT people claim that HFT fulfills a market-making function where it matches up a buyer and a seller very rapidly, while pocketing a small fraction of the bid-ask spread. They don't tell you that they use e.g. quote stuffing to artificially increase this spread, thus earning more money. Have a look at this graph from Nanex (they do a lot of very interesting analysis of HFT algos). Note that the entire width of the graph is 200 milliseconds.

Notables:

Set a record number of trades in 1 second: Over 12,000.
Over 1.5 million trades by end of day. SPY - often the most active, rarely breaches 1 million/day.
Nasdaq quotes went radio silent for 2 hours, but traded just fine.
Many other stocks were affected right before and after it finally opened (AAPL, NFLX, INTU for example)

Check out some of the whitepapers Nanex releases on what they see from all the market data.

Nanex Analysis

Trading is all about the banks taking out the stops.

And the HFT algorithms, in order to do so, deliberately manipulate the prices of shares, by stuffing the channels with quotes for bids and offers that the algorithms have no intention of executing.

Under long-standing law, any pattern of orders intended to manipulate the price of a security, as opposed to actually buy it or sell it, is illegal.

Where are the damn COPS? Where are the indictments, and the handcuffs, and the perp walks? Why aren't there bankster fraudsters currently doing time in federal PMITA prison for this?

Of course, you know the answer. The regulators have been co-opted. The government is looking the other way. And the "little guy" gets screwed. Again.

For more information, you'll want to look at Nanex.net's "Flash Crash Analysis" page.

Just from some quick Googling:

100% perfect trading streak

61-day perfect run

There are some in-depth analyses of the "Flash Crash" recently, for instance at:

Flash Crash Analysis

There's lots of gaming going on with high frequency trading, or really high frequency price pinging, bids and asks which are tossed out and canceled to simply mess with the quote queues. High frequency algorithms can flood the queues to get artificial imbalances and quote delays. There might even be some arbitrage possibilities based on differences between different quote systems time stamp transactions. Some timestamps are the time of the quote when queued, and others are the time the quote leaves a queue. This can lead to price inversions or other information queuing distortions.

According to Eric Scott Hunsader, the founder of Nanex the Chicago data firm that first identified strange patterns, "This surge in orders may not have been intended to cause the general market rout. Instead, it may have been intended simply to slow down some markets so that traders could profit by arbitrage with other exchanges."

There's way too much potential for gaming the queues if there is no cost to fake a bid or ask. When the cost is zero you get the same thing we have with spam email. If email cost a fraction of a penny to send, spam would drop drastically. If bids cost a tiny amount and were forced to remain open for the time a bid could electronically circle the globe, then that small bit of friction would eliminate many of the system's instabilities. And, all price queues should use the same time-stamping method.

Here are a few good links to more information:

http://dealbook.blogs.nytimes.com/2010/09/27/troubling-trades-found-ahead-of-flash-crash/

http://www.nanex.net/FlashCrashFinal/FlashCrashSummary.html

http://www.thestreet.com/story/10876642/4/the-5-dumbest-things-on-wall-street-oct-1.html

Unlike Reuters "fat finger" trader theory, the folks at NANEX (who provide the data feed for the exchanges) actually looked at the data.

Their original report is quite fascinating to read.

http://www.nanex.net/FlashCrash/FlashCrashAnalysis.html

Before you make snap judgements ... take a look at the root cause.

Sorry, but RTFA. These trade "offers" aren't genuine offers at all. They are extraneous noise introduced into the system. And, people are only speculating about the reasons.

Hell - it's remotely POSSIBLE that some of these algorithms run only because some geek likes looking at the video output! Looky the pictures: http://www.nanex.net/FlashCrash/CCircleDay.html

Alright, so I don't really think for a moment that some autistic nerd does this just to look at the pics. Maybe the noise IS only there to make the competition's algorithms work a few nanoseconds slower. Or, maybe someone is playing with ideas to manipulate the market, and these are dry runs for practice. OR, maybe an outsider (like China) is already set up with a New Jersey or Manhattan data center, and they are already manipulating the market in ways that we haven't detected.

Whatever - it doesn't look to me like this should be permitted.

Areed. I spent 3 months trying to find a good solution. If you've got buckets of money, NxCore ( http://www.nanex.net/NxCore/NxCore.htm - prices start at $500/month) and any of the brokers that support a FIX API (for which you can expect to pay a hefty fee, too; Interactive Brokers (IB - http://www.interactivebrokers.com/ ) for example charge a one time $500 fee, OANDA ( http://www.oanda.com/ ) charge $600 for the first two months then an ongoing subscription fee if you trade $12mil/month or something).

For those people not wanting to pour money into it, as good as you can get is Interactive Broker's Trader Workstation (TWS), and JBookTrader (http://code.google.com/p/jbooktrader/) or a custom trading platform that talks to their API. TWS is a pain that lacks automated login (for security reasons) and auto-exits every 24 hours (for... err... security reasons?), but it gets the job done. Data feed can be an issue still, though; IB offer up to 100 symbols at a time, and a basic historical data service, but some people dislike the fact they drop price ticks during busy market times (over 10 prices per second) and the historical data service is paced so you can only do a limited of number of requests (about one every ten seconds I believe).

In short though, AC is right; use Windows, it may well be less painful. Really.