Market Data Firm Spots the Tracks of Bizarre Robot Trading

jamie spotted a fascinating story at The Atlantic about "mysterious and possibly nefarious trading algorithms [that] are operating every minute of every day in" the stock market: "Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there's no way they'd ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants." Spotting the behavior of these bots was possible by looking at much finer time slices than casual traders ever see — cool detective work, but as the story points out, discovering it is just the beginning: "[W]e're witnessing a market phenomenon that is not easily explained. And it's really bizarre."

Comments

Here's an explanation for you:

[Pojut]

The "market" is a fucking scam.

There, that wasn't so hard, was it.

Re:Here's an explanation for you:

[eldavojohn]

The "market" is a fucking scam.

There, that wasn't so hard, was it.

Well, in the article they say that one firm's explanation is that high frequency traders are injecting quotes into the system because they know about them and don't have to sort through them when they are posted ... but their competitor's bots have to look at that data and sort out the real data that are actual useful quotes instead of the outliers which are quotes that will never be taken.

So scam is close but spam might be a better word for this.

I also get a kick out of how periodically in this article they remind us that high frequency trading is good for the market and these people that don't do anything that act as middle men are actually good for the market because they up availability or "eliminate inefficiencies" (that's my favorite). And they're all taking money out of this magical unending bucket of cash ... quant funds and high frequency traders are so 1929 I don't even know where to begin.

Re:Here's an explanation for you:

[vlm]

I also get a kick out of how periodically in this article they remind us that high frequency trading is good for the market and these people that don't do anything that act as middle men are actually good for the market because they up availability or "eliminate inefficiencies" (that's my favorite)

Maybe they started with an intelligent explanation that seems to fit reality, like we're watching a very confrontational version of simulated annealing among multiple competing firms using real money, but you run that thru the "english to journalist" filter and get the gibberish you describe. You have to realize journalists are the guys that flunked out of Calc I in their freshmen year and then spent the rest of their schooling drunk or stoned, as gatekeepers to the masses they are always going to be epic fails.

http://en.wikipedia.org/wiki/Simulated_annealing

Its fairly perceptive to note that journalist style gibberish is often used by people trying to scam. There are plenty of (often self serving) religious / philosophical arguments that claim markets are always scams, etc. Need to very carefully consider cause vs effect and correlation vs causation or else you just send up with cliche instead of insight.

Re:Here's an explanation for you:

[omnibit]

I'll probably be modded down for being counter consensus but so many delight in crying foul when they don't understand a concept.

The markets bring together buyers and sellers (who would have thought!). It just so happens that a group of math and programming whizzes know how to capture the minor fluctuations in market sentiment.

Human day traders (attempt) to make a living out of playing the bid/ask game but usually their volume is so minute that it has almost zero bearing on liquidity. Markets need liquidity to avoid gapping - when spreads become large and there is a disconnect between buyers and sellers. Volatility is exacerbated by a lack of liquidity.

As for the scam nonsense, attempting to profit on capital markets is a perfectly legitimate form of business. No body was swindled. Most people cannot program for HFT and therefore think it is hocus-pocus, with amoral corrupt businessman profiteering at the 'expense' of the rest of us. All they're doing is capturing volume at a faster rate than human traders. Most investors aren't interested in short time horizons - they make investment decisions independent of how quickly they can turn a stock - they simply limit a price, form an expectation and sell when they deem fit (e.g. for the price hits a target, some event happens, etc).

If HFT are spewing out thousands of orders a second - let them. It's up to other HFT to adjust their parameters, or the exchanges to limit the number of orders to keep server integrity.

Despite popular belief - the markets don't function as casinos (though exceptions remain, a la China). Sentiment and expectations do run away, we only need look back at the 2008-09 crash. But that per se does not indicate a casino like behavior - it just means few people ever believed the world economy would tank as hard as it did.

The excoriation of HFT might be fun, but it's all for nought. They're making money and you're jealous. Tall poppy syndrome reigns supreme.

Re:Here's an explanation for you:

[Anonymous Coward]

Lobstah man gets pissed off and says Well, OK, that's all very confusing or interesting or both, probably to try and rip me off, but how far away am I from the damn pier, two hours or three hours?

And this is where a trader would figure out that the current price of lobster was $4 a pound, the boat carried 1,000 pounds of lobster, and the lobsterman is 2 hours and 37 minutes from the dock. The trader immediately buys a put option to deliver 1,000 pounds of lobster at $4 a pound. One minute before the lobster boat docks, the trader begins to execute the put option, driving down the local market price for lobster to $3 a pound. The trader meets the boat at the dock, purchases the lobster on the boat for $3 a pound, and completes the execution of the put option by delivering the lobster to the lobster pound. The lobsterman makes $3,000 and the trader makes $1,000.

In the real world, the lobsterman takes care of the problem by using the trader's lifeless remains as lobster trap bait. In the financial world, the trader is hailed for discovering inefficiencies in the lobster trading market, and receives a hefty bonus at Christmas.

I mostly agree! But let's soften it a little.

[Petersko]

"The "market" is a fucking scam."

I think I'd prefer to say that the market has a purpose, and that purpose has absolutely nothing to do with maintaining wealth for the casual investor. Once you abandon the idea that the market gives a damn about the solidity of retirement accounts or the portfolios of the masses, then it's easier to accept that the purpose of the market is to move money around and around in a big circle, while slowly siphoning it off into the pockets of particular groups.

Stocks are a massive game of hot potato. Whoever is holding the stock with the game is over gets burned.

I say it's not necessarily a scam because it should be clear to anybody looking in that this is how it works. Like the rake at a poker game, if you wait long enough the house has all the money. This fact isn't hidden - you just have to wake up to the implications.

Re:I mostly agree! But let's soften it a little.

[gtbritishskull]

I disagree. As the parent said, the job of the stock market is to marry capital to seekers of capital. A retirement account is a collection of capital that you want to make money on. If you can afford the risk of the stock market (you have enough time for variations in the stock market to even out) then it is the best place to put your money because you will get the highest return. But, those people "nearing retirement age in 2008 and 2009" that "watched their retirement plans go out the window" were stupid. If you are about to retire, you should have a large percentage of your money in bonds. Because, you can't afford the risk of the stock market. That shows that their retirement accounts were mismanaged, not that the stock market for some reason should not be used when saving for your retirement.

Re:I mostly agree! But let's soften it a little.

[CodeBuster]

The value of a share of stock is derived from it being a share, albeit a small one in practice, of ownership in a business. The price of that share, in the long run, will reflect the proportional value of the benefits that would ordinarily accrue to the owner of that business. It is very easy to see why shares in a viable business, however small individually, are NOT worthless. Suppose, for example that the "worthless" shares of a viable and profitable business were selling for $0.01 per share. Don't you think that someone would come along and buy up all of the shares at that price? Even if the buyer's only intent was to liquidate the company and pocket the resulting profits he would still be interested in buying the outstanding shares at that price because if he acquired control of the company, perhaps by becoming the 51% owner, then he could force that kind of liquidation. This is why the long term share price in the marketplace tends to reflect the true present value of the underlying business. A share of something is worth something; It is not worthless. Now in the short run people can and do play psychological games in the marketplace which is why the moment to moment price of a stock is essentially random. However one must not confuse the result of individual games (i.e. I buy and you sell; game finished) with the iterated version which is played continuously for years, decades and even centuries. The individual stock investor does best by doing his homework, looking at the qualities of the business that cannot be feed into a short term computer trading algorithm, and then investing for the long run. This practice has very little to do with gambling.

Gamble all you want, but try to avoid spreading the lie.

This one gets thrown around a lot here on Slashdot, where the investing (particularly stock market investing) == gambling meme is often taken for granted. However, this analogy, like most, is a rather crude approximation of what is actually happening when one invests. If you are interested in a more in-depth treatment of this subject, there is an excellent essay on investorguide.com which covers this very topic, investing vs gambling.

Re:I mostly agree! But let's soften it a little.

[Black Parrot]

The "scam" here is the massive one where America thought the purpose of the market was to provide retirement savings- Thus people dumped all their money into the market in hopes of having big retirement payouts.

Various replies disagree with you, but it has certainly been marketed to the public as a "Make Money Fast" game for ordinary people.

Look at the surge in the DOW since the 90's- that's everyone's retirements going straight into the market. You know how many people nearing retirement in 2008 and 2009 watched their retirement plans go out the window?

And here's the real motivation for all those Republican politicians who want to "save" Social Security by moving the money to the stock market. A sudden two-trillion dollar flood of money inflates share prices, the savvy rich people cash out, the correction hits, and the savvy rich people use their inflated profits to cash back in. The people who will actually need Social Security when they retire get left holding the empty bag. This privatization plan, like all others, is just a scam to move ordinary people's money into rich people's pockets.

Nope, it's right on

[RingDev]

I read an interview a few weeks ago about these trades. When we're talking about the majority of all stock trades being done by these incredibly fast bots, where people are looking for every possible advantage, there are many tricks. One of them is to flood out a huge quantity of bogus bid/sell offers in sufficient enough bulk that it may cause your competition's bot to slip a few micro seconds. Just enough for your own bot to snipe a fraction of a cent advantage.

If you are interested in the 'Cyber-War'. Forget China, head to Wall Street.

-Rick

Re:Nope, it's right on

[countertrolling]

If you are interested in the 'Cyber-War'. Forget China, head to Wall Street.

Why should Americans have all the fun? Could be Chinese bots... I hear they like money, also...

Re:Nope, it's right on

[sumdumass]

Lol.. Are you sure it's to cause your competition's bot to slip a few micro seconds? Or could it be someone who simply wants to avoid the lag of checking the prices before participating in a transaction so he simply sets the bots to always submit the preset buy / sell limits and if it's in range, the trade is accepted, if not, it's simply rejected.

That would shave more then a few microseconds from the competition compared to attempting to bog them down which could also bog their transaction down at the same time.

Re:Nope, it's right on

[lgw]

This looks far more like steganography. There's a lot of data in these patterns, and it has nothing to do with trades. Someone is communicating using a very odd channel!

Re:Nope, it's right on

[inKubus]

It is that, but they could just be trolling as well. In a market with billions of shares and millions of actors, there are bound to be mistakes and typos. Someone puts an autocommiting ask out for a stock with an extra zero and there's no one looking there, no problem. But if there's an electronic bot trolling that price range, it can lock the order before they have a chance to retract. Now there are rules and appeals to mitigate this, but if it was just a small thing then it might be overlooked. Of course, the obvious thing is annealing, like you mentioned.

Free Market = good; Capitalism = Usury

[spun]

Usury is the sin of lending money for unfairly large amounts of interest. Capitalism is an economic system of lending money for as much profit as possible. Capitalism makes labor subservient to money. It lets people expand their power over others, not by working, but by lending. This unfair adjudication of risk and reward, and the subsequent consolidation of power into fewer and fewer hands, is why many religions, at one time or another before the rich took them over, considered usury a fairly serious sin.

The rich do not have to work to earn a living, they just sit back and let the money roll in. Supposedly the return they get is for the risk, but there is no risk involved. The rich can buy politicians, laws and experts who, in practice, reduce the risk to near zero. The average investor faces at least some real risk, but not the truly wealthy.

Re:Free Market = good; Capitalism = Usury

[OakDragon]

Capitalism is an economic system of lending money for as much profit as possible.

Capitalism is more properly defined as a system in which the means of production are privately owned (as Wikipedia has it). It may have some aspects that conform with your definition, but that's not the whole of capitalism.

Re:Free Market = good; Capitalism = Usury

[dcollins]

"I'm not religious at all, but even I know this is the 'correct' interpretation."

This sounds like total bullshit.

The article you link to doesn't say anything like this. In fact, it says the opposite in the second sentence:

Usury... originally meant the charging of interest on loans. This included charging a fee for the use of money, such as at a bureau de change. [Wikipedia, "Usury"]

Wow, that's better

[Itninja]

WoW has rules against using scripts, bots, and 3rd party programs to play for you. Failure to abide by the rules get you banned.
The stock market trading system has no rules against scripts, bots, and 3rd party programs to buy millions Every time I think about how WoW regulates the artificially increasing of fake wealth while the stock market has no regulation regarding the artificially increasing of actual wealth, I die a little inside.

Re:Wow, that's better

[trout007]

You are exactly right but you are missing the point. There are no government regulations on WoW. It's operates completely to make money by pleasing it's paying customers. If they don't self regulate the game and the players don't consider it fun anymore they stop paying. The difference is the Exchanges are regulated by the government. So they only people they have to please is the SEC. Without government oversight they would have to operate more like WoW and pay attention as people abandon their markets that they consider rigged.

Also the government almost forces people into the stock market through tax laws. If the government didn't continually devaluate the dollar you could just save your money in a bank and you wouldn't lose purchasing power. If you keep your money in the market in a brokerage account they tax every dividend and profit you make. So they set up IRA's and 401k's to lock you into the stock markets. All so their powerful friends can leech off the hard work of millions of people.

Emergent Behavior

[PIPBoy3000]

I suspect that a fair amount of this is emergent behavior - complex patterns from simple rules. For example, if two bots are making test purchases of a stock, one penny greater than the last buy, up to a fixed, you end up getting these odd patterns. The two programmers may not have planned the interaction at all, though they have these weird Game of Life sort of patterns in the data.

Re:MUDs and the Stock Market

[Wonko the Sane]

Exactly.

Facinating

[m.dillon]

It looks to me like the orders are trying to match against dark pool bids/asks, and/or all-or-nothing bids/asks. Another possibility is that they are trying to extract non public information from the trading system by purposefully loading the system down and timing responses.

High frequency trading bleeds money away from institutional investors (by sussing out dark pool bid/ask levels) and from market makers (by stealing ETF rebates for volume). Also, most brokerages use fairly simple algorithms to handle market orders which can be sussed out by the more sophisticated algorithms used by the HF traders.

None of this will really effect the retail investor, it amounts to a penny or less on some transactions. Frankly, people have it easy these days where the bid/ask spread is a single penny. When I began trading in my late teens the bid/ask spread was in fractions and was considerably more than a penny. Retail investors get much better pricing these days.

-Matt

Free Market Checklist

[copponex]

You have proposed a solution to introduce more accountability, transparency, or ethical considerations into the free market. Wall Street will not accept your proposal because your solution:

(x) reduces profits gamed from the current flaws
(x) introduces accountability
(x) introduces transparency
(x) introduces ethical considerations

Re:The reasons are actually well known

[hamburger lady]

problem is, since every other large-scale HFT algorithm does the same thing the benefits are lost. of course, they all have to keep doing it to keep the new equilibrium going.

why hasn't this whole market fallen apart yet?

High Frequency Trading Should Be Banned

[careysub]

In the absence of sensible regulation there are many abuses of the "free market" that effectively destroy it and turn it into a rigged game to benefit the already rich and powerful. Monopolies. Cartels. Price fixing. Trading on one's own account ahead of a customer.

These special access high-speed connections to the stock market exchange are market fixing tools, pure and simple. They allow the trading firms to skim the market for their own profit, thus defrauding every market participant in the world who lacks these powerful and privileged tools.

Requiring all buys to be held for a "long" time (a minute?, an hour?) would kill a lot of these shenanigans. Also requiring the link to go through a regulated buffer that introduces a random delay of a second or so would also take the wind out of their sales (pun intended). Or maybe we just impose a fee on each transaction so that they aren't free. Sub-millisecond trading loses a lot of luster if you automatically incur a charge equal to 0.1% (or something) of the stock's value.

Re:Failover testing

[hvm2hvm]

Yes, I am wondering the same thing, here's a quote: "And it certainly gets at a central mystery surrounding them: if trading firms aren't sending out these orders, how are they getting into the market?"

Is there a server with a simple API that receives these quotes or WTF is going on? Can I just send some packets to the server and have my quote put up? How can they not know who is sending the requests?

The whole article reminds me of those documentaries on discovery that show you something simple like a cloud that looks like a giraffe and they keep asking "is this just a cloud or is there something that we don't understand about the giraffe cloud?"

When is the game over?

[snowwrestler]

Stocks are a massive game of hot potato. Whoever is holding the stock with the game is over gets burned.

When is the game over? Do you mean when a company declares bankruptcy? (the game is over for that stock) Or when the market falls? (it goes up and down constantly) Or is the entire stock market going to crash and burn? (end of American society as we know it)

I agree that the goal of the stock market is not to maintain wealth--if you just want to maintain, you can't beat inflation-protected Treasuries. The stock market is a way to grow wealth, and the winning strategy is not a secret: dollar cost averaging and low-load index funds. It's not a get-rich-quick scheme, but it will grow wealth if given enough time.

If you're wheeling and dealing individual stocks, yeah, it's more like gambling. But that is only one way to play the stock market.

Re:A Solution to this and the eBay 'sniping' probl

[afabbro]

I've never really understood the complaints about eBay sniping. Set your maximum bid at the actual maximum that you want to pay. Whether someone snipes or not, if your bid is the highest you will win. If it's not, you won't.

You are right in principle, but...let's say I see something now and decide I'll pay $50 max for it. If it sells for $50.01, well damn, I would have paid $50.01. I might not have paid $60, but one cent more?

It's really hard to find the exact to-the-penny point where your "no, I won't pay that" mode is tripped. Virtually everyone will pay a few cents more than their maximum bid - and hence, snipers flourish and cause angst. It's not a case of paying 20% more - that's obvious - it's a case of paying .001% more. Most people can't focus their "maximum that you want to pay" that finely.

Re:A Solution to this and the eBay 'sniping' probl

[Cajun Hell]

I don't see why this is a big deal, though. If you bid $50.00 and it sells to someone else for $50.01, all that happened is that you failed to buy something. For you, that's a neutral outcome, not a bad one. The sniper bought the item they wanted and the seller got a fair price. Everyone either won or broke even. No harm happened to anyone. What's the problem?

The problem ...

[atomic brainslide]

... isn't that the mysterious bidders are "testing" the market to see if anyone is selling or buying at outrageous prices. the problem is that the bids being placed are not placed in good faith -- this is against the law in the USA.

the crazy, high-frequency bids are placed and then cancelled at high speed. they act as place holders waiting in line for the price to move in their favoured direction. however, since the vast majority of the time the bids are cancelled, they never execute. this results in the mirage of liquidity and the inevitable "Flash Crash" where sellers come in and all the buyers instantly disappear.

finally adjudicated?

[HornWumpus]

Finally adjudicated? As in bankruptcy?

WTF are you babbling on about?

The * is worth whatever someone will pay for it.

That's right blair1q Enron really was worth all that money way back when (even though it was all fraud).

The money made was green and spent just the same (as long as you were not part of the fraud).

Stocks must be liquid for markets to work at all efficiently.

It's much harder to raise capital for a private corporation vs a public one.

There are several reasons for this but stock liquidity is definitely a feature for all investors (including but not limited to those that get in on IPOs).

It should be noted that most holders of IPO stock were previously holders of private stock (Founders, Angels, Vulture Capitalists etc), not Wall street insiders.

It should also be noted that IPO are 'Initial Public Offerings' not 'Only Public Offerings', companies raise capital with new stock offerings all the time.

I will agree with you that speculators are just gamblers who lower the signal to noise ratio in prices.

I'd tax any market gains from positions held less then a year the same a gambling winnings.

if this is a form of spam ...

[JoeBuck]

... that is, if people are doing this kind of thing to gum up the works for their competition, one answer is to assess a very small fee per trade, less than a penny. This would be completely negligible to a normal investor, but could be quite expensive to those trying to saturate the system for the benefit of their trading algorithm. Market-makers like Goldman Sachs would also wind up paying significant amounts, but given their privileged position which basically gives them a license to print money it's only fair. The fees collected could go into an insurance fund to help cover the next financial meltdown, and if it slows down trading a bit, that may well be a good thing. Complex nonlinear systems have a tendency to go unstable, and damping is one way of decreasing this possibility.

Intent

[zogger]

The intent is to game the system by creating bogus artificial demand-or lack of demand-in large enough quantities to influence trades below. Therefore,because they can do it at such a huge volume, and they know in advance what they are doing, they can use the split they have created to leverage that into a sort of arbitrage all day long. I am *guessing* right now they have to use a partner trader/bot to do the actual "real" trades following the bot shilling. Like secret partners in a poker game.

My opinion, crooked leeches, parasites, this sort of trading should be outright banned. I'd also like to see sales tax put on trades, we simply don't need this high speed trading at all, and that would be the simplest solution to this whole mess.

Would it reduce churn and volatility? Yes it would, not eliminate it, but slow it down enough to make it so actual human beings had to stop and think on what they want to do, and it would force a return to investing in a company, rather than this casino action we have now.

also see this, it's just a high tech variation: http://en.wikipedia.org/wiki/Front_running