Slashdot Mirror
Stock Market Manipulation By Millisecond Trading
cfa22 writes "Nice piece in the NY Times today on ultra-fast trading on the NYSE and other markets. The 'algos' that make autonomous trading decisions have to be fast, but I wonder: Is network speed ever a bottleneck? Can anyone with inside experience with millisecond trading provide some details for the curious among us regarding hardware architectures and networking used for such trading systems?" According to the article, high-frequency traders generated about $21 billion in profits last year.
In one or two decades we might be able to let all the stock trading to be done by the machines while we focus on doing the non-specialized work ourselfs! Ow wait... wasn't it supposed to go the other way around?
I interviewed at a millisecond (market-making) trading firm in Chicago. They claimed that when a hedge fund, etc. would buy or sell a stock, that one large purchase or sale would typically signal another. Whichever firm could get their quote up the fastest would make the buy or sale, and it's a winner-take-all system. The first market-maker to adjust their price would benefit. Thus, server speed is THE essential bottleneck. Needless to say, they keep the location of their server a secret.
Ultra fast trading is an interesting idea and done right it can lead to successful short term returns, but if you take a Ferrari around a hairpin at 120mph, you're still going to hit the wall and die.
Here's what happens when that particular Ferrari hits the wall:
http://tech.slashdot.org/tech/08/09/10/203233.shtml
A firm I worked with recently tore down an arbitrage network (they were getting out of the business as it was not core) which comprised of a great deal of Layer 2 dark fiber between sites in NYC and an external data center in NJ, Force 10 fabric switches with multiple paths to server clusters, and a great many Sun X-series servers running Linux. This arbitrage network bypassed the standard corporate (i.e. Cisco-based) network as they wanted exclusivity, higher bandwidth and as much speed as possible. Still, there were issues and the whole environment was scrapped since the actual returns did not match the expectations or cover the costs.
When I looked over the shoulders of the designers (they didn't want too much support from the regular network engineering team) they were concerned with raw performance and not as much with security or other daily operational issues. I would characterize it as the difference between, say, a NASCAR Sprint Cup car and your regular transportation. The former is purpose-built solely for performance while the other has to contend with safety requirements, daily functionality, and a lower common denominator for use.
I think, therefore I am - Rene Descartes; I yam what I yam, an' that's what I yam - Popeye
Can someone explain to me the benefit to society of this kind of activity? I get how the stock market is beneficial, generally allocating resources according to the merit of the business ventures involved, investing capital where it will produce goods/services/jobs, and so on. So despite being a social lefty, I'm not anti-capitalism or anti-stock-market; it has risks and flaws but it works. But how does this kind of stock trading benefit anyone other than the traders themselves?
http://alternatives.rzero.com/
Allston Trading occasionally speaks at my university, and they've said that network bandwidth can be a big bottleneck. They needed to install servers across the street from the NYSE to attain the edge they needed.
As far as who the profits go to, ALlston (and I suspect many similar organizations) keeps most of their profits internal, and exercise big profit-sharing programs for their employees. It's actually quite an interesting idea, as this group of almost entirely Computer Scientists are using their expertise to make some good money as a cell in an atmosphere dominated mostly by business-types.
As someone who works with people who do this, I can tell you they spend a lot of money on very powerful machines, and then try to place said machines within walking distance of the exchange's computers. I have been told that running a server at the office is too slow, even if its in the same city. Also, millisecond is the wrong word. Their trading is measured more closely in microseconds.
"Going to war without the French is like going deer hunting without your accordion." ~General Norman Schwarzkopf
In theory it adds liquidity to the market and reduces trading costs for other players.
Also, many of the successful trades will be based on having better information than the market price (over time, it would be very difficult to have much success by being lucky, especially with multiple automatic systems active), the execution of those trades makes the market price more accurate.
Nerd rage is the funniest rage.
I suppose that's the same guys that are always getting the "first post" on /.
I work in the finance industry, and know a few things about this business. It can be very profitable indeed. Since the HF trades are typically finished at the end of each day (or even minute), they are not required to hold much cash (capital) to support their positions. Thus the business is unusual in the finance world for making a profit on, essentially, zero capital. Of course, it costs a lot of money to stay in the arms race.
The article hints at two kinds of HF strategies, and they really are distinct. First, there are the "rebate" strategies that collect those credits for providing markets. Then, there are the "predatory" strategies that try to find the price points of buyers and sellers as described. Other HF strategies include pairs trades (Exxon goes up so RIG will soon), inter-exchange arbitrage where a stock is traded on multiple exchanges, and index arbitrage such as trading the elements of the S&P 500 against the index futures (which has been around almost forever).
Other algorithmic trading includes strategies meant to take on positions slowly (or quickly) and efficiently. A famous old category are the Volume Weighted Average Price (VWAP) strategies that try to trade a little bit at a time throughout the day, so that the average trade price is close to the day's average. Other algos try to take advantage of mean reversion or trends during the day.
There is huge demand for technical people in this industry (I probably get one headhunter call every two weeks), almost all of it in NYC or Chicago. There's demand for network engineers, statisticians, programmers, and traders, and high pay for quality. Surprisingly few programmers these days are really acceptable to the business, because the code has to be so fast and efficient, and almost no one studies that any more.
This kind of activity is an abuse of the free stock market system.
This activity does not generate wealth. It doesn't create something from nothing. And it doesn't add value to society. If they generated 21 billion, then 21 billion was necessarily lost by others.
People should look down on this kind of business and method of trading.
This is an outstandingly bogus article, what happens when arts graduates attempt to understand anything except celebrity gossip.
>It is the hot new thing on Wall Street,
The first algotrading I encountered was in the early 1990s at Deutsche, and they senior guys there told me of some of the mid80s stuff they'd done.
Not new.
>a way for a handful of traders to master the stock market,
Although algotrading is not exactly mass market, it is about as exclusive an activity as getting drunk.
>peek at investorsâ(TM) orders
That's not algotrading.
>and, critics say, even subtly manipulate share prices.
A major topic in algotrading is actually market impact modelling, ie working out how to make prices move less when they trade.
>Nearly everyone on Wall Street is wondering how hedge funds and large banks like Goldman Sachs are making so much money so soon after the financial system nearly collapsed. High-frequency >trading is one answer.
Actually they're working out how to employ these guys since they've often been shafted by the GS bonus scheme.
>software that a federal prosecutor said could âoemanipulate markets in unfair waysâ â" it only added to the mystery.
He was fed this line by GS as part of a dispute over an algotrader's pay. There is no suggestion that GS was being "unfair", just addressing the issues caused by GS having an inferior tech infrastructure for AT than may competitors.
>"This is where all the money is getting made,â said William H. Donaldson".
I'd love to see the original quote before editing. Bet it was a lot less sexy.
>For most of Wall Streetâ(TM)s history, stock trading was fairly straightforward:
For an arts graduate the writer is terribly ignorant of history, as well as trading.
For instance is he not aware of how the Kennedy family got rich as part of causing the crash of 1929 ?
>Joseph M. Mecane of NYSE Euronext, which operates the New York Stock Exchange. âoeMarkets need liquidity, and high-frequency traders provide opportunities for other investors to buy and sell.â
It's more complex than that. Some ATs are consumers of liquidity, others provide it, and there exist models that imply that heavy AT activity drives liquidity away.
>Average daily volume has soared by 164 percent since 2005, according to data from NYSE. Although precise figures are elusive,
164% is a surprisingly "precise" figure, which PR bunny fed that to him ? Wonder why the PR wouldn't give more ?
>"The result is that the slower-moving investors paid $1.4 million for about 56,000 shares, or $7,800 more than if they had been able to move as quickly as the high-frequency traders."
That's really quite amazingly precise. So precise that I believe not a word of it.
Edit/Delete Message
Dominic Connor,Quant Headhunter
Anyways, back on the original question, no, network speed is not so crucial once all of your packets are properly timestamped.
RTFA:
One second after the market opened, shares of Broadcom started changing hands at $26.20.
...
While markets are supposed to ensure transparency by showing orders to everyone simultaneously, a loophole in regulations allows marketplaces like Nasdaq to show traders some orders ahead of everyone else in exchange for a fee.
...
Automatic programs began issuing and canceling tiny orders within milliseconds to determine how much the slower traders were willing to pay. The high-frequency computers quickly determined that some investors' upper limit was $26.40. The price shot to $26.39, and high-frequency programs began offering to sell hundreds of thousands of shares.
On the one hand, you can call this 'perfect price discovery'
OTOH, that specific set of behaviors fundamentally breaks the traditional way that the imperfect markets have worked.
And just as importantly, it represents an unfair trading advantage that you or I will never have.
Allowing this behavior doesn't further market activities, it just allows a few players to accumulate wealth at everyone's expense.
Seems to me that the solution is to close the loophole that allows this to happen.
[Fuck Beta]
o0t!
GS sees both the buy order and the sell order.
They can sell before large sell orders. Then buy again after to avoid a loss.
They can buy before large buy orders. Then sell right after to take a gain.
It's called front running and it's illegal.
But the government is not enforcing the law.
My theory is they are letting illegal activity go on to hold up the market prices.
I could be paranoid. But I've been in this thing for 28 years now and I've never seen such goofy market behavior and repeated evidence of price manipulation at market close on thin volume-- and yet there are no investigations.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
All you needed to stop were the bailouts. The behavior at that point would have corrected itself since many of these companies would have been out of business.
It goes like this. For a given security there are two prices the BID (what you're willing to pay for more of a stock) and the ASK (what you're willing to sell some of the stock you already have for). Most normal folks have only one up at any given time, you're either buying or selling. Then, if a BID equals an ASK you've got a trade.
There are entities in the market known as "market makers" who do both at the same time, however. They're usually not in the market to load up on a stock or unload their stock, so they try to balance the number of orders filled on a given security by raising their ask or dropping their bid in order to maintain equilibrium. Let's look at an example of an inefficient market without adequate liquidity. Let's say there's only one market maker -- he might bid $1.00 but ask $10.00, and clean up on anyone who tries to do business in his market.
So my hypothetical firm sees this huge gap and says, "Sweet. Let's start making market in here as well!" -- and I pad my bid a little bit, say $1.05 and shave a bit off the ask, call it $9.95. All of a sudden the original market maker is cut out - none of their bids end up buying stock, none of their asks end up selling it. So they decide to close the gap even further. 1.10 and 9.90. My firm doesn't like getting cut out of this sweet action so we close the gasp further and you have an arms race towards price discovery where the gap between the bid and the ask are separated by a very narrow margin, constantly being adjusted by what actual demand for the security is like.
Having multiple folks providing liquidity in the market means when you decide you're going to load up on STCK because you think their new product launch is going to do good things for the company your orders are unlikely to drive up the price very much so you can enter the position without paying a big premium to do so. Likewise, a year later you decide you're satisfied with your profits and are ready to exit the position. In a market without this liquidity selling your stock would drive down the price further and cut into your profits on that side.
Efficient price discovery is the purpose of the market. Liquidity is essential to that end. Traders provide that liquidity and, overall, the market benefits because of it.
You're confusing volatility with market liquidity. A market that's very liquid has lots of trades, so the value of a security (i.e. the most recent sale price) is always known and up-to-date. It doesn't have to be a volatile market to be liquid. For example, the market for government securities is very liquid, but if interest rates are stable it's also very non-volatile. (And even when it's "volatile" it's not volatile like a tech stock.) Needless to say an asset in a liquid market is typically easy to sell and convert into cash, which is the other definition of liquid.
The housing market is non-liquid. There have been only a half-dozen houses sold in my neighborhood over the last three years, and none of those houses is quite like mine. So nobody knows exactly what my house is worth. "Appraisers" are just professional educated guessers, who try to use judgment and experience to replace the pricing information provided by a liquid market. And as we've seen, a non-liquid market can be very volatile. Also, a house is a non-liquid asset; even a simple house sale to a family member takes weeks and hundreds of dollars to arrange.
But there are some securities (private placements, for one example) that trade very rarely. There actually are appraisers for these types of securities, because with so little trading activity the market price can become stale. These are very non-liquid markets. But as long as you can find a buyer and are willing to accept the buyers price they're no less liquid than IBM stock. You sell them, the trade clears, you get cash the next day. So they're liquid assets (more liquid than a house, in any case). And it's not even meaningful to talk about volatility in these markets because there is so little trade activity.
We did trading similar to this a few years ago, though we used a simple, pure-arbitrage strategy. We were trading the morning of 9/11 (pre market-open - the market never opened that day), and didn't pull the plug when the news hit. We were actually testing feasibility, and had not yet installed our server in a rack in Manhattan. My partner was trading (actually, "overseeing" is a better term) in a day-trading room about 30 miles N of NY. (Later we moved our server to the same building as the Island ECN, which at the time owned the ITCH technology mentioned in the article.). Everyone in the room stopped trading, their jaws dropped, staring at the monitor in the corner. BTW, the consensus in that room that day was that the market was not going to re-open. EVER.
My partner, though, is sitting in the back of the room, trading away (or, watching the computer trade away), wearing headphones, which already had the other guys suspicious. (The headphones didn't have soothing music, but vocalized order flow information - it was too fast to keep up with visually. My partner did have to intervene when things went awry, and the headphones told him when he needed to act or if things were humming along nicely.)
I figured later that we did about 5% of the pre-market NASDAQ trades that morning.
This gave me some consternation for some time. We made a bundle that morning, and the subsequent increased market volatility was the start of a winning streak. We did thousands of trades a day, and simply did not have losing days. Was there a purpose or value to our trading?
I eventually concluded "yes". We provided market liquidity when it was sorely lacking. The people who sold to us at, say, .50 outside of the market would have sold to somebody else at $1.00 outside of the market had we not been there as a counter-party. We stood there catching hot knives and handing them off, while providing the benefit of a quick and certain sale. This helped reduce spread and volatility.
Indeed, over the next couple of years, our typical profit/share shrunk from .10-.50 to 1-2 pennies, and competitors entered the market and the war of technological escalation started.
Now, as for the "flash orders", which apparently started last year, I don't see the benefit of that to the market. It just front-running in exchange for a fee paid for the privilege.
Yes, no questions were answered.
"high-frequency traders generated about $21 billion in profits last year"
Many of those trades took money from the guy who comes home at night and trades for his 401K. With every winner in the stock market, there is a loser, and it is big banks like Goldman Sachs that are usually winners.
I suspect that talking about "market liquidity" is just avoiding the issue: The U.S. financial system is corrupt.
No, it doesn't.
I think our entire society (Except stock traders) would be a hell of a lot better off if you were required to own stock for six months.
People might actually start purchasing stock based on actual company performance. They'd start expecting to make money via dividends, aka, company profit, instead of random fluctuations in the price caused by CEO manipulation.
Like the CEO, oh, firing half the workforce to cause an upward stock price bump for two months, so they get their bonuses. Oh, and incidentally cripple the company, but what the fuck do they care, they're out the door to another company.
Stock ownership is company ownership. It is not a fucking bingo game. Although if that existed by itself, that would be fine...the problem is that the idiotic bingo games get play by the board, which starts operating the company for the benefit of the bingo game, instead, of, I dunno, the actual company.
As a lefty, I'll complain when companies put profits ahead of employees, but hell, they've stopped doing even that. At least that system worked somewhat. Make the workers too unhappy, abuse them too much, and you don't make as much money.
Now the people running the company are rewarded solely based on an idiotic bingo game, which bears no relationship to how much money the company makes. And it doesn't matter how much the workers abused, because the actual business of actually producing goods and services is irrelevant to the paycheck of the people at the top.
Until the entire company collapses, at which point they walk out the door with a shitload of money and head to another company, whose board will happily hire them because it will make their stock go up. Which they can sell to poor unsuspecting suckers, and walk away with a lot of money to pour into the next business, building it up as an actual industry, until they can hire a stock-pricing oriented CEO and suck all the cash out of that stock, too.
If corporations are people, aren't stockholders guilty of slavery?
As an investor (which is the term the article is using--among whom are you and I), you had better not be trading like these traders. These traders are Market Makers, which are specialized traders that provide liquidity (a very important thing) to the market. If you don't have real market makers, then the stock market is going to look like what Mortgage Backed Securities look like right now: there will be periods of time when nobody wants to touch the stuff, and if you are an investor, you are truly screwed if you need to buy or sell your security.
These market markets are looking to capture a spread (the cost to buy or sell)-which by the way is much better for the investor than 5, 10 or 20 years ago--capturing a fraction of a penny a billion times a day. The key component to a market maker's success or failure is understanding supply and demand. This was true even before the computers came in. The computers can only detect it much faster, so you get what you mentioned first--better price discovery. It's not that the investor is getting screwed out of 20 cents, it's that they're not getting the 20 cent discount they used to when the market makers were humans standing in the pits.
As an investor, you really shouldn't care if you get Broadcom at $26.20 or $26.40. Yeah, it's nice if you get it for the cheaper price, but the reason you're buying Broadcom is because you think it's a good investment, and if $0.20 ruins that investment for you, then I guarantee it's not a good investment.
The $0.20 might ruin a day-trader's day, but day-traders are not investors nor market makers. You and I are not day traders. The only thing the electronic market makers ("high-frequency traders") ruin are slower, human-based market makers' profits and day-trader's profits. It's certainly not worth an article in the New York Times.