Algorithmic Trading Rapidly Replacing Need For Humans

DMandPenfold writes "Algorithmic trading, also known as high frequency trading (HFT), is rapidly replacing human decision making, according to a UK government panel which warned that the right regulations need to be introduced to protect stock markets. Around one third of share trading in the UK is conducted by computers fulfilling commands based on complex algorithms, said the Foresight panel in a working paper published yesterday. Nevertheless, this proportion is significantly lower than in the U.S., where three-quarters of equity dealing is computer generated. The Foresight panel, led by Dame Clara Furse, the former chief executive of the London Stock Exchange, argued that there are both benefits and severe risks to algorithmic trading. There was 'no direct evidence' that the computer trading in itself increased volatility, it said, but in specific circumstances it was possible for a series of events with 'undesired interactions and outcomes' to occur and cause massive damage."

Comments

Re:Not replacing, just adding on top

[Hatta]

It's not replacing humans, it just improves profit making for those who want to trade

By siphoning value away from those who want to do something productive.

Absolutely

[Anonymous Coward]

As someone who worked at Goldman Sachs, I can completely attest to this; a lot of the software was automated, but what the trading group was always asking for was basically an autopilot system; they wanted to sit back and just let the money roll in. One of them, I remember it distinctly, said that he'd love it if he didn't have to watch all the various windows all the time because he "had better things to do".

This was the same group of guys who, one of them told me "if I could kill you, not get caught, and make money because if it...I wouldn't think twice".

Fun times...

so SkyNet is really a Wall Street computer?

[peter303]

The computer that "takes over the world" wont come from a mad scientist's workshop or the military-industrial complex. Instead it will emerge out of Wall Street. There are few stronger motives for Artificial Intelligence than to make lots of money.

Re:so SkyNet is really a Wall Street computer?

[jd]

Ok, here's one option. You write N herustics and M algorithms (not the same things, guys) and you apply the following rules. (To make it easier to read, I'll use "program" to refer to any herustic or algorithm. This is assumed to cover any implementation, including neural nets, genetic algorithms, ELIZA bots, etc.)

• Any fresh program is given an initial budget of a fixed amount.
• A program is "taxed" (to simulate the natural-world overheads of a given form and lifestyle) per day.
• Any time a program goes into the red (ie: it has "died" of starvation), it is killed off from the system.
• Any time a program does better in one area than another, it forks off a copy with one instance in one domain or sub-domain, the other in the other domain or sub-domain.
• If a program exceeds a certain amount of money, it forks off a copy and splits the money available between them.
• Whenever a program is forked in the same domain, it is modified slightly. Forks across domains are left as-is.
• All programs are run hard real-time (ie: they have an absolutely guaranteed number of CPU cycles per unit of wall-clock time).
• One machine (physical or virtual) shall be responsible for some specific domain and all sub-domains within it. Transfers to a different domain equals a transfer to a different machine.
• If a machine cannot add another program and maintain hard real-time guarantees, no program may be transferred in or be started via a fork within that domain.
• The tax rate shall be proportional to the spare capacity on that machine (so a full machine shall have a higher tax than an empty one, reflecting the sort of resource scarcity you'd have in the real world).
• A program generator can use some certain percentage of the tax collected to create fresh starting points, up to but not exceeding the number that it can afford to give the standard starting money to.

This will generate all the conditions that applied in the very early days of hominids, using the tax system to replace all of the ecological, environmental and biological factors that simply don't have a direct translation in electronic terms, and the program generator to simulate the fact that in the natural world the precursor species is also evolving and therefore throwing out new lines all the time.

I predict that, using the description you have, some measure of sentience at least one program will be achieved within 2.5 - 3.5 million years.

Re:Not replacing, just adding on top

[h4rr4r]

What is this something productive that the stock market does?
Other than issuing new stock, when is it productive for the companies invested in?

Seems not much different than trading cards, now that most companies don't give dividends.

Re:clever humans can introduce "black swans"

[Nursie]

Only if they work for the right people. Otherwise they'll be arrested and thrown in jail.

Re:Not replacing, just adding on top

[Anrego]

Agreed.

The way it was supposed to work.. where people would invest in ideas they thought were good in the hopes that they would take off and make a profit... made sense.

It's so abstract from that and there is so much skimming off the top (don't give me "market liquidity" crap...) now that we ideally should just wipe it and start fresh from the basics, with regulations in place to prevent the kind of bullshit that is happening at present.

Problem is the people who could make this kind of thing happen are bringing in so much money from the way things currently are, that they have little motivation to do so.

This is bullshit.

[brxndxn]

HFT does not help the market in any way. It does not promote the investing of capital. Going into and out of a company in less than a second is ridiculous. Steps need to be taken to stop HFT in its tracks before the whole market is ruined.

This will fix HFT:

1. random delay in all trades.. stick a 100ms to 1000ms delay before all trades are posted on the market
2. tax all trades by a miniscule percentage.. give straight to government debt
3. enact a rule that all trades stand.. erroneous trades made by a computer algorithm will never get rolled back

Re:This is bullshit.

[Ruke]

You assert that HFT does not help the market in any way. I'd be inclined to agree, but that's really just my gut feeling. Can anyone provide any kind of source, one way or the other, saying that HFT is necessary, or good, or terribly evil? I'd like to hear what actual economists think of it, rather than just laymen.

Re:This is bullshit.

[LordNacho]

I Googled it. Here's a report about what some researchers think:

http://www.tradersnarrative.com/what-if-hft-is-actually-good-for-the-market-4723.html

"That public perception may need to be adjusted according to this recent research by finance PhD candidate, Jonathan Brogaard (Kellogg). The paper looks at the effect of HFT on equity markets and through analysing the strategies used by these firms it considers their profitability, impact on market liquidity and volatility.

There is evidence that high frequency trading contributes to price discovery and liquidity. There is also evidence that it has a minimal impact on volatility and may even reduce it. There is also no evidence that they engage in front-running. HFT demand for liquidity (42.7%) is slightly higher than their supply of liquidity (41.1%) and they provide the inside quote about 65% of the time."

Re:This is bullshit.

[j. andrew rogers]

Can anyone provide any kind of source, one way or the other, saying that HFT is necessary, or good, or terribly evil? I'd like to hear what actual economists think of it, rather than just laymen.

The empirical impact of HFTs on market behavior and non-HFT traders has been studied, the following paper for example:

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1641387

From the abstract: "These findings suggest that HFTs' activities are not detrimental to non-HFTs and that HFT tends to improve market quality."

This is in line with the findings of most serious studies. The intuitions of most laymen fail them because they do not understand market dynamics.

Re:This is bullshit.

[dannys42]

The problem with studies like that, and much of the analysis of the stock market, is that it's all done on the numbers. The researchers may be absolutely correct in their study. However, what the current state of the stock market does do is encourage the "wrong" state of mind for many businesses. It's my belief that going public ruins more companies than it helps for exactly this reason (this would be a human study not a numbers study). People start making decisions based on what they thing the investors want that quarter. And to me, that's the surest way to kill a business, as you've now taken away from your focus on your customers and your employees. So while HFT may increase liquidity and all that good stuff, it's doing so at the cost of the long term health of the company. (Not all businesses fall into this of course, but it appears to be more common than not).

I may not be a serious day trader, but as an (albiet modest) investor, I want a company that thinks long term. And it's my understanding the stock market was originally created for investment purposes. What sort of "investment" is it to put money into company for a second or even a day or a week? I actually think the time window for making trades should be more like one a month or even a quarter or maybe twice a year. With a longer period between trades, I understand people may feel uneasy about the commitment, but I think that's exactly what it should mean to "invest." However, if it's a big enough deal to people, then perhaps you could also have a (shorter) window for backing out (with perhaps a small penalty).

Re:This is bullshit.

[bluefoxlucid]

In communism, the government holds reins on the market and dictates activity.

In free-market capitalism, big money holders find ways to hold reins on the market and dictate activity.

Communism is arguably better, except that it's really identical aside from trading "vicious, self-serving market leaders" for "brain dead, incompetent market leaders." It turns out putting idiots in charge is just as bad as putting tyrants in charge.

It is thus sensible that we would occasionally look at a situation, knee jerk "That's Not Fair(TM)," and then calmly and rationally begin to investigate the unfairness to determine if we can make it fair without doing too much damage. Every time we try to regulate something, there is a cost ... but there are already economic costs, like the out-of-control prices of college tuition and books, or the decoupling of rental property rates from rental operating costs (if there is more money in the area, rent prices go up and the poor get pushed out--in other words, it will always be expensive to live decently, and the middle class will likely continue to shrink while the poverty line continues to raise). Fixing any of these things is dangerous; they're already broken, but you can do far worse.

HFT is one of these things. It has an economic cost I'm not very interested in (I dislike the practice, but not on any specifically well researched basis), but to regulate it away would also have economic costs. You have to determine if those costs are in excess of the social and economic value gained by eliminating HFT before you even begin to regulate it.

Re:This is bullshit.

[Bob the Super Hamste]

My understanding is that HFT get to see incoming orders before they are processed and then react to them. So lets say that some orders come in one says I would like to sell 1000 shares of stock x at $10.00 a share and one of the other orders comes in and says I would like to buy 1000 shares of stock x at $10.01. In this case the HFT puts in an order to buy 1000 shares of stock x at $10.00 a share and then puts in an order to sell 1000 shares of stock x at $10.01 a share. They have effectively skimmed $10.00 out of the system. Now go and do this several million times an hour and bingo you now are skimming truck loads of money out of the system. they probably make even less per share but this is just an example with easy numbers.

Re:This is bullshit.

[LordNacho]

This is not possible in all markets, and is definitely fishy. But also, because most markets do not allow this, it is not the main business of HFT traders.

Additional regulations

[Halo1]

Algorithmic trading, also known as high frequency trading (HFT), is rapidly replacing human decision making, according to a UK government panel which warned that the right regulations need to be introduced to protect stock markets

Like making it illegal for humans to beat the algorithms?

Re:Not replacing, just adding on top

[MightyMartian]

In other words, the system will become so complex that we will quite literally be unable to ever quite figure out what's going on, until, of course, it all collapses, kills trillions of dollars in value, renders most economies smoking ruins, and then everyone will finally ask "Why the fuck did ever let that happen?"

Re:Not replacing, just adding on top

[Tsingi]

In other words, the system will become so complex that we will quite literally be unable to ever quite figure out what's going on, until, of course, it all collapses, kills trillions of dollars in value, renders most economies smoking ruins, and then everyone will finally ask "Why the fuck did ever let that happen?"

Umm, I think we've already been there.

Awesome...

[RobinEggs]

I've never been convinced that HFT is anything but a scam to make institutional investors more money without doing more research or making more socially responsible investment decisions.

The company worth truly investing in, in the sense that you hope it survives and hope it continues to grow as opposed to only making you lots of money, is the one that will treat the environment, their employees, their supply chain, and their customers with respect while paying investors and owners a respectable return.

HFT algorithms don't give a fuck about any of that, exactly like the stereotypical Wall Street broker doesn't care about any of that; in fact HFT algorithms were written when brokers realized they could make more money in corrupting and managing young mathematicians than in doing their own jobs. HFT just further emphasizes empty, short-term speculation without regard to the product sold, the behavior of the company, or the future potential of the company. It enables the irresponsible greed of people who just want to make a dollar in the next day to become the irresponsible greed of people who just want to make a dollar in the next 0.0000000001 seconds.

Re:Awesome...

[FoolishOwl]

If investment decisions are better made by computer program than by human investors, what justification is left for private ownership of capital? And what's the argument against planned economics?

Can we have another look at the idea of democratically deciding upon our social priorities?

Re:Not replacing, just adding on top

[h4rr4r]

That was the idea once upon a time. These days I disagree.

Otherwise stock would have voting rights that mattered and pay dividends.

Re:Not replacing, just adding on top

[TheRaven64]

don't give me "market liquidity" crap.

It's not all crap. There is value in liquidity. A company that wants to expand needs to be able to finance this expansion. This typically involves either issuing more shares or getting a loan. The former is preferable to the company (no interest to pay), but needs investors. Having some speculators in the market makes it possible for the company to issue the stock and sell it immediately to speculators, who later sell it on to long-term investors once they've had more time to judge the risks.

The problem is that we now have a lot more speculators and they dominate the system, so the ability of a company to raise capital no longer depends on whether it's a good long-term investment, but on the opinions of the speculators.

Re:Not replacing, just adding on top

[h4rr4r]

It is a rigged second market. With fees no other secondary market has, it has regulations that prevents market participants from being on even playing ground and on top of it no one is really interested in non-institutional investors.

Re:Algorithmic Trading is not trading

[Erbo]

It is absolutely this. I would go so far as to say that no individual investor has a chance in the markets when going up against these HFT algorithms run by the big banks and trading firms. No matter how good your analysis or how closely you watch the market like a hawk, you're going to be screwed, blued, and tattooed.

And this comes at the same time as The Fed and Chairsatan The Ben Bernank have pretty much destroyed all "safe" investments through manipulation of interest rates, forcing people to turn to the stock market if they want any hope of any kind of significant return on their money...and where they can be fleeced by the HFT algorithms and the bankster fraudsters. Not to mention destroying the purchasing power of those dollars via "quantitative easing" (read: money printing) games.

Probably a good time to invest in precious metals. No, not gold, silver, and platinum. I'm talking steel, lead, and brass...in appropriate forms, of course.

Re:Not replacing, just adding on top

[H3lldr0p]

I think they use a number of financial tools and hold a number of positions on a given stock, none of which look at the long term stability, profitability, or sustainability of the company but instead focus on the day to day noise in order to maximize when to trade which option they currently hold the most money in.

These people look for the gaps. The gaps of knowledge, the gaps of valuation, and the gaps of naivety of those just getting into that sort of trading. They at not looking at the value of the employees, the value of the products, or the value of the actual company.

Just look at the oil markets and how volatile and uncoupled from supply and demand they've become. We're sitting at the lowest demand for gasoline in fifty years but it's still being traded at two to three times what many feel is the actual value of the good. Look at what it has done to demand for the product and for the larger economy. That is what speculation has gotten us.

Re:Not replacing, just adding on top

[ipoverscsi]

The purpose of the stock market is to provide price discovery. If you had perfect information at all times you would know the price of a good and the stock market would be pointless. But because perfect information is impossible, the stock market crowd-sources the gathering of information so that the true price can be discovered.

Determining the price of a good is something only a human can do. Price is a value quantified, and determining value requires sifting and filtering of information and the application of significant amounts of gut instinct. Computers cannot set prices since they don't have any concept of value -- they have neither needs not wants.

Computer assisted trading -- trades where people set stops and buy limits -- is okay because the human has done the work to determine the valid price ranges a priori; the computer simply executes the bid on behalf of the user.

High Frequency Trading, however, should be illegal since it does not involve human value judgements at all. It simply allows a computer to front-run actual humans and siphon off people attempting to perform a useful act -- that is, price discovery.

Re:Not replacing, just adding on top

[scamper_22]

well, it does it better and cheaper than humans doing the same thing.

Most of the financial industry is just that... people looking at some trends and data and taking actions. It's one of the reasons most actively managed funds don't beat the index year over year. They really don't do anything useful.

They have layers and layers of financial people and associates and advisers... to basically do nothing productive. They basically act as proxies to the actual funds.

So we replace them with a some algorithmic systems. Someone still has to program the system and you typically get some very skilled people designing the algorithm.

At its 'most honest and useful level' it might be a simple algorithm to track dividend paying stock based on their average X year dividend payout. Granny might like this kind of system.

Of course it can be more 'trader' based and just look for patterns and buy and sell quickly...

But again... the point is there were humans doing the same thing. The computer just does it better and more efficiently. There is just no reason to employ many people in the financial services industry when they don't do anything that can't be done better via a computer.

These folks wouldn't blink an eye if they could automate a manufacturing worker's job. There's really no reason we should worry about the automation of 'skilled' or 'educated' labor.

And I don't really buy the line that human oversight will somehow act to prevent feedback loops... Computers are surely vulnerable to it. But so are people. So are regulations.

And they've already begun putting in various stop measures in case things go crazy. We could work towards the 'trader based hft' by maybe throwing in random delays or something... but I think the stock exchanges can work towards monitoring.

Re:Not replacing, just adding on top

[RobinEggs]

if a company doesn't pay out a dividend, it keeps the cash. Investors can then adjust their valuation accordingly.

Which suggests that investors react rationally to the presence or absence of dividend; I think the truth is that all but the most conservative investors simply ignore dividend today, and it's a problem in more ways than I can count.

For one thing, stock ownership is supposed to translate to company ownership; in reality when you buy a stock without a dividend you have no way of making money unless you abandon the company you supposedly own at some future time. What kind of owner doesn't benefit from profits unless the company is not only growing but the owner also leaves the company in some proportion (i.e. sells shares)? That's completely, irreconcilably moronic if you ask me.

"Both benefits and severe risk"

[GameboyRMH]

Benefits: More money transferred to the very wealthiest individuals as traders who can't afford HFT servers (physically as close to the trading floor as possible - at these speeds, light is too damn slow) are at a severe disadvantage.

Severe risks: Potential for total economic collapse to take place in the blink of an eye.

I punch those numbers into my calculator and it makes a frownie face.

Re:Stock markets are just legalized gambling nowad

[Overzeetop]

Yeah, for long term investors it's like being the house when the guy sitting at the table playing black jack and knows what the dealers down card is and what the next 10 cards in the shoe are. Usually, the house takes those guys out back and breaks their knees, but it this case, the guy at the table happens to already own the knee-breaking-guys.

Re:Not replacing, just adding on top

[gstoddart]

Explain how value is "siphoned away".

The large trading houses, who have a direct wire into the exchange, do not pay fees on transactions like the rest of us do.

By doing all of these high volume trades, they more or less get to cut in line and gain benefits of being tied directly into the trading system.

This allows them to reap huge benefits by having a computer do something that you and I would not have access to.

This is literally a mechanism where the trading houses can game the system and skim off the top. You and I could not do this, because we would both be paying fees on the transactions, and because we wouldn't have the same level of access as they do.

Value is siphoned away, because they're exploiting their better access to more or less take their profit before anybody in this (supposedly free and fair) market has a chance to.

Personally, I think the practice should be illegal -- they really do gain access to profit taking that is only made possible by their special role in the way the system works.

Re:Not replacing, just adding on top

[lgw]

Market makers reduce the bid-ask gap. I get to buy a stock cheaper than I would without them - but they split the difference. I get to sell a stock for more than wothout them - but they split the difference.

The entire business model is splitting the difference, and thereby making it cheaper for the small trader to trade. This is especially visible recently in options trading, where everything is a thin market: bid-ask gaps are far smaller than 10 years ago, thanks to algorithmic trading.

I'm about to sell my used car to a dealer for 2/3s of what he will sell it for - the bid-ask gap is huge - I could really use some market makers here!

Re:Not replacing, just adding on top

[frank_adrian314159]

I think we've already been there.

Umm, no. This is something that is still yet to come.

However, the OP was wrong about us asking why we let this happened. Instead, we will declare the brokerages too big to fail, bail them out, make up stories about irresponsible individual, non-computerized traders who "gamed the system" and "caused the crash", and then use the resulting deficits from the bail-out and supporting people through the crash as an excuse to cut benefits to actual people.

Oh... uh, wait... maybe we have been there before...