How and Why Wall Street Programmers Earn Top Salaries

msmoriarty writes "Given the level of interest in the recent highest-paid programmers discussion, our reporter decided to do a follow-up looking into the languages and skills needed to work on high-frequency trading systems. There's actually a pretty wide range of languages/tools used, but Linux is the 'default' OS and, not surprisingly, the 'ability to work under pressure when the traders are screaming at you' is a must-have skill."

Comments

Perversion of Capitalism

[NFN_NLN]

"In high-frequency trading (HFT), programmers eke out every last incremental tick in performance to build algorithms that battle other algorithms for computational supremacy and millions in profits -- and earn a lot in the process."

Skimming money off billions of micro-transactions. Ahh, yes... forget investing in ideas and backing well managed companies... this is the way capitalism was envisioned.

Re:Perversion of Capitalism

[zippthorne]

Look, superman III had a lot of lessons to teach. It's really too bad on villains watched it...

Re:Perversion of Capitalism

[Anonymous Coward]

Skimming money off billions of micro-transactions. Ahh, yes... forget investing in ideas and backing well managed companies... this is the way capitalism was envisioned.

Oh, admit it; this has been every computer programmer's dream since they saw Superman III as a child.

Re:Perversion of Capitalism

[russotto]

I call BS. You don't need much skill or talent in writing code that effectively does this in the end:

account.balance *= 10.0;

The challenge isn't making the code do that. The challenge is making it look like the code is intended to do something entirely different, and that part is merely an unavoidable side effect.

Re:Perversion of Capitalism

[fliptout]

Oh, please, spare me. I see this sentiment everywhere, and it's nonsense. Traditional investing still works. At the end of the day, investors are still going to look at how much money a company makes and assess how much risk that company's stock poses. If HFT causes blips in a stock's price, the the market will eventually correct the price.

Re:Perversion of Capitalism

[NFN_NLN]

This guy nails it - http://boards.straightdope.com/sdmb/archive/index.php/t-601887.html

"It's not about acting on market information. It is purely arbitrage. A mis pricing allows one to buy and sell simultaneously and lock in the difference minus trading costs.

In the old days, traders used to do this in the trading pit. Now it's computers closest to the exchange feed.

Tied in with this is the automatic trading. In the case of that big intra day fall, a wrong trade was entered. I forget the details but it was big enough to push down the market xx amount, which triggered automatic sell programs from non-arbitrage automated computer selling, which triggered a market sell off, which in turn triggered more selling until the market circuit breakers kicked in.

During the mandatory no trading period, the original bad trade was discovered and reversed. This IIRC also triggered automated buy programs and the whole thing went in reverse. The bad thing is that the market whipsaw really hammered some real end investor trades as collateral damage.

I remember watching the Hang Seng Index the day that Soc Gen announced Jerome Kerviel's fraud and liquidated the positions. It was a full trading day of massive market swings for big losses to big gains several times throughout the day. It was almost all computer generated programmatic trading."

Re:Perversion of Capitalism

[MarkvW]

We pay our taxes to these guys as much as we pay them to the government.

Re:Perversion of Capitalism

[Anonymous Coward]

"Perversion of Capitalism"

No it;s exactly what you get in the real world where market values (profit) drive everything. We see it all the time in offshored jobs, destroyed lives, rolling back of the welfare state, the election of extreme right wing conservatives like Stephen harper.

This "there is some pure capitalism we have to get back to" bullshit is just that - bullshit. The left was born from captialism killing workers, it caused two world wars and then then there was the cold war. This idea that is some 'benign' capitalism we have to get back to is just utter american ignorance.

Re:Perversion of Capitalism

[bjourne]

I think you are missing the point completely. It is not that money is being moved around in what basically amounts to a huge zero-sum game. One daytrader has better computers or lower ping to the nyse and beats out another trader who hasn't. That's really not a problem. The problem is the huge amounts of resources that is wasted on this game and the impact we are letting it have on our lives. The worlds brightest minds are spent in the game. You may not see it as a problem that the best mathematicians and programmers are working in the finance industry instead of developing a cure for cancer, affordable space shuttles, electric cars, aids vaccine or whatever because the salaries are much higher there so obviously that is what the market wants and the market is always right. But I do, I think it is a waste. But the worst problem is the importance we are giving to the stock market game. The idea was that the stock price should reflect the progress if its company. Now it's the other way around. It doesn't matter what the company does, if the stock price is high, then that's good otherwise it is bad. Oh and if the price of most stocks are low, and most players in the game have lost, then that is really bad. It's a depression coming and because the game was busted the rest of society will have to clean it up.

Re:Perversion of Capitalism

[Anonymous Coward]

Be careful when you read that long post; who knows who made it and with what purpose. Try to determine for yourself which sentences ring true and which are shaped as a straw-man.
This is from the Wikipedia article http://en.wikipedia.org/wiki/Definitions_of_fascism: Umberto Eco defines fascism with the following features:

The features of fascism he lists are as follows:

• * "The Cult of Tradition", combining cultural syncretism with a rejection of modernism (often disguised as a rejection of capitalism).
• * "The Cult of Action for Action's Sake", which dictates that action is of value in itself, and should be taken without intellectual reflection. This, says Eco, is connected with anti-intellectualism and irrationalism, and often manifests in attacks on modern culture and science.
• * "Disagreement Is Treason" - fascism devalues intellectual discourse and critical reasoning as barriers to action.
• * "Fear of Difference", which fascism seeks to exploit and exacerbate, often in the form of racism or an appeal against foreigners and immigrants.
• * "Appeal to a Frustrated Middle Class", fearing economic pressure from the demands and aspirations of lower social groups.
• * "Obsession with a Plot" and the hyping-up of an enemy threat. This often involves an appeal to xenophobia or the identification of an internal security threat. He cites Pat Robertson's book The New World Order as a prominent example of a plot obsession.
• * "Pacifism Is Trafficking with the Enemy" because "Life is Permanent Warfare" - there must always be an enemy to fight.
• * "Contempt for the Weak" - although a fascist society is elitist, everybody in the society is educated to become a hero.
• * "Selective Populism" - the People have a common will, which is not delegated but interpreted by a leader. This may involve doubt being cast upon a democratic institution, because "it no longer represents the Voice of the People".
• * "Newspeak" - fascism employs and promotes an impoverished vocabulary in order to limit critical reasoning.

Re:Perversion of Capitalism

[Anonymous Coward]

Look, what am I supposed to do.

I'm a chip designer. I used to work on beautiful hardware, you might be using some of my products to play games even today. Turns out, ASIC design is screwed as a business in so many ways, and it's not because of outsourcing. People just don't want the hardware all that much, and they don't pay a lot for it. The chips are priced by cost of manufacturing, not cost of R&D, and so R&D is basically cut to the bone on salaries.

Outside of financial companies (and FPGA vendors, who have been trying to sell this idea unsuccessfully to everyone on the planet for the last 10 years), nobody seems to have recognized that bringing stuff into hardware (FPGAs, for instance) can make money through performance. So that's where I work.

Re:Perversion of Capitalism

[Opportunist]

Patience, my friend. Soon we'll pay more of our taxes for these guys. It's all a matter of time and things will get sorted out.

Then we'll finally have the small government everyone dreams of. Because there won't be any money for anything but a small government left after the locusts had their share.

Re:Perversion of Capitalism

[TheRaven64]

Anyone with a CS degree from somewhere other than the back of a cornflakes box will have seen the algorithm for decomposing multiplication into shifts and adds. They will typically also have studied the idea of pipelining, and will see that the proposed 'optimisation' will require multiple trips through the pipeline, so they'll be able to explain why it's a bad idea. They will probably also know that instruction cache misses are one of the biggest performance killers in code for systems with a typical memory hierarchy, and so bloating the code from one instruction up to 3 is not such a great idea.

Those who have studied compilers (sadly, not all of them - well, not too sadly, since it means there's a skill shortage in that field, which is great for me) will also know that compilers for modern pipelined architectures will actually do the opposite transform, and turn this 'optimised' version into the original, with a single multiply.

The ones who have done an advanced architecture module will also be able to tell you that ARM is the one exception to this, where the fact that you get the shift for free and the latency for add is shorter than for multiply means that this may be marginally faster.

The ones that did any kind of software engineering module (a requirement for accreditation in the UK, not sure about the USA) will know that obfuscating your code for a marginal performance gain based on probably-obsolete assumptions about the underlying architecture is a really good way of writing unmaintainable code that runs slowly and ends up being thrown away and completely rewritten.

Re:Perversion of Capitalism

[mikael]

A search for topics on "stochastic calculus" will be enlightening.
Basically it's a form of one-dimension Brownian motion or card-counting with random noise rather than cards. You've got a share price you want to watch because it's constantly moving up and down; it's got an upper and lower bound as well as maximum/minimum deltas, so you know there are good times to buy and good times to sell. The electronic trading system gives you the option of buying and selling and canceling orders at different times in the very short-term future. Like the throw of a coin, the price can go up or down (theoretically, the coin could land on its edge, and the share price could remain static, but there's enough noise in the system to prevent that from happening).

As time goes on, the probability of a lucky streak or a straight run of downs or ups becomes infinitely small. So the trading algorithm has to determine the optimum times of buying and selling. The system can send in an order to buy at time t0, and another order to sell at time t1. If at some time in-between, the price falls, the orders are canceled, otherwise if the price exceeds the target profit, the shares are sold anyway. Because the computer systems are so closed to the actual trading system, all of this can happen thousands of times faster than any remote punter could.

Re:Perversion of Capitalism

[TheLink]

The HFT traders who know what they are doing are NOT doing arbitrage. Because they get a 30 millisecond advantage:

http://www.nytimes.com/imagepages/2009/07/24/business/0724-webBIZ-trading.ready.html

http://www.nytimes.com/2009/07/24/business/24trading.html

When the favoured companies screw up they get the trades rolled back.
When they get beaten by "normal humans", they prosecute the humans: http://www.computerworlduk.com/news/security/3244186/norwegian-traders-convicted-for-outsmarting-us-stock-broker-algorithm/
When they beat the humans, they keep the profit as "rightfully earned".

Re:Perversion of Capitalism

[Your.Master]

You're not paying any attention to what he said.

He's not castigating people for taking these jobs, nor is he even saying that those jobs should just vanish into thin air. He's pointing out a societal problem that leads people to making these choices. Basically, he's agreeing with everything you said about researching the cure for cancer paying less than HFT, and calling that the problem. Surely you can imagine a society where curing cancer is paid like HFT and HFT is paid like curing cancer, even if you can't imagine how to get there from here. The only thing you said that countered the GP rather than reinforcing it was the argument that HFT applies upward salary pressure.

The narrative where people should not be allowed to make their own choices in favour of your choices, is yours alone. At best, it's mistaken, irrational hyperbole. At worst, it is a battle-cry against change and progress and any analysis whatsoever about the status quo.

Re:Perversion of Capitalism

[Pseudonym]

Many slashdotters are pushing 40 these days, and did their CS degrees back in the day when pipelines were shorter and even Booth multipliers were relatively slow.

The shift/add combination probably won't take "multiple trips through the pipeline" on any modern non-embedded core. If it's an x86 or x86_64, the compiler will probably generate a single LEA instruction (i.e. it won't pollute the I-cache), which is probably compiled to several uops which are issued together. Rather than round-tripping through the entire pipeline, the intermediate values will just get circulated around the CDB, which is fairly quick.

If you're as old as I am, chances are that you may not know this because Tomasulo schedulers weren't in vogue during your CS degree.

But as you are a fellow compiler writer, you must know that even on a modern CPU, the most optimal strategy depends on a lot of context. Using the multiplier isn't always a good idea, for example, since it's usually a more scarce resource than other functional units. Even a relatively recent AMD K10 core only has a single integer multiplier unit (which admittedly only costs a couple of cycles, but there's still only one), and a single floating-point combined multiplier/divider/square root unit (for non-SIMD operations, anyway).

You probably also know that simple integer operations can be effectively "free" if you really know what you're doing, since they can be used to fill pipeline delays from other operations.

But your central point is valid: This is the sort of optimisation that the programmer should not be wasting their time with. Leave the micro-optimisations up to the compiler, because it will almost always do a better job.

Re:Perversion of Capitalism

[gidyn]

it caused two world wars and then then there was the cold war

Silly me, I thought the second world war was started by fascists, and that the cold war had something to do with communists. If only my history teachers had known that an evil cabal of capitalists was behind it all.

Traders

[Anonymous Coward]

"the 'ability to work under pressure when the traders are screaming at you' is a must-have skill.""

Yeah, most of the 'traders' I've met were complete assholes, probably because they realised their high salaries were more down to luck than skill.

Fortunately when I screw up air traffic controllers just have to start rerouting airliners with hundreds of people on board so they don't crash into each other, so I clearly couldn't justify being paid a Wall Street salary. I can't imagine the stress level of having a 'trader' screaming at me.

Re:Traders

[artor3]

It's not even luck, either. It's down to your ability to hob-nob with other psychopaths. My friend's brother-in-law was a trader five years back (pre-recession) and lost something like $800 million (of other people's money) in bad speculations. He was fired for it, but hired into a new firm thanks to his "connections" a few weeks later, at a higher salary than his previous job. Oh, and of course he kept the bonuses he made at the first firm before his investment went bust.

Wall street is the world's greatest argument against the notion that capitalism rewards people in proportion to their skill and hard work.

Re:Traders

[Jane Q. Public]

The thing is: Wall Street anymore is not very "capitalist", in the historic sense of the term.

"Capitalism", in general, refers to making capital investments in goods that in turn will make a profit. Often, that means indirectly and over time: you spend $1M to buy a machine that makes thousands of widgets a day, and sell the widgets for $5.00 each.

However, with the current obsession with turnover and a quick dollar, what you end up with on Wall Street is really much more pure money speculation, which doesn't have much in the way of capitalist underpinnings. In fact it is a lot more like gambling, which has been around a lot longer than capitalism has.

We simply haven't been seeing the long-term investment in capital goods, which in turn power a robust economy, and at one time made the US the greatest economy in the world.

Until we see a return to something resembling real capitalism (and it still does happen, just not so much on Wall Street anymore), we will continue to have economic problems. A corollary to that is something I have been saying for a long time: Washington needs to stop concentrating on viewing Wall Street as some kind of driver for the economy. That's a false economy. Reasoning: anything that can lose a large percentage of itself overnight is not a stable thing on which to base a real economy. Keynesian "created" fiat dollars can disappear just as fast as they are created, as we saw quite clearly in 2008.

And for similar reasons, I believe HFT should be outlawed. It isn't even a real market anymore; it's simply a contest to see who can spot a discrepancy in perceived value the fastest.

Re:Traders

[timeOday]

I agree HFT is rubbish, but where do you see justification for long-term investment in capital goods in the US? It seems that industry is being moved to where it can be operated more cheaply - there is no economic justification for more investment here. In fact there's a surplus of "capital," at lest in the sense of invested money seeking good returns. (Overall the S&P has returned almost exactly 0% over the last 10 years.) So we keep getting bubbles due to over-investment, first in .com, then housing... what is the "next big thing" where we could invest to bring real growth?

Re:Traders

[Anonymous Coward]

When people talk about 'return to a capitalistic base'. They are talking about the sort of thing where 500 people build things on assembly lines making things. That sort of thing only exists in very low wage countries these days. In higher wage countries those jobs are automated.

I was watching a 'how its made' show. They showed a factory that was picked up 100% brick by brick and moved to another country from the US. Why? The factory was build for building things using humans. The labor here costs too much. It is that simple. You can buy 2-3 workers in another country for the same as here. Most of the shows they have 20 people working in a small factory that pumps out 100k in items a day. Most of the things they show are nearly automated are so close its not even funny.

Wall street is not a real economy but they have *MAJOR* effects on our economy. To minimize them is not right either. But the fed is made up from people that come from there. So they do what they know...

Also the dollars disappearing was a result of bad loans unfolding. It was also an effect of 15 years of unwinding of the 1930s laws that protected the rest of us from those silly antics. In the name of 'simpler paper work'. It was an effect of low hold rates placed upon the banks to create the ability of banks to loan money on the cheap. In many ways it was an echo of the 2000 crash. Which was an echo of the 1992 crash. Which were a direct result of the 1986 crash and wanting to 'free up cash to drive the economy'. They unhinged the very things reigning in wall street. To keep wall street from creating a 1930s crash. Which once those reigns were unshackled in 1997 they did. The 2008 crash is almost exactly the same as the 1930s one.

HFT is just a game to see who can screw each other faster. Making millions off the backs of others all in the name of 'liquidity'. They could put and end to it by doing 3 things. Making people take delivery of a % of commodity goods they buy. Also by limiting the number of times per day you can put in an order for a good say start with 3. Creating a minimum owned time. You buy something you must keep it for at least 1 day.

HFT is creating demand that doesnt really exist and in effect creating liquidity that doesnt exist. As my econ teacher said on the first day talking about government regulation or lack of it. You can cheat the market but eventually the market fixes itself. 2008 was a major correction. There is another bubble building in the commodities market. Whoever gets caught on the other end of that one is really going to get their shorts burnt.

This 'false' liquidity pushes prices up and creates cash that does not exist. It is why you are seeing 100+ oil, and 1600oz gold. Never mind the printing spree the gov went on. Plus the 'possibility' of default. The last 2 we can not really gauge the effects because of the first one. I read in the WSJ of 1 guy controlling 75-90% of the gold market 2 years ago. With only 1% of the cash needed to do so. Something is very wrong there...

Re:Traders

[Jane Q. Public]

"I agree HFT is rubbish, but where do you see justification for long-term investment in capital goods in the US? It seems that industry is being moved to where it can be operated more cheaply - there is no economic justification for more investment here."

Industry WAS being moved to where it can operate more cheaply. And there is plenty of justification. If you're American.

First, let me say this and get it out of the way: China's economy has been slowing. It's not exactly something they advertise, but it's true nevertheless. Further (and I won't claim this is directly related): there is a new Chinese "consumer class" that scarcely existed before, and they are hankering for goods that are not made at home. But having said that, I'm not going to belabor it, and instead I will concentrate on the U.S.

We have learned some things over the last couple of decades. One of the things we have learned is that regardless of whether it results in cheaper goods, outsourcing is bad for the economy. We know for a fact (it is not a guess or just a theory anymore) that it costs jobs at home, and those jobs are more valuable to our economy than the cheaper goods are.

Further, outsourcing is not "fair trade", because it artificially bypasses exchange rates. It is easy enough to say that "U.S. labor can't compete with foreign labor for the same amount of pay". Which is a true statement... but it's only half the story. It's a lie by omission. Because we aren't really competing on an equal basis. Let me construct a simplified example to illustrate the point (numbers just pulled out of the air to make the point):

Company X wants to hire people to manufacture widgets. It can hire unskilled Americans for $10 per hour, or Crotobaltoslavonian workers for $2 per hour. But here's the thing: In America, that $10 buys about 5 pounds of rice. In Crotobaltoslavonia, that $2 buys 10 pounds of rice. So in actual purchasing power, you're paying that Crotobaltoslavonian the equivalent of $20 per hour in America. Americans literally can't compete for that amount because it doesn't pay them enough to eat... in THEIR economy.

And that's the whole point. Economies are different. That's just the reality of the situation. And that's why we have exchange rates... to keep trading FAIR between countries. But hiring labor in a foreign country, in effect, bypasses that exchange rate, and so it is not a "fair trade". Americans are hurt in the process... even if rich Company X gets to keep a few more dollars. We are exporting even more dollars than what they keep as a result... and that doesn't help our economy, either. So it hurts in several different ways, not just one.

Savvy companies actually realize this, but they just don't care. Which is pretty unethical, if you ask me. They are willing to sacrifice your $10 so they can make an extra $1 in eventual profits. And you should be angry about that.

Plain and simple: we need to bring the manufacturing jobs back home. Partly because of the direct monetary imbalance that outsourcing causes, and partly because we need to keep that capital investment (in machines and buildings and other capital goods that can actually MAKE things) here at home, rather than letting it be done overseas at our expense.

"In fact there's a surplus of "capital," at lest in the sense of invested money seeking good returns. ... So we keep getting bubbles due to over-investment, first in .com, then housing... what is the "next big thing" where we could invest to bring real growth?"

The bubbles aren't due to a surplus of capital, per se. I mean they are, but the proximate cause of that surplus is indisputably government monetary policy. They have been keeping interest rates artificially low to "stimulate" the economy (BEFORE the 2001 and 2008 bubble burstings). Easy money means more capital. But it wasn't invested in capital goods, intended to

Re:Traders

[Jane Q. Public]

I meant to add:

Q: How can we bring jobs back home?

A: By directly -- and heavily -- taxing companies that outsource labor and manufacturing. This avoids the pitfalls of tariffs: we would not be blocking trade from coming into or going out of the country. However, we would be demanding compensation for our economy, from the companies that have been so harming it by their practices. That's about as fair as it gets.

(It's also not a subsidy by the government... so it avoids those problems, too. Instead, it is a source of revenue TO the government.)

Re:Traders

[Bacon Bits]

You realize all that would happen then is that the company itself would relocate overseas. It might have very large offices in the US, but they would all be "branch offices". All this tax would do is punish companies that are assholes enough to outsource labor, but not assholes enough to evade taxes by leaving the country.

What do you do then? Require businesses that do business here to be based here? Ultimately, it's a walled garden approach. Someone will root it and break the system.

No, the best option is to let them outsource. Eventually -- and I mean in probably a century -- local economies will realign to the global economy and outsourcing will not save the amounts of money it does today. This is what is happening in China and India. The problem with businesses screwing people over is that people aren't as stupid as business would like them to be. People figure it out, and start demanding silly things like equality and justice, and those are things that governments tend to take an interest in.

Re:Traders

[Opportunist]

Wall Street can be a driver for economy, but we have to get rid of the locusts. The "investors" that try to squeeze money quickly from companies, bleed them dry, dump them and repeat this bloodletting with the next. To do this, there is very little actually needed: Reward long term investment and the creation of goods and services (and jobs!), and punish short time gambling. For this, all you have to do is to install a hefty tax on short term trading and reward holding stock. One could, e.g. create a tax that is crippling for daytrading, but progressively diminishes the longer the time between buying and selling that stock gets. Make dividends paid according to the actual productive performance (productive! I.e. based on the creation and selling of goods, not shifting money around and dabbling in banking, insurance and real estate) tax exempt or at least lowly taxed.

That way investors would be keenly interested in investing long term in a company that produces goods and services. Which is, essentially, what Wall Street and investment is good for. If it's done for everything else, it's an economical parasite and should be eliminated from the system.

Re:Traders

[Registered Coward v2]

I meant to add: Q: How can we bring jobs back home? A: By directly -- and heavily -- taxing companies that outsource labor and manufacturing. This avoids the pitfalls of tariffs: we would not be blocking trade from coming into or going out of the country. However, we would be demanding compensation for our economy, from the companies that have been so harming it by their practices. That's about as fair as it gets. (It's also not a subsidy by the government... so it avoids those problems, too. Instead, it is a source of revenue TO the government.)

All that does is add a "tax" on every product made in the US. It would essentially raise the cost of goods produced in the US to an artificially high level since now US labor and production is "competitive" to out sourced labor and production. As a result, the cost of goods increases and less people are able to buy them. You don't magically create more wealth, you just inflate the costs of goods - less people buy and the workers you helped are out of a job again.

The key is productivity - if you can make more, even at a higher labor cost, it's worth producing it here. Trying to keep jobs by tax policy is counterproductive.

Stress

[MichaelSmith]

I used to co-locate in the same building as the local stock exchange. One day, very late, I took the elevator down to the car park, which was where the computers were. There was a guy in the elevator who looked absolutely wrecked. He was sweaty, shaky and not taking things in. He got off at my level and stumbled off towards a porsche which appeared to be similarly young and in equally bad condition.

The thing is, I work in air traffic control, where the stakes are even higher. The difference is that the operational people have an absolutely obsessive approach to managing their workforce. Traffic controllers are just not allowed to get upset or stressed. In many environments they have unlimited sick leave.

The difference, I suppose, is that traders personally stand to take home a lot of money. You could do this in any field: offer ridiculous compensation for ridiculous effort. But if you work it out, I doubt the long term returns justify what this does to people.

I am an HFT programmer

[Anonymous Coward]

I'll be posting anonymously, but I think many here have a very poor understanding of what we do. Most of that is because we do tend to be a very secretive group, but if you were to sit down with some of us, you would see that we really do very normal (and useful!) things in the market.

I work on the algo and core infrastructure. I wrote price feeds that take 1/5th of a microsecond in C++ and (a little slower) in Java. I understand in fine detail how cache and the the PCI-e bus works. I have a very good understanding of algorithms and the constant-time tradeoffs. I know when to make something simple, when to use and avoid threads, and I can debug in minutes and push out a new version in the seconds before market open (not many people can handle that level of stress well). I read the C++ and Hotspot assembly, and know how to program for superscaler architectures specifically. If you really need me to, I can even crank out some VHDL code.

On top of that, I understand market microstructure and derivative pricing. I work 12 hours a day on average and do 100 hour weeks. I am on call during Asian hours and need to come in sometimes on holidays when other markets are going nuts and we need to plan.

I also hope to make $500,000 this year.

You always hear about Google programmers being the best in the industry, but I've been to a couple Google interviews and turned them down both times because the engineering quality just isn't there. I'd put the average HFT programmer up against the best in Google anyday.

Ask away, and I'll answer to the best of my ability.

Re:I am an HFT programmer

[MichaelSmith]

I can debug in minutes and push out a new version in the seconds

You must be taking a huge gamble by doing that. I don't see how your new version could be considered safe to use if it is deployed that fast.

Re:I am an HFT programmer

[demonlapin]

I also hope to make $500,000 this year

That's all? Demand more, if your skills are what you say. You should be pulling in $2M/yr minimum.

Re:I am an HFT programmer

[Anonymous Coward]

how do you feel about having all that talent and technical ability, yet produce absolutely nothing of value to society and instead spend your time allowing psychopaths to beat other psychopaths by fractions of a second, all to the detriment of everyone else?

Re:I am an HFT programmer

[NFN_NLN]

...we really do very normal (and useful!) things in the market...

You can't tell me that a company changed direction in 1/5th of a microsecond. You're not allocating capital investment to companies based on actual merit, you're skimming money from other investors.

John Templeton and Warren Buffet didn't get rich through micro-transactions. They allocated capital to companies, allowed them time to prosper and *if* they did, then they were rewarded.

You've convinced me of your technical skills but not that "[you] really do very normal (and useful!) things in the market".

Re:I am an HFT programmer

[Anonymous Coward]

Another HFT programmer here. I once had to make a run-time modification to an algorithm to keep about $100 million from going at a lower price than what the traders wanted. Sometimes market conditions change so fast that the traders demand the ability to make rapid adjustments to the algorithm. They're willing to take the risk. They can't wait for the safe development cycle.

Re:I am an HFT programmer

[pauljlucas]

This means that you can prove the correctness of the code, presumably automatically, so presumably very quickly...no need for peer reviews and test deployments.

If that's true, then why wasn't it apparently done in the first place to prevent the bug that he supposedly fixed?

Re:I am an HFT programmer

[Anonymous Coward]

$500,000 divide by 52 weeks = $9,615 / week
$9,615 divide by 100 hours = $96.15

Re:I am an HFT programmer

[Jayson]

To some extend yes, but the order management and routing systems have checks and there are circuit breakers to prevent total trainwrecks. It is a cost/benefit thing. How likely am I to lose $10k immediatey verus make $50k on the market close? One very important skill is in being able to estimate how likely you think you are right.

I wouldn't do this for huge programs or where the lost can be gigantic or I couldn't evaluate the risk.

Trying to cut corners, I did lose $1.6 million one day because I had a bug in my code. A lot of people have these stories when working on some high-risk projects (something HFT place usually try to stay away from). I learned from my mistake and more than made up for it a week later.

Re:I am an HFT programmer

[Antique Geekmeister]

I can tell you that the investors who bought the stock _first_ after the company's announcement of better-than-expected annual earnings was published, and started to push up the value of the stock, might indeed make a lot of money. Being the stock trader that handled the trades for them is also a reliable source of revenue. Being able to sell your stock clients a slight advantage in profits, one that you can measure, can easily bring in an unreasonable amount of extra business.

The business is very strange, and I'm afraid vulnerable to changes by the SEC in how such transactions are allowed. It's inherently unstable: the feedback loops are nearly impossible to trace because they're hidden behind the concealed trading algorithms of numerous companies, and it's very easy for a set of very modest delays to feed back and cause a massive positive feedback loop, unless the system is very heavily damped. Right now, it's not very damped, and this sets the possibility of "high frequency trading" become "high frequency oscillation" until it slams against the floor of the stock market assets.

Re:I am an HFT programmer

[Anonymous Coward]

$9,615 / (40 hours + (60 hours overtime * 1.5)) = $73.96 hourly wage

Re:I am an HFT programmer

[Anonymous Coward]

Indeed, it must be very stressful working 12 hours a day, 8.3 days a week...

Re:I am an HFT programmer

[pyite]

Where do you get these price feeds from (or route the prices to) ? surely you could save 5e-7s just by using shorter cables or putting the mic and speakers closer to the traders.

Nearly all market data is transmitted from the exchanges via IP multicast. Typically you will have servers in each exchange to trade on that exchange, but you also will have links pulling in market data from every other relevant venue as well.

See, for example, Spread Networks who made a lot of money by digging a really straight trench from Chicago to New Jersey in order to get CME data to the NJ metro area as fast as possible.

Re:I am an HFT programmer

[Mr. Freeman]

"They can't wait for the safe development cycle."

This, from the people in charge of trading amazingly large amounts of money in a market which influences the global economy in a big way. (remember that billion vs. million mix-up awhile back that caused some pretty big problems until it was fixed)

You call it risky, I call it reckless. You try what you're doing in any other field and you'd be fired pretty damn quick.

That said, I'm not so much angry at you as I am at the people who ask you to do this.

Re:I am an HFT programmer

[Anonymous Coward]

Yet another HFT programmer here. Let me address your concern about the risk to the market. HFT systems have tons of controls which govern our ability to trade. We have what we call 'doors', which act like circuit breakers. When a door is closed, it prevents any further trading. Doors can be triggered by any number of conditions (abnormally high P&L, too much exposure, etc.), and there are strict compliance policies governing how and when they can be reopened. We also have throttle-style controls ("speed bumps") which effectively limit how many trades we can make in a given time window. These controls can be configured per security, per name, per exchange, or globally. They are extremely robust, and we maintain them meticulously.

We take these matters very, very seriously. We are at the mercy of the exchanges: if we screw up and don't contain the damage, they will revoke our ability to trade, effectively shutting us down.

Re:I am an HFT programmer

[Bing Tsher E]

Warren Buffet didn't get rich through micro-transactions.

Warren Buffet gets rich by loudly proclaiming that estate taxes should be increased. Then he buys distressed companies out from under the family of the founder when they prove impossible to pass on to the next generation.

And he buys companies distressed in other ways to 'fix' them or to chop them up and sell the parts. Truly an American Hero, worthy of the adulation he gets as the poster boy 'good capitalist' by the left.

Re:I am an HFT programmer

[Anonymous Coward]

Not HFT here, but I'm a finance quant. Testing is a critical part of banks' infrastructure--for the obvious reasons but also regulatory reasons. For example you'll find derivative pricing models subject to high levels of scrutiny. The standards may be lower at hedge funds and other unregulated (in the legal sense) entities.

But you have to be able to fix stuff in the middle of the day. Sometimes things break, or market are going crazy, or maybe the trade you thought you booked wasn't the actual trade you signed. And you need to push out a fix ASAP because you might be losing money, or otherwise causing huge amounts of risk.

Quants are hired to be able to do this kind of coding AND NOT SCREW UP. A lot of the work we do (not all, but a lot) could be done by Joe Programmer if it weren't for the requirement that you can't screw up. Trading is a dangerous occupation and that's why they try to hire the best people.

Re:I am an HFT programmer

[bertok]

Just because you're good at what you do doesn't mean that what you do is good.

Microseconds? Why not nanoseconds? How about femtoseconds? Why wouldn't that make just as much sense?

What could possibly change in the underlying value of a corporation made up of flesh and blood humans and capital with decades of depreciation in a fucking microsecond? Here's a hint: nothing. You are not investing, or trading, but simply racing other gamblers. Investing doesn't benefit from microsecond response times, and trading doesn't need it either. People could buy IPO shares just fine over the phone. Nobody ever needed a microsecond response time to buy a thousand bushels of wheat, and never will, because bread is baked daily, not a million times a fucking second.

If politicians had two braincells to rub together, they'd enact a law to prevent trades faster than some tick, say, an hour. Your 'trading' company would go out of business in a week, and nobody would care. Farmers would still sell their wheat, and bakers would still buy it, but without you leeches skimming off the top.

Re:I am an HFT programmer

[jampola]

I concur. Stuff like this makes me angry. It's just stupidly reckless. Reading OP's essay, it sounds quite glamorous in a geeky sense but seriously, I don't think any amount of money would woo me into that kind of recklessness with any kind of responsibility attached to to. In regards to what happened a few years back, if I recall, that was some guy who accidentally entered an extra 2 0's assuming that he needed to include the decimal at the end. Not completely his fault, since I would've thought the system he was using had some kind of system where it spits out the amount in words? Or maybe not since my trading platform allows for one click trades.

Re:I am an HFT programmer

[rdnetto]

What can you tell us about the trading algorithms? I understand that it's mostly trade secrets, but if you could even say what broad area it's in, I'd appreciate it.

Re:I am an HFT programmer

[Alex Belits]

I am most likely better than you at each and every aspect of software (and HDL) development you have mentioned. Except, of course, "debugging in minutes" -- that kind of irresponsibility would get me fired. I also have to work long hours, and have to have clear understanding of complex issues unrelated to software.

Except I do embedded software and FPGA development for professional audio equipment. Each device I worked on, each firmware release, each line of code, does something useful for many, many people. Some of those people don't even know that audio equipment, leave alone software, is involved with what they are hearing. Large fraction of my work ends up being free/open source, too -- platform, drivers, etc.

I also don't have any problems with posting here under my real name. Or with telling you, and people like you, to die in a fire.

Re:I am an HFT programmer

[Renevith]

Do you ever buy or sell stock? Perhaps indirectly, through a mutual fund or 401k type plan? If so, then you benefit from high liquidity in the market. HFT and other Wall Street shenanigans do skim from the top, but they also provide liquidity. It's almost certain that the liquidity benefits small market players more than the skimming hurts them. In other words, the money they're skimming comes from the banks and brokers rather than you and me.

For example, take the stock of Red Hat (http://finance.yahoo.com/q?s=RHT). Yahoo Finance right now shows that, as of the last time the market was open, I could buy 100 shares for $42.56 (that's the "ask" or best current asking price), or I could sell 300 shares for $42.09 (the best available "bid"). That's a bid-ask spread of about 50 cents. That spread is a hidden cost to either buying or selling stock: If you buy and then sell RHT, you will have paid about 50 cents per share just for the privilege, even if nothing in particular happens to the company. Let's split that 50-50 and say that every stock transaction in RHT (buy or sell) costs you 25 cents per share in implied fees.

Those bids and asks are set by individuals and companies who are competing. They want to get a good deal for either buying or selling the stock, but they also know that if they set asking price too high or their bid too low, they'll never make any trades. The more competition there is, the tighter the bid-ask spread will be. HFT and other algorithmic approaches allow firms to set prices on tons of stocks without requiring human attention for each one, which dramatically increases the competition and thus tightens the bid-ask spread.

In this example, if you outlaw HFT and similar trading strategies, maybe RHT will have a spread of $1 intead of 50 cents. Maybe you'll be happy that HFTers aren't making ther 5 cent skim off the top anymore, but it'll be cold comfort when you're paying 25 cents more on each transaction and it's just going to a different Wall Street firm.

If you think I'm exaggerating the effect of computerized trading of the spread, have a look at slide 8 (page 4) of this study: http://fisher.osu.edu/~diether_1/b822/trading_costs_2up.pdf. Starting in 1960, the average bid-ask spread has ben dropping steadily every decade to a small fraction of what it used to be.

Background: I am an actuary trained in quantitative finance. I've never worked in Wall Street or done any HFT or other algo trading.

Re:I am an HFT programmer

[TheRaven64]

You haven't really explained why that's a bad thing. If the spread is more, then that means you can't make money from he noise as easily, you have to actually invest your money based on your expectation of the long-term performance of a company (or invest it in unusually volatile stocks, if you really like gambling). It seems that this would make it easier for companies that had good long-term business models to get investment, which would be good for the economy overall.

Re:I am an HFT programmer

[s73v3r]

Losing money is also defined as not making as much as possible. If you were making $1000/minute on Monday doing something and only $100/minute on Friday (many bots only run for the first 5 to 10 minutes of the market); then that is also defined as a loss of $900/minute.

I find that statement quite disgusting, and also indicative of why we're in the trouble we are as a country.

Re:I am an HFT programmer

[JavaRob]

Err... he didn't say he does 100 hour weeks *on average*, just that he does them.

I.e., his average week is 84 hours (assuming he works 7 days a week; otherwise 72 or 60 hours a week), and sometimes he works as much as a 100-hour week (7 14-hour days, or some other split).

Come on now,

Re:I am an HFT programmer

[NoOneInParticular]

Why would information about a company only change on some tick, say, an hour? Wouldn't that be unfair to firms that are not a phone call away? Why not make it 24 hours, or a quarter? Open the trade floor once a year for 15 minutes. You make a mistake, wait a year. Probably often better than waiting an hour.

So, what could possibly change in the underlying value of a corporation in a microsecond? Nothing, you are right. The stock market is however not trading the underlying value (as this value is unknown and unmeasurable, and given that the underlying value also includes expectations about the future, it probably doesn't exist at all), but rather the information about the underlying value. And that information can change by the microsecond. Each trade establishes a new datapoint about the underlying value of a corporation. So one single person, or algorithm, that thinks that this particular stock should trade a quantum higher or lower, makes a statement about the underlying value of a corporation.

Your idea of a trade an hour would make that tiny bit of information aggregate into a big thing. Big things lead to massive swings in the stock market. People get scared and do wild stuff. So, if anything, HFT has removed volatility, not added to it. Yes, everyone also found out that if one person makes a stupid bet, all those suckers that use stop-losses to protect their investment get creamed. It used to be the market maker that would clean out the suckers, now its the algorithm. Nothing has changed. Don't use stop-loss, use options for protection.

In the past, it used to be who had the fastest horse. The New York stock exchange and the Chicago stock exchange could be out of sync for weeks. If you traveled from New York to Chicago with some important news, and beat the other guy, you could make millions. This news, when brought by horse, leads to massive changes and immediate bankruptcies as there is no time to react to such large bits of news. But.... buggywhip makers rejoiced. Then the telegraph came. It become important to have the fastest morse signallers. Then the phone came, and then it became quiet for a while. The limits of the technology were reached and the playing field was leveled. Trades could now be done by professionals and regular people. Then the computer came, and the network. We are now again in a speed race that will last until some physical limit will be reached. Lots of energy goes in there, but not more than the gains. When this all settles down, there will be a steady source of arbitraders that will remove inefficiencies in nano-seconds. Brilliant engineers will leave the field, and another level of artificial volatility is removed from the markets, leading to less artefacts created by the discrete nature of trading. Time is made continuous. And that makes this whole trading thing more stable.

Is it worth billions? Well, yes. The cumulated effects of the inefficiencies in the market apparently add up to such sums. If you leave these in the market, they will be translated to unnatural volatility.

Re:I am an HFT programmer

[hexagonc]

What could possibly change in the underlying value of a corporation made up of flesh and blood humans and capital with decades of depreciation in a fucking microsecond? Here's a hint: nothing.

Not exactly true. If companies were islands isolated from each other and other external factors then you might have a point. However, as we all know that is not true. The value of a company can indeed change within microseconds and less because some external factor outside of that company's control might affect its value. For example, suppose some disruptive event occurs in the world, such as a coup in Iran or a UFO landing on Washington. The value of a lot of companies will change instantly (well, really at the speed at which information can flow, which is on the order of nanoseconds). I imagine our HFT friends here would want to be amongst the first to act upon this information, either minimizing loss or maximizing gain from such an event. This doesn't strike me as voodoo or particularly nefarious/underhanded/unethical, especially since there would probably be obvious losers depending on the type of event.

Re:I am an HFT programmer

[Alex Belits]

What about it? He is a loathsome being that causes harm to other people, there is nothing wrong with wishing him to die in a fire.

yeeeeeah

[Anonymous Coward]

I'm past the whole "a-holes can yell at me if they pay me enough" phase.

from an HFT developer's view

[Anonymous Coward]

I'm a developer at an HFT firm and can say that it is some of the most interesting work I've done in my career. You need to wear many hats for the job.

On a daily basis I multitask between making low-level kernel modifications to reduce system latencies, to refactoring our high level marketdata->prediction->execution engine. It's a never ending balance/conflict between making things as fast as possible while trying to maintain good design principles and not take shortcuts in the sole interest of efficiency.

To those that say HFT is an anathema to the economy, I think there is some merit, but there are two sides to every story. I've definitely met some other HFT shops that focus on some really shady and morally questionably tactics (essentially DDOSing markets to gain an advantage). However, the amazing advancement in network technology and all these crazy RDMA and 10G user-space network stacks is literally due to the HFT community, and I'm already seeing these new technologies start to penetrate other markets. This is capitalism at work.

Also, believe it or not, HFT has reduced spreads due to competition, which means you pay less when you want to buy or sell a stock RightNow instead of using a limit order (and while spread reduction is good for investors, it sucks for us because it is not as profitable as it used to be a few years ago).

The "Billions" in profits going to HFT firms right now previously went to the big investment banks and the old classic broker taking orders over the phone and charging a ridiculous market spread.

word on the street...

[kervin]

As some of the interviewers in the article hint at, HFT is a small niche that's getting way more than its share of attention.

I'm not saying HFT isn't dangerous, or something to keep an eye on. But if you're looking for a job in Manhattan financial sector, you'll more likely be working in web or other "high level" stacks.

Java is *huge*, .Net is also very popular. Web is, of course, very important with just about every stack, from Ruby to ASP.Net represented. I see a surprising about of MatLab for analysis ( I guess I just wasn't expecting to see any with QuantLib and similar floating around ).

How long to programmers stay in that type of job??

[Cutting_Crew]

I get e-mails ALL the time from these people up in NYC. If i so much as wink at my resume online differently I can always be certain that I will be receiving many opening jobs in the trading and banking industry. In fact, throughout my career, this has always been the case. I live in Florida. Why in the world would they be trying to recruit me from florida and why have I always gotten e-mails from them. Is it because they have a lot of people leaving? Do people get frequent burnout? Are they just spamming?

The last "recruiter" to e-mail me wanted to know if I was looking for a 'great' opportunity in the banking industry writing C++ applications with a big whopping salary of $100,000 with a 12 month contract. Since i was tired of these people wasting their time trying to contact me I e-mailed them back and asked why they thought I would just get up and move from Florida to New York for a 1 year contract and i tried to get confirmation of a starting salary of $100,000 a year. He e-mailed back stating that $100,000 was indeed the starting salary with great 'benefits'. So i e-mailed back and informed him that i was already making $75,000 there and no state taxes so $100K really wasn't that great of a deal, especially considering that I would have to make 4 times my current salary that I am making now just to come close to breaking even in NYC. Incidentally I never got another e-mail back from him.

When you have a family, there is no way I am giving up a house to move to NYC to live in a one bedroom closet for the same price as my 4 bedroom house. - with less income coming in.

you know who else posted bullshit analogies

[decora]

on internet forums?

that's right. Adolph Hitler.

He wrote hundreds of pages 'exposing' the 'truth' behind 'power'.

It was called 'the protocols of the elders of zion'. of course, the whole thing was bullshit. made up by some anonymous author, possibly the Russian Tsar's secret police, to support yet another pogrom.

But Hitler took this and ran with it. Over and over this document, and many others, including his book Mein Kampf, were given out by the tens of thousands. People were happy to 'forward' these 'revelations' to others, often tweaking details here and there, or changing the attribution of the author(s).

And what happened in the end? Trillions of children were killed. I'm a student of history too. I have over 5 billion books published. On the internet. You can look it up.

enough lies please

[decora]

we all understand what 'arbitrage' is. when the synthetic CDO market calls their deals 'arbitrage' we all know its fucking bullshit.

when the sales guys in the brochures talked about the 'AAA' ratings on these pieces of 'arbitrage', it was all bullshit.

when Lloyd Blankfein calls it 'hedging, not betting', its fucking bullshit. ]

there is absolutely nothing, whatsoever, 'valuable' behind a credit default swap. it is a bet. that is a fact, and its not rocket science, and its not a conspiracy theory, and its not "the ignorant and alarmist" decrying some nefarious boogey man. its the basic fucking fact of what fucking happened.

I beg of you. stop lying. nobody believes you anymore. this is like the scene in Shattered Glass when Peter Saarsgard has to finally explain to Hayden Christiansen that the whole charade has ended.

the financial industry has no clothes. we all know it. there is no point in pretending.

Re:enough lies please

[pyite]

there is absolutely nothing, whatsoever, 'valuable' behind a credit default swap. it is a bet. that is a fact, and its not rocket science, and its not a conspiracy theory, and its not "the ignorant and alarmist" decrying some nefarious boogey man.

You're completely wrong. If you buy a bond from company X, it certainly makes sense to have insurance that if company X goes out of business, you still get your money. And hence, the credit default swap was born. The fact that someone may use the instrument to speculate is a separate issue.

People speculate on everything. It's what you do when you stock up on cans of soup when it's on sale. You speculate that the price is going to go up next week.

Re:enough lies please

[obarthelemy]

there's a strong moral risk: if you can both buy a company's debt, and insurance in case they don't repay it, why not keep doing that endlessly ? you make money both ways, nobody is really interested in the underlying debt quality, and bankers make fat commissions on both sides of every deal.

oh, wait...

Re:enough lies please

[UnknowingFool]

As a derivative, CDS are really under no regulations at this time. They were sold under hedge funds but those in themselves are derivatives. They are supposed to be regulated under Commodities Futures Trading Commission. In the 1996s the head of CTFC, Brookely Born started learning about OTC (over the counter derivatives) which were essentially unregulated. Under the mandate given by Congress, her agency was supposed to regulate them. She got enormous push back from practically all the major players including Fed Chairman Alan Greenspan against any sort of regulation. What got her attention was not so much the market wished to be unregulated was that they fought furiously against any disclosure about big the market was.

Greenspan and the banks, pressured Congress to essentially gut her agency of any regulatory power. Still she warned Congress back then that it had power to take down the financial markets as the banks were not required to disclose how much money they owed or to whom they owed money. In 2008, she was proven right when one after one investment banks like Lehman Brothers and Bear Stearns started to collapse due to over-leveraging.

Because they are unregulated those that invested in them essentially had no idea how sound the purchase was. At the time of their collapse, Bear Stearns were leveraged $13.40 trillion dollars in derivatives but only $395 billion in assets. It was only after their fire sale to JP Morgan that saved them. Since disclosure was not required, how many funds and companies had money in Bear Stearns and Lehman Brothers that they thought were "insured" against risk.

funny you mention potatos - see NYMEX

[decora]

there is a great book that just came out, The Asylum, by Leah McGrath Goodman , which explains how the potato market became a cluster fuck of manipulation and greedy assholes absolutely stealing from the ordinary person.

it also explains why certain industries were banned from trading this shit. why? because you cant operate a society where the price of basic commodities fluctuates by several hundred percent a year just so that a handful of a few dozen traders can make massive amounts of money through manipulating the market.

how much cocaine do traders use?

[decora]

and how many prostitutes do they kill, on average, per year?

Wall Street

[br00tus]

I worked for a Fortune 100 financial company in Manhattan. Some of the nastiest people I ever worked with were there. I also never worked anywhere where the company made more clear to me that I was a disposable cog. Not that that isn't the case elsewhere, but management usually tries to at least conceal it. I worked 60 hours a week, including weekends, and I was on the low end compared to others who were working to move up in the hierarchy.

There was a limited and fixed bonus pool, so if you got less money, others got more. I'm sure this was a plan by management - the firm was so wealthy, they felt it better their workers be divided against one another than working together. They did this in a variety of ways - staff versus contractors, contractor firm versus rival contractor firm and so forth. This encouraged people trying to rip one another apart during weekly meetings, code reviews and the like.

The office politics can be strange too. I used to be on conference calls with a programmer with a huge ego, and who people deferred to because the program suite he wrote was important for the firm. The program was shoddy though, it had massive memory leaks and the like. But he and the team under him were able to throw it together quickly and the business people were happy with him so he was a golden boy. It wouldn't have been so bad if he wasn't always denigrating everyone else's work with a holier-than-thou attitude. I wasn't in a position to say anything though. Lots of stuff like this happens. A lot.

It was not all bad. It was a large environment. Stuff I would have done maybe once a year at a smaller company I was doing every day there. I was surrounded by dozens of people who were sharp and knew what they were doing. There was a feeling of camaraderie among some of us. It is hard to explain the change in quality due to the sheer size of the company, with its ability to spend massively if needed.

One thing I will say - unless you are there to work 24/7/365 and try to make it to be one of these "highest paid programmers", there is no reason to be there. Some of my co-workers joined straight of college, which seemed dumb to me. Anyone who takes a job for less than six figures is a fool - if you don't have the skills to make at least $100k, there is no reason to work there. It is not the place to come in at a low level and hang around.

To repeat a point - anyone who takes a job on Wall Street right out of college is a fool. They recruit heavily on campuses for the same reason Microsoft and Intel and Electronic Arts do: so they can take advantage of suckers with no work experience who don't know any better. Thankfully I didn't make this mistake. I was able to put everything I saw there into perspective. Going from college straight to Wall Street as a programmer is a dumb move for most people.

Re:Ada!

[n8r0n]

I worked on defense systems in Ada for a number of years. While I love working in C# and Java, and sometimes Objective-C, I have to say that Ada had some really great features (some of which clearly influenced later languages).

It might seem like Ada would be a great language for financial services, as it was designed with security and reliability in mind. But, alas, the financial sector is really less interested in those features, and vastly more interested in skimming their clients' money while telling them how lucky they are to have the liquidity. I don't remember Ada including a library for that (system.scheming.ponzi, maybe?).

Re:Who likes to be screamed at

[jcr]

I've heard stories like this, but I never encountered anyone yelling at me in all the time I spent in the financial industry. This was over several years in NYC and Chicago, at Salomon, UBS/Warburg, JP Morgan, and Phibro energy. The only time anyone ever did yell at me at work, I just turned my back on him, went to my desk, and calmly packed up my stuff while he and his boss were frantically apologizing and begging me to stay.

-jcr

Yet Another HFT Article

[hsk17]

Disclaimer: I work at an HFT firm.

The implied accusations are flying out of the page like daggers. I wish you, Slashdot readers, could see the world through my eyes. As techno-savvy as you are, you somehow love to hate on HFT without having any idea what it is. Don't get me wrong -- I really don't care if you hate it. What bothers me is that haters have NO IDEA what HFT is doing and basically project their hatred for finance onto it.

I have to say, this article is pretty level-headed. I was expecting more baseless accusations. Of course, the article throws around the typical "HFT was blamed for the huge drop in the stock market in May 2010..." If you cared to look at the linked WSJ article, you would have read that Waddell's desk had sold 75,000 E-mini contracts at the start of the flash crash. If you cared to look at the CFTC report that officially investigated the flash crash on May 6th, you would have read that CFTC blames the flash crash on some trader who executed a large sell order worth $4.1 billion dollars -- why, isn't that just about 75,000 E-minis?

You would have also read that HFT firms actually mitigated Waddell's mistake. They were there to absorb the thousands of E-minis and so dramatically lessened the initial impact. It's really quite admirable the amount of precision coders needed to invoke in order to create a system that executes so quickly and at scale during such a turbulent period. I was hoping that the discourse here would be more along those lines.

Unfortunately, the amount of volume that Waddell executed was too much risk for the traders to bear, so they started getting out of their positions. In fact, no one could handle a trade of such size. It was as if someone predicted the collapse of the US economy and bet $4.1 billion on it. The ensuing chaos was purely the after-effects of the initial destruction caused by Waddell.

New technologies can be used for bad. I bet there's plenty of bad traders manipulating the markets and using speed as an unfair advantage. We need to police HFT, for sure. But I'm also sure that people are using guns to kill other people out of malice, using cars to traffick illegal drugs, and using airplanes to destroy buildings. HFT is a style of trading. It's a technique, not a strategy. The sooner we realize this, the more progress we will make as a society in implementing policies and regulations.

You guys all hate on HFT, but you are really the ones benefitting from this technology. In market making there's a spread -- the difference between the lowest ask price and the highest bid price. Take a look at the most liquid stocks. They are probably trading at 1 cent wide spreads. Compare that to years ago when spreads used to be dollars. Go to a bank and look at the currency exchange rates. I just did a look into Bank of America's spreads. 100 euros gets you 135.35 dollars. Based on recent trading prices on the public exchanges, 100 euros should be able to fetch closer to 143.62 dollars. BoA is charging a spread that is more that 5% of the value of the product! I don't blame them -- the landscape in the currency markets discourages technological innovation and competition.

This phenomenon isn't true just for currencies. It's true for most products that are not regulated or traded publicly. You, the average investor, are being ripped off dearly investing into these opaque markets. The size of spreads is truly a symbol of capitalism. If there's competition, the spreads are tight. If there's monopoly, the spreads are wide.

You complain about HFT being super fast and "shaving off transactions" as if we somehow have access to your accounts and embezzle money a la Office Space. That's like complaining about WalMart having such efficient systems and internal logistics that your cereal is getting too cheap. Yes, I do believe that some traders use speed unfairly, and yes, WalMart probably did shady things we don't know about, but my point is that not every trader is bad. Technology

Re:Yet Another HFT Article

[benhattman]

Sorry, but I don't buy it. I posted something similar above, but please explain how pushing transaction speed from 50 microseconds to 44 microseconds actually benefits any normal person. I honestly don't believe it does. But, the HFT who has that faster algorithm becomes tremendously wealthy while the 10% slower algorithm is put out of business.

You can state that you are providing liquidity to the world, which you are, but compared to the liquidity we had perhaps 10 years ago, I really don't see how you're work makes anyone's life better except perhaps your own. And it all brings with it the risk that my own stock holdings may be decimated in literally the blink of an eye. The fact is, your line of work is looked down on because you are essentially gaming a system. Comparing it to cars used in traffic is not accurate. HFT is more similar to the people who used their credit cards to buy $1 coins for the frequent flier miles, and then deposited them in the bank.

http://www.creditcards.com/credit-card-news/mint-closes-loophole-ends-credit-card-coin-sales-frequent-flyer-flier-miles-1263.php

Re:Yet Another HFT Article

[Xyrus]

I don't think people here have a problem with HFT technology or understanding it. It's how it is being used (and abused) that make people angry.

The anger directed at HFT is really misplaced. It should be on the banks and firms who use it, shady practices, and dark markets to bend the world over so they can make more money. I think people are also angry with the fact that these companies continue to rake in the cash and profits, while simultaneously bitching about taxes. A recent report cited companies making considerable profit, and yet unemployment rates are not improving all that much. Isn't it their mouthpieces in congress who claim that lower taxes will spawn job creation?

The anger comes from getting economically ass-raped seven different ways till Sunday, and then having to pay the sociopathic assholes for the honor of being ass-raped. That's why HFT, CDOs, big banks, and financial institutions have been the target of all this venom and hatred. It's not the technology or the ideas or the products. It is what was done with them that pissed people off.

That being said, you don't work for some benevolent altruistic company. You don't work for a company who is concerned about the greater good, or furthering the progress of humanity. You aren't performing some great majestic service for mankind. You work for an HFT company, who's only concern is to make assloads of money. You're upper management could give a fuck less about "balancing the markets" and even less about wiping out grandma's savings and throwing her out in the street. As long as it pulls in profits, they'd be happy kicking babies off the top floor of their 30 story trade tower.

So don't come on here and pretend that HFT is some mechanism intended for the benefit of all. HFT is what it is, and that's a money making tool. The fact that it has some fringe benefits for the rest of us serfs is purely a unintended side effect.

Re:Yet Another HFT Article

[soks]

Hsk17 is in fact quite correct and you sir cannot be. There are only two ways to "peak at the future prices." Either your computer is faster than everyone elses, in which case the price reaches your logic/screen faster (if you call this cheating... umm... I don't know what to say to you) OR what some exchanges offer is called "flashing."

First thing about flashing, not all exchanges offer it. In fact America's largest futures/options exchange (think crude oil, corn, carbon credits) does NOT offer any form of flashing. As for the exchanges that do offer flashing, it is simply an added service that any of their clients can sign up for. It is a special high speed connection that, if your firm has the technology to handle it, allows you to see orders as closely as possible to when the exchange receives them. This is not cheating, anyone can do it, you just need to have the funds to support the required technologies. No doubt if you know what you are doing you can get those technologies relatively cheap (likely a special NIC w/ whatever hardware can support it (linux!)).

As for risk being related to holding time, that is just wrong. Risk is generally considered in terms of how far can a price go in a given amount of time. An option contract that is farther out (read: longer potential holding time) will cost more due to the increased chance of volatility (read: price change, up or down, good or bad) through that longer period. Thus more time = more risk, as represented by the higher price of a longer contract.

As for what wallstreet can and can't afford, sure, there are imbalances in our economic system that allows certain industries to work with more money than they should. However imbalance usually comes from the use of force against one group to make them do something they wouldn't normally do, thus giving someone more favor (money) and perhaps creating an entire industry that simply shouldn't exist. I urge you to consider my proposition; Any time you see trading in a brand new high volume industry you should not say "trading is bad, look at this false industry" but you should instead say "what is the industry that utilizes this trading from the buy side." That is, who uses this type of trading for their actual business, as a client, not a trader.

See when an exchange offers traders to trade something like corn futures contracts, that means there are people out there (real, hard working people) who benefit from entering said contracts to mitigate price risk. The ability to mitigate price risk allows seasonally impacted industries to average out their costs and provide more stable prices to their consumers. Once this need for a contractual service exists it is the exchange who goes out to market makers (a type of trader) and says "hey, we have people trading, we need you to guarantee them realistic and competitive prices to trade at." They go out to brokerages (not really considered 'traders' but they do trade) and say "hey, we have products and if you bring us clients you can charge them fees for the complex task of processing, placing, and managing orders on our exchange." Market makers are speculators and there are many other types of speculators of which some may or may not be HFTs. If HFTs were harming the market for other brokers or speculators then the exchange would simply slow down their transaction rate (which would be business suicide) and even this would not stop the HFT market. That is, you will still want to have the fastest reaction time to market events possible and you can likely game transaction rate limiting systems as well.

Either way, it's just competition, and if it interfered the exchanges would not allow it.

Discrete time

[hakonm]

Why not divide time into discrete blocks, e.g. one each minute or so? All bids placed in the current time block are then traded simultaneously at the end of the current block. If traders can only look at data form previous time blocks, I think all this BS could be avoided.

The Penn effect; Community Reinvestment Act

[tepples]

Further, outsourcing is not "fair trade", because it artificially bypasses exchange rates.

Then perhaps the problem is that the exchange rates are lower in the first place. The "Penn effect" is an observation that less industrialized countries have cheaper currencies than purchasing power parity alone would predict. This is traditionally explained by the Balassa-Samuelson model that developed countries are more efficient at producing tradable goods and services. Some goods and services are "local": some goods spoil quickly in intercontinental transit, and personal services such as health and beauty care require face-to-face interaction. Others are far more tradable: durable goods have a shelf life far longer than the weeks it takes for a container ship to cross an ocean, and many of us are aware of offshore outsourced services. The value of a market's currency is related to the market's efficiency at producing tradable goods and services because local sectors have to increase their wages to keep workers from defecting to a tradable sector. See more detailed discussion of this effect. So once a country industrializes, its currency will become more valuable, and wages will rise.

Plain and simple: we need to bring the manufacturing jobs back home.

And according to the Balassa-Samuelson model, the way to do that is to make manufacturing here far more efficient than manufacturing there. This is already the case for automobiles, which I admit are less than perfectly tradable due to the cost of intercontinental shipping (case in point: the major car makers headquartered in Japan have plants in the United States). But you don't see cars of international brands being made in, say, Vietnam because the infrastructure isn't there yet.

You saw billions being spent to acquire companies that had never earned a dime of profit, and did not even have a good plan for doing so.

I'd argue that the eventual plans for the dot-coms were 1. to build up a user base whose demographic was attractive to advertisers, and 2. to offer services at a fee.

Later, banks and finance companies were making bad loans on real estate for a number of reasons

I've been told that one of the justifications for the rise in subprime mortgage lending involved mandates from the U.S. government to make home ownership easier for lower-income people, especially those in groups that were traditionally targets of racist discrimination. See Community Reinvestment Act.

Re:The Penn effect; Community Reinvestment Act

[Hognoxious]

I've been told that one of the justifications for the rise in subprime mortgage lending involved mandates from the U.S. government to make home ownership easier for lower-income people, especially those in groups that were traditionally targets of racist discrimination. See Community Reinvestment Act.

Told by Fox News, I take it.

If you'd bother to read the article you linked to, that myth is well debunked. Stick a fork in it already.

Now if you can explain how the act caused such a fuck up in commercial property, when it only applied to residential, then we'd all be happy to hear it.

Re:Liquidity

[benhattman]

Liquidity is a commodity of diminishing returns. If I put in a sell bid on stocks and you say it will take one month, the team than can sell in a day instead is indeed providing a valuable service. And going from taking a day to trade down to an hour or say five minutes is quite valuable too (though not nearly as much so). But the moment your liquidity is faster than my ability to be informed about it, additional liquidity has ZERO value. It takes me several seconds to click sell on a website, and watch as the site refreshes to inform me the transaction has occurred.

In the 1990s, these people were creating value. Today, they are exclusively leeching money from the rest of us.

Re:HFT borderline illegal

[hsk17]

You're confusing flash orders, which is indeed illegal -- they allow firms to get a peek at customer orders before the orders hit public exchanges.

But being fast as a competitive advantage, given that everyone receives the same information -- that's something entirely different. By the same argument you're making, Google should be fined because they have infrastructure that's vastly superior to the average person. If the average person wanted to compete with Google, he or she would need to build too many things.

I'm being a bit facetious, obviously, but you have to distinguish these concepts.

No, it's not about slavery.

[198348726583297634]

It's about the world we want to live in, which stands in ever-starker contrast to the world we do live in.

Do you want to live in a world where the best and brightest throw their efforts away with such mundane, trivial shit?

Again, not where-are-we-now, but where-do-we-want-to-be.

Do you want to live in a world where, thanks to some tricks of law and circumstance, a handful of people have such domination over billions?

Again, not where-are-we-now, but where-do-we-want-to-be.

You don't have to be a slaver (wtf??) to say, the world we live in is wrong, because a lot of people are suffering for what seem like really lousy reasons.

You do, however, have to be a decent human being.

Re:No, it's not about slavery.

[russotto]

It's about the world we want to live in, which stands in ever-starker contrast to the world we do live in.

Do you want to live in a world where the best and brightest throw their efforts away with such mundane, trivial shit?

I don't want to live in a world where it's my business where other people choose to place their efforts (nor theirs mine, obviously). And that's why I think that particular argument against HFT -- that it drains talent from other "more important" fields -- is fatally flawed.