Slashdot Mirror

← Back to Stories (view on slashdot.org)

How and Why Wall Street Programmers Earn Top Salaries

Posted by Soulskill on from the are-they-holding-the-economy-ransom dept.

msmoriarty writes "Given the level of interest in the recent highest-paid programmers discussion, our reporter decided to do a follow-up looking into the languages and skills needed to work on high-frequency trading systems. There's actually a pretty wide range of languages/tools used, but Linux is the 'default' OS and, not surprisingly, the 'ability to work under pressure when the traders are screaming at you' is a must-have skill."

21 of 791 comments (clear)

  1. Perversion of Capitalism by NFN_NLN · · Score: 5, Insightful

    "In high-frequency trading (HFT), programmers eke out every last incremental tick in performance to build algorithms that battle other algorithms for computational supremacy and millions in profits -- and earn a lot in the process."

    Skimming money off billions of micro-transactions. Ahh, yes... forget investing in ideas and backing well managed companies... this is the way capitalism was envisioned.

    1. Re:Perversion of Capitalism by NFN_NLN · · Score: 5, Interesting

      This guy nails it - http://boards.straightdope.com/sdmb/archive/index.php/t-601887.html

      "It's not about acting on market information. It is purely arbitrage. A mis pricing allows one to buy and sell simultaneously and lock in the difference minus trading costs.

      In the old days, traders used to do this in the trading pit. Now it's computers closest to the exchange feed.

      Tied in with this is the automatic trading. In the case of that big intra day fall, a wrong trade was entered. I forget the details but it was big enough to push down the market xx amount, which triggered automatic sell programs from non-arbitrage automated computer selling, which triggered a market sell off, which in turn triggered more selling until the market circuit breakers kicked in.

      During the mandatory no trading period, the original bad trade was discovered and reversed. This IIRC also triggered automated buy programs and the whole thing went in reverse. The bad thing is that the market whipsaw really hammered some real end investor trades as collateral damage.

      I remember watching the Hang Seng Index the day that Soc Gen announced Jerome Kerviel's fraud and liquidated the positions. It was a full trading day of massive market swings for big losses to big gains several times throughout the day. It was almost all computer generated programmatic trading."

    2. Re:Perversion of Capitalism by Anonymous Coward · · Score: 5, Insightful

      "Perversion of Capitalism"

      No it;s exactly what you get in the real world where market values (profit) drive everything. We see it all the time in offshored jobs, destroyed lives, rolling back of the welfare state, the election of extreme right wing conservatives like Stephen harper.

      This "there is some pure capitalism we have to get back to" bullshit is just that - bullshit. The left was born from captialism killing workers, it caused two world wars and then then there was the cold war. This idea that is some 'benign' capitalism we have to get back to is just utter american ignorance.

    3. Re:Perversion of Capitalism by bjourne · · Score: 5, Insightful

      I think you are missing the point completely. It is not that money is being moved around in what basically amounts to a huge zero-sum game. One daytrader has better computers or lower ping to the nyse and beats out another trader who hasn't. That's really not a problem. The problem is the huge amounts of resources that is wasted on this game and the impact we are letting it have on our lives. The worlds brightest minds are spent in the game. You may not see it as a problem that the best mathematicians and programmers are working in the finance industry instead of developing a cure for cancer, affordable space shuttles, electric cars, aids vaccine or whatever because the salaries are much higher there so obviously that is what the market wants and the market is always right. But I do, I think it is a waste. But the worst problem is the importance we are giving to the stock market game. The idea was that the stock price should reflect the progress if its company. Now it's the other way around. It doesn't matter what the company does, if the stock price is high, then that's good otherwise it is bad. Oh and if the price of most stocks are low, and most players in the game have lost, then that is really bad. It's a depression coming and because the game was busted the rest of society will have to clean it up.

      --
      Football Odds
    4. Re:Perversion of Capitalism by Anonymous Coward · · Score: 5, Interesting
      Be careful when you read that long post; who knows who made it and with what purpose. Try to determine for yourself which sentences ring true and which are shaped as a straw-man.
      This is from the Wikipedia article http://en.wikipedia.org/wiki/Definitions_of_fascism: Umberto Eco defines fascism with the following features:

      The features of fascism he lists are as follows:

      • * "The Cult of Tradition", combining cultural syncretism with a rejection of modernism (often disguised as a rejection of capitalism).
      • * "The Cult of Action for Action's Sake", which dictates that action is of value in itself, and should be taken without intellectual reflection. This, says Eco, is connected with anti-intellectualism and irrationalism, and often manifests in attacks on modern culture and science.
      • * "Disagreement Is Treason" - fascism devalues intellectual discourse and critical reasoning as barriers to action.
      • * "Fear of Difference", which fascism seeks to exploit and exacerbate, often in the form of racism or an appeal against foreigners and immigrants.
      • * "Appeal to a Frustrated Middle Class", fearing economic pressure from the demands and aspirations of lower social groups.
      • * "Obsession with a Plot" and the hyping-up of an enemy threat. This often involves an appeal to xenophobia or the identification of an internal security threat. He cites Pat Robertson's book The New World Order as a prominent example of a plot obsession.
      • * "Pacifism Is Trafficking with the Enemy" because "Life is Permanent Warfare" - there must always be an enemy to fight.
      • * "Contempt for the Weak" - although a fascist society is elitist, everybody in the society is educated to become a hero.
      • * "Selective Populism" - the People have a common will, which is not delegated but interpreted by a leader. This may involve doubt being cast upon a democratic institution, because "it no longer represents the Voice of the People".
      • * "Newspeak" - fascism employs and promotes an impoverished vocabulary in order to limit critical reasoning.
    5. Re:Perversion of Capitalism by Anonymous Coward · · Score: 5, Interesting

      Look, what am I supposed to do.

      I'm a chip designer. I used to work on beautiful hardware, you might be using some of my products to play games even today. Turns out, ASIC design is screwed as a business in so many ways, and it's not because of outsourcing. People just don't want the hardware all that much, and they don't pay a lot for it. The chips are priced by cost of manufacturing, not cost of R&D, and so R&D is basically cut to the bone on salaries.

      Outside of financial companies (and FPGA vendors, who have been trying to sell this idea unsuccessfully to everyone on the planet for the last 10 years), nobody seems to have recognized that bringing stuff into hardware (FPGAs, for instance) can make money through performance. So that's where I work.

  2. Traders by Anonymous Coward · · Score: 5, Insightful

    "the 'ability to work under pressure when the traders are screaming at you' is a must-have skill.""

    Yeah, most of the 'traders' I've met were complete assholes, probably because they realised their high salaries were more down to luck than skill.

    Fortunately when I screw up air traffic controllers just have to start rerouting airliners with hundreds of people on board so they don't crash into each other, so I clearly couldn't justify being paid a Wall Street salary. I can't imagine the stress level of having a 'trader' screaming at me.

    1. Re:Traders by Jane+Q.+Public · · Score: 5, Interesting

      The thing is: Wall Street anymore is not very "capitalist", in the historic sense of the term.

      "Capitalism", in general, refers to making capital investments in goods that in turn will make a profit. Often, that means indirectly and over time: you spend $1M to buy a machine that makes thousands of widgets a day, and sell the widgets for $5.00 each.

      However, with the current obsession with turnover and a quick dollar, what you end up with on Wall Street is really much more pure money speculation, which doesn't have much in the way of capitalist underpinnings. In fact it is a lot more like gambling, which has been around a lot longer than capitalism has.

      We simply haven't been seeing the long-term investment in capital goods, which in turn power a robust economy, and at one time made the US the greatest economy in the world.

      Until we see a return to something resembling real capitalism (and it still does happen, just not so much on Wall Street anymore), we will continue to have economic problems. A corollary to that is something I have been saying for a long time: Washington needs to stop concentrating on viewing Wall Street as some kind of driver for the economy. That's a false economy. Reasoning: anything that can lose a large percentage of itself overnight is not a stable thing on which to base a real economy. Keynesian "created" fiat dollars can disappear just as fast as they are created, as we saw quite clearly in 2008.

      And for similar reasons, I believe HFT should be outlawed. It isn't even a real market anymore; it's simply a contest to see who can spot a discrepancy in perceived value the fastest.

    2. Re:Traders by timeOday · · Score: 5, Insightful

      I agree HFT is rubbish, but where do you see justification for long-term investment in capital goods in the US? It seems that industry is being moved to where it can be operated more cheaply - there is no economic justification for more investment here. In fact there's a surplus of "capital," at lest in the sense of invested money seeking good returns. (Overall the S&P has returned almost exactly 0% over the last 10 years.) So we keep getting bubbles due to over-investment, first in .com, then housing... what is the "next big thing" where we could invest to bring real growth?

    3. Re:Traders by Jane+Q.+Public · · Score: 5, Interesting

      "I agree HFT is rubbish, but where do you see justification for long-term investment in capital goods in the US? It seems that industry is being moved to where it can be operated more cheaply - there is no economic justification for more investment here."

      Industry WAS being moved to where it can operate more cheaply. And there is plenty of justification. If you're American.

      First, let me say this and get it out of the way: China's economy has been slowing. It's not exactly something they advertise, but it's true nevertheless. Further (and I won't claim this is directly related): there is a new Chinese "consumer class" that scarcely existed before, and they are hankering for goods that are not made at home. But having said that, I'm not going to belabor it, and instead I will concentrate on the U.S.

      We have learned some things over the last couple of decades. One of the things we have learned is that regardless of whether it results in cheaper goods, outsourcing is bad for the economy. We know for a fact (it is not a guess or just a theory anymore) that it costs jobs at home, and those jobs are more valuable to our economy than the cheaper goods are.

      Further, outsourcing is not "fair trade", because it artificially bypasses exchange rates. It is easy enough to say that "U.S. labor can't compete with foreign labor for the same amount of pay". Which is a true statement... but it's only half the story. It's a lie by omission. Because we aren't really competing on an equal basis. Let me construct a simplified example to illustrate the point (numbers just pulled out of the air to make the point):

      Company X wants to hire people to manufacture widgets. It can hire unskilled Americans for $10 per hour, or Crotobaltoslavonian workers for $2 per hour. But here's the thing: In America, that $10 buys about 5 pounds of rice. In Crotobaltoslavonia, that $2 buys 10 pounds of rice. So in actual purchasing power, you're paying that Crotobaltoslavonian the equivalent of $20 per hour in America. Americans literally can't compete for that amount because it doesn't pay them enough to eat... in THEIR economy.

      And that's the whole point. Economies are different. That's just the reality of the situation. And that's why we have exchange rates... to keep trading FAIR between countries. But hiring labor in a foreign country, in effect, bypasses that exchange rate, and so it is not a "fair trade". Americans are hurt in the process... even if rich Company X gets to keep a few more dollars. We are exporting even more dollars than what they keep as a result... and that doesn't help our economy, either. So it hurts in several different ways, not just one.

      Savvy companies actually realize this, but they just don't care. Which is pretty unethical, if you ask me. They are willing to sacrifice your $10 so they can make an extra $1 in eventual profits. And you should be angry about that.

      Plain and simple: we need to bring the manufacturing jobs back home. Partly because of the direct monetary imbalance that outsourcing causes, and partly because we need to keep that capital investment (in machines and buildings and other capital goods that can actually MAKE things) here at home, rather than letting it be done overseas at our expense.

      "In fact there's a surplus of "capital," at lest in the sense of invested money seeking good returns. ... So we keep getting bubbles due to over-investment, first in .com, then housing... what is the "next big thing" where we could invest to bring real growth?"

      The bubbles aren't due to a surplus of capital, per se. I mean they are, but the proximate cause of that surplus is indisputably government monetary policy. They have been keeping interest rates artificially low to "stimulate" the economy (BEFORE the 2001 and 2008 bubble burstings). Easy money means more capital. But it wasn't invested in capital goods, intended to

  3. Stress by MichaelSmith · · Score: 5, Interesting

    I used to co-locate in the same building as the local stock exchange. One day, very late, I took the elevator down to the car park, which was where the computers were. There was a guy in the elevator who looked absolutely wrecked. He was sweaty, shaky and not taking things in. He got off at my level and stumbled off towards a porsche which appeared to be similarly young and in equally bad condition.

    The thing is, I work in air traffic control, where the stakes are even higher. The difference is that the operational people have an absolutely obsessive approach to managing their workforce. Traffic controllers are just not allowed to get upset or stressed. In many environments they have unlimited sick leave.

    The difference, I suppose, is that traders personally stand to take home a lot of money. You could do this in any field: offer ridiculous compensation for ridiculous effort. But if you work it out, I doubt the long term returns justify what this does to people.

    --
    http://michaelsmith.id.au
  4. I am an HFT programmer by Anonymous Coward · · Score: 5, Interesting

    I'll be posting anonymously, but I think many here have a very poor understanding of what we do. Most of that is because we do tend to be a very secretive group, but if you were to sit down with some of us, you would see that we really do very normal (and useful!) things in the market.

    I work on the algo and core infrastructure. I wrote price feeds that take 1/5th of a microsecond in C++ and (a little slower) in Java. I understand in fine detail how cache and the the PCI-e bus works. I have a very good understanding of algorithms and the constant-time tradeoffs. I know when to make something simple, when to use and avoid threads, and I can debug in minutes and push out a new version in the seconds before market open (not many people can handle that level of stress well). I read the C++ and Hotspot assembly, and know how to program for superscaler architectures specifically. If you really need me to, I can even crank out some VHDL code.

    On top of that, I understand market microstructure and derivative pricing. I work 12 hours a day on average and do 100 hour weeks. I am on call during Asian hours and need to come in sometimes on holidays when other markets are going nuts and we need to plan.

    I also hope to make $500,000 this year.

    You always hear about Google programmers being the best in the industry, but I've been to a couple Google interviews and turned them down both times because the engineering quality just isn't there. I'd put the average HFT programmer up against the best in Google anyday.

    Ask away, and I'll answer to the best of my ability.

    1. Re:I am an HFT programmer by Anonymous Coward · · Score: 5, Informative

      $500,000 divide by 52 weeks = $9,615 / week
      $9,615 divide by 100 hours = $96.15

    2. Re:I am an HFT programmer by Mr.+Freeman · · Score: 5, Insightful

      "They can't wait for the safe development cycle."

      This, from the people in charge of trading amazingly large amounts of money in a market which influences the global economy in a big way. (remember that billion vs. million mix-up awhile back that caused some pretty big problems until it was fixed)

      You call it risky, I call it reckless. You try what you're doing in any other field and you'd be fired pretty damn quick.

      That said, I'm not so much angry at you as I am at the people who ask you to do this.

      --
      -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
    3. Re:I am an HFT programmer by bertok · · Score: 5, Insightful

      Just because you're good at what you do doesn't mean that what you do is good.

      Microseconds? Why not nanoseconds? How about femtoseconds? Why wouldn't that make just as much sense?

      What could possibly change in the underlying value of a corporation made up of flesh and blood humans and capital with decades of depreciation in a fucking microsecond? Here's a hint: nothing. You are not investing, or trading, but simply racing other gamblers. Investing doesn't benefit from microsecond response times, and trading doesn't need it either. People could buy IPO shares just fine over the phone. Nobody ever needed a microsecond response time to buy a thousand bushels of wheat, and never will, because bread is baked daily, not a million times a fucking second.

      If politicians had two braincells to rub together, they'd enact a law to prevent trades faster than some tick, say, an hour. Your 'trading' company would go out of business in a week, and nobody would care. Farmers would still sell their wheat, and bakers would still buy it, but without you leeches skimming off the top.

    4. Re:I am an HFT programmer by Alex+Belits · · Score: 5, Insightful

      I am most likely better than you at each and every aspect of software (and HDL) development you have mentioned. Except, of course, "debugging in minutes" -- that kind of irresponsibility would get me fired. I also have to work long hours, and have to have clear understanding of complex issues unrelated to software.

      Except I do embedded software and FPGA development for professional audio equipment. Each device I worked on, each firmware release, each line of code, does something useful for many, many people. Some of those people don't even know that audio equipment, leave alone software, is involved with what they are hearing. Large fraction of my work ends up being free/open source, too -- platform, drivers, etc.

      I also don't have any problems with posting here under my real name. Or with telling you, and people like you, to die in a fire.

      --
      Contrary to the popular belief, there indeed is no God.
    5. Re:I am an HFT programmer by Renevith · · Score: 5, Insightful

      Do you ever buy or sell stock? Perhaps indirectly, through a mutual fund or 401k type plan? If so, then you benefit from high liquidity in the market. HFT and other Wall Street shenanigans do skim from the top, but they also provide liquidity. It's almost certain that the liquidity benefits small market players more than the skimming hurts them. In other words, the money they're skimming comes from the banks and brokers rather than you and me.

      For example, take the stock of Red Hat (http://finance.yahoo.com/q?s=RHT). Yahoo Finance right now shows that, as of the last time the market was open, I could buy 100 shares for $42.56 (that's the "ask" or best current asking price), or I could sell 300 shares for $42.09 (the best available "bid"). That's a bid-ask spread of about 50 cents. That spread is a hidden cost to either buying or selling stock: If you buy and then sell RHT, you will have paid about 50 cents per share just for the privilege, even if nothing in particular happens to the company. Let's split that 50-50 and say that every stock transaction in RHT (buy or sell) costs you 25 cents per share in implied fees.

      Those bids and asks are set by individuals and companies who are competing. They want to get a good deal for either buying or selling the stock, but they also know that if they set asking price too high or their bid too low, they'll never make any trades. The more competition there is, the tighter the bid-ask spread will be. HFT and other algorithmic approaches allow firms to set prices on tons of stocks without requiring human attention for each one, which dramatically increases the competition and thus tightens the bid-ask spread.

      In this example, if you outlaw HFT and similar trading strategies, maybe RHT will have a spread of $1 intead of 50 cents. Maybe you'll be happy that HFTers aren't making ther 5 cent skim off the top anymore, but it'll be cold comfort when you're paying 25 cents more on each transaction and it's just going to a different Wall Street firm.

      If you think I'm exaggerating the effect of computerized trading of the spread, have a look at slide 8 (page 4) of this study: http://fisher.osu.edu/~diether_1/b822/trading_costs_2up.pdf. Starting in 1960, the average bid-ask spread has ben dropping steadily every decade to a small fraction of what it used to be.

      Background: I am an actuary trained in quantitative finance. I've never worked in Wall Street or done any HFT or other algo trading.

  5. yeeeeeah by Anonymous Coward · · Score: 5, Insightful

    I'm past the whole "a-holes can yell at me if they pay me enough" phase.

  6. enough lies please by decora · · Score: 5, Insightful

    we all understand what 'arbitrage' is. when the synthetic CDO market calls their deals 'arbitrage' we all know its fucking bullshit.

    when the sales guys in the brochures talked about the 'AAA' ratings on these pieces of 'arbitrage', it was all bullshit.

    when Lloyd Blankfein calls it 'hedging, not betting', its fucking bullshit. ]

    there is absolutely nothing, whatsoever, 'valuable' behind a credit default swap. it is a bet. that is a fact, and its not rocket science, and its not a conspiracy theory, and its not "the ignorant and alarmist" decrying some nefarious boogey man. its the basic fucking fact of what fucking happened.

    I beg of you. stop lying. nobody believes you anymore. this is like the scene in Shattered Glass when Peter Saarsgard has to finally explain to Hayden Christiansen that the whole charade has ended.

    the financial industry has no clothes. we all know it. there is no point in pretending.

    1. Re:enough lies please by obarthelemy · · Score: 5, Insightful

      there's a strong moral risk: if you can both buy a company's debt, and insurance in case they don't repay it, why not keep doing that endlessly ? you make money both ways, nobody is really interested in the underlying debt quality, and bankers make fat commissions on both sides of every deal.

      oh, wait...

      --
      The Cloud - because you don't care if your apps and data are up in the air.
  7. Yet Another HFT Article by hsk17 · · Score: 5, Informative

    Disclaimer: I work at an HFT firm.

    The implied accusations are flying out of the page like daggers. I wish you, Slashdot readers, could see the world through my eyes. As techno-savvy as you are, you somehow love to hate on HFT without having any idea what it is. Don't get me wrong -- I really don't care if you hate it. What bothers me is that haters have NO IDEA what HFT is doing and basically project their hatred for finance onto it.

    I have to say, this article is pretty level-headed. I was expecting more baseless accusations. Of course, the article throws around the typical "HFT was blamed for the huge drop in the stock market in May 2010..." If you cared to look at the linked WSJ article, you would have read that Waddell's desk had sold 75,000 E-mini contracts at the start of the flash crash. If you cared to look at the CFTC report that officially investigated the flash crash on May 6th, you would have read that CFTC blames the flash crash on some trader who executed a large sell order worth $4.1 billion dollars -- why, isn't that just about 75,000 E-minis?

    You would have also read that HFT firms actually mitigated Waddell's mistake. They were there to absorb the thousands of E-minis and so dramatically lessened the initial impact. It's really quite admirable the amount of precision coders needed to invoke in order to create a system that executes so quickly and at scale during such a turbulent period. I was hoping that the discourse here would be more along those lines.

    Unfortunately, the amount of volume that Waddell executed was too much risk for the traders to bear, so they started getting out of their positions. In fact, no one could handle a trade of such size. It was as if someone predicted the collapse of the US economy and bet $4.1 billion on it. The ensuing chaos was purely the after-effects of the initial destruction caused by Waddell.

    New technologies can be used for bad. I bet there's plenty of bad traders manipulating the markets and using speed as an unfair advantage. We need to police HFT, for sure. But I'm also sure that people are using guns to kill other people out of malice, using cars to traffick illegal drugs, and using airplanes to destroy buildings. HFT is a style of trading. It's a technique, not a strategy. The sooner we realize this, the more progress we will make as a society in implementing policies and regulations.

    You guys all hate on HFT, but you are really the ones benefitting from this technology. In market making there's a spread -- the difference between the lowest ask price and the highest bid price. Take a look at the most liquid stocks. They are probably trading at 1 cent wide spreads. Compare that to years ago when spreads used to be dollars. Go to a bank and look at the currency exchange rates. I just did a look into Bank of America's spreads. 100 euros gets you 135.35 dollars. Based on recent trading prices on the public exchanges, 100 euros should be able to fetch closer to 143.62 dollars. BoA is charging a spread that is more that 5% of the value of the product! I don't blame them -- the landscape in the currency markets discourages technological innovation and competition.

    This phenomenon isn't true just for currencies. It's true for most products that are not regulated or traded publicly. You, the average investor, are being ripped off dearly investing into these opaque markets. The size of spreads is truly a symbol of capitalism. If there's competition, the spreads are tight. If there's monopoly, the spreads are wide.

    You complain about HFT being super fast and "shaving off transactions" as if we somehow have access to your accounts and embezzle money a la Office Space. That's like complaining about WalMart having such efficient systems and internal logistics that your cereal is getting too cheap. Yes, I do believe that some traders use speed unfairly, and yes, WalMart probably did shady things we don't know about, but my point is that not every trader is bad. Technology