How and Why Wall Street Programmers Earn Top Salaries
msmoriarty writes "Given the level of interest in the recent highest-paid programmers discussion, our reporter decided to do a follow-up looking into the languages and skills needed to work on high-frequency trading systems. There's actually a pretty wide range of languages/tools used, but Linux is the 'default' OS and, not surprisingly, the 'ability to work under pressure when the traders are screaming at you' is a must-have skill."
"In high-frequency trading (HFT), programmers eke out every last incremental tick in performance to build algorithms that battle other algorithms for computational supremacy and millions in profits -- and earn a lot in the process."
Skimming money off billions of micro-transactions. Ahh, yes... forget investing in ideas and backing well managed companies... this is the way capitalism was envisioned.
"the 'ability to work under pressure when the traders are screaming at you' is a must-have skill.""
Yeah, most of the 'traders' I've met were complete assholes, probably because they realised their high salaries were more down to luck than skill.
Fortunately when I screw up air traffic controllers just have to start rerouting airliners with hundreds of people on board so they don't crash into each other, so I clearly couldn't justify being paid a Wall Street salary. I can't imagine the stress level of having a 'trader' screaming at me.
Not surprising. Wall street is basically a hot bed of criminals, so it's not surprising to see them using an operating system that is built from stolen Microsoft technology.
That's a twist ending even M. Night Shyamalan would be proud of...
I used to co-locate in the same building as the local stock exchange. One day, very late, I took the elevator down to the car park, which was where the computers were. There was a guy in the elevator who looked absolutely wrecked. He was sweaty, shaky and not taking things in. He got off at my level and stumbled off towards a porsche which appeared to be similarly young and in equally bad condition.
The thing is, I work in air traffic control, where the stakes are even higher. The difference is that the operational people have an absolutely obsessive approach to managing their workforce. Traffic controllers are just not allowed to get upset or stressed. In many environments they have unlimited sick leave.
The difference, I suppose, is that traders personally stand to take home a lot of money. You could do this in any field: offer ridiculous compensation for ridiculous effort. But if you work it out, I doubt the long term returns justify what this does to people.
http://michaelsmith.id.au
I'll be posting anonymously, but I think many here have a very poor understanding of what we do. Most of that is because we do tend to be a very secretive group, but if you were to sit down with some of us, you would see that we really do very normal (and useful!) things in the market.
I work on the algo and core infrastructure. I wrote price feeds that take 1/5th of a microsecond in C++ and (a little slower) in Java. I understand in fine detail how cache and the the PCI-e bus works. I have a very good understanding of algorithms and the constant-time tradeoffs. I know when to make something simple, when to use and avoid threads, and I can debug in minutes and push out a new version in the seconds before market open (not many people can handle that level of stress well). I read the C++ and Hotspot assembly, and know how to program for superscaler architectures specifically. If you really need me to, I can even crank out some VHDL code.
On top of that, I understand market microstructure and derivative pricing. I work 12 hours a day on average and do 100 hour weeks. I am on call during Asian hours and need to come in sometimes on holidays when other markets are going nuts and we need to plan.
I also hope to make $500,000 this year.
You always hear about Google programmers being the best in the industry, but I've been to a couple Google interviews and turned them down both times because the engineering quality just isn't there. I'd put the average HFT programmer up against the best in Google anyday.
Ask away, and I'll answer to the best of my ability.
They're paid not to steal (or at least not get caught).
Chas - The one, the only.
THANK GOD!!!
I'm past the whole "a-holes can yell at me if they pay me enough" phase.
I'm a developer at an HFT firm and can say that it is some of the most interesting work I've done in my career. You need to wear many hats for the job.
On a daily basis I multitask between making low-level kernel modifications to reduce system latencies, to refactoring our high level marketdata->prediction->execution engine. It's a never ending balance/conflict between making things as fast as possible while trying to maintain good design principles and not take shortcuts in the sole interest of efficiency.
To those that say HFT is an anathema to the economy, I think there is some merit, but there are two sides to every story. I've definitely met some other HFT shops that focus on some really shady and morally questionably tactics (essentially DDOSing markets to gain an advantage). However, the amazing advancement in network technology and all these crazy RDMA and 10G user-space network stacks is literally due to the HFT community, and I'm already seeing these new technologies start to penetrate other markets. This is capitalism at work.
Also, believe it or not, HFT has reduced spreads due to competition, which means you pay less when you want to buy or sell a stock RightNow instead of using a limit order (and while spread reduction is good for investors, it sucks for us because it is not as profitable as it used to be a few years ago).
The "Billions" in profits going to HFT firms right now previously went to the big investment banks and the old classic broker taking orders over the phone and charging a ridiculous market spread.
As some of the interviewers in the article hint at, HFT is a small niche that's getting way more than its share of attention.
I'm not saying HFT isn't dangerous, or something to keep an eye on. But if you're looking for a job in Manhattan financial sector, you'll more likely be working in web or other "high level" stacks.
Java is *huge*, .Net is also very popular. Web is, of course, very important with just about every stack, from Ruby to ASP.Net represented. I see a surprising about of MatLab for analysis ( I guess I just wasn't expecting to see any with QuantLib and similar floating around ).
>Linux stole from minix and unix
Boy oh boy, this story is bringing out the softies tonight.
Do you get paid to do this clown act or do you just do it to attract the children?
--
BMO
1 - A lot of stuff you hear about HFT (using supercomputers, using functional languages, using clusters, using nosql databases, etc...) are either back office, B-rate HFT firms, or BS. In my experience at the higher end places I've worked, Java and C/C++ are the standard for any trading system or order routing system. To push something out in single-digit microseconds, you really need the control of those languages (yeah, Java doesn't quite have the same level, but you can get it close enough by knowing enough about HotSpot). (back office are those that do the behind the support stuff like keeping track of risk, order flow, etc.. while front office does the actual algos and trading systems)
2 - We tend to rewrite everything at the upper levels. A lot of firms will use things like 29 West or other messaging, order routing, parsing libraries, but when you get to head of the pack, you tend to do it all yourself. This goes doubly true for the language APIs. I can't remember the last time I used STL, Boost, or the JDK util classes.
3 - We don't front run, we try to not predict over any appreciable amount of time. We have very high Sharp ratios (risk adjusted return) because we don't do those things. HFT is very low risk and we like to keep it that way. There are some HFT firms that can a full year without a losing day.
4 - In HFT the line between a front-office developer and a trader is very blurry. I'm often responsible for developing my own trading algos and running them all myself. Various shops have different break downs, and some don't even employ traders beyond the front office programming staff.
A while back, I hired a former HFT programmer. He was bright, but not six-figure talent.
If you're the best and brightest in any field, you can make a very nice salary. HFT requires some special skills, but in my experience hiring programmers, intelligence, hard work, and ability to learn quickly are more important than specific resume points.
I get e-mails ALL the time from these people up in NYC. If i so much as wink at my resume online differently I can always be certain that I will be receiving many opening jobs in the trading and banking industry. In fact, throughout my career, this has always been the case. I live in Florida. Why in the world would they be trying to recruit me from florida and why have I always gotten e-mails from them. Is it because they have a lot of people leaving? Do people get frequent burnout? Are they just spamming?
The last "recruiter" to e-mail me wanted to know if I was looking for a 'great' opportunity in the banking industry writing C++ applications with a big whopping salary of $100,000 with a 12 month contract. Since i was tired of these people wasting their time trying to contact me I e-mailed them back and asked why they thought I would just get up and move from Florida to New York for a 1 year contract and i tried to get confirmation of a starting salary of $100,000 a year. He e-mailed back stating that $100,000 was indeed the starting salary with great 'benefits'. So i e-mailed back and informed him that i was already making $75,000 there and no state taxes so $100K really wasn't that great of a deal, especially considering that I would have to make 4 times my current salary that I am making now just to come close to breaking even in NYC. Incidentally I never got another e-mail back from him.
When you have a family, there is no way I am giving up a house to move to NYC to live in a one bedroom closet for the same price as my 4 bedroom house. - with less income coming in.
You do it for the bonus, not the salary. But either way, you don't do it for a 1 year contract.
-Malakai
A Dragon Lives in my Garage
POSIX: Portable Operating System Interface for UNIX.
Linux tries to keep pretty close to to it. 'nuff said.
It's also no secret linus learned OS design a large part from hands on operation with minix, and that the minix file system was the first file system linux used.
yeah but the cost of housing costs about 3 or 4 times as much for what i am living in now. how much does a 3,000 sq ft home cost in the NYC suburbs anyway? I am also aware of the fact that if i want to maintain that same sort of lifestyle then i am not going to be living in city which is not going to work with kids - but maybe if people do end up working 100 hour weeks, then it is probably more suited to people that have no family at all to take care of. So what are we talking about here? A 1 hour commute one way? No thanks. I also doubt that the bonus would still not make up for what I would get to break even - just a hunch.
Unfortunately, us systems/network engineers in the HFT industry don't get the same level of respect (or pay) as the developers... It's better than other industries, but yeah...
Edmund White
http://flickr.com/ewwhite
When I worked on the Canadian Automated Air Traffic Control System (CAATS) Ada skills were at a premium, and I was recruited from Australia on $80,000, in 1998, so that's probably $150K plus in 2011. The project had nearly two hundred Ada engineers recruited from around the world, on similar, or better, salaries. Plus, it was a great project, great team, moderate pressure, and in one of the worlds most beautiful cities!
I shifted to C++ and then .Net, but I think the Ada market has kept high salaries, even while it has been shrinking, especially for those with a security clearance in the US.
I'm a software visionary. I don't code.
on internet forums?
that's right. Adolph Hitler.
He wrote hundreds of pages 'exposing' the 'truth' behind 'power'.
It was called 'the protocols of the elders of zion'. of course, the whole thing was bullshit. made up by some anonymous author, possibly the Russian Tsar's secret police, to support yet another pogrom.
But Hitler took this and ran with it. Over and over this document, and many others, including his book Mein Kampf, were given out by the tens of thousands. People were happy to 'forward' these 'revelations' to others, often tweaking details here and there, or changing the attribution of the author(s).
And what happened in the end? Trillions of children were killed. I'm a student of history too. I have over 5 billion books published. On the internet. You can look it up.
> Using a specification
> "stealing"
No. Just no. You may as well claim the same thing about Windows, because Windows idiots hop up and down about how POSIX compliant Windows is.
>It's also no secret linus learned OS design a large part from hands on operation with minix
You have no idea what Tanenbaum had to say about this, do you?
http://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate
>Implying that a file system specification has any bearing on how a kernel is designed.
Don't know if you're trolling or just stupid. I suspect both.
--
BMO
we all understand what 'arbitrage' is. when the synthetic CDO market calls their deals 'arbitrage' we all know its fucking bullshit.
when the sales guys in the brochures talked about the 'AAA' ratings on these pieces of 'arbitrage', it was all bullshit.
when Lloyd Blankfein calls it 'hedging, not betting', its fucking bullshit. ]
there is absolutely nothing, whatsoever, 'valuable' behind a credit default swap. it is a bet. that is a fact, and its not rocket science, and its not a conspiracy theory, and its not "the ignorant and alarmist" decrying some nefarious boogey man. its the basic fucking fact of what fucking happened.
I beg of you. stop lying. nobody believes you anymore. this is like the scene in Shattered Glass when Peter Saarsgard has to finally explain to Hayden Christiansen that the whole charade has ended.
the financial industry has no clothes. we all know it. there is no point in pretending.
there is a great book that just came out, The Asylum, by Leah McGrath Goodman , which explains how the potato market became a cluster fuck of manipulation and greedy assholes absolutely stealing from the ordinary person.
it also explains why certain industries were banned from trading this shit. why? because you cant operate a society where the price of basic commodities fluctuates by several hundred percent a year just so that a handful of a few dozen traders can make massive amounts of money through manipulating the market.
and how many prostitutes do they kill, on average, per year?
I interviewed once for a trading firm. They were honest at the interview and told me that someone would probably scream at me for something that wasn't my fault. I declined.
I had another friend who did take a job up there (different firm and not a programmer). She was ready to quit and was complaining about how mean they were. I tried to console her and told her she had no way of knowing what it would be like. Then she told me that they actually made her cry during the interview but she took the job anyways.
-- Don't Tase me, bro!
[Posting anonymously since we are talking money, etc.]
I am a senior engineer @ Google. I work regular 40 hour weeks, am rarely on-call and work on projects that really interest me. I work with a great team where I have a lot of fun with my co-workers, enjoy fantastic free meals, great health insurance and numerous other benefits (such the latest and greatest geek toys, and working from home any time I want). And yes, I am sure to make $500K this year.
Maybe you should consider accepting the Google offer?
Such talented people doing such silly and meaningless work
Lived in NYC, worked there in Fortune 10 (yes ten) companies as a Sys Admin, got out when I got a chance and a nice offer far-far away in a land of sanity, relaxation, no state taxes, affordable housing, and am never coming back to NYC. The quality of life anywhere else is so much better in all the ways than the daily grind in NYC no matter how much they pay you. The disgusting NYC subway system shows the true underbelly of the city.
NYC financial recruiters (aka head hunters) and companies always get their talent (aka willing slaves) on short term 6-12 month contracts and never or hardly ever as employees unless they are doing a direct transfer from another financial company and want to retain their corporate title and position Associate, AVP, VP, Director, Managing Director, etc. Their favorite ploy is to dangle conversion to full-time employment and eligibility for bonuses as a way to whip the consultants and contractors to work long overtime hours on straight-time (aka no overtime pay).
After years of being away from NYC these recruiters still contact me with contract offers for NYC and I just keep telling them to take a flying fuck off a bridge and remove me from their contacts database.
Been there, done that, never again, it isn't worth any kind of money to live in the technology ghetto that is NYC.
I work in a financial firm, but not in the HFT department, so while my salary isn't at some of the levels mentioned, I'm doing pretty well. I don't code really fancy math and I'm not determining prices or anything, but what think is the critical skill in my group is being able to think on your feet when people are calling you about something gone wrong with or how to pull off for the first time TODAY a deal valued at north of $50M, and being able to pull it off on a regular basis and with confidence. There is constantly lots of pressure and dealing with people who are under even more pressure. You have to know the systems involved so well that you can make changes to code which moves millions of dollars around, test them, then release them right away and still sleep at night. I've dealt with lots of other programming shops who seem terrified to actually release code, and that just won't fly at a financial trading firm. You get used to a lot of responsibility and performing under pressure without a net. It takes a lot of confidence and experience. You also have to have the business sense that when a situation comes up that the code doesn't handle and the business types tell you not to work on it because they've decided not to ever do that again, you make sure it's the next thing you get done because it'll get mentioned again someday, but on that day it'll be a high energy financial type calls you and tells you the go/no go decision has to be made within the next 30 minutes for that day.
You are also dealing with data files received from firms all over the world, and you'd be amazed at how primitive some of those firms are and their files just make random changes from time to time and you've got to fix for those changes NOW and get things running again, not an hour from now.
So it's about the ability to work with confidence, under stress, and be able to maintain intimate knowledge of broad systems which must deal with all sorts of real world exceptional cases.
I know this isn't going to sound all that special to people not in that situation, but it really does raise the stakes a lot.
The last I read, 70% of all volume in the NYSE is HFT. We need another word for this because it is not capitalism. It is taking advantage of quarks in the system. Everyone knows these quarks exists and that they're distorting the system, yet these quarks are deliberately left as is.
POSIX: Portable Operating System Interface for UNIX.
Linux tries to keep pretty close to to it. 'nuff said.
What's your point? OS/400 - i5/OS - i operating system has been POSIX compliant for many years.
I worked for a Fortune 100 financial company in Manhattan. Some of the nastiest people I ever worked with were there. I also never worked anywhere where the company made more clear to me that I was a disposable cog. Not that that isn't the case elsewhere, but management usually tries to at least conceal it. I worked 60 hours a week, including weekends, and I was on the low end compared to others who were working to move up in the hierarchy.
There was a limited and fixed bonus pool, so if you got less money, others got more. I'm sure this was a plan by management - the firm was so wealthy, they felt it better their workers be divided against one another than working together. They did this in a variety of ways - staff versus contractors, contractor firm versus rival contractor firm and so forth. This encouraged people trying to rip one another apart during weekly meetings, code reviews and the like.
The office politics can be strange too. I used to be on conference calls with a programmer with a huge ego, and who people deferred to because the program suite he wrote was important for the firm. The program was shoddy though, it had massive memory leaks and the like. But he and the team under him were able to throw it together quickly and the business people were happy with him so he was a golden boy. It wouldn't have been so bad if he wasn't always denigrating everyone else's work with a holier-than-thou attitude. I wasn't in a position to say anything though. Lots of stuff like this happens. A lot.
It was not all bad. It was a large environment. Stuff I would have done maybe once a year at a smaller company I was doing every day there. I was surrounded by dozens of people who were sharp and knew what they were doing. There was a feeling of camaraderie among some of us. It is hard to explain the change in quality due to the sheer size of the company, with its ability to spend massively if needed.
One thing I will say - unless you are there to work 24/7/365 and try to make it to be one of these "highest paid programmers", there is no reason to be there. Some of my co-workers joined straight of college, which seemed dumb to me. Anyone who takes a job for less than six figures is a fool - if you don't have the skills to make at least $100k, there is no reason to work there. It is not the place to come in at a low level and hang around.
To repeat a point - anyone who takes a job on Wall Street right out of college is a fool. They recruit heavily on campuses for the same reason Microsoft and Intel and Electronic Arts do: so they can take advantage of suckers with no work experience who don't know any better. Thankfully I didn't make this mistake. I was able to put everything I saw there into perspective. Going from college straight to Wall Street as a programmer is a dumb move for most people.
I left IT to drive a big truck. I work only 168 days in a year. I make $170,000
Any recruiter pitching a $100K job in the banking industry is not worth talking to. In my first wall street gig back in 1990, I was making $100/hour writing options pricing apps on NeXTSTEP, with barely six months of hands-on experience on the platform, and no finance or accounting background at all.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Disclaimer: I work at an HFT firm.
The implied accusations are flying out of the page like daggers. I wish you, Slashdot readers, could see the world through my eyes. As techno-savvy as you are, you somehow love to hate on HFT without having any idea what it is. Don't get me wrong -- I really don't care if you hate it. What bothers me is that haters have NO IDEA what HFT is doing and basically project their hatred for finance onto it.
I have to say, this article is pretty level-headed. I was expecting more baseless accusations. Of course, the article throws around the typical "HFT was blamed for the huge drop in the stock market in May 2010..." If you cared to look at the linked WSJ article, you would have read that Waddell's desk had sold 75,000 E-mini contracts at the start of the flash crash. If you cared to look at the CFTC report that officially investigated the flash crash on May 6th, you would have read that CFTC blames the flash crash on some trader who executed a large sell order worth $4.1 billion dollars -- why, isn't that just about 75,000 E-minis?
You would have also read that HFT firms actually mitigated Waddell's mistake. They were there to absorb the thousands of E-minis and so dramatically lessened the initial impact. It's really quite admirable the amount of precision coders needed to invoke in order to create a system that executes so quickly and at scale during such a turbulent period. I was hoping that the discourse here would be more along those lines.
Unfortunately, the amount of volume that Waddell executed was too much risk for the traders to bear, so they started getting out of their positions. In fact, no one could handle a trade of such size. It was as if someone predicted the collapse of the US economy and bet $4.1 billion on it. The ensuing chaos was purely the after-effects of the initial destruction caused by Waddell.
New technologies can be used for bad. I bet there's plenty of bad traders manipulating the markets and using speed as an unfair advantage. We need to police HFT, for sure. But I'm also sure that people are using guns to kill other people out of malice, using cars to traffick illegal drugs, and using airplanes to destroy buildings. HFT is a style of trading. It's a technique, not a strategy. The sooner we realize this, the more progress we will make as a society in implementing policies and regulations.
You guys all hate on HFT, but you are really the ones benefitting from this technology. In market making there's a spread -- the difference between the lowest ask price and the highest bid price. Take a look at the most liquid stocks. They are probably trading at 1 cent wide spreads. Compare that to years ago when spreads used to be dollars. Go to a bank and look at the currency exchange rates. I just did a look into Bank of America's spreads. 100 euros gets you 135.35 dollars. Based on recent trading prices on the public exchanges, 100 euros should be able to fetch closer to 143.62 dollars. BoA is charging a spread that is more that 5% of the value of the product! I don't blame them -- the landscape in the currency markets discourages technological innovation and competition.
This phenomenon isn't true just for currencies. It's true for most products that are not regulated or traded publicly. You, the average investor, are being ripped off dearly investing into these opaque markets. The size of spreads is truly a symbol of capitalism. If there's competition, the spreads are tight. If there's monopoly, the spreads are wide.
You complain about HFT being super fast and "shaving off transactions" as if we somehow have access to your accounts and embezzle money a la Office Space. That's like complaining about WalMart having such efficient systems and internal logistics that your cereal is getting too cheap. Yes, I do believe that some traders use speed unfairly, and yes, WalMart probably did shady things we don't know about, but my point is that not every trader is bad. Technology
I agree completely. I don't think it's feasible to outlaw HFT, but I'd settle a STET tax to put a damper on it.
XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
What about you? How do you feel about doing this? You can't really know what other HFT programmers feel.
What a great idea! I would have to think about it for a bit, but my first impression is close to love at first sight.
You missed one criticism though, the claim that HFT is a "waste of talent". I'll cover that one for ya :
Is there really such a limited supply of talent in the world? Talent is created by intellect, motivation, and experience. We may ignore experience as it's self creating. If banks hire fresh math & physics PhDs with significant programming experience for $100k plus maybe a $50k bonus, that'll just creates more motivation to study math & physics.
We're therefore left with the claim that intellect is limited supply. Is this true? Yes and no. Intellect isn't in nearly such short supply as people imagine. It's also heavily impacted by people's motivation during their youth.
Conversely, there simply isn't enough money in research for all these people. You've seen these online discussions saying : For God's sakes, please don't do a PhD in the humanities!! If advertising and finance weren't slurping up all the excess scientists, then we'd be seeing the same discouragement levied at PhDs in the sciences. And that'd be extremely bad for our society!
Ideally, we want the quantitative fields and algorithms to take over more of the traditionally "powerful" professions, encouraging young people to think more quantitatively and algorithmically, and pushing our society forward.
As an aside, do you realize that anti-semitism was heavily influenced by Jewish bankers, like the Rothschilds, refusing to finance wars when they had interests in both wars? Ain't necessarily all love & roses for those that help society progress.
The Christian religion has been and still is the principal enemy of moral progress in the world. -- Bertrand Russell
Investing your pension on the stock market is either very smart, or very stupid. It has nothing to do with how solid a company is, but everything with how HFT reacts and what people's sentiments are.
Say what?
If you do your pension saving for two weeks or what?
Can't see how a comment this stupid can be coming from someone who work with HFT.
HFT doesn't matter at all for someone doing pension savings.
You save for 40 years and HFT matters how during those 40 years?
Not many pension savers are day trading their pension ..
Are you heading west? Can I get a ride to Fresno?
If you mod me down the terrorists will have won
For HFT, I get proposals of 200k+ salaries all the time, and not fixed contracts.
Your recruiter wasn't very good.
HFT is somewhat illegal, since it could be considered insider trading, since you have time to process information before it becomes available to other people thanks to a faster network infrastructure.
I'm married, in my 50's with college age kids. I have knowledge and experience in Linux, C++, signal processing and trading. I have a nerdy interest in Cuda and GPU algorithms.
I'm sad because I could be involved in this exciting field but I'm simply too old to up and relocate to the places where this exciting and interesting is being done. With kids in college, and friends and relatives where I live I'm simply not as mobile as I once was.
Ah, to be 25 and single in this environment knowing what I know!
All that and yet your job (and mine in a different field) is ultimately less important than that of a first year nurse, a private soldier or the guy in the control room that operates the line in a large factory or steelworks. Vast amounts of money is streaming through but a trading halt is not going to kill anyone or hold up the production of a physical commodity anywhere. It's not even a 24 hour operation. The "mission critical" bullshit attitude burning people out is from false importance being granted due to using the wrong metrics plus poor management that would rather burn out their staff and wear the inevitable mistakes instead of taking a professional attitude. It costs millions to shut a steelworks down for a few weeks for maintainance but it gets done to avoid expensive failures. A stock exchange which isn't even a 24 hour operation is not magically more sacred, despite egos and a bizzare type of Feudalism insisting it is.
If you're a HFT programmer, you usually don't have traders in your ear, you have a bunch of quants in your ear. They usually don't yell (much).
The programmers that get yelled at by traders do more line of business applications and datapulls (kind of a business analyst role), because traders rely on them to make trading decisions and want their information "right now, you fucking asshole". These programmers usually stick to .NET or Java, though with the invention of Silverlight I'm seeing a lot more of that in the arena lately.
Full disclosure, I have worked in finance for 10+ years and currently work at a hedge fund as a systems architect/engineer.
The price is always right if someone else is paying.
Why not divide time into discrete blocks, e.g. one each minute or so? All bids placed in the current time block are then traded simultaneously at the end of the current block. If traders can only look at data form previous time blocks, I think all this BS could be avoided.
Dead on. That's exactly what happened and exactly what caused the current recession. It's not a lack of goods or supply. It's a lack of money and demand.
It had been cushioned for a while since people were living on their substance for the last decade (or rather, for the last 2-3 decades). They didn't regenerate as much as they spent, but they tried to keep their living standard. So they spent more than they earned, and that's what kept the system afloat for a while. But we now hit the moment when that overspending doesn't work anymore because people simply cannot continue that way. They don't have anything left. Worse, they're in deep debt by now, and even reversing it would not result in an immediate recovery because they'd first of all have to recover the debt before starting to spend again.
Money has accumulated in the hands of a few, who (as you mentioned) do not spend it now because there is little, if any, incentive to do so. Why invest when there's nobody to buy the services you would offer? No, I didn't say "goods and services", I said services. Because that's what drives our economy. Services. Our economy (with "our" being "every country's in what's considered the "western world", don't feel left out, I mean you too!) depends heavily on services. Mostly because they're just absolutely awesome cash cows. It's selling the pure labour, without having to expend resources. Unlike agriculture or the manufacturing of goods, services do not require you to invest any (or at least many) resources. You don't sell what's in or on your ground, you sell your workforce. The beauty about it is that you can sell as much as you can put people behind it, and if there's no shortage in today's economy, it's workforce. At some point you cannot buy more land to grow something, and you might not be able to acquire more raw materials to produce something, but there's always unemployed people who you can pay a few pennies and have them create a service for you to sell.
Unfortunately, services are also a very, very delicate commodity. First of all, they're near impossible to export. Unless you have tourists. You cannot send a haircut abroad, unless the person from abroad wanting it comes to you. And second, and more critically, they're the first thing people start to cut when money gets tight. A new haircut would be nice, but I can do without. Fixing the plumbing would be nice, but that dripping faucet is not really on top of my list. First I need food. Then I need shelter. Then I need a few other things before I start to care about services.
This is why our economy is on its belly. We need money in our people. But don't expect things to pick up any time soon, even if you somehow manage to give them more money than they spend on "existing". They'd spend a good deal of the near future recovering from debt.
And that's exactly why this ain't happening.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Further, outsourcing is not "fair trade", because it artificially bypasses exchange rates.
Then perhaps the problem is that the exchange rates are lower in the first place. The "Penn effect" is an observation that less industrialized countries have cheaper currencies than purchasing power parity alone would predict. This is traditionally explained by the Balassa-Samuelson model that developed countries are more efficient at producing tradable goods and services. Some goods and services are "local": some goods spoil quickly in intercontinental transit, and personal services such as health and beauty care require face-to-face interaction. Others are far more tradable: durable goods have a shelf life far longer than the weeks it takes for a container ship to cross an ocean, and many of us are aware of offshore outsourced services. The value of a market's currency is related to the market's efficiency at producing tradable goods and services because local sectors have to increase their wages to keep workers from defecting to a tradable sector. See more detailed discussion of this effect. So once a country industrializes, its currency will become more valuable, and wages will rise.
Plain and simple: we need to bring the manufacturing jobs back home.
And according to the Balassa-Samuelson model, the way to do that is to make manufacturing here far more efficient than manufacturing there. This is already the case for automobiles, which I admit are less than perfectly tradable due to the cost of intercontinental shipping (case in point: the major car makers headquartered in Japan have plants in the United States). But you don't see cars of international brands being made in, say, Vietnam because the infrastructure isn't there yet.
You saw billions being spent to acquire companies that had never earned a dime of profit, and did not even have a good plan for doing so.
I'd argue that the eventual plans for the dot-coms were 1. to build up a user base whose demographic was attractive to advertisers, and 2. to offer services at a fee.
Later, banks and finance companies were making bad loans on real estate for a number of reasons
I've been told that one of the justifications for the rise in subprime mortgage lending involved mandates from the U.S. government to make home ownership easier for lower-income people, especially those in groups that were traditionally targets of racist discrimination. See Community Reinvestment Act.
Liquidity of an asset is a service that said psychopaths provide.
Uh, the pension saver doesn't choose how it is invested at all - this might be a language issue.
In the US the word pension refers to a fund set up by an employer to pay the employee retirement benefits at some point in the future. It isn't an employee-owned account, and the employee has very little control over how it is invested.
And in the US, employers invest their pensions in hedge funds all the time. And where do you think those funds make their money? Hint - it isn't from building factories and selling widgets.
So is it traded? Or you mean to keep some cash within the fund for payments?
Here in Sweden we got one part where you choose your funds, that wouldn't lead to multiple buys and sells unless you want to?
There is one other part to though, I don't know how that one work.
If it suck I guess the system should be scrapped and people get to choose how they want to invest their pension money themselves.
I do understand that hedge funds must have someone else to play with (or well, must and must, but to be of any use) and that there will probably be some of them who are successful and some who aren't.
One would still be free to choose whatever one wanted to invest in a hedge fund or not.
You mean the HFT algorithms ride the selling and buying of the hedge funds? Or you mean they gain from their 20% or so of the gains of the hedge funds?
For pension savings buy and hold with re-invested dividends probably work quite well? At least if you got decades and move some over to interests the last 5-10 years before pension.
Discrete auctions could work. I can't think of any a priori reason why periodic auctions wouldn't work. They might even make the market place more efficient.
However, if you're willing to listen, I'm inclined to disagree.
First, investors get utility out of a continuous market. It's like having a convenience store open continuously from 8:30AM to 4:00PM, not in periodic intervals.
Second, that system still favors super-speed traders. Think about it. People will submit their quotes, but the market won't match them until after a period of time. That means the last entrant has the most advantage. As a result, you'll still want the fastest programs, best connections, the best programmers, etc. Even if you make everything hidden until the match (as in a dark pool) you can still process worldly information and wait till the last microsecond.
Third, that system will widen the spread, the difference between the lowest ask price and the highest bid price. Think about it. Let me stretch out the time so that it makes more sense in an everyday context. Imagine you're doing a lemonade stand business. You buy from suppliers, sell to consumers. If you were forced to only buy and sell once a month, instead of every day, what kind of limitations would you face? You would probably be extremely wary of overstocking, because the oranges might go bad (do they? let's just say they do for now). You could probably get away with charging more per drink because people know it happens so rarely. As a result, the costs of having to deal with the increased risk of holding the unsold oranges for longer are passed to the consumer.
There are dark pools, which you can read about. In such exchanges, all quotes are private from all participants. Only the machines see the quotes and generate matches. Obviously, you can probably back out information by analyzing your own trades, but because a lot of information is hidden, many institutions prefer to do volume on dark pools. However, as much as dark pools solve issues, they are somewhat bad for the market as well, because a lot of trading is done secretly. Average investors like yourself -- or anyone, for that matter -- has no visibility into the quotes -- the price discovery -- that are being generated.
There are different flavors of exchanges, all trying to solve the problem of risk transfer. You're not alone in advocating auctions. But it might not have the intended effect.
The SEC needs to step in and mandate the release of instantaneous pricing information. In much the same manner as they required companies to make releases of material information available on an equitable basis some years ago.
If I get my stock quotes with a 20 minute delay, everybody should get theirs with the same delay.
Have gnu, will travel.
A couple of things:
One or both of these people are effectively cutting in to the head of the line. What if I'm willing to pay 5.34 for those shares as well? But you are set up for HFT and I'm not. I'm getting screwed.
The most equitable trading system would match the 5.30 sell offer with the 5.34 buy offer and make the deal for both parties at 5.32.
Have gnu, will travel.
It seems to me that the HFT industry and the wall street crooks who employ it are desperately trying to portray HFT as the sophisticated work of some extremely talented programmers who are just smart enough to help brokers earn more money, while in reality the whole system seems to be half inside trading (having access to information other traders don't know yet and methods other traders cannot employ) and half DoS attacks on regular traders (the same order will not work for them because HFT is preventing it). Therefore, it should (IMO) be illegal and perhaps such spin posts exist mostly to counter sensible attempts to ban it.
Disclaimer: I do not own or trade shares of public companies.
"I love my job, but I hate talking to people like you" (Freddie Mercury)
I doubt the long term returns justify what this does to people.
What does society measure one's success by? Yearly salary (regardless of lifespan), or perhaps whether the subject will die happily at the age of 95 while having 30 grand-grandchildren? In fact, we glorify those who die young - after becoming famous for some work that took a huge toll on their lives - much more than those who live long in spite of it ...
"I love my job, but I hate talking to people like you" (Freddie Mercury)
have studied goldman sachs, there are actually several books out now that specifically focus on GS, and several other books that mention its place in the crisis.
Occasionally I receive calls for HFT jobs from recruiters
who represent elite hedge funds. Here is the typical exchange.
RECRUITER
You’ll be working with astronomically smart people. One
group uses crystalline cohomology to obtain the best
polynomial time approximation algorithms for
intractable problems in HFT. The engineer who did this
was an embryo prodigy who taught himself calculus
within ten to the negative sixty-seven seconds of conception.
ME
Is that the work you have in mind for me?
RECRUITER
No. You’ll be cleaning the group’s digital bed pans.
ME
Perhaps you should recruit a Nobel Laureate.
Thanks for calling.
It's about the world we want to live in, which stands in ever-starker contrast to the world we do live in.
Do you want to live in a world where the best and brightest throw their efforts away with such mundane, trivial shit?
Again, not where-are-we-now, but where-do-we-want-to-be.
Do you want to live in a world where, thanks to some tricks of law and circumstance, a handful of people have such domination over billions?
Again, not where-are-we-now, but where-do-we-want-to-be.
You don't have to be a slaver (wtf??) to say, the world we live in is wrong, because a lot of people are suffering for what seem like really lousy reasons.
You do, however, have to be a decent human being.
So, in the US the way a pension basically works is:
1. Employer promises to pay employee some amount of money when they retire.
2. Employer hopefully makes that payment.
Now, that was how it worked until a while back when companies started defaulting on those promises and employees had no recourse (spend 30 years working for a company that goes bankrupt to find out a big part of your total compensation is gone). So, now companies have to actually set aside some amount of money in some kind of fund to try to cover some of these obligations. The problem is that the regulations are pretty weak.
Now, when creating a pension fund a company can either set up a fund that makes a low, safe return, or a high, risky return. Well, the company isn't the one losing their shirt if the investment goes south, but if the return is low the company will be putting a lot more of its own money into the fund. So, companies tend to make risky investments with pension funds.
Now, private retirement accounts do exist in the US. They take the form of IRAs, 401ks, and 403bs, which are all tax-deferred accounts. In these cases the employee actually owns the money and decides (sometimes within limits) how it gets invested. There are usually tax penalties for taking money out early, but the money belongs to the employee and they can cash the whole thing out any time they want to. These tend to be invested in more sane investments, since the employee actually has a stake in the outcome and usually it is illegal for individual of ordinary means to invest in the kinds of financial games that make people millionaires or broke overnight..
I make the majority of my income buying and selling illegal drugs. (Mostly moving marijuana from California to the Midwest/South.) Posting anon for obvious reasons. I make close to 300k a year, and only work about 10-20 hours a week. You can imagine that the ratio of blood pressure to income is one of the best in this occupation.
The vote is tied up by the boomers? Hahaha. I hate to destroy your fantasy, but the boomers aren't the ones propping up the Conservative party. Do you think it's a coincidence that Alberta, the Conservative stronghold, has the lowest median age of any of the major provinces? Do you think it's a coincidence that Alberta's many young, successful professionals, businessmen and tradespeople are voting for Conservative candidates?
It doesn't even make any sense if you think about it. An aging and retiring population has a lot to gain by supporting the NDP. On the other hand, the NDP has absolutely nothing to offer someone who has a significant source of income.
You're just miserable that you didn't get your way, so you're trying to blame it on those stupid boomers and their backwards thinking, right?
By the way, jobs still fall of trees for young people. It was YOUR choice to get a BA in Fuck All instead of researching whether or not employers will care. People with talent, qualifications and social skills are just as busy today as they were in 2007.
Anyone think it was a little strange that half of the quotes in the ADTMag article were just pulled from Slashdot comments? I mean, there are some great comments on here and all, but that seems like some lazy reporting to me.
Bravo, well said! :-)
So... are you running for office?
Thank you for what you & the other HFT people posted.
Your perspectives are very interesting; especially about the workplace dynamics, the value flow from big banks etc.
This is one of the more interesting threads I've read on Slashdot for a while.
p.s. about the "leech" complainers, *shrug* if I thought they had a plan for building a better world, I'd ask to hear about it.
and completely unnecessary in an ideal world where humans weren't concerned with screwing each other over.
If we eliminated everything "completely unnecessary in an ideal world where humans weren't concerned with screwing each other over", we'd be living in working communism. That won't happen until the tribulation, comrade.
Isn't liquidity good?
__
Men with no respect for life must never be allowed to control the ultimate instruments of death.
GW Bu
Sure, but only up to a point. If for example, you limit trades to once per minute, but break ties by who submitted a buy order/sell order first, then there would still be emphasis on getting your offers in as quickly as possible, (i.e. within the first few microseconds). Some, but not all of of the existing HFT practices would change. the HFT would become every-block traders, and would still still try to be first to the transaction.
Breaking ties by last to submit wins would produce something similar. Randomly breaking ties would be best, but people who accuse the system of being based, even if it were not.
Stylish sheet to fix many problems in Slashdot's D3: https://gist.github.com/801524
You misread his comment. He said he normally works twelve hour days (presumably five days a week, so 60 hours a week), but sometimes works up to 100 hours a week.
I don't want to live in a world where other people's actions shape the decisions I make throughout the day, either.
Curiously enough, that's exactly why I'd love to see this sort of frivolous, deeply harmful shit stopped. These people are reckless, and the actions they take day after day make my life harder in a million tiny, indirect ways, along with your and virtually everyone else's life too.
(Note: you may not agree or see it that way, but the world doesn't need your consent or acknowledgement to be profoundly broken and in desperate need of structural repair. So if you're getting ready to type some sort of "no I'm cool everything's cool" reply, just spare us.)
It's just another one of those world-we-want vs world-we-have situations. I want a world where a bunch of shits in a room thousands of miles from me aren't chipping away the economy a billion tiny transactions a day.
In fact, I'd go so far as to say, that's the price you pay for living in society. You give up some freedom (can't wreck shit for millions), and in exchange you get police, fire, sanitation, the whole ball of wax.
Do I think I should be able to tell them not to do that, so long as we're all sharing this same society?
You'll get one guess as to the right answer here.
Now are you starting to understand?
If that was not a deliberate attempt to turn me into a strawman then you need to work on your reading comprehension skills. I was writing about overly inflated levels of importance and never wrote anything about not adding value/anything and especially did not write the text you put in quotes.
Suppose I am the greatest programmer in the world applying the fastest algorithm from my home.
I would still lag about 1 sec behind the competition with my "fast" internet connection.
Since we are speaking microseconds here, is not the main advantage being the closest or even within the final exchange computer?
The competition is obviously biased in favor, not of the best programmers, but of those privileged enough to be located nearest to the apex of the whole system.
These particular banks or trading offices can very safely and easily skim the cream and slowly leech the rest of the world, they do not even need the best programmers.
And presumably this whole looting scheme would collapse if some minimal Tobin tax (say 0.001%) was applied to all ttransactions.
sorry, this just bugs me. i've worked with coders from asia, europe, and all over the states. from open source zealot hackers in the bay, to buttoned-up microsofties in Seattle, IBM, health care software in Atlanta, Film/TV/games in Northern Virginia, telecom software integration in Madrid, Spain. Worked with remote people in Montreal, and remote people in various places in India. While I haven't haven't worked in NYC, where you are, I know people at Google Manhattan, and a couple quants there, one of whom is an MIT+Berkeley PhD old friend, fellow grad student, and fellow coworker.
and, dude, sorry, but...
NO ONE CALLS THEM "ALGOS"
It's like going to silicon valley and calling SF "San Fran". ugh. just uncomfortable for everyone in earshot.
The reason computer chips are so small is computers don't eat much.
That's pretty amazing. You are making not quite double my salary in Engineering.
Is it worth it?
A work that expires before its copyright never enters the public domain and thus enjoys eternal copyright protection.
It was my first thought also. Then we will probably see sniping (ala eBay) and the HFT technology will be used to be first or last into the discrete block. However with the volume reduced so will the impact of HFT on volatility.
Phillip.
Property for sale in Nice, France
"Stealing democracy, one billion dollars at a time."
just hover with your mouse over the text "NYSE Euronext" on the map.
Windows has some add-ons that make it more posix-compliant, but they are implemented in user-space. It's an afterthought at best. On the other hand Linux has a Posix interface in the kernel, and only breaks away from the specification when there is a really good reason to.
The Tanerbaum Torvalds debate is about the design decision of the respective kernels. This does not imply that Torvalds did not consult Tanerbaums Operating System Books or code when he first began his project. The use of the same filesystem clearly shows the influence. Aristotle and Socrates did not agree on everything, yet Socrates was a clear influence upon Aristotle.
But I think all your objection is just because I didn't make my first post all that clear. If you want to say linux "stole" technology (factually copied or adapted), then you should start by looking a Unix and Minix systems. I'm not saying that any copyright or patent was violated, or that linux has added no novel contributions of it's own. On the contrary it's the fastest evolving software project on the planet.
My, we're disingenuous here.
You're attempting to sound reasonable. But in fact, you're using the same stupid argument that SCO, Microsoft, Dan Wallace, Darl McBride, Robert Enderle, ADTI, etc, all used against Linux to paint it as intellectual theft of Unix.
You are the one that said "stole" first, bucko.
You are intellectually dishonest.
--
BMO
You're right. I should have used the word "disclosure".
Abusus non tollit usum. Some people have used a similar argument to try to do bad thing in the past, however that simply isn't relavent to the accuracy. (Software Patents shouldn't exist anyways (When's the last time someone from Microsoft said that?)) Should I believe Linux sprang fully formed, a system in itself from the loins of Linus? Now for some words from the big cheese himself.
I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).
PS. Yes - it's free of any minix code, and it has a multi-threaded fs.
So it's own code, but not exactly a clean room implementation, nor 100% original. The linux kernel has taken BSD code and incorporated it in the kernel. How exactly does this work unless they two kernels work in a similar fashion? "Stealing" in the context of intellectual ideas is always metaphorical and may or may not have anything do with copyrights or patents.(which again, I am no great fan of) e.g IThree years I built some raised gardens with two by twelves. In the next two years each of my neighbors neighbors stole the idea.
Lastly I'll leave you with a bit of advice, though I know you will likely ignore it. Adjectives describing personal qualities never settle an argument, and usually derail it. It's better just to try to find the common ground first. Here's the question. Does linux owe it's current form and existence more to Minix and Unix than any other operating systems? If not show me the evidence, if so then just a matter of settling to which degree.
>filesystem hierarchy
Filesystem hierarchy is trivial. It is neither patentable nor copyrightable. It is not intellectual property owned by anyone. Copying a filesystem hierarchy is like copying a list of phone numbers from a phone book. Mere listings of facts are not and never were copyrightable. If you are going to say "stealing" with regards to intellectual property, you have to have intellectual property there in the first place.
This is also why following the POSIX standard is not copyright infringement. The POSIX standard is a list of facts that an OS needs to match in order to be called POSIX compliant. How one implements those facts is up to the OS author. There is simply no theft being done.
>cleanroom
Cleanroom implementation is required for originality? Since when? Since when is something ever created in a vacuum? Since when is a "cleanroom implementation" not influenced by outside forces or the thing you're trying to emulate (Phoenix BIOS implementation of the IBM BIOS as a classic example).
Your requirements for something to be called "original" is ridiculous and an impossible bar to pass.
>"Stealing" in the context of intellectual ideas is always metaphorical
No, no it's not. It's not metaphorical. This is the same argument that SCO tried to use against Linux. Either there is identical code or patent infringing algorithms or there aren't. After all these years from 2003, there has yet to be a single line of code shown to be "stolen" from Unix.
Stop redefining words with your own private definitions. This verbiage gaming that you are doing is done by crooked lawyers and politicians everywhere.
>Should I believe Linux sprang fully formed, a system in itself from the loins of Linus?
Argument from incredulity. Also ignoring the history of Linux. It was very rough and very small in the beginning. It was nowhere "fully formed." This "fully formed" bullshit is part of the "Big Lie" that ADTI was using.to discredit Linux as if to say "there's no way Linux could be this functional, there /must/ be stolen code in it."
>Does linux owe it's current form and existence more to Minix and Unix than any other operating systems?
One thing you should get through your brain is that Minix is a microkernel and Linux is monolithic. Your failure to understand this FUNDAMENTAL STRUCTURAL difference in design leads you to think that these are similar. They are not. Your continued arguing in spite of this fact tells me you are basing your entire argument upon a false assumption. For an argument to be valid, the whole chain must be valid. Your argument fails on its most base of assumptions.
There is not a single line of code that is from Minix or Unix in Linux. As shown in court.
>If not show me the evidence, if so then just a matter of settling to which degree.
>Asking me to prove a negative.
My we're full of logical fallacies today.
SCO tried for years to show that Linux had SysV code (you know, actual AT&T Unix, not FreeBSD) in it. There is no "there" there. Your argument has been used by many other people with very large financial interest and consequences if proven true. Yet for years, those with a financial stake were never able to prove a single thing in court. SCO wanted $5 billion from IBM for "stolen code" and were unable to show a single line. They even tried to reopen the USL can of worms and failed.
If you are going to continue to argue SCO's bullshit case all over again, then I suggest you ask Darl McBride how well that worked out for him.
And calling someone disingenuous when someone is indeed disingenuous is not ad-hominem. This is especially true when the alternatives are worse.
--
BMO