How Linux Mastered Wall Street

itwbennett writes "Linux has become a dominant player in finance thanks to its ability to pass messages very quickly, said Linux kernel contributor Christoph Lameter. 'The trading shops saw that the lowest-latency solutions would only be possible with Linux,' Lameter said. 'The older Unixes couldn't move as fast as Linux did.' One key attribute was the TCP/IP stack, the configuration of which determines how fast a message can be passed between two systems. Linux also offers financial firms the ability to modify the source code to further speed performance. 'It depends on how daring the exchange is,' he said, noting that NASDAQ uses a modified version of the Gentoo Linux distribution. Lameter will discuss how Linux became widely adopted by financial exchanges at the LinuxCon conference in Vancouver this week."

Comments

NASDAQ uses Gentoo?

[suso]

Oh dear god our economy is a whole lot more fragile than I ever imagined. Brings a new meaning to "emerge world".

Re:NASDAQ uses Gentoo?

[basketcase]

So, what do I add to USE or CFLAGS to be able to call myself rich using imaginary money? CFLAGS="-fmake-money"? USE="federal_reserve"?

Re:NASDAQ uses Gentoo?

[Oxford_Comma_Lover]

Nah, they should switch to Macs. There's nothing like running a TCP/IP stack that reuses a modified copy of old FreeBSD code without changing the comments. =)

Re:NASDAQ uses Gentoo?

[BitZtream]

Thats kind of the point of BSD licensed code ...

They'd be much better off switching to FreeBSD since its known to have the fastest IP stack in existence, hence why its used for high end networking gear. Let me know when you see your BigIP running something other than FreeBSD.

I dont care of WallStreet likes linux

[FudRucker]

I still wont trust WallStreet with my money, which would be about like trusting an alcoholic with the beer at a party...

Re:I dont care of WallStreet likes linux

[migla]

I still wont trust WallStreet with my money, which would be about like trusting an alcoholic with the beer at a party...

I don't know how it is over there, but over here in Sweden everyone has money in the stock market as all pensions were moved there some years ago. We're all supposed to choose where to invest them and to care about the wellbeing of Wall Street.

Kind of unfair advantage given to the propagandists of one economic system over another, in my opinion.

thanks for whoring quants

[decora]

lets just ignore the fact that the Great Recession was directly enabled by the PHDs who turned their eyes askance at what the Gaussian Copula Function code was being used for. Not your problem right? You just make a tool, not your responsibility how others use it.

you had five kids to feed.

Re:thanks for whoring quants

[tunapez]

The problem is that measures of uncertainty using the bell curve simply disregard the possibility of sharp jumps or discontinuities and, therefore, have no meaning or consequence. Using them is like focusing on the grass and missing out on the (gigantic) trees. In fact, while the occasional and unpredictable large deviations are rare, they cannot be dismissed as âoeoutliersâ because, cumulatively, their impact in the long term is so dramatic.

The Godfather tried to warn them. Don't know if I'm pleased or saddened that he lived long enough to see his incredible tools turned into "weapons of financial destruction". [Emphasis mine]

Luckily, he will be spared the repeat performance. Nothing's changed, they're still "printing money" every day. [emphasis theirs]

Re:thanks for whoring quants

[nog_lorp]

equally the fault of

Not how blame works. Google "blame is not a zero sum game". The influences (in part or wholly) responsible for any event are infinite, and so are the factors leading to those influences (i.e. it's not Goldman Sach's fault their parents didn't raise 'em right ;)

And if you really think people with no financial know-how who were misled by predatory lenders (with the responsibility of providing those people sound advice) are as responsible as those who carefully architected massive fraud, you must be a troll (intentionally or not).

Customizable Kernel

[Warlord88]

As mentioned in the summary, linux allows the firms to modify the OS kernel to serve their purpose. For example, the performance of trading algorithms would considerably degrade if context switching is allowed. So you can modify the kernel so as to dedicate certain cores for the main algorithms to which the OS can pass a very limited number of signals.

I know for a fact that at least one bank employs this in their high frequency trading group and probably all of them do.

Re:Customizable Kernel

[Jah-Wren Ryel]

That would be an interesting and a not too hard a project, I am assuming. So this in turn would imply that there would already be an option in the kernel to do it right? So why not improve the performance of gaming or computation systems by recompiling the kernel to do this?

I suspect they are using some form of real-time extensions to the linux kernel. One thing they all share in common is little to no OS functionality is available to applications that are running real-time because, it is almost always the case that as soon as you ask the kernel to do something for you, all bets are off as to when it will get done or even when you will get scheduled back on to your cpu.

So, in order for a game to take advantage it would need to isolate out whatever parts of the code are performance and dead-line critical so they wouldn't need to interact with the OS. And then the user would have to install and boot a linux kernel with the real-time stuff turned out.

Also, FWIW, real-time is not about speeding up computation, it is about hitting your deadlines exactly when they need to be hit - not too soon, not too late. In many cases you trade off computational performance in exchange for the ability to meet those deadlines.

Re:Really?

[Superken7]

Yeah, they must have chosen it because it was free, even though they cite the fact that it was the best solution to their needs because of the lower latencies compared to alternatives and because of its flexibility.

Since when do people like you bring a relevant argument to the table? Oh, that's right. You're an AC.

Oh, Linux, how you've forsaken us

[RobinEggs]

This is the very last place I like seeing Linux.

The article is saying (obviously) that Linux is the chosen platform for high-frequency trading, i.e. algorithm-dominated trading that has everything to do with manipulating and responding to the market in nanosecond time frames for a quick buck and nothing to do with making stable, long-term investment decisions.

I'd rather see evidence of a Linux machine in Hitler's bunker than hear about Linux helping Wall Street punks get even further from real, useful activities than they used to be.

Re:Oh, Linux, how you've forsaken us

[vux984]

All HFT does is take advantage of arbitrage opportunities that a human is too slow for.

Exactly right but think about that for just half a second.

Who was benefitting from the fact that the arbitrage opportunity was not exploited? In each case it was the buyer or seller.

So the "profit" the HFT takes, is taken directly from the buyer or seller in each transaction.

Everytime I place a buy order, without HFT, i might have got the shares for a few cents less. Everytime I place a sell order, without HFT, I might have gotten a few cents more for the shares.

"HFT takes advantage of arbitrate opportunities that a human is too slow for" ... meaning my transactions would have closed without the "liquidity" HFT provided within seconds.

So how do you reach any conclusion other than "HFT" is leeching profits from the actual buyer and seller, while providing nothing of value?

Re:Oh, Linux, how you've forsaken us

[I(rispee_I(reme]

When you buy low and sell high, that stabilizes prices. Since High Frequencies Traders are profitable, they therefore contribute to the stability of the market.

This is exactly the fairytale middlemen tell themselves so they can sleep at night after making ludicrous amounts of money for producing nothing of value.
Their only goal in life is not to the the last person holding the bag or the bottom tier of the pyramid. That's why latency is so important, and that's why the "free market" is a myth- as long as your latency is higher than someone closer to the exchange, there can be no level playing field.

I've thought about this at some length, and barring "spooky action at a distance" to negate the effects of latency, two ideas commend themselves to me:

1) A fixed interval of latency imposed on all trades that is much larger than the maximum latency differential. This seems like it might help things, but it also seems like sweeping the problem under the rug- there would still be some advantage to lower latency in trades, after all. The "high frequency trades" would just occur as close to that fixed interval as possible.

2) An alternate currency used to pay those whose "profitable" actions can be repeated arbitrarily in a given interval. It takes no more effort to sell a million shares short than a billion, but (for example) an ear of corn cannot be multiplied effortlessly in a given interval. It seems to me that by paying the middlemen (who do not produce anything of real value) in the same coin as the farmer (who does), the farmer's money is devalued. Let me anticipate the "the middlemen would simply exchange their currency for the farmer's currency" reply by saying that in doing so, they would empower the farmer. After all, his currency is scarce, and the middlemens' currency is not.

I'm sure I'm an imbecile who doesn't understand the subtleties of Wall Street, but then again, evidence seems to suggest that so is everyone else.

Re:Oh, Linux, how you've forsaken us

[couchslug]

Freedom doesn't mean "just freedoms you approve of".

The best quote on the subject is from the OpenBSD camp:

"But software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia."
Theo de Raadt
      cvs@openbsd.org mailing list, May 29, 2001

Accuracy ?

[Alain Williams]

As late as 2007, Wall Street exchanges were still largely run on Unix, such as HP's AIX and Sun Microsystems' Solaris

I don't think that IBM will be pleased to be told that HP produced AIX!

Set the exchanges to a clock.

[inhuman_4]

This high frequency trading is stupid. Everyone knows that it is a scam that is just making the markets more unstable. Yet no one does anything about it. IMHO the markets should have a clock speed like a CPU. All of the trades enter a queue and the queue gets executed once a second. This would limit each trading day to X number of ticks per day. This would go a long way to removing high frequency crap from the system. Of course people will then try to improve short term predictions rather than long term like they should. But it would be a step in the right direction.

Yes, we are heavily linux

[proud american]

I work at a major wall street bank. We used to be heavily Sparc/Solaris/C++. Over time the Intel platforms became much faster and much cheaper than the Sparc ones. There was some early concerns about reliability but it was not warranted. The boxes are so fast now we are almost exclusively using virtual linux boxes too.

We are doing a lot of Java these days. The JVM's are much improved. It is very easy to write large heavily multithreaded Java apps to replace the our large C++ distributed systems. The Java development, build, debug, and deployment tools are great.

One can spend time arguing the merits of C++ vs Java. The reality is in most cases the C++ development time is slower, and the coding patterns used do not produce code that is faster than Java. C++ development and deployment across different platforms is a pain.