Slashdot Mirror
How Linux Mastered Wall Street
itwbennett writes "Linux has become a dominant player in finance thanks to its ability to pass messages very quickly, said Linux kernel contributor Christoph Lameter. 'The trading shops saw that the lowest-latency solutions would only be possible with Linux,' Lameter said. 'The older Unixes couldn't move as fast as Linux did.' One key attribute was the TCP/IP stack, the configuration of which determines how fast a message can be passed between two systems. Linux also offers financial firms the ability to modify the source code to further speed performance. 'It depends on how daring the exchange is,' he said, noting that NASDAQ uses a modified version of the Gentoo Linux distribution. Lameter will discuss how Linux became widely adopted by financial exchanges at the LinuxCon conference in Vancouver this week."
Oh dear god our economy is a whole lot more fragile than I ever imagined. Brings a new meaning to "emerge world".
I still wont trust WallStreet with my money, which would be about like trusting an alcoholic with the beer at a party...
Politics is Treachery, Religion is Brainwashing
lets just ignore the fact that the Great Recession was directly enabled by the PHDs who turned their eyes askance at what the Gaussian Copula Function code was being used for. Not your problem right? You just make a tool, not your responsibility how others use it.
you had five kids to feed.
As mentioned in the summary, linux allows the firms to modify the OS kernel to serve their purpose. For example, the performance of trading algorithms would considerably degrade if context switching is allowed. So you can modify the kernel so as to dedicate certain cores for the main algorithms to which the OS can pass a very limited number of signals.
I know for a fact that at least one bank employs this in their high frequency trading group and probably all of them do.
Yeah, they must have chosen it because it was free, even though they cite the fact that it was the best solution to their needs because of the lower latencies compared to alternatives and because of its flexibility.
Since when do people like you bring a relevant argument to the table? Oh, that's right. You're an AC.
This is the very last place I like seeing Linux.
The article is saying (obviously) that Linux is the chosen platform for high-frequency trading, i.e. algorithm-dominated trading that has everything to do with manipulating and responding to the market in nanosecond time frames for a quick buck and nothing to do with making stable, long-term investment decisions.
I'd rather see evidence of a Linux machine in Hitler's bunker than hear about Linux helping Wall Street punks get even further from real, useful activities than they used to be.
I don't think that IBM will be pleased to be told that HP produced AIX!
It's sad to see how much money Wall Street firms were able to generate using this open source platform. We have yet to see what the open source communities get from these welches!
Development and administration at Fortune 100 companies in Manhattan is different than any other place I encountered, including other large companies. There is a lot of message-oriented middleware to patch together different systems.
You'd see a lot of strange stuff - a batch job printing from an IBM mainframe would be routed to the Unix print server, and be sent off to a junky old printer in some foreign country. Not always easy to debug when there is a problem.
Where I was, there were a ton of these old programs written in FORTRAN, COBOL and whatnot which had had business logic put in them for decades sitting on these modern IBM mainframes. Some of the business logic within it was probably lost long ago, it all just "worked", with a lot of the output routed to more modern equipment and technology. I guess they figure if anything ever goes wrong, they have almost unlimited money to throw at the problem so they don't worry about it.
You also have things happen. A business group has their developers write some program, it goes production on a machine or two, and then for whatever reason it generates a lot of money. Suddenly you have millions, sometimes even billions of dollars going over one production machine in a day. Everything happens so fast that it was never planned out to be scalable, and the main developer is too busy tweaking the program to make it make more money than to be scalable etc. If you're lucky, its market is closed during the week and you get to work on adding in additional levels of redundancy to the machine which suddenly has billions flowing through it every day. Despite the lack of planning, you better bet people will be flipping out if the machine goes down during the day, and the traders hear that their trades aren't going through due to "computer problems".
At the Fortune 100 financial I was at, Windows was considered a joke. Even the local head of the Windows team admitted that the Unix side was where things were really happening. It was just more flexible, focused on high availability and so on. With Linux coming in so much on the Unix side, that flexibility has only increased. I'm sure whatever RHEL or SUSE edition being run on most servers is so heavily modified internally by the various companies internal engineering teams, that it doesn't look like a RHEL or SUSE anyone here has ever seen. And RHEL and SUSE bend over backwards to get the business - which can be on tens of thousands, even hundreds of thousands of machines around the world.
This high frequency trading is stupid. Everyone knows that it is a scam that is just making the markets more unstable. Yet no one does anything about it. IMHO the markets should have a clock speed like a CPU. All of the trades enter a queue and the queue gets executed once a second. This would limit each trading day to X number of ticks per day. This would go a long way to removing high frequency crap from the system. Of course people will then try to improve short term predictions rather than long term like they should. But it would be a step in the right direction.
about the crisis. including
On the Brink by Henry Paulson, who was CEO of Goldman until 2006, and sec. of treasury during the crash
Too Big to Fail, Andrew Ross Sorkin
etc etc etc.
if Merrill hadn't been bought by BoA, and Morgan Stanley hadn't been bailed out by the Japanese banks, then Goldman would have fallen soon after.
Goldman got its credit default swap deals with AIG payed off, 100 cents on the dollars, when they werent even worth a fraction of that... payed off when taxpayers bailed out AIG. if AIG had gone down, goldman would have gone down, crash, explosion. none of its 'big shorts' (or 'hedges' as lloyd blankfein likes to describe it, but alot of other people dont) would have worked.
I work at a major wall street bank. We used to be heavily Sparc/Solaris/C++. Over time the Intel platforms became much faster and much cheaper than the Sparc ones. There was some early concerns about reliability but it was not warranted. The boxes are so fast now we are almost exclusively using virtual linux boxes too.
We are doing a lot of Java these days. The JVM's are much improved. It is very easy to write large heavily multithreaded Java apps to replace the our large C++ distributed systems. The Java development, build, debug, and deployment tools are great.
One can spend time arguing the merits of C++ vs Java. The reality is in most cases the C++ development time is slower, and the coding patterns used do not produce code that is faster than Java. C++ development and deployment across different platforms is a pain.
It makes sense if you are using Java for the GUI. Presumably they aren't using Java in their custom-kernel-hacking.
"First they came for the slanderers and i said nothing."
The GPL only requires you that you distribute the source code to those that have received the binary. So, if you modify GPL code for personal use, and do not distribute the binary, you're not required to distribute the source code.
i have a few Gs in a savings account but i could pull it out any time i wanted it, and there would more than likely be a few bucks in interest added to it...
I used to be like that when I was younger. A few Gs in a savings accounts. It is fucking sad, and reckless as a few Gs can evaporate quickly in an emergency. I say this in retrospect because I've been there. Luckily I grew up. Even in these times, working in software is one of the most profitable and safe careers there is. Anyone who works in software and only has a few Gs in savings is doing something very stupid with his finances.
Structured Finance and Collateralized Debt Obligations - Janet Tavakoli
Lost Trust - Lang Gibson (CDO expert who worked for Merrill Lynch)
The Trillion Dollar Meltdown - Charles R Morris
A Colossal Failure of Common Sense - Lawrence McDonald (former Lehman bond trader) and Pat Robinson
Confidence Game - Christine S Richard
EConned - Yves Smith
Diary of a Very Bad Year - Anonymous Hedge Fund Manager + Keith Gessen
The Big Short - Michael Lewis
All the Devils are Here - Bethany McLean and Joe Nocera
The Zeroes - Randall Lane
On the Brink - Henry Paulson
Crash of the Titans - Greg Farrell
How I Caused the Credit Crunch (novel) - Tetsuya Ishikawa (ex-Goldman worker)
Mark Williams - Uncontrolled Risk
Vicky Ward - Devil's Casino
Chasing Goldman Sachs - Suzanne McGee
And Then The Roof Caved In - David Faber
Inside Job - Charles Ferguson (film)
Panic - Andrew Redleaf
Too Big to Fail - Andrew Ross Sorkin
The Sellout - Charles Gasparino
Street Fighters - Kate Kelley
"1. Synthetic products were not significant, and certainly not a significant cause of the crisis. Cash products are the ones that created excess demand for real mortgages."
--- The books that mention Synthetics, especially Tavakoli's, do not agree with this.
"2. "Gambling" is a subjective term with negative connotations. I claim that any definition of "gambling" you can post will either be far to broad, or not include asset-backed securities or their synthetic counterparts. "Investing" is usually the prefered term for risk-taking in financial products. "Gambling" usually refers to games of chance, and betting on games of skill or sports."
-- Gambling is not my phrase alone, several of these authors use the word, most notable Tavakoli and McDonald, both of whom have decades of experience in high finance. There is also a definition of gambling and speculation and how they compare to legitimate businesss activity or hedging, given in Edward Chancellors ' Devil Take the Hindmost ' -- a history of financial bubbles
"3. Synthetic CDOs are based entirely on real cashflows."
The 'flow of cash' in a CDS does not come from someone who bought a product, it comes from one counterparty who is gambling that a certain debt will never be payed off, to another counterparty, neither of which need to have any relationship, whatseover, with the debt they are betting on. This is described in Janet Tavakoli's book in great detail. Synthetic CDOs are made of CDS, thats why they are gambling.
"Second, I'm sure it's completely false that ratings agencies didn't look at the underlying loans."
Michael Lewis ver batim quotes an internal memo from ratings agencies in The Big Short, where a manager tells the workers that they do not have any underlying loan data, and that the products need to be rated anyways.
"False, but points to a truth. First, ratings are always paid by the issuer (note I didn't say "banker" since banks aren't the only issuer). Remove the words "very highly": the issuer doesn't get to pick their rating. They can haggle "
They 'haggled' by saying that if the rating agency didnt rate the bond highly enough, then the bank would go to another ratings company. This is well documented in books like The Big Short.
"I can assure you that the books were written to generate sales, rather than provide an accurate historical record"
Wow. I dont know what to tell you. If you beleive the entire industry of journalism, including hundreds of articles describing in detail the dozens of CDO frauds like ABACUS, Magnetar Capital, etc, and that its all a conspiracy theory then I guess I have nothing to argue against it.
"And so did the CDO desks that made tons of money for their companies. And I think you gave credit to the desks where it's due: "
Every thing ever written about the fall of Merill Lynch points directly to the CDO desk. The same could be said of Lehman and others. In fact, that is Goldman and Detuschebanks' defense of some of their acti
before it started. lets take hedge fund manager Bill Ackman for example.
he was telling anyone who would listen that the Monoline insurance companies were based on fraud. they were insuring stuff they were never supposed to have insured, risky stuff, and then lying to investors about it.
what happened to him? He got investigated by the SEC. he could have gone to jail for a long time. the monolines didnt investigated, the ratings agencies didnt get investigated, the New York State Insurance regulators didnt get investigated for allowing this.... Bill Ackman got investigated.
the monolines were like mini-AIGs - they enabled the fraud of the CDO market to continue even after AIG stopped insuring certain CDOs at the end of 2005. the monolines continued, and helped push the bubble up. then they all collapsed and died in 2009 - they were too small to get bailed out. their only mistake was not being bigger.
its described in Confidence Game by Christine S Richard
the 'bullshit morality' of Ackman , well, it actually made him a lot of money.
the same for the 3 main groups in the Big Short .... they tried to warn the government about what was happening, and journalists, and they were largely ignored. their 'bullshit morality' made them a lot of money too.
of course some people dont think these 'big shorts' behaved morally..... because they profited from the crash (Yves Smith thinks they helped it... because Synth CDOs were made up of their bets (CDS))
anyways.
there were others. Janet Tavakoli in particular wrote some textbooks on CDOs , describing a lot of the fraud that was inside the business. Her 'bullshit morality' might not get her the fame and TV spots, but then again, her type of 'bullshit morality' is the foundation of all modern civilization.