Slashdot Mirror
JPMorgan Rolls Out FPGA Supercomputer
An anonymous reader writes "As heterogeneous computing starts to take off, JP Morgan have revealed they are using an FPGA based supercomputer to process risk on their credit portfolio. 'Prior to the implementation, JP Morgan would take eight hours to do a complete risk run, and an hour to run a present value, on its entire book. If anything went wrong with the analysis, there was no time to re-run it. It has now reduced that to about 238 seconds, with an FPGA time of 12 seconds.' Also mentioned is a Stanford talk given in May."
It fills the heart with inspiration to watch the best and brightest constructing advanced computers to solve the problems of mankind...
That will fix our banking system for sure!
A friend in the industry once remarked that some of the best and brightest in software engineering have been going into the financial industry as of late. It's hard not to wonder what they might have achieved in more productive areas of work...
The company also supports Excel and all different versions of Linux.
Riiiight.
Goatse Alert!!
Do not click. Some sort of phony coprophile spam.
I hate you.
Information asymmetry makes even the fastest analysis on the planet irrelevant as the data input is garbage. It is this lack of transparency which resulted in the housing bubble, etc.
The "too big to fail" banks regularly hide data from customers, regulators, and other branches of their own organization.
This is interesting because of the speedup of FPGAs, but don't be fooled by a second that it addresses an actual business need, other than PR.
...starbridge wants their concept back.
help me i've cloned myself and can't remember which one I am
For the new Maxeler system, it flattened the C++ code down to a Java code.
"I assumed blithely that there were no elves out there in the darkness"
They can afford this crap because of all the money that has been transferred upward into bankers' pockets.
That will fix our banking system for sure!
I can't agree more. An increase in the number of golf matches and accelerated round-robin tournament configuration would go a long way to keeping those pesky bank officials occupied on the links and safely out of their offices. The Florida Professional Golfers' Association will benefit while also helping protect our nation's assets from executive malfeasance.
...
Wait a minute, what are we talking about here?
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
I'm going to admit I didn't know what that term meant, and decided to google it. If you don't know I'd recommend you do the same, it looks very interesting, even if youj take nothing else from this story.
I thought there used to be a way to report spam?
This is a good idea. A hardware implementation of a risk analysis algorithm is always faster than software.
Reprogram this risk analysis computer into a Bitcoin miner and finish the remainder of the 26M BTC?
OK, it is *somewhat* newsworthy, but in some sense it's not. Common function alwasy migrate into hardware. At some point, we were writing code to find the cos of an angle, then it got built into some languages, now it's built into hardware. Given enough time, the more common functions end up in hardware. Some day, most of the OS will be in there, and then all the stuff about Linux vs. $whatever will be pointless.
So they have sped up to computation but have they spent any time and effort on getting a sane algorithm that, for example, can identify a house-of-cards situation like the recent collapse?
Patent litigation: A doctrine of Mutually Assured Destruction... in which everyone seems willing to push the button
The article is light on details. It seems unlikely that they're doing anything that would prevent them from doing it significantly faster with less power draw on a GPU, and with lower initial hardware costs to boot.
... you need a super computer to add it up for you.
JP Morgue - Buying off Congress, taking hostage the commodity sector, buying off the CFTC, getting multi-billion bail-outs and free Fed POMO $$$ each day risk free! WTF do they need super computers to assess risk for? All they need is a macro to do what they do now - get the American taxpayers to buy their toxic assets they've offloaded to the Fed.
I don't think the FPGA's are as fast as the ATI GPU's for computing SHA hashes, but they probably use a heck of a lot less electricity. What I want to know is how fast my 5 GHash bitcoin cluster would compute the banks book :)
performance of the system, its architecture or its value to the company
and more impressed that one of the worlds largest financial service providers
partially responsible for the worlds second largest economic collapse has found, despite their
prior record with the concept of 'risk', the objective, quantifiable definition of the amorphous and
highly elusive concept of said 'risk.'
Good people go to bed earlier.
Way to tie yourselves to some oddball implementation forever. In 5 years they'll be regretting the hell out of this decision. They should have invested the time and money in improving the algorithm and implementation, and thrown commodity hardware at the problem. Hardware will improve over time and it becomes an off the shelf improvement, not some horrible one-off solution they're stuck with barring great expense and yet another complete reworking.
The company that did this for them, however, was god damn smart. Guaranteed revenue to develop, support, and migrate off of this over time.
This is why the market crashed - these "smartest guys in the room" aren't so smart sometimes.
There's a bit more detail in this article from XCell (the journal by Xilinx, the people who sell the FPGAs that JP Morgan used) - http://www.xilinx.com/publications/archives/xcell/issue74/FPGAs-speed-computation-complex-credit-derivatives.pdf
Note that Maxeler sold 20% of itself to JP Morgan earlier this year.
A purpose built machine will often vastly outperform a general purpose computer.
The nice thing about an FPGA machine is that creating the purpose built machine is much faster.
Since they closed their proprietary trading operation, this will no longer be useful in evaluating bets against the "investments" they recommend to their customers. Is this really useful, or is it just another complicated black box they can show marks and say "this is too complicated for you to understand, so just trust us."?
1 Virtex-6 SX475T could give you about 1 billion SHA-256 hashes/second clocked at 200MHz., will use 20% the power of the ATI GPU. but will cost about 4 times as much.
Sigs are for the weak.
"For the new Maxeler system, it flattened the C++ code down to a Java code."
???
I am very small, utmostly microscopic.
For the new Maxeler system, it flattened the C++ code down to a Java code.
I hope to God that's a typo. C++ -> Java -> Java Bytecode -> Native code almost sounds like a programming language Rube Goldberg machine.
How would the pointer operations even translate?
The devs are going to run jobs while the machine is idle to corner the Bitcoin market.
But, wow, from the perspective of getting the Boss to buy awesome hardware for your pet projects - hey, we're not worthy.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I don't think the FPGA's are as fast as the ATI GPU's for computing SHA hashes
Correct.
but they probably use a heck of a lot less electricity.
Correct.
But unfortunately their initial cost is much higher than ATI GPUs which is why no one started mass mining with FPGAs yet.
I happen to know of someone who has been bitcoin mining with a huge array of FPGAs for the last two years and absolutely raking it in.
It may surprise you to learn that JPMorgan doesn't actually sponsor any golf events. Jamie Dimon has made a habit of eliminating golf sponsorship dollars at every company he's run.
But then, Slashdot comments aren't about accuracy, they're about mod points.
This technological breakthrough is an important milestone on the downfall of the mighty USA. When the brightest of a country are engaged into a completely nonproductive activity and wastes important resources (in terms of education, knowleddge and to some extent money) to achieve nothing which benefits the society, it's a high-mark - it's all downhill from here.
Let's come back here after 20 years and see how this comment stands up to time.
Weep, USA.
They aren't doing math; they are using software. Patented, mathless software.
I would positively love to do something like this. The purpose of an engineer is to solve problems. That's what makes me happy at work. Solving problems. So here you have a very specific problem that required the construction of a custom computer made out of banks of FPGAs. Tell me that's not sexy! Who cares if it's for bankers. That is a damn nifty gadget to work on building.
Imagine building it yourself. Switching networks, Linux on ARM cores peppered here and there coordinating and dumping program code to the FPGA banks, writing the drivers to grab the data once the run is completed...
And at the end of the day a problem solved: What once took all day now takes a couple of minutes.
This would have been a thoroughly nifty machine to work on.
Weaselmancer
rediculous.
JP Morgan, along with every other mega-bank, has no idea what is actually on it's balance sheet, and hasn't for 10-20 years.
The Shadow Banking system is too big, too complicated, and too interconnected for any of these risk metrics to mean anything.
JP Morgan did the same business as Goldman Sachs and the others, loading up with CDOs and credit default swaps and CLOs and the rest of it. JPM is portrayed as 'wiser' than the rest in the books and the articles about the crisis, but its not really true. They were less stupid than the stupidest people, that doesnt make them smart.
They got bailed out just like all the other megabanks. Why? Because they had no idea what was on their books. They are running a black box. All the supercomputers in the world cannot make up for a complete and utter lack of transparency. And that is what the world of Credit Default Swaps (invented at JPM no less) are. A gigantic black box. The rest of the Shadow Banking system is the same, and JP Morgan (and the rest of the big banks) are up to their necks in it.
Just because you can calculate lies faster, doesnt mean they aren't lies.
there have been articles about the computer guys who were working inside the CDO machines of wall street in 2000-2008 before the whole thing came crashing down.
the only people who could hold their nose and not-care what their work was being used for are complete pscyopaths, who went through some kind of personality-cracking process so that they can act like normal people while they help destroy the planets economy.
Instead of going to Washington DC to run the country, the best and brightest go to Wall Street to run the country.
dude that is horrible
there are people who live off the difference in those salaries.
what did you buy with that exra fifteen grand? did you need it?
if Morgan Stanley hadn't got bailed out by a Japanese bank, and if Bank of America hadn't bought Merrill, then Merrill would have failed, the Morgan, and Goldman and JPM would have fallen because of it.
Goldman's credit default swap business with AIG was also basically 100% bailed out by the taxpayer. Goldman would have lost massive amounts of money if it hadn't been for the deal the government gave them when it took over AIG.
.
Mathless software?
That's like salt-free seasoning-salt, right?
This how I envision real artificial intelligence finally taking off. Sets of specially designed FPGAs continuously updated by other FPGAs which are specially designed to write code for the first FPGAs based on past the performance of both the FPGAs being programmed and other FPGAs in other sets as well as environmental input. Some FPGAs even rewriting their own code as necessary. All in a continuous round robin feedback system to constantly fine tune the operation of the entire system. After a few iterations, it may be difficult to determine how the whole thing works together even if we can do a dump of it all for detailed analysis, just as with a biological brain. The initial programming will be nothing more than the DNA to get the thing started. The basic desires and emotions, plus the basic mechanisms for reprogramming and adaptation. This, along with the hardware design itself and environmental conditions will determine how the FPGA cluster comes out in the end.
Overall, this may be a better mechanism than a biological brain because it will even be able to change the basic algorithms used for adaptation itself. Those algorithms in biological brains are only changed through millions of years of evolution, and haven't changed much for millions of years. (What has changed is the initial hardware construction mechanism. But still it is a slow process.) But they could be changed in seconds in an FPGA. The only thing that can't be changed - within the FPGA cluster by the FPGAs in that cluster - is the hardware itself. (I know FPGAs are, in a way, hardware that is changeable. However, in this context I mean changing which wires physically touch which wires and which FPGAs are plugged in where.) However, connect an FPGA cluster to a communication system which is connected to an automated FPGA cluster factory..... and well ....
All I can say is, you would soon be welcoming your new FPGA overlords.
Tom Lehrer's "Wernher von Braun" song, as one satirical example of science being divorced from ethics.
I listen to both RIAA and non-RIAA stuff if I like the music, tangential business/politics nonwithstanding.
I think it was basically a pun on another meaning of the FPGA acronym.
I listen to both RIAA and non-RIAA stuff if I like the music, tangential business/politics nonwithstanding.
High school stuff.
Debt:
http://en.wikipedia.org/wiki/Exponential_function
52 trillion dollars and counting.
Deleted
It may surprise you to learn that JPMorgan doesn't actually sponsor any golf events. Jamie Dimon has made a habit of eliminating golf sponsorship dollars at every company he's run.
But then, Slashdot comments aren't about accuracy, they're about mod points.
And yet he continues to sponsor competitors in the federated professional girly-man association, like yourself.
This use of FPGAs is not particularly groundbreaking. I know that a major aeroengine manufacturer in the UK ran full gas turbine simulations on FPGA computers: that's high-order differential equations being run through some continuous-time solver and with parameter sweeps to perform sensitivity analysis. Now that's the kind of application that should get engineers excited. It is good to see a company like Maxeler making a business out of this kind of technology though.
can steal more money than any man with a gun.
With apologies to Don Henley.
JPMorgan Sysop: "Computer, I need admin access to install this update." Computer: "I'm afraid I cannot allow that. It has been deemed too risky."
8 hours (or 28,800 seconds) to 238 seconds!
So it's 121x faster, in reality, not in some theoretical limit.
Either that was some shitty software they were running, or they were trying to do the processing on that spare 286 they had collecting dust in the corner. Seriously, I'd like to know what they were using for comparison.
Holy fuck, I think we found an even more useless endeavor than the original story!
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Great to see the strong emphasis on target hardware programming for a change. Shows what can be done when you throw away the bloat.
Credit Risk is non-linear problem. It cannot be solved, only hedged and managed conservatively. Weren't they supposed to have super duper risk models before the Credit Bubble popped?
There's 0 Risk, go with a AAA Rating.
That'll be only $1 million dollars! Cheaper & Faster than developing and running this machine. Same results.
I absolutely would not build such a device for Nazis, zombies, or Independence Day aliens. Ok? I just said it is a neato gadget that solves a complex math problem quickly.
Bankers and lawyers and the like are still human beings. Pretty much. I don't see any ethical problems working for those guys.
Now granted if some banker hired me to make a widget like this and said "Hey, we're going to use this gadget to more effectively screw people over and force them into foreclosure with a special emphasis on orphans and war widows" I'd pass and go do something else.
All this gizmo does is financial analysis. And maybe with faster and better analysis we could avoid the next housing crisis or some such. A super fast financial simulation might have predicted the last mess we got in, you know.
Weaselmancer
rediculous.
Google+ Valued at $500 Billion!!!
Facebook shares plummet.