Slashdot Mirror
JPMorgan Rolls Out (Another) FPGA Supercomputer
An anonymous reader writes "JP Morgan is expanding its use of dataflow supercomputers to speed up more of its fixed income trading operations. Earlier this year, the bank revealed how it reduced the time it took to run an end-of-day risk calculation from eight hours down to just 238 seconds. The new dataflow supercomputer, where the computer chips are tailored to perform specific, bespoke tasks (as explained in this Wall Street Journal article) — will be equivalent to more than 12,000 conventional x86 cores, providing 128 Teraflops of performance."
So they can project how much money to borrow from the Federal Government the next time they have lent beyond sane limits to property speculators or invested in schemes even Mandelbrot wouldn't be able to simulate.
A feeling of having made the same mistake before: Deja Foobar
for all you people who don't really think in seconds when seconds is > 60.
How much is your data worth? Back it up now.
These banks aren't just siphoning money, they are also siphoning talent away from more important projects. The people working on these things could be brilliant physicists or engineers, if they weren't sucked into the dark side.
You can get petaflops with a fraction of 12,000's x86 price, just use GPUs...
This is not really news.
Nope. OpenCL 1.2 was released recently. That's news.
;)
But it's certainly popular to blame banks with computers for everything. It's been happening for as long as I can remember.
As a boy being forced to attend church, I remember sermons on the evil computer somewhere in a room, nicknamed "The Beast," calculating everything everyone did. Boy were those people wrong. Turns out the computer was named google.
using FPGA's instead of x86 would probably consume a significant amount less electricity. using manpower is good in terms of the many men being paid for their efforts instead or a few ceos just pocketing the money as extra bonuses
equivalent to more than 12,000 conventional x86 cores, providing 128 Teraflops of performance.
128 Tflops / 5 Tflops = 25.6 AMD dual gpu video cards.
25.6 x $700+/- = $17,920
I wonder what they paid for it.
This story got me thinking that many of the tasks routinely executed on personal computers (perhaps cryptography, video decoding, and such) may benefit from including a FPGA in PCs to serve as a programmable coprocessor. Much like graphics-intensive software can come with shader code to offload processing to the GPU, couldn't a video codec or an implementation of SSL or whatever come with code that would allow an FPGA to do part of the work?
I googled around and found that at least CERN has done something of the sort, but that was over seven years ago. There was a story on Slashdot about something of this sort, but it's even older than the CERN publication. Is anyone working on this sort of idea? If not, why? Is it simply a matter of cost, or is there some other issue that makes this impractical?
Maybe I just suck at googling...
I spent two great years working for J. P. Morgan Chase, starting in 1999, followed by a year with the merged JPMC, so I have some knowledge of how this new system will be used and how it fits into the business process of running a bank. I can't discuss details about that, but I just wanted to share my congratulations with the JPMC team for tackling that thorny issue.
You have to understand that investments can't be made until those risk analyses are done, so cutting 7-8 hours off the run time will earn the company millions over the course of a year. We're talking about the kind of investment loans where even a 4-5 hour overnight "float" of capital to help someone seal a bigger deal can be worth a significant amount of interest and profit.
Remember: the big investment banks are dealing with numbers that cause spreadsheets to overflow. You can't even visualize the data with standard desktop tools. You wouldn't believe the totals I saw come out of some reports, and I wish I could forget them. Such numbers are not meant for the grasp of mere humans living on a working wage.
I do not fail; I succeed at finding out what does not work.
Talks about this stuff and some low latency stuff too...
http://techcareers.jpmorgan.com/downloads/Field%20Programme%20Gate%20Array%20factsheet.pdf
Now JP Morgan can raid future MF Globals all that much faster, while hiding their shenanigans at the COMEX.
How much money are they spending in manpower, electricity and consumables by calculating risk? how about make a super computer to figure out how to solve the world debt.
Everyone knows the answer to that question already. Learn to live with less resources for each person, or figure out how to have less people.
In the movie, Margin Call (2011), the investment bank that saw the danger of their risk exposure first was prompted to sell and start a massive panic that sunk a competitor. The movie is loosely based on the Lehman crisis. "John Tuld: There are three ways to make a living in this business: be first, be smarter, or cheat. "
Sounds like I should get a job in the financial industry. /subscribes to "Giant Boats"
That's 38 more teraflops than Skynet, and I thought I was afraid of big banks before...
before credit default swaps grew to dozens-of-trillions of dollars business?
before Commodity Index Funds?
before the stock exchanges and commodities exchanges got rid of their open outcry trading pits and went electronic?
before JP Morgan bought Bear Stearns (with a Fed loan)?
before JP Morgan got bailed out by taxpayers?
no offense dude... but what on earth does 1999 have to do with 2011?
So JP Morgan can calculate VaR faster. so what? they didnt calculate it fast enough in 2008 - they took that big fat payout like everyone else, they were up to the arms in CDOs and subprime like everyone else, they had their fingers in every pie. They arent "earning" anything, they are "stealing" it from the taxpayer, with an enforced monopoly based on corruption in the federal government. JP Morgan should not exist.
I do not buy your thing about spreadsheets, considering how CDOs were often calculated by people called 'F9 monkeys'. and one CDO deal could be worth a billion dollars.
If bankers had a clue they would use these supercomputers to generate bitcoins!
It is a FPGA supercomputer, don't you need to reconfigure it first?
GENERATION 25: The first time you see this, copy it into your sig on any forum and add 1 to the generation.
Actually global exports are over $300 billion more than global imports. So we are running a trade surplus with *someone*.
"I assumed blithely that there were no elves out there in the darkness"
More importantly, it makes it impossible for anyone to dispute their results. After all, no one else has exactly the same system so no one is better qualified to evaluate their conclusions. "No, we made the best possible choice at the time. You would know that if you had OUR analytic engine, but since you don't your speculations are baseless."
The reason we subjugate ourselves to law is to better procure justice. If law does not accomplish this purpose then it m
Amen... now that I understand rehypothecation, (having cake, borrowing against same cake, borrowing (again) against same cake... an unlimited number of times, then eating cake, is all legal thanks to "the city of london")
Any company with a division or branch in England can use this trick to hide/steal spectacular amounts of money. Because of the interlinked nature of the markets, Nothing is safe from this, anything in an "account" of any sort (stock, bank, savings, checking, etc) is only as good as the personal bond between yourself and the person who actively manages it. Since none of us have that connection to anything stock related... it's all a big Ponzi scheme, ready to implode.
Unfortunately, it may take all of civilization with it.
It will be unwound, messily, but it will unwind and society will go on. Do not buy into all of the hype about civilization coming to an end. There will always be people with excess capital that they want to invest. There will always be people with good ideas who need funding to get them off of the ground.
All of our supply lines have had the slack engineered out of them in the name of "efficiency" in the last 20+ years. The US doesn't make everything it needs, let alone any given state... without international credit, next day air, and a continuous flow of liquid energy, it's all going to crash.
Eventually we'll recover... but the days of ever faster extraction of resources to be "consumed" then buried in a landfill, is rapidly approaching it's end. Unfortunately, we don't know how to work any other way.
Let face it JP is doing some Bitcoin Mining on the side with this hardware! loll Off course that would be less of a scam that what they really do with it!
you could build in a serial interface or shared memory interface for printing status or debugging, or debug over JTAG by tapping the signal flow inside the FPGA with signal tap (altera) or chipscope (xilinx).
NetBSD: the cathedral vs the bizzare.
What a waste. These banks can build the most impressive hardware in the world, perform calculations in the petaflops, and still have absolutely no clue what risk is involved in the business they're doing because their assumptions and data are all wrong. If by some freak accident they were to get the right answer, they would conceal it from their clients and investors anyway because their incentive is to take big risks - they get enormous rewards if they are right and lose little if they are wrong. They are incompetent and amoral, which is simply not a technical problem.
Typically, you write a testbench that can, in fact, printf() (sort of). Basically, you end up running a timing-level simulation of the FPGA, or sub-blocks thereof. You're really not developing a piece of software, you're developing a small ASIC. In any event, after you run timing simulations through your testbench where you put in known inputs and verify that you get the expected outputs, you're ready for anywhere from a few minutes to a few days (depending on the size of the FPGA) of compilation to get your code turned into a bitstream to program the device. Then you run the same inputs from the testbench and see if you get the same output. At this point, you hope that you remembered to build various debugging registers into your design so you can have a prayer of finding problems. You can also stop the clock and scan out the value of every register on the chip through something called JTAG. You can then import that back into your simulator to try to figure out what has gone wrong. Then restart at the "few days of compilation" stage.
This is why so few people do reconfigurable computing....
Go Badgers! -- #include "std/disclaimer.h"
and they do their damn best to keep it quiet. Look at how quickly stories about Congress and their staffs using insider information was quashed. It went from being front page to gone in days, as if it didn't exist. Similar to how Fast and Furious vanished.
We need an OKS, Occupy K-Street, Wall Street is fully enabled by Washington. They just pay their dues and Washington insiders reap the real rewards.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
There (is) new evidence of a payoff for certain kinds of customers from J.P. Morgan, which has been working since 2008 to adopt Maxelerâ(TM)s hardware and software to help assess its trading risks. The companies say the approach has allowed the financial-services company to quickly examine tens of thousands of possible scenarios for how its investments might be affected by events in financial markets, reducing the time for running certain scenarios from hours to a few minutes....quickly examine tens of thousands of possible scenarios for how its investments might be affected by events in financial markets,
Yeah sure, they examined them, chop chop, and what did this tell them? Nothing, unless all this refers to is guessing what other hedge funds will do in response to each other in a time frame that is not longer than a few milliseconds. .
Beyond that narrow (and getting narrower all the time) interpretation, there is no "prediction" about the market to be had, since the market is determined by what happens in the real world which is pathologically and permanently unpredictable.
Oskar Mencer, Maxelerâ(TM)s chief executive and founder, says the technology can prove its worth to J.P. Morgan in situations like handicapping whether debtor countries might default on obligationsâ"allowing the bank to make hedging transactions to reduce the financial impact. âoeThey can ran different default scenarios two days ahead,â Mencer says. âoeThey can hedge against those scenarios very effectively.â
So you get what they're trying to do, right? They're trying to use historical data - about say, defaults, or just generally about what real world actors do under circumstance X when conditions Y and Z hold, and assign probabilities to them so they can make a decision and "beat the market".
The problems here are manifold.
One is, a perfect record of all the financial events in the world, however you want to define them, is not enough to form a statistical basis for analysis. Why? Because there's no reason to believe that such events are the product of Random (not "random") variables, which is a axiomatic of all statistical analysis.
Two, the events they're concerned with interact with each other and random (not "Random") world events in ways which are so complex they are fundamentally unpredictable. Only systems whose events of interest can be causally isolated from other events, however broadly you have to define events to make that happen, are amenable to statistical analysis.
In the real world, where literally every social and financial event could and does effect all others, you have the impossible task of having to model everything in order to predict anything accurately and consistently, or even on a, you know, regular basis.
The real world of human and financial events is not amenable to analytic predictive methods . Full stop.
This is not to say there won't be winner and losers where winner's predictions come true and losers predictions go bust. This is to say they don't know who they are in advance.
The unfolding of any given course of financial events is a one-time historical event happening forwards in time and is ultimately infinitely sensitive to any other real world event or set of conditions, either knowable or unknowable. There is no basis to form a probability distribution of likely futures because there is no clearly definable event or circumstance- every event and circumstance is one-off and sui genris.
But this s not REALLY what they're trying to achieve, at least, we can hope not.
What they're trying to achieve is to guess how OTHER institutions who understand the world as they themselves do are likely to act, and beat them to it or o
You can run chipscope (Xilinx) to see what's going on inside. You can also run simulations before you instantiate your hardware in the FPGA. In a simulation, you can indeed do the equivalent of "printf".
Oolite: Elite-like game. For Mac, Linux and Windows
... when JPMoran is broke due to "fat finger" bugs in the trading software...
"I love my job, but I hate talking to people like you" (Freddie Mercury)
Sadly, both stories lack details on how the FPGAs are used in the computing architecture. Instead the spend great lengths on listing telephone number like, meaningless speedup comparisons with conventional hardware. A typical drawback of FPGAs is that they cannot accommodate as many floating point units (FPUs) per chip as current GPUs and that FPGAs run at about 10x lower clock speeds. Their advantage however, is that the internal chip architecture can be reconfigured to match the algorithm, so that all FPUs run at maximum efficiency. At the end of the day, it really depends on the algorithm, whether it's run best on FPGAs, GPUs or standard CPUs. This is also the reason why one cannot say that an FPGA is X times faster than a GPU: it really depends on the algorithm.
Maxeler, the manufacturer of the machine, had a booth at SC11. The basic component is the MAX3 card, a PCIe 2.0 8x card with up to 96 GB of DRAM on board. The boards are optimized for data stream processing. This is not unlike how GPUs are architectured.
Up to 4 of those boards are located in a MaxNode, which can then be networked via 10Gbit Ethernet or InfiniBand. Multiple MaxNodes can be put into a MaxRack, which can also be seen in the WSJ article. The MAX3 boards can be connected via a custom MaxRing network, which provides a bandwidth of 8 GB/s.
Computer simulation made easy -- LibGeoDecomp
/facepalm
Yet another misinformed soul parroting whatever it is he heard or read. Overpopulation is not the real problem, it's the fact that we're living in an outdated society with outdated structures (poor space management). Really, a lot of things in our society are inefficient and there's a lot of waste. We produce so much waste it makes me physically ill to think about, especially when you consider the problem could be mostly avoided with better management of our resources.
Inefficiencies like eating meat when insects are a far better at turning grains into protein?
The "Food and Agriculture Organization" (FAO) are making recommendations for the reduction of traditional meat and instead advocate eating insects. Why? To combat the growing demand for protein in world with too many people, which is only growing. This is only one example of one resource limitation... and they are all limited.
I don't know why you think we have a shortage of people or why the existing people need to reduce their quality of life to accommodate them.
And since you like George so much, here is a quote from your hero asshole: "It's irresponsible to have more than one child. Have one. Have one child, replacement value for yourself, that's all." - George Carlin
http://www.guardian.co.uk/environment/2010/aug/01/insects-food-emissions