Slashdot Mirror
Beowulf In Business
"Cnet has a story on how businesses are starting to use Beowulf for those heavy duty tasks," writes NIB. The story notes, "Beowulf wouldn't be good for a program that executes large numbers of transactions, such as an airline
reservation system. It would, however, work for business tasks such as deciding how to design an assembly line
or which mix of currencies to buy, said International Data Corporation analyst Dan Kusnetzky."
One big one is simulations for financial calculations. One such is roughly this: the price of some class of security is sensitive to interest rates. So you want to see what happens if, at several time steps, the interest rate goes up or down some small amount. Evaluating the different 'paths' of interest rates over time lends itself to parallel processing.
Large volume OLTP has a huge amount of inter-connection at the data level. You have to lock and unlock all the records to maintain ACID properties. Beowulf is a shared-nothing approach and doesn't have facilities for sharing all the disks with appropriate concurrency control.
The largest airline systems all use IBM's Transaction Processing Facility (TPF), which is a specialized real-time OS for mainframes. TPF shares disks amongst all processors in the cluster and pushes the locking down to the individual disk controllers (specialized microcode). Where I work, we get about 200,000 physical I/Os per second to the disk farm, using TPF on eight mainframes and a several TB of disk.
Still, there's nothing about large-volume OLTP that Linux couldn't do, it's just a matter of programming. I, for one, would like to see it happen.
I think the problem with the transaction systems is that they top out on a different bottleneck. CPU isn't the major gating factor. Multithreadde applications will take great advantage of this type of system. One application that I worked on in a previous life was a creditcard limit verification system for a major player. They had a 1 second transaction turn around specification. We ended up setting it up with discrete machines with a failure rollover mechanism involved. Much of the coordination we had to design would have been far easier in a coupled system like Beowulf.
There's a movement on to put together a large Beowulf cluster for the Boston Geekfest in October. One of the things we're trying to come up with is a good demo that actually shows something to the crowd. We've had ideas from the realtime rendering of POV scenes to decryption (yeah, right, watch it hum for 20 hours and then spit out the true key) but haven't come up with a "killer demo app". Email if you have any ideas.
I just read the article. As a manufacturer of "turn-key" Beowulf systems, here was my reply to the author:
Stephen,
I just read your story about Beowulf systems. While the story was well written and informative, there are some points that you have missed.
1) The definition of a Beowulf requires and "open source" OS (See "How to Build a Beowulf" by Sterling, Becker, Salmon, Savarese) Therefore, systems built from True 64 are NOT Beowulf systems.
2) You missed my company, Paralogic Inc. We sell turnkey Beowulf systems. In fact rather than "several" as reported by IBM, we have several dozens of installed production systems at companies like Lucent, Amerada Hess, Conoco, Procter and Gamble, government sites like NASA, NRL, and the Air Force, and many Universities. (see www.xtreme-machines.com)
3) There is a rather huge barrier to entry because of the technical nature of these machines. As far as I know, we are the only company who will offer support for Beowulf clusters. Without support the market can never enter the mainstream.
4) There have been quite a few other people who contributed quite a lot of effort to the Beowulf technology other than IBM and VA Linux. Although all contributions are welcome, these guys are a little late to the party and we hope they stay.
Sincerely
Douglas Eadline, Ph.D.
President
Paralogic, Inc.
PEAK PARALLEL PERFORMANCE