Slashdot Mirror
LinuxBIOS, BProc-Based Supercomputer For LANL
An anonymous reader writes "LANL will be receiving a 1024 node (2048 processor) LinuxBIOS/BProc based supercomputer late this year. The story is at this location. This system is unique in Linux cluster terms due to no disks on compute nodes, using LinuxBIOS and Beoboot to accomplish booting, and BProc for job startup and management. It is officially known as the Science Appliance, but is affectionately known as Pink to the team that is building much of it."
I could serve up a shitload of porn with this. Seriously. No, really. A whole lot of porn.
Anybody know if this is a reference to Pink Floyd, if so then I appreciate this team all the more :)
Pardon my ignorance, but would this be considered a Beowulf cluster? I mean everyone on /. talks about them so is this it, finally? A real live, Beowulf cluster? If so, imagine a beowulf cluster of beowulf clusters.
Or, a beowulf cluster of beowulf clusters of beowulf clusters. The possibilities are infinite (literally.)
Think nothing is impossible? Try slamming a revolving door.
It is a good thing it doesnt run windows, because every year they would have Science Appliance Compliance issues, with updates and $$$
But if you'd replace the expensive high-performance interconnect with a cheap ethernet, then it would be a Beowulf cluster.
Which one is pink?
Just like the Iraqis and the Chinese, I do all my nuclear weapons testing on my Playstation 2 Supercomputer!
And how many FPS will they get in Quake?
Let's just hope they do something good with this. I'm tired of reading about how supercomputers are used for military war simulations.
Does anybody know other applications that supercomputers are being used for. I know some do weather predictions.
I'm sure a diskless cluster has been done before. Maybe not with the linuxbios, sure, but who cares how you accomplish the remote booting?
as it is an OS by Apple and IBM (well gone but still)
Diskless X clients have been attractive due to the lack of remote configurations and disk/data failures.
Clusters suck up a lot of electricity because of the hardware they support.
I'm sure the next step involves skipping the extra motherboard components (IDE, USB, AGP, etc.) and making the CPU/Memory mount to a TCP/IP Switch backplane. Better yet, drive the thing with low power CPU's so it won't sound like a helicopter prior to take off/reqire a new nuclear plant to power it up/create a new market for Frigidair.
I wonder why LinuxBIOS hasn't taken off. I've debated ordering one of their "kits." It seems to me the 3 second boot time of LinuxBIOS should be a selling point for some obscure Linux vendor, but no one really offers it yet.
I really imagine a machine with an 8MB EEPROM/ROM that can be updated as needed, but provides a boot environment and login screen - while spinning the disks in the background. This would make an excellent product.
Why hasn't anyone done this yet?
Curious
A former client who worked at a Cancer Center used a cluster to simulate radiation treatments.
When Linux runs on the smallest devices all the way up to boxen like this 10 tera-FLOPs beast and this SGI supercomputer that just set a memory bandwidth speed record with 120GB per second (faster than a Sun SunFire 15K, Cray C90, IBM p690, etc) on a single system image? Scale it any which way you like ;-)
Rock on with your bad self.
This sounds like some kind of dual-processor rackmount type solution. Why not go all the way and use something like compactPCI? You can fit 21 cPCI blades into 8U of rackspace.
A standard blade could have up to a couple gigs of ram, a powerpc or p3/p4 cpu, 100BT or 1000BT ethernet, etc, etc.
You boot the things using bootp/tftpboot and then run linux off a ramdisk.
We're using cPCI at work to run VoIP softwitches. Currently we're at over a million calls an hour on a wimpy 450MHz processor.
I don't envy the developers... After every revision of LinuxBIOS, they get to reflash 1024 motherboards, which could take a while...
I'm enormously massive and powerful, pink, and made of penguins. What am I?
enough to fill about 17 football fields.
And I thought the new 48 node cluster at work will soon be able put out some flops... outclassed, outgunned and outperformed. I used to get excited about hearing of other's beowulfs. Now I am only jealous. :)
BTW, if you see a post that says 'Damn...' and nothing else, thats cuz this damn keyboard has this enter key that gets in the way.
You'll have to do better than that...
You guys get more and more creative by the day, don't you?
Why Bother! www.top500.org shows Power4 faster than any linux offering, intel or not.
in fact for many years straight macintosh related processors DOMINATE the top 500 supercomputer cluster sites of the world.
Check out the top500 list yourself and search for linux, or whatecer you please . YOu will see other unix osses predominate, and that PowerPC realted computers also achieve the fastest speeds.
True, two low cost p3s with three gige cards each, all bridged might seem ok, but if you want your work done in 1 hour instead of one month you cannot yet use linux or these non-RISC systems.... that is unless the us gov is spending the money and has no cost constraints such as LLNL
Why does the /. post say 2048 processors yet the article says it has 1024 processors?
http://saveie6.com/
For those of you not familiar with the "Weekly World News" publication, it is a tabloid you'll find at most american supermarkets which will feature highly elevating stories such as "mom gives birth to four-headed quintuplets". The above story is just another one of their fictions. This is what tabloids do. They sell fiction. They appeal to the mentally ill-challenged, gulible minus habens.
Yahoo features those articles in their TV/Gossip/Entertainment section. So you don't have to spend money at the supermarket. Go yahoo.
Nothing to see here, move along.
Extraordinary Vacations. Exceptional Prices
This one, brpoc, is different it is completely stable. You never get NFS wedges. Jobs launch in flash. Plus if you do reboot the whole thing is back up in seconds (literally).
Bproc is an incredibly light weight job submission system. It is so light weight and fast that it changes how you think about sumbitting jobs. Rather than designing long duration jobs and tossing them on queue, you can just run tiny short jobs if you want with no loss to overhead. It makes you re-think the whole idea of batch processing.
when the jobs run they appear in the process list of the master node. That is if you run "top" or "ps" the jobs are listed right there. In fact from the users point of view the whole system looks like just one big computer.
Some drink at the fountain of knowledge. Others just gargle.
I will personally track down and slaughter the first person to mention a popular clustering architecture, and how one might imagine it...
Code, Hardware, stuff like that.
It will be interesting to see if anyone builds a massive HyperTransport switch (probably a local switch for a blade frame with 1000bt between blade frames). The opteron looks like it could run without much in the way of chipset support (build in memory controller), and skipping all the unnecessary I/O would be pretty simple.
Of course, dumping all the heat would be an issue...
Cluster Overview:
* 2050 Intel 2.4GHz Xeon processors
Now when people complain about the United States government being responsible for global warming they will have some good hard facts to use.
I've been waiting for this. A Whole-Linux clusterbox ready to add cpus as needed to make some good science. I've done aero, astro, chem, the "unnamed", and bio, but now it's all about the perfect ocean boat. Where can I get the distro?
All of the software that Pink will be based on is GPL'd and open source. This includes BProc, LinuxBIOS, Supermon, beoboot, Linux (of course), the BProc Job Scheduler, V9FS, etc... All of these projects are available from the cluster research lab web page and most if not all (not sure about V9FS currently) are available via Sourceforge for users interested in the most recent, bleeding edge CVS images.
dude thats the dumbest thing ive ever hear even if the LANL was getting a 2083 processor the BA has way to much strain that was put on it inadvertently through the software nodes this will create a backlash the likes of which you have never seen on network TV. I should know. I was a child star, when i was younger and trhe cash was great. I wouldn't change it for the world anyhoo in closing i would like to say that i agree
"The Science Appliance" as it is dubbed will use dual processor AMD based nodes.
Scary part is that this will be one of the top 5 supercomputers in the world.
Scary because you could buy all the hardware off the shelf for about half a million dollars.
On a lighter note:
"The Linux NetworX cluster will be used solely for unclassified computing, including testing on ASCI-relevant unclassified applications."
I think they mean text mode quake.
I guess they got tired of "Global Thermo-Nuclear War"
If voting were effective, it would be illegal by now.
It is called Monte Carlo simulation of the radiation transport. Basically one tracks propagation of the high-energy particles as they progress trough the matter (human tissue). e.g. if you simulate a brachytherapy source (radioactive "seed" implanted in tissue), the code "creates" photon with energy characteristic for a given isotope. The direction vector is chosen by a random number generator (RNG). The RNG also decides at what point along the photon's trajectory an interaction with surrounding matter should occur according to the physical probabilities.
After the interaction, there is a bunch of scattered particles (photons, electrons and positrons) and the code continues to track them until the energy of the n-th generation particle drops below certain energy (10 keV for electrons) at which moment particle's energy is deposited as a dose.
The object of the simulation is to obtain precise dose distribution. In order to achieve good statistics one needs to run millions and millions of histories.
Beowulf clusters are ideal for this job because histories are independent and there is no need
for the fast shared memory and a fancy interprocess communication.
I had a pleasure to assemble 24-node 1.6 GHz AMD cluster and we achieved sub-minute simulation times, a result that makes this technique suitable for everyday clinical practice.
Yes, various practices that fall under the moniker of 'technical trading' have been around a long time. By some counts, since right after the 1930s. By others, before then. Software assisted trading is in some ways new, but in the past the same result happened, aleit slower, through agents.
To give you a point...
Sure, ill tuned risk management systems fuck up. Plus, they're extremely important to the world economy. That's why Greenspan bailed out a certain well known hedge fund very recently.
I was not asserting that "much of the research in this area died suddenly". On the contrary, research in risk management is hitting a rather furious pace. Please re-read what I wrote, and this time pay attention.
The volume of trades taking place without human interention causes huge swings at the moment. We're seeing this now, and have been for a couple of years.
Bonus points if you come up with a theory why seeking short-term gains are going to cause exactly the "double dip" so many cheerleaders at Bussinessweek and Fortune are trying to recant.(
If you get bored, you could actually respond to what I was asserting, which was that a commercially viable trading system would rapidly stabilize any advantage it had, because it would spread to everyone who had a serious interest in tracking new developments in this area. That's what's still oddly on topic for the parent post.
Thank you, drive though.
-j
I forget what 8 was for.
"IABCOT!"
TIA,
AC, YABA, IG.
I think that mauve has the most RAM.
Oil companies like to have serious computer power too for prospecting and resevoir modelling.
In organic chemistry, you can do some serious molecular simulations ranging from pharmaceuticals through to the actions of enzymes and catalysts.
The fluidics side can even extend through to air-flow modeling (aircraft to cars) and combustion.
A supercomputer is a single system image. Some people call large clusters "supercomputers," but technically they're wrong.
Says who?
Once upon a time 'supercomputer' meant 'any computer made by Seymour Cray', and this was reasonable, because he (probably) invented the concept. Then there was the mid-80's loose but widely-accepted definition 'any computing system that can do more than 200 MIPS'. Then MIPS went out of fashion and processors got faster and it was 'anything that does more than a GigaFlop'. Or there's the US Department of Commerce definition which was 'any computing system that does more than 195 Mtops (Million theoretical operations per second)' during the 80's, which then got changed to 1500 Mtops and is probably something different now.
Note that most Linux cluster systems would meet the requirements of most of these - indeed, most single-CPU computers today would meet most of these requirements, which is how Apple manages to get away with calling the G4 a 'supercomputer'.
Really, these days 'supercomputer' means absolutely anything you want it to be, although if I had to define it, I think probably the fairest definition would be 'anything that can run the LINPACK benchmark suite and get on the Top500 list'.
Nice try at creative redefinition though.
Imagine the thermal output of 2050 Athlons....
But doesn't it say " build, integrate and deliver a 1,024-processor Linux cluster " rather than 2048 cpu cluster.
What else would you need a super computer of this size for? Is there really an application that requires it, outside of real time virtual reality?
Bet you couldn't make up the dumb team names and stats with that budget though. Or get so many screens to display dumb meaningless graphs (more or less)... and let's not even get started on the Seti Forums and fansites ;)
Conversion Rate Optimisation French / English consultant
http://jailbait.sourceforge.net/
Company went under, API Networks. We had Linux booting out of 2MB of flash in less than 10 secs. The firmware that controlled the deal was less than 400K and allowed for complete disaster recovery, ramdisks, booting successive Linux kernels, and other firmware. Alas it wasnt finished in time... Alternatives still exist to this day besides LinuxBIOS. Any openfirmware vendor like codegen offers the capability of booting Linux from flash. So does Redhat's redboot bootstrap loader which is part of their eCos microkernel.
Last time I checked, the LB project was hacking off flash chips to bootstrap these things when they've failed. Basically no recovery procedures. I would ask anyone considering this option that they would consider the question "Am I better off using BOOTP?" . 8MB of flash is unreasonable and it speaks volumes about the product. Scyld could do it less than 2MB, what does that tell you??? Before anyone asks "why don't you finish it", Unfortunately Linux is no longer my job; Just a hobby.
Peter
www.alphalinux.org
I built a 116 node cluster at my last job, and it was a diskless node setup that booted from the network and used NFS-root. Although linuxBIOS would have been cool it's not needed if a machine has SRM {yes as in alpha} or PXE network cards.
If the only reason they went to LinuxBIOS was to boot off of the network maybe they should examine the date on the first copy of the HOWTO-NFS-Root and realise it has been going on for a LONG TIME!
any working links that can clue one such as I on what exactly this is. I can guess by context but then I am stuck with this feeling of silliness at the result... you said "Linux" and "BIOS" right?
I mean, you haven't seen him post any poorly researched diatribes lately (hell in the last eight months...) now have you?!!
Look at the first result on a google search for LinuxBIOS.