Slashdot Mirror
IBM Creates New Fastest Beowulf Cluster
shawnb writes "It seems that IBM has created the world's fastest Linux cluster built from lots of small servers (64 IBM Netfinity 256 servers). The Netfinity servers are linked together using "special clustering software and high-speed networking hardware, which causes the separate units to act as one computer, delivering a processing speed of 375 gigaflops, or 375 billion operations per second." They also go on to say that this is the fastest Linux supercomputer, "it will only rank 24th on the list of the top 500 fastest supercomputers. " "
The cool thing about this is not its ranking on the top 500, but that the machine can be built so cheaply and *relatively* easily. Any school, country, or organization that wants supercomputer power can now have it.
I am not certain what problems this machine cannot solve (some types of problems may not be solvable with a Beowulf architecture), but so many are in its realm that certain export restrictions start looking pretty useless.
Jesus christ, do you live here? WERE A FUCKING UNION. THE UNITED KINGDOM! ENGLAND AND WALES ARE TWO OF THE COUNTRIES IN THE UNION! Therefore you dont need a PASSPORT. Ever wondered why the British flag is called a UNION JACK? England and Wales are countries in their own rights. They are not states or provinces.
Oh it's an american forum is it? Does that mean I have to become very FAT, very IGNORANT, wear a BASEBALL CAP, drink PISS WEAK BEER, go on about how EVERYTHING IS BIGGER BACK HOME, have the DRESS SENSE OF A DEAD WEASEL, wear a BUMBAG everywhere and watch WWF WRESTLING and think its real. Only one word to describe that, PONY.
Now you are asking for a slap :)
Is it entirely realistic to compare clusters and supercomputers like this? I mean, while both are theoretically quite similar (multiple processors, and most supercomputers apparently have separate memory for each processor), the question arises: how effectively can these separate units co-operate? With a cluster you're using more or less run-of-the-mill networking equipment, while your average Cray has lots of (very expensive) high-speed buses connecting these components. The result: stuff that can be split into separate subtasks (3D rendering á la Toy Story springs to mind) can run lots faster than processing where each operation relies heavily on the results of previous calculations (meteorological and some other physical simulations at least - most meteorologists I've talked to find talk of replacing supercomputers with clusters highly amusing). With what sort of ranking has this 24th position been calculated?
Moderators, please moderate the parent posting up. It is at the very least Funny, and may even be Insightful/Informative.
thank you
Altough "flaimbait" may also have been appropriate.
Uhhhh...
And how many of the top 500 (hell, 5000) are running an OS from M$?
Duh-oh!
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
http://www.zdnet.c om/intweek/stories/news/0,4164,2470425,00.html
(Hmm, wonder how many products made in the early 90's have OO all over their literature?)
straight from the horses mouth.
Dear IBM, if you are reading this, want to know how to make IBM look really in touch? Have IBMers posting to Slashdot. Intelligent stuff, and maybe a few trolls, some anti-Microsoft flamebait. When Slashdot posts some story about ripping the cover off a box and tweaking the PROMs, you could write in about how you did it back in the day with a Burroughs for $1M. It's nothing new; dig out some old JCL and I'll bet you have some that already says /.
Where can this software be obtained?? Does anyone have any good experiences to share? I'm looking into clusting some Linux servers, but have found lackluster information on the topic
dude, I think it is intended to be comical. Look at the evidence. He mentions Beavis and Butthead, he claims that Wales is in England, Some of those languages he names don't look real to me. The guy even admits he uses Caldera at home. Why is it that humor on slashdot goes unnoticied unless it carries hundreds of :-)s ? Are we really that irony-deficient ? Are we really unable to laugh at ourselves ? Or maybe the Slashdot readership really is collectively stupid.
Relax. Someone criticises Linux in an obvious troll. BFD. Get over it. Move on. These are not the trolls you are looking for.
more proof that only linux is a scalable, enterprise os and is controlling the datacenter market!
ibm's beowulf shows that linux cannot be beat and that intel hardware is light years ahead of anything sun/sgi are producing.
the revolution is here.
I don't see the word Beowulf mentioned in the press release. Why do the linux zealots have to call every cluster a BEOWULF cluster?
You need to chill out dude, this post is NOT FOR REAL, the guy is JOKING. Its funny, LAUGH. You remind me of a sad Amiga advocate who carries on singing the praises of his beloved machine long after it has died in the marketplace :-)
Price could become an issue with NT - licences for all the machines (despite being small compared to h/w) adds up - plus client access licences.. with any free O/S you don't have to worry whether you're clean on the license front.
Don't forget Mandrake Linux 7 has full support for Welsh/Cymraeg!
Congratulations, you've the funniest post I've seen on Slashdot for eons.
Why use Linux instead of pc operating systems like nt? One word: performance. The folks at NASA who build supercomputing clusters report that they get double the performance on a cluster just by moving from nt to Linux. Given that, why would anyone pay the money for a proprietary pc operating system from Redmond when Unix/Linux is freely available?
Check news.cnet.com for an article on the same thing that makes more sense. It says there are 256 machines for a total of 512 processors in the linux beowulf cluster.
m l?tag=st.ne.fd.lthd.1003-200-1580530
http://news.cnet.com/news/0-1003-200-1580530.ht
Can anybody moderate that one up? It's a link to a better article than the one mentioned in the header.
wow, imagine a beowulf cluster of these first posts... er, maybe not.
Last time I looked Wales was part of England. I don't remember seeing any passport controls from the UK. Same goes for Scotland and Northern Ireland.
I'm a bit confused here. First you say that this 1000 x Pentium II 350 system should be faster because there are more processors, even though they are slower, and then you admit that Intel processors don't play well together? Try using an argument that is not self-contradictory (let alone one that uses the right info on the system you're trying to run down).
What is this crap ? It does not appear to be English. Were you aware that slashdot is an AMERICAN forum, therefore the official language is ENGLISH ? Also, why do you bother answering this moron, he is obviously trying to get a reaction. The best thing to do is ignore them, they will go away eventually, leaving Slashdot for the true linux fan.
Jealousy is a very destructive force.Your being threatened by open source ware would be comical if it wasn't so pathetically self serving.Can you even admit to yourself that you are on the chain of large corps like Sun?People don't like to be around self serving greedy folks.For good reason!I prefer to look at the brighter better side of people that want to cooperate and help each other.Cynics never develop great ideas,but are the first to shoot at those who develop them.For your own good seek spiritual help.More crabs aren't needed on this earth.
If you have a Beowulf class system you're probably going to use MPI or PVM to parallerize your applications, altough there are other options. Beowulf isn't a particular software parallerization method; it was the name of the original 16 node cluster at CESDIS composed of commodity hardware running Linux. Clusters similar to that one are called Beowulf class systems, or just Beowulf systems for short (and rightly so, because they are running lots of code that came out of the Beowulf project, for example, most of the Linux network drivers)
Get a clue... Most beowulf systems use PVM or MPI for message passing.
Hi - i'm tangentially involved with this project, and it's DEFINITELY NOT BEOWULF. The name does indeed come from the University of New Mexico's mascot, the lobo. I have passed on the fact that this discussion is happening to the people in charge there.
I think it has more to do with the "Imagine a beowulf cluster of these" trolls on ever thread. I agreee, redundant, but in this case a +1 funny should also be appropriate seeing how it is on topic.
http://www.linuxplanet.com/linuxplanet/reports/162 6/1/
I doubt those 64 ibm machines are as fast as the 3000 Google pentiums.
Nice one boyo ;)
From a semiotic perspective, the Linux/open source narrative constructs appear unsound. Indeed, they are like some kind of post-cold-war echo of the death-throes of Communism. I have spent a long time analysing the whole phenomenon using holistic, semiotic, and macro-psychological techniques, and have come to some very worrying conclusions. For obvious reasons, I will have to remain anonymous, but should you really need to find out who I am, it will not be too difficult, given that this forum is powered by the notoriously insecure Linux OS, and its bastard stepchild, Perl.
It is obvious even to a mental retard, or the average 15-year old Slashdot linux zealot, that the problems with Linux and Open Source are the same as the problems with any other Communistic/Leftwing ideology, and they will fail for the same reasons that the USSR collapsed, namely Human Nature. It is one of the fundamental laws of human nature to be self-interested. Richard Dawkins explains this very well in his book "The Selfish Gene". Linux and Open Source are genetically doomed to failure, because quite simply, they do not put food on the table. It is a simple equation Humans need to eat to live. Linux/Open Source does not feed them. Microsoft does. Who would you rather work for ? A bunch of hippies in a commune paying you in the invisible and worthless currency of "Kudos", or a go-ahead money making corporation like Microsoft with deep pockets, and highly skilled workers.
Now before all the high school and college attending long-haired left wing Crypto-anarchist Linux zealots on this forum get all hot and bothered about this, and start accusing me of trolling (whatever that is), I suggest you pause for a few seconds and think before you post.
Chances are, I have been around this industry for many many more years than you have, and have probably seen more trendy/fashionable technologies and processes come and go than you have had hot dinners. If you stopped ranting and raving for a few minutes, you may learn something from my years and years of experience and hard-won wisdom, I will try to impart some key points for your edification.
It is not always the best technologies that win through.
For instance, a project I was lead marketeer for way back in the early 90's wanted to use C++ as their programming language, because they claimed the C compiler we used at the time was buggy and unreliable. However, as a marketer, I knew that the customer did not care about this, as long as the product could be described as "object oriented", I also knew that re-writing our code in a new very over complex language such as C++ could put weeks on our time to market. I knew that the developers just wanted to play with some new language currently perceived as "cool". So I decided that only a small subsection would be written in the ludicrously complex C++, while the core product remained in C. In this way I pragmatically kept some key developers on board, learning new skills, while minimising the risks from using an insanely complicated language like C++. I also made sure our product mentioned "Object Oriented Technology" many many times in the marketing literature. You moronic Linux zealots should demonstrate some real world expertise like mine before you start sounding off on your high horses.
The right tools for the job are essential.
It's all very well going on and on about how great Linux is, but you are forgetting one thing: Not EVERYONE has a PC. Some of my clients have legacy networks of Sun hardware. Since Linux is an X86 based product, what are my clients supposed to do, throw out perfectly good kit just because they want to run the trendy new OS ? Quite apart from the fact that Solaris is far more solid and reliable than Linux, and will continue to be, since Sun control all the changes made to the source tree. Unlike the anarchists free-for-all that characterises Linux developments. And then there is the issue of text editors. How can a cryptic editor like VI or Emacs possibly compete with something of the professional quality of Notepad.exe, which I am using to write this information-packed response ?
Now some may think I have a cynical view of the whole shareware movement, but this is not so. I actually use Linux (Caldera OpenDesktop) at home, because that is an OS which we are studying. But in terms of "desktop acceptance" the shareware Linux has a long, long way to go.
Another problem with open source is its intensely annoying and consumer unfriendly "community spirit". Is it just me or do others immediately think of the hippy teacher "Van Driessen" from MTV's "Beavis & Butthead" whenever they see yet another bleeding-heart liberal apologist perform some lame act of Linux Advocacy ? It makes me want to puke green vomit all over my $2000 Giorgio Armani Suit. When will they realise, America is based on the absolute FREEDOM of every individual. Our constitution specifically gaurantees us this. How can I pursue my right to happiness if I have some unwritten obligation to pay something back to the "community" just because I installed a shareware operating system ? What the Linux advocates seem to forget is that most Americans are (like me) God-fearing, libertarian Christians, for whom the whole relativistic Linux philosophy is an anathema. Another reason for Linux's impending doom.
Linux must be banned
The Europeans have the right idea. Certain parts of England, e.g Wales have BANNED LINUX COMPLETELY, due to its incomplete support of the Welsh language. (In Europe, no software product may be shipped, unless it will display its text in all 22 "official" languages of the EU, including Icelandic, Catalan, Ebthaar (Latvian dialect), Gnrdhtplag, and Welsh). I would not normally advocate copying Europe, as it tends to be a breeding ground for destructive ideas (communism, nazism, The common market etc etc) But in this I make an exeption.
Linux contains obscenity
Did you know the Linux source code contains profanity ? How can this possibly be justified ? How can I trust a multi-million dollar enterprise to a kernal whose comments use words like "sucks" and "bites" (and much, much worse) ? Also, the spelling in some of the comments is atrotious, which again asks questions about intelligence of many of the so-called hacker community.
The issue of provenance.
When I buy a Microsoft product, It has a hologram. This proves I have a genuine Microsoft certified product, with a serial number and everything that that entails. When I buy a Linux product, there is no hologram, and therefore no way of knowing where it came from. For all I know some "warez-d00d" could have pirated the distribution. Would you trust your e-commerce enterprise to a pirated version of Linux ? I thought not.
No-one likes a sore loser
This seems to be a typical characteristic of the average whining Slashdot reading Linux lunatic. Nowhere is this typified better than by studying the posturing of the Linux zealots on the DOJ case against Microsoft. It is almost as if they want one of America's most successful corporations to fail. Again, the overtones of Communism and anti-corporatistic tendancies are clearly in evidence.
Security of Linux (or lack of it).
How can American corporations (which are without doubt the best in the world) possibly trust their "information crown jewels" to an OS written by a Non-US Citizen ? Now I have no hard evidence that Linus Torvaldes is a Communist hell-bent on the destruction of the USA, the Great Satan by taking down its essential e-commerce infrastructure, BUT, can you PROVE to me that this is not the case ? I thought not. Can you imagine what would happen if for example a Option trading program running on a Linux system went crazy and started taking contrary positions in the market place ? Can you say Black Wednesday ? Can you say "Global Financial Meltdown" ? I thought you could. The depression of 1929 would look like a Sunday school picnic compared to the bloodbath that would ensue if Linux was ever to be unleashed on the trading floors of America. Linux should stay where it belongs, in the bedrooms of the hobbyists and sysadmin wannabes, and on the desktop of effeminate European hackers with body piercings, tattoos and black clothing. Please keep it out of corporate America, we don't want to be infected by this dangerous virus.
And now onto another issue. I have noticed that Linux, unlike the superior Windows product, has no control panel settings to enable Internet censorship. Which makes me ask the important question: How can we protect the weaker members of society, and our children, from the daily tide of filth and degradation that spews forth from the internet like the outpourings of some infernal sewer right onto our desktops in an unrelenting stream of obscenity and liberalism ? Linux actively promotes moral relativism, and I am aware of at least one case of a normal morally strong American, who became gradually more and more liberal after exposure to the dangerous Linux kernel. This person eventually grew a pony tail, became bisexual, and started voting Democrat. Are these the values we want to promote in our once-great country ? I though not.
What can I do ?
If you have read this far, you must be wondering what you can do to help us fight this menacing un-American "operating system". The way forward is clear. If you support the values of the USA, namely the Constitution, Christianity, the right to bear arms, and the right to pursue happiness, you have only one option. Boycott the Linux kernal. Refuse to do e-business with any corporation which wants to store your credit card details on a potentially insecure Linux machine. Educate yourself - find out why Linux can never be secure because it runs Sendmail - the official worlds most insecure program. Consider taking direct action against Linux wherever it rears its ugly, text-based configuration files and command-line interface. Just say no to Linux wherever you can. Vote with your wallets. Remember Linux is one step along the road to the Communistic New World Order as prophesied by Nostradamus, and promoted by the Masonic-backed KKKLinton administration. If we don't fight this now, how long will it be before our families are rounded up into forced labour camps only to be forced at gunpoint to write yet another Gnome based front end to a cdplayer application, or to create a new theme for the KDE desktop with no financial rewards whatsoever ? Have we learned nothing from the events at Waco ?
A lot of what I say might sound like I'm stirring up trouble, but it's the truth. To misquote that famous libertarian, Malcolm X. "As long as those explosive ingredients remain, you have the potential for explosion on your hands". Linux is the Semtex of the liberal revolution. Don't be fooled.
thank you
Try http://www.netlib.org/benchmark/top500.html P.S. The Cray T3E1200 at No14 now has 916 Processors, so should now be at number nine.
eh?! I found this wacky arse website with "anti-linux" pics..http://www.polyester.net..Check this out..
Click to en-large the anti-linux movement.. erm. ok.. They got something called "geek satire" also..
Oxygen with Linux? Science fiction could become science fact
Rare meteorite promises glimpse into creation and the source code of Windows2000
Hackers suspected as Microsoft 747 explodes in mid-air due to blue screen of death.
Cisco integrates routing, switching and underwear drawer technology.
IBM plans to change the shampoo world.
Microsoft Unveils Plans for X-Brush tooth brushes.
In case you didn't read the header: How does a first post get marked as redundant?
According to this news.com article it's not 64 Netfinity 256's (does there even exist a Netfinity 256 model?), but 256 2 CPU Netfinity servers. (Maybe the new Netfinity 4000's... ?)
"it will only rank 24th on the list of the top 500 fastest supercomputers. " - don't they remember the first beowulf clusters that ranked at 250 or lower? It's like "my car only drives at mach 3" ;-)
I wonder if we can run Linux on this. Oh, wait we can.
Then wonder if we can make a Beowolf of these things! Oh wait, that's what they are doing with em...
Ummm, ahhh, ummmm... Grits anyone?
That's seems pretty amazing to me, that one can 'throw' 64 readily available systems together to create the 24th most powerful system in the world.
Thank you. Drive through. (:wq)
From the article they say that clusters max out at 64 machines, limiting their size - but also it's claimed that the cluster acts like a single machine, so my question is, why can't you cluster the clusters to use 4096 machines. Is it simply a case of (lack of) bandwidth linking the machines together?
It's probably a software limitation, but probably not a bad one. Large clusters get unwieldy quickly, and network latency and bandwidth is the bane of any parallel programmers existence. Communication between the nodes of a cluster is several orders of magnitude slower than referencing internal memory, and any no real parallel program has autonomous nodes. It's no coincidence that Donald Becker, a major contributor to Beowulf on Linux, also wrote huge chunks of the kernel networking, tons of network card drivers, and a network channel bonding implementation for Linux.
My point is that you can create a cluster with thousands of nodes, but doing so is an administrative and technological nightmare. For most parallel problems, it's much easier (and generally more efficient) to have a smaller number of more powerful nodes.
--
odds of being killed by lighning and
Odds of being killed by lightning and winning the lottery in the same day: 1 in 2^55
Argh!!! I meant to say under three quarters of a million... It's not even early morning, so I don't have that excuse...
--
This pure processing power here but the biggest problem is the fact when you have lots of data hardrives are your biggest enemy. We need faster hardrives. When I see the harddrives speeds reach the the level of processor speeds then we will really be cooking
http://theotherside.com/dvd/
I wonder if there are _any_ Microsoft clusters in that top 500 and if on some miraculous chance there are where they stand... Hmmm, 64 of each running side by side - a new benchmark for Mindcraft to screw up? :)
Mike
No, it wouldn't, unless you put all of your program into the kernel. You don't want to pay the price of going into and out of the kernel.
Ideally you'll implement everything userlevel, including your networking, so you never go into the kernel.
The UP machines might have a bus for each CPU, but the SMP machines' busses are still a lot faster than having to go through Ethernet for inter-CPU communication. Two SMP machines will beat four UP machines any day, at least until we get faster networking (or more importantly, lower network latency).
Since these are 733MHz PIII machines, I suspect they have 64bit PCI as well. That gives you 266MB/s bandwidth on the bus.
If you bothered to reag the RC5 FAQ, you'd see that they've been asked this question enough times for it to have an item in the FAQ all of its own. They say, basically, that the RC5 distributed network is pretty much the same as a Beowulf network in terms of processing efficiency for RC5, so a Beowulf would perform much the same as the same number of machines operating independently on the project.
This isn't true scalability in the way that server manufacturers mean. In that, I mean you can't just take MySQL, Apache, Oracle, or any other application and code it to Linux's API's and then install it on a Beowolf cluster and watch it magically take advantage of all the nodes. This is a completely different beast, whose only commonality with the desktop and server Linux OSes is the fact that there is a common kernel running on each machine.
Microsoft could do the same thing, install the NT kernel on a bunch of machines and run a program on top of it to actually pool machine resources, if they felt like it, but that's not at all their target market. Really, no pure OS vendor is going to approach the super computer market. There's no demand for just OSes... people want machines. Since people are kind enough to develop Beowolf and Linux, vendors can play around with the idea of using Linux clusters.
AFAIK, Mosix can run over any TCP/IP, so ethernet included.
--- Hindsight is 20/20, but walking backwards is not the answer.
Just thought I would mention that a cluster of Linux computers is by definition a Beowulf Cluster.
Not completely, there is also mosix.
--- Hindsight is 20/20, but walking backwards is not the answer.
Thats why I wrote "IMNSHO", and I didn't think long enough about it. You're absolutely right.
My background is variational methods (the theory of it) and I should have been more specific in narrowing the applications down.
numeric applications of all kind for solving partial differential equations
finite elements, particle methods esp. in 3-dimensions (very computing intensive), finite difference (nobody uses that anymore IMNSHO), whatever they use in molecule modelling.
For what is that used:
crash simulation (cars etc.), airplane design, engine design, meteorologie (- very ugly because of 3-d problems) etc, fluid dynamics, nuclear weapons simulation (- somewhat ugly for other reasons).
There's no way there will be ever enough computing power even for very non esoteric applications. For instance imagine doubling the density of weather stations in three dims, this results in 8 times more input-data which causes 64 times more computing (theoretically) and perhaps much slower convergence of your solving algorithms.
I belive you're wrong there. Although beowulf clusters are nice for applications that are computationally intensive but don't require much bandwidth, a lot of problems require a lot of bandwidth. This is where SMP boxes rule since bandwidth between processors on a single motherboard/backplane is a LOT larger than across a network. For these types of problems, the time required is more function of bandwidth rather than cpu so a smp box with fewer processors but more bandwidth between the processors will beat even a large beowulf/clustering solution.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
yup. top500
Satan, oscillate my metallic sonatas.
Of course that's only an educated guess on my part. If I'm incorrect, note that Myrinet is coming out with single unit 64 and 128 way switches later this year. That should help improve the interconnect situation for larger clusters a great deal. Prices will be dropping too, possibly putting Myrinet in reach of groups with smaller budgets.
--Jason
--Jason
Unfortunatly one chip isn't actually going to produce 733MFlops on Linpack. A PIII-500 gets about 200, which is 40% of the TPP. Dunno much about the 733Mhz chips (except the cache runs at full processor speed, but if it's only 512kb it's not going to offer much improvement), but I'll be nice and say it gets 75% of the TPP. Ok, I'm probably being REALLY nice there. That leaves us with ~240GFlops.
Of course, for a press release 375GFlops looks alot better
Jason
How about the fact that your post and the other post that pointed this out have both been marked redundant, which, while not inaccurate, defeats the entire purpose of having a redundant rating.
Or in typical slashdot fashion clustering+linux=beowulf, regardless of the realities of the situation.
Leaving out harddrives, etc. would just slow you down. Extra load on your communications channels is not something you want in a distributed mp machine.
Am I the only one around that doesn't know what kind of a machine a "Netfinity 256" is?
In fact, I took a glance on the IBM web site and I couldn't find any such machine...
Am I blind or simply stupid?
Wouldn't an implementation of PVM or equivalent in the Linux kernel improve the performance of parallel computing greatly? :-)
There is a kernel-httpd in the development branch of the Linux Kernel already - So wouldn't a feature in the kernel to enable parallel computing or clustering be the next big step?
Well, I think the new Netfinity boxes should have some 64/66 slots, as well as some 64/33 slots... the 66MHz would buy you a good bit of performance, since you can transfer the data just that much faster (sometimes you have to wait for the bus, after all...)
"It's tough to be bilingual when you get hit in the head."
the IBM cluster is 256 boxes, so if they are SMP, you have 512 procs
Assuming they are using the Netfinity 8500R (8-way SMP), there could be a total of 2048 processors.
well I am not sure how big a pentabyte is so
I went with 10K Gigabyte which means all you need is 134 of those new 75 Gig hd's
So how about a 140 dual processors alpha systems with 500Mb ram, 75 Gig hd, dual fiberchannel
networking with a cray as the controlling host.
Oh pardon the drewl. ahhh ahah PORK CHOPS!!!!!
;P
Elephant: a mouse built to government specs
I think there is a significant clue that it is a Beowulf or Beowulf derivitive in that the name of the machine is Los Lobos, 'The Wolves'.
Of course it could just be a refence to the UMN Basketball Team The Lobos
Work for Change & GET PAID!
Gigabit should do the trick. I believe that 32-bit, 33 MHz PCI is 132 MB/s. Gigabit is 100 MB/s, so as an off-the-shelf solution it would be pretty close to any proprietary one you could implement. 64-bit 66MHz PCI is another matter.
Gamingmuseum.com: Give your 3D accelerator a rest.
Just thought I would mention that a cluster of Linux computers is by definition a Beowulf Cluster. It was the term used to describe them, which is why M$ came out with thier version of doing the same thing with NT...WolfPack. Don't u just hate those guys?
Byw, if anyone wants a copy of Extreme Linux (the beowulf clustering CD) that was made a few years back, drop me a line at nicksworld@netscape.net
I'll give you a place to download it, or if you send some $$$ I'll send you a CD copy of it.
"Imagination is the only weapon in the war against reality." -Jules de Gautier
well, if you want to be semantic, a beowulf cluster of beowulf clusters is kind of redundant.
Instead of referring to the post itself as redundant, maybe the moderator was referring to the content of the post. Oh heck, I don't know.
One more drink, and I'll move on. --Dave Matthews Band
Check the stats for Team IBM
Dunno if this cluster is being used in the effort, though.
Any sufficiently advanced civilization is indistinguishable from Gods.
I wonder if 64 RS/6000s running LinuxPPC would have a better or worse price/GFLOP ratio?
The Ohio Supercomputer Center has a similar cluster here
Though I don't see anything about its speed in terms of gigaflops
And here's one press release here.
But how fast will it crack RC5? Does the processor underlying all this have that nice shift-n in one clock cycle?
threadeds blog
The Los Lobos supercluster is part of the National Science Foundation's National Computational Sciences Alliance program, which gives scientists remote access to the fast machines needed for scientific research.
Interesting that the Wired article claims that the machine has 64 nodes but that UNM press release says 256. I wonder what else Wired got wrong...
-JeremyH
The latest copy of the paper I have seen is a pre-print, so even if I did know the specific details of the experiments the authors used, I would be uncomfortable going into details.
I do know that they compared the NCSA NT Supercluster with the Linux cluster at Sandia National Laboratory. This is the second "large" cluster within the Alliance and sort of provides a counterbalance to our NT cluster.
You are quite right in that there should be no big difference between operating systems for applications that are largely computation intensive. The big differences would come from applications that are heavily file intensive or communications intensive. Both clusters use a high performance interconnect called Myrinet, but the NT Supercluster uses a messaging layer developed here at University of Illinois called Fast Messages which provides very low latency and high bandwidth. The last I heard, the Sandia Linux cluster used TCP/IP over Myrinet and I do not believe this offers as low of latency as FM.
To be honest, keeping the machines up for long periods of time has never really been a problem. My NT desktop system, on which I do a lot of developement and other "weird" work, can easily have uptimes of 2-3 months, which is about all I really demand out of a desktop OS. (This is in distinct contrast to my wife's home machine which runs Windows 98 and crashes multiple times per day.)
But, your point about remote administration is quite valid. Actually, I'd extend this to being "any remote access" to NT as being the crux of the problem. Launching jobs on a group of remote NT machines and getting the stdout back to the submitting machine has definitely been a pain with this project. With a Unix operating system, you could simply rsh each of the components of your parallel job. With NT, we ended up using a piece of commercial software that provides this functionality, although there are issues with it that make it not as nice of a solution as you would have out-of-the-box with Unix.
Another related idea about NT being oriented to "the user sits at the console" as opposed to using the machine from remote comes when a job abnormally terminates. The NT debugger, Dr. Watson, tends to pop up and wait for the user to press "OK" to continue. If you're not sitting at the physical console, this is a problem. We've had to use some workarounds to deal with this situation.
So, all-in-all, while there have been some studies that suggest NT performs slightly better for some types of scientific applications and some interesting results have been obtained by using NT, there have been many many days when I have really wished we had used Linux instead. Fortunately there are BOTH NT and Linux clusters, so scientists can choose which one they think will work best for their particular application.
Well, this is not a very tech article. It doesn't say how fast is the high speed network.
Anyway, I'm managing a 60 alpha XP1000 cluster connected with a 100Mbit network and I'm using a Cisco Catalyst. If you check their page, you'll see that the Catalyst 8500 can have 128 port switch for 10/100 Mbit. While only 64 for Gigabit connections.
Indeed you should see how much your parallel jobs are communicating with each other and how much traffic they have to support. Sure you don't want to spend a fortune in gigabit connections if you're sending out very few packets.
I'm pretty confident that a 100Mb will do for most of you out there!
Check this page, for some more on the LosLobos!
http://www.unm.edu/~ paaffair/news/news%20releases/Mar21hpcc.html
Indeed they are 256 dual processor 733 MHZ Intel IA-32.
Check the UNM press release at
http://www.unm.edu/~ paaffair/news/news%20releases/Mar21hpcc.html
Ahaha... Well then, they are no doubt fast, and faster than the 70 machine DEC Alpha 533 for sure... But still... Fastest cluster in the world? Faster than 1000 p2 350's? Well, let's do the stastical math on the clustering, and I believe it would say no... (70 x 2) x 733 = 102,620 vs. 1000 x 350 = 350,000. Even though those boxes are dual processor, the speed gains aren't that impressive when you realize that both processors share the same BUS (unlike the coming Dual Athlon boards, where each processor will have it's own Bus to the North Bridge) whereas in this cluster, though there are many more separate processors, they all have their very own Bus. Intel's processors, like the company itself, don't share nice and don't play well with others, so I can't be impressed by the fact that they have 140 processors, limited to half their transaction rates by having to share a BUS for concurrent processes, though I'd take one of the boxes, or the whole cluster if IBM wanted to give it to me.
"Inveniemus Viam Aut Faciemus" 'We will find a way... Or we will make one!' --Hannibal of Carthage
"This low-cost supercomputer will allow researchers and developers access to computational power they previously could not afford," according to this yahoo story .
Jeff
If they're not using dual procs, then I find it hard to believe that it's more powerful than this.
Quote from site: The FSL cluster (called "Jet") currently consists of 276 nodes, organized into three long banks. The nodes are unmodified, off-the-shelf Compaq Alpha systems with 667 MHz processors and 512 MB of memory.
The original number is correct. 375 floating point operations per second is laughable. A pocket calculator can probably do better.
I'm only partly sarcastic when I ask just how one keeps 128 NT computers up all at once? Based on NT's standard downtime ratio I'd figure you'd spend all your time running from machine to machine rebooting them. I mean you can't properly administer an NT box remotely ... someone needs to hit CTRL-ALT-DEL after all. ;-)
http://www.top500.org Lists the top 500 super computers in the world. ASCI Red, ASCI Blue-Pacific SST, ASCI Blue Mountain are the top three. ASCI=Accelerated Strategic Computing Initiative.
24th doesn't sound bad to me, it's a start anyway.
From the article they say that clusters max out at 64 machines, limiting their size - but also it's claimed that the cluster acts like a single machine, so my question is, why can't you cluster the clusters to use 4096 machines. Is it simply a case of (lack of) bandwidth linking the machines together?
Not to mention High Availability Linux, and the Failover protocols being developed as we speak...
That didn't answer my question. A scientist doing research... What kind of research?
------- What exactly is real?
Is there a list of these computers to see what's ranked 24?
------- What exactly is real?
http://www.netlib.org/benchmark/top500/top500.list .html I wonder how many 486SX's it'd take to make that list. HerrGlock
Cav Pilot's Reference Page
UNIX - Not just for Vestal Virgins anymore
hmmmm, A nation wide network of computers made to connect researchers toghther... Take away the millitary, and they used to call this ARPANET. Ok, maybe not the supercomputer thing, but wasn't NCAC on ARPANET? Then again, I could have all of my facts wrong.
Since all of the cliche's are taken, I submit this:A computer that fast just gets me thinking. There is no mention of hard drive space (It could obvously be over 1 TB (Terabye) with that many machines,) We'll have to go with processing power. How many simultanious streams of mp3s can it play?
There are exactly 42,935,718 letter sized sheets in a square mile.
Sheesh, beat me to it! :-)
...gee, but can it run Linux?
There is much cruelty in the universe, John.
Yeah, we seem to have the tour map.
Just in case you didn't get what I was implying that friend was me and I just didn't want to look like an idiot. By the way thanks for the link.
Well isn't IBM a wealth of cool new things. Sounds like it will be a cool computer(s?). Also if you want to see a computer that can go to 1 teraflop, click here. Ehh ehh, sorry the computer is making me horny. Also I know a guy who knows a guy that wants to know what a Beowulf cluster is. I can't explain it very well so could you tell me what to say. (that sounds authentic, right?)
Oh well, Have a nice day all,
Andrew
Sounds Like Microsoft was feeing left out of the bashing other OS (Particulary Microsoft).... Oh well..sucks to be them... -Mike
"Gravity cannot be held accountable for people falling in love." -Einstein
Does this mean it can run Xgalaga ?
--- Bouh !!! ---
Apart from (maybe) calculating in different currency's here, there's the issue that for a node in a beowulf cluster, it is not neccesary to have RAID (== costly), or even a harddrive. Nor is it very likely that they used dual processors.
Because of the crappy multiprocessing with x86 it would hardly be of benefit over using only single-processor boxes, while being much more cost-effective.
Anyway, thats just my 2,00 * 10^-2 Eur.
Oh yeah, Thats what i meant...billion
I'm an applied mathematician, there's nothing pure about me.
delivering a processing speed of 375 gigaflops, or 375 billion operations per second
I think they mean 375 floating point operations per second. There's a difference.
I'm an applied mathematician, there's nothing pure about me.
I have a friend at SGI that said SGI's bid for this was almost at cost and it was like $1 million. IBM is losing a lot of money on this deal but they get the publicity and the fact that all the geeks at UNM will see big blue's logo all over it.
Also, the cost of the boxes is almost unimportant with something like this, you have to take into account the actual construction of the network (usually the most time-consuming part of any super computer) and the main cost is the support contract. You have to have people available to fix this guy on a moment's notice whenever it breaks.
I guess that would depend largely on the software run, however I believe that SMP will be useful on the nodes; simple SMP systems are dirt cheap nowadays, and they cost actually less than two UP boxes (1 powser supply, 1 MB, 1 bus, 1 Network interface, etc ...)
Also, it mentions the limitations of networking; can't you link together 3com (now defunct, I know) switches in a stack to make larger switches? If not, I'm pretty sure that we'll have larger switches in the next few years.
--
Because the joke is so damn obvious. Christ sakes my mother could have thought of that one.
What possessed someone to mark the above post as 'flamebait'?
I literally read it 3 times through to find the 'flamebait' there: nada. The only moderation down, which could have had a sliver of merit would have been 'overrated' but this is ridiculous. Hopefully this gets caught in meta-moderation...
Chris
San Francisco values: compassion, tolerance, respect, intelligence
Beowulf != any of these. Beowulf is the idea that one can take commodity, off the shelf (COTS) components and build a powerful machine at a price far less then a comparable commercial offering.
Codes run on Beowulf, and really any parallel machine, typically use MPI, PVM, or custom message passing libraries. The beowulf idea includes the use of MPI & PVM, among other freely available software packages. Codes that run on shared memory machines typicall uses the shared memory device of MPI, shared memory, or pthreads.
For CPU intensive tasks the Beowulf idea is great. Codes that perform lots of disk I/O suffer, as adding higher performance (i.e. SCSI) disks increases system cost greatly. Communication intensive tasks perform the worst on beowulf style clusters compared to commercial computers, as the interconnect on beowulf-style clusters can't compare. For a relatively large increase in cost, one can use Myrinet. With Myrinet bandwidth and latency begin to approach that of the switch found on the IBM SP series of machines.
With high bandwidth, low latency interconnect technologies that scale well (e.g. Myrinet), one can build a cluster that outperforms a comparable commercial offering at, say one quarter to one eigth the price. The difference at that point is software. There's really not alot out there to configure and administer beowulf-style clusters, and commercial implementations of some packages beat the pants off of their freely available counterparts (compilers, for example). Until the software situation changes there is still reason to buy your big iron from IBM, SGI, and Sun.
--Jason
Actually, I'm not making a category mistake (I even noted in my original post that the things I was using as examples were not necessarily interchangeable), but you've made my point for me.
As you note, the real power of distributed/parallel computing comes from the message passing libraries, most commonly MPI or PVM. Beowulf per se is almost nothing more than a label for the generic concept of distributed computing on Linux. The same thing can be done with any other reasonably modern networked computer you have lying around, even those running Windows - you can even mix OSes in a cluster, although this introduces new and interesting problems. (There are a serious lot of underutilzed cycles sitting out there on the corporate world's desktops if they're not running OpenGL screensavers...)
BTW: If the phenomenal success of Sun's E10000 Starfire has taught us anything at all, it's that where I/O is important, a big honkin' SMP box kicks cluster butt! Seriously, the interconnect technology between boxes just *can't* be fast enough to compete effectively with a huge multi-level crossbar packet switch like the ones in the E10K. Sun and the other SMP vendors can win here because they own the domain in which the simpler problem resides...
Don't assume by this that I'm against Beowulf clusters at all - they are a great and amazing thing, but there's more than one way to skin a cat, and Beowulf isn't the only path to Linux distributed computing.
"The future's good and the present is nothing to sneeze at." - Roblimo's last
The article doesn't say, but despite what you'd think by reading the rantings of the ill-informed 3l337 d00dZ on slashdot, Beowulf isn't even a very good clustering technology for most problems.
There are far more serious, industrial-strength solutions out there, things like MPI, PVM, LINDA,and IBM's own HACMP. (Note these cover a lot of ground and are not necessarily even comparable to one another.)
Beowulf (or any of the others listed above) is not automatically the correct distributed computing methodology. Selecting the proper solution for the job at hand is far more complex than you might imagine. There is a lot more developer activity on some of these than there is on Beowulf - MPI in particular is maturing rapidly and is used for solving big/tough problems in many of the largest companies in the world. (No particular MPI advocacy or bias, it just seems like I run into it more often than the others...)
"The future's good and the present is nothing to sneeze at." - Roblimo's last
True, but they are using 2-way boxen...
e s/Mar21hpcc.html
http://www.unm.edu/~paaffair/news/news%20releas
"The National Computational Science Alliance (Alliance) will take delivery of a 512-processor Linux supercluster within the next month - a move that will give this nationwide partnership the largest open production Linux supercluster aimed at the research community. The new supercluster, called LosLobos, will be located at the University of New Mexico's (UNM) Albuquerque High Performance Computing Center (AHPCC), one of the Alliance Partners for Advanced Computational Resources sites."
"It's tough to be bilingual when you get hit in the head."
Well, if you can keep most of your working set in the 2MB L2, you can stay off of the bus more often, so there won't be so much holdoff or contention with the other proc... but a switched fabric is better for SMP than a bus... far more costly, though. Can't wait for a nice dual athlon board my self... Where's the 70 from? Looks like you are referencing the Alphas there (which use the same bus as the Althlons)... the IBM cluster is 256 boxes, so if they are SMP, you have 512 procs, and like I said, if your software is designed properly, you shouldn't be heading to memory constantly anyway (though this *is* unavoidable sometimes).
"It's tough to be bilingual when you get hit in the head."
There are 4096-machine clusters, but the marginal performance gain per machine drops for many kinds of computation because of the 'segmented' archictecture. That is, the limited bandwidth inter-cluster relative to the bandwidth intra-cluster makes programming the whole cluster a problem of finding a 64-way very loosely-bound approach, each segment of which is 64-way loosely-bound (each segment of which is N-way tightly bound because of the possibility of SMP). Finding algorithms to split up problems this way is very difficult, and in some cases, impossible. For a given problem, there is a network width/speed for which the limiting factor is the processor speed (e.g. you're not losing performance to overhead); this is the case for many more problems at switched N bps than at shared N bps (in/out-side a cluster).
On the other hand, there are algorithmic techniques for masking (network) latency (e.g. time-skewing), so it's possible to make better use of 'loosely-coupled' (relative to the algorithm's interconnect requirements) compute elements (machines/clusters/etc) than you may think.
-_Quinn
Reality Maintenance Group, Silver City Construction Co., Ltd.
I really think that more attention needs to be payed to clustering technologies. I first got started with my cluster about 18 months ago. Beowulf was fairly new, and almost considered to be a black art. After reading and tinkering for a few weeks, I was amazed at how easy it actually is to get things running in parallel. Now, don't get me wrong, like many things its easy to do but hard to do *well*...nevertheless, I'd sure like to see more activity in this area. (IMHO, SMP is going to run out of steam quickly)
All you slashdotters with three or more systems in your basement! Go! Get them networked! Load MPICH or PVM! This should be your mantra:
"If SuperID can do it, then I certainly can"
So to answer your question, no. MS clustering is really more for failover then load balancing. The load balancing works but not nearly as well as you like. The basic problem is that Win2k/NT is not designed to cluster at its core. Linux can be made to do that and thus has a destinct advantage.
-- soldack
Interesting.
Were the Linux cluster users using gcc/g77? It is well known that (at least for most scientific codes) you can get 50-100% speedup by switching from the GNU compilers to commercial ones from Portland or elsewhere.
If there is still a difference, then the next thing to try is the latest dev kernels, which have better SMP (if SMP nodes are used), and significantly faster disk io through the elimination of double caching.
Since most scientific apps should spend most of their eating user CPU cycles, I wouldn't expect there to be very much difference between one OS and another, however node uptime and more established remote admin are points in Linux' favour for big clusters.
It doesn't seem like a particularly techincal article, so someone probably just glossed that particular detail over. Your average Joe won't remember it anyway and someone may have been trying to avoid confusing the reporter.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Why does the "Doesn't Scale Well" MS FUD persist in the face of stories like this? Is there an NT cluster anywhere in the top 500 much less the top 100?
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I dunno... My guess, if any computer were to download all of the internet, and be able to categorize it, it would be cracked of all its porn.
Just one man's guess...
The inter-machine communication requirements for genetic programming are low. Basically each machine can operate independently, and at the end of each generation, transmit only the fitness of each individual evaluated back to the server - very little data has to be sent over the network. See http://www.genetic-programming.com/machine1000.htm l
In fact, they're using a very standard 100Mbps ethernet, while my impression is that the IBM supercomputer will be much more tightly coupled. The GP cluster is rate at 0.37 Thz, about the same as the IBM machine.
On a side note, genetic programming will be an ideal for distributed.net; it's disappointing that GP isn't being attempted there.
Now, these Netfinity machines at best are 550 MHz Xeons (according to the best model I saw in IBM's Homepage) so I seriously doubt they outrun the Beowulf cluster of 1000 Pentium 2 350MHz machines (and the controlling host) being used for Genetic Programming, were they in fact clustered together to achieve the type of speed benchmarks IBM was after, rather than being used for a useful purpose as these are. Do they even outrun the 70 machine cluster of 533MHz DEC Alpha's that had previously been used for Genetic Programming? I doubt it. I tried to submit the Genetic Programming thing as an article once a while back, but it was rejected for some reason or another, even though someone posted about it in a forum like this once long ago, and people seem to continuously forget about this amazing cluster and what possibilities it presents to the computing world. Imagine if you told it to try to create a better version of itself? Once we have the storage capacity (the Petabyte, theorized to be necessary to store the totality of a human consoiusness) what would happen if you give it a pipe to the internet and told to to absorb data, correlate it with data it already has, "remember" or "forget" the data as is considered relevant based on things it already "knows"? Anyway. that's beyond the point... Which is this IBM cluster isn't amazingly new or ground breaking at all, and I have to doubt IBM's claim as fastest.
"Inveniemus Viam Aut Faciemus" 'We will find a way... Or we will make one!' --Hannibal of Carthage
Come on, IBM! We want to know how fast this thing is in BogoMIPS.
While I'm all in favor of breakfast cereal based supercomputers, particularly the linux driven variety, like everything else, there are many unforseen problems in implementing such a device. I've built seven such clusters myself (including a personal 60 box cluster of old 486 and P1s. It's mighty nifty), and each time I run into the same fscking problem.
Squirrels.
That's right, squirrels.
It has nothing to do with software conflicts, or processers overstepping each other. That can all be taken care of with a little bit of clever coding and hacks/workarounds. But that doesn't take care of the squirrel problem. Everytime I finally work around all the network tomfoolery, and get the power for umpteen boxes managable, like clockwork the squirrels come.
And they come not in single spies, they come in batallions.
It's never the same. Sometimes they just chew the wires. Sometimes they try and make off with a box or two. What the hell does a rodent need with a computer?!?!? Wait, I don't want to know the answer to that. My repeated attempts at hunting down and exterminating the wascally bastards are met with comic hijinxs and failure. And as far as I'm aware, there aren't any open sourced squirrel repellant systems. I can't trust a proprietary system to not conflict with the many many tweaks I've made to the system. But alas, I'm stuck with my ACME catalogue and a variety of clever devices which only fail and fail again, each one making me look successively worse.
So let's hope IBM can manage a good rodent-security system, and release it back into the community. God knows I've tried. I'm sure they will realize the importance of this issue after the first few attacks. This is a much overlooked problem, but we need a solution. And as soon as possible.
thankyoutheend.
A better account of what's going on -- complete with a description of how Beowulf is used -- can be found at http://www.linuxplanet.com/linuxplanet/reports/162 6/1/
it is on top500.org
You jest, but actually there is an NT supercluster within the National Computational Science Alliance. See here for more information. I was part of the original group which developed this clustering technology while it was a research project in the computer science department. Now that it has been deployed as a real computation resource, one of my projects is to make it available to the national computational grid which the Linux article discusses.
Deploying an NT cluster was certainly a challenge in some ways that would have been easier with Unix, but not impossible. Some of our collaborators have published results favorably comparing the performance of the NT supercluster to that of Linux clusters, so there seem to be good reasons to continue building at least some technology like this on NT.
Sorry couldn't resist.
More race stuff in one place,
than any one place on the net.
There is no mention of Beowulf in the article,
just "special clustering software"
Well to really drive this redundant topic into the gound. How about, maybe the moderator thought the beowulf cluster of beowulf clusters was kind of reapetedly redundant and then the repeated repeating of repeating the original repeating of the beowulf of beowulfs was being repeated and this was eyed, seen and viewed as redundant, then the moderating moderator moderated the repeated post outlining and stating in the verbage of the written post that in his/her point of view and from his/her mindset he/she thought that the repeated repeating of repeating the original repeating of the beowulf of beowulfs was being repeated and this was eyed, seen and viewed as redundant. So the moderating moderator moderated the repeated post outlining and stating in the verbage of the written post that in his/her point of view and from his/her mindset he/she thought that the repeated repeating of repeating the original repeating of the beowulf of beowulfs was being repeated and this was eyed, seen and viewed as redundant. This being said, the moderator more than likely tagged the repeatedly repeating repeat posts stating the redundancy of the redundancy was redundant.
More race stuff in one place,
than any one place on the net.
it will only rank 24th on the list of the top 500
I like the comment that its "only" 24th. As though being only the 24th richest person alive, or only having the worlds 24th fastest car would also be something to be sneezed at.
Anyway, is there a list of world supercomputer rankings?