Slashdot Mirror
Debian Cluster Replaces Supercomputer For Weather Forecasting
wazza brings us a story about the Philippine government's weather service (PAGASA), which has recently used an eight-PC Debian cluster to replace an SGI supercomputer. The system processes data from local sources and the Global Telecommunication System, and it has reduced monthly operational costs by a factor of 20. Quoting:
"'We tried several Linux flavours, including Red Hat, Mandrake, Fedora etc,' said Alan Pineda, head of ICT and flood forecasting at PAGASA. 'It doesn't make a dent in our budget; it's very negligible.' Pineda said PAGASA also wanted to implement a system which is very scalable. All of the equipment used for PICWIN's data gathering comes off-the-shelf, including laptops and mobile phones to transmit weather data such as temperature, humidity, rainfall, cloud formation and atmospheric pressure from field stations via SMS into PAGASA's central database."
that would be HURD
Snowden and Manning are heroes.
it's a cluster, in other words a herd of computers :-)
can anything that is "free" put a dent in ANY budget? if something gets bloated as it ages - dump it and go to OLD VERSION. shiiiiit.
and it's not even finished, why would they put it on a production cluster?
i would suppose that debian is quite a versatile distro for any purpose...
-- from a debian user... who actually started quite late with potato....
Actually, Debian is intended for servers and runs on more architectures than any other distro. The whole reason for the long testing cycle on Debian is to make sure it's as stable as possible so it can sit on a server and need little or no attention for days, weeks, or even months at a time.
I hated Debian at first because it wasn't friendly, but I looked into it more and realized it was the best choice I could make for my production servers. I can set them up and check once a week or so and they're still chugging along without need of intervention.
I wouldn't use Debian on my desktop (I use Kubuntu), but it can't be beat for servers.
It's NOT a desktop distro. Especially compared to Mandr* or Ubuntu or many others out there.
Debian, like most distros, is what you want it to be. Debian is used regularly as a server OS.
I doubt they have X installed on these machines.
I won't join Slashcott. OTOH, If Beta goes live, I just won't be back until it's fixed. Sorry Dice.
How different can Debian really be compared to RedHat in terms of stability? They both use the Linux kernel, and GNU tools, and follow the LSB, no?
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
Actually I don't like Debian much as a desktop machine, but I love it as a number cruncher OS. I've had a 10 machine openmosix cluster going for several years now, problem free.
Stability is a major thing with Debian, and my experience has been that this is quite true.
What was the age and the specs of the SGI being replaced?
Going by Moore's law, a factor of 20 performance improvement takes about 6 to 8 years. If the SGI was at least that old, this isn't news -- it's just the state of the art these days. In other words, small clusters capable of weather forcasting are relatively run-of-the-mill.
Of course, props to linux for being the enabler in this case.
Most weather prediction centers have adapted their weather forecast models to use Linux clusters. By running an operational forecast model on a cluster, it's easy for forecasters to scale the models so that they can be run (albeit slowly) on desktop machines, and are easily worked on by real meteorologists (versus IT professionals). At my university, we use a large cluster of machines on a RedHat enterprise system, and then able to scale the models and run them on multiple processors using MPICH compilers and batch jobs. Really, using a Debian cluster is no different then using a RedHat cluster. My colleague has access to the NOAA machine, which has more processors then you can shake a stick at... he talks about some code that takes 3 days to run on his personal workstation that takes 2 minutes on 40 processors. With the relatively low cost of a linux cluster, weather forecasting models can be run quickly and efficiently on numerous processors at a local level. With the ease of use of a Linux machine versus some of the supercomputers, it puts the power in the meteorologists to make those changes to the model so that it can improve forecasts.
Debian works out just fine for these kind of tasks. Here in the Netherlands the national compute cluster Lisa runs on Debian (http://www.sara.nl/userinfo/lisa/description/index.html) with 800~ to a 1000 nodes (I think the page needs updating by now).
All I can say is that I enjoy running Debian servers and RHEL clients at my work... and you're a douche...
oh, wait...
Do not mock my vision of impractical footwear
More importantly, is it Vista capable?
Debian is sure not a desktop distribution. Ubuntu would be one. I run Debian on all my servers.
"Freiheit ist immer auch die Freiheit des Andersdenkenden" - Rosa Luxemburg, 1871 - 1919
I tried: /lib/libearthquake-2.3.so.0 is not a symbolic link"
apt-get -f -y install gweather
But it failed with something about "ldconfig:
Is libearthquake in unstable?
Interactive Visual Medical Dictionary
The binary package management really says it all.. you shouldn't be running anything but compiled source on a performance cluster.
/bin/ls though? I don't think it matters to anyone on a high performance cluster. Just so long as the cluster apps are optimised then the rest is just noise - better to have a system that's less work for your administrators so they can concentrate on what's important.
Wow - how many performance clusters do you run again?
Not that I run a "performance cluster" as such - but I do run a bunch of machines that are very busy, all on Debian.
You know what? We compile the couple of programs where CPU is the bottleneck from source. We also compile Cyrus IMAP from source because we apply a pile of patches, but if someone else was packaging up all those patches in upstream, I'd be happy for them to be compiled there. Disk IO is the issue with Cyrus, and a custom compile won't help with that.
Yeah, we build our own kernels as well - that's another point that's worth the effort to customise.
I run ubuntu-server on my servers. Keeps packages in sync with my desktop
so I can test things locally, but has slight differences in defaults.
Not sure why you call Debian a desktop distro, it's much more useful as a server.
You obviously doesn't have a clue about Debian. Debian has been a fine server since 1995, and I still choose it before RHEL anyday. I have always found it strange that everyone went for RedHat, when they could have Debian. Mark S. saw the advantages of Debian and based Ubuntu on it, Ubuntu is a server and a desktop distro, based on Debian. It has made more people realize the strength of the Debian approach.
Exactly, My job is running high performance computing clusters. You don't need to put much effort into your cluster distribution at all. As long as it's stable, and gets the job done, why mess with it.
/bin/ls don't matter a bit.
The things I (and my co-workers) put a lot of optimization effort into is the kernel and our apps. You're exactly right.. 99.9% of our CPU cycles go into getting work done, and that 0.1% used by
I don't get why every one is saying it's not for desktop. It runs great on my T60. Everything works fine: sleep, mouse, red nub, wireless, sound, screen brightness, Blue-tooth. I guess I have never used the media buttons...
Money is the root of all evil?
My HP calculator is Vista Capable. It has a sticker that says so.
It is difficult to get a man to understand something when his job depends on not understanding it.
What, only one?
They never said it can't be used for desktop. They are just saying that it isn't primarily aiming for desktop use and works very well on servers, debating the original posters claim.
Debian is pretty great, but notived that you need a pretty big virtual machine
allocation if you are to use it within vmware as the log files clog up the system pretty quickly.
there is a setting somewhere but by default is expects a normal hard drive and not for example a 1gb allocation.
"an eight-PC Debian cluster"
"[we] wanted to implement a system which is very scalable"
8 PCS ? 64 cores at most. And they call that scalable ? Come on, today's top500 top machines scale on 10'000 cores. They're 15 years late.
From TFA What was even surprising to us is that Intel FORTRAN is also free of charge ...
I bet Intel are surprised too. Their compilers are not that free of charge. The people at the Philippine government's official weather service are hardly "not getting compensated in any form"
http://www.intel.com/cd/software/products/asmo-na/eng/219771.htm
The article lack, as usual, information about what those machines actually do when they compute together.
What I want to know is: Do they have a big 64 bit addressable RAM image spread over all nodes, communicating with pthreads, like I prefer? Or perhaps they have several 32bit RAM images communicating with some special message protocol. Or perhaps they just have lots of quite independent but equal programs running, as an ensemble. Or perhaps some kind of pipeline where the different parts of the calculation run on different machines.
All those free and commercial producers of supercomputers, why don't they tell us clearly how they are supposed to be used? Personally, I prefer one big image, because I am a physicist kind of person, knowing that this simple computational model will save me lots of work, and also work fast in practice, as long as I do not write too stupid code, i.e. with tight nonlocal interdependencies. But from what I see, many of them appear to use 32bit operating systems, which makes this impossible, and they thus have to use message passing protocols, which make everything much more complicated. For instance: I can do a big 3D wave simulation by having a big 3D array spanning several machines, and updating it piecemal. However, if I have to cut it into 64 sub-cubes, and using message protocols to glue their edges together, then the work required to do this is a significant road block, and extra code like that also introduces bugs.
Could this be solved by something as simple as using an NFS file, memory mapped piecemal to different machines to do automatic cross-machine data sharing?
Kim0
Debian isn't - and never has been - a desktop distro. If you want a desktop distro built on Debian architecture, you get Ubuntu, or Knoppix, or one of a dozen others. Debian's unique selling proposition is a combination of stability, which is very important to production servers, and a rigorous commitment to free software. Packages don't make it into Debian Stable until they have been thoroughly tested. Debian also has the best package management system in the industry.
Frankly, I wouldn't run a server with anything else.
I'm old enough to remember when discussions on Slashdot were well informed.
Hmmm... 0 to Gentoo/Slack fanboyism in like... 4 comments.
I know people who know a fair amount about running clusters. None of them want the headache of dealing with the random-ass unexpected conflicts that arise out of having the explosion of possibilities for custom compiling for each server. Also, nobody wants to use their precious "performance cluster" cycles compiling every update. If you really need to compile tweaks (for the important stuff only), you do it offline, once, and then build a *binary* package to distribute to your nodes.
I do the same with debian actually. Well Ubuntu comes from Debian so there wont be a big difference I guess :)
"Freiheit ist immer auch die Freiheit des Andersdenkenden" - Rosa Luxemburg, 1871 - 1919
I never said it doesn't work on a desktop. I use it here on mine right now. It's just not as easy to setup as Ubunutu I think.
"Freiheit ist immer auch die Freiheit des Andersdenkenden" - Rosa Luxemburg, 1871 - 1919
This makes me very happy.
"The bad machine doesn't know he's a bad machine."
Nope, it's one big, happy family :)
Every time I want to install Ubuntu on some random machine, it fails. I always have to go back to Debian.
I have Debian currently installed for my father and my sister. Spares me the headaches of Windows problems. The only support I need to deliver to them is giving information about performing tasks.
since it is a cluster that would be correct, a herd of hurd...
Politics is Treachery, Religion is Brainwashing
I felt a great disturbance in the force - as if millions of geeks had ROFl'ed and then were silent.
"It doesn't cost enough, and it makes too much sense."
10 computers over several years? Greenpeace are gonna be hunting you down like sailors to whales!http://forumpix.co.uk/i.php?I=1205492846
The article mentions the Global Telecommunications System (GTS) It would be cool to know
how they get their GTS data, probably use a satellite downlink. There is a GPL GTS switch that's developed for Debian:
http://metpx.sourceforge.net/
How fortunate that apt/dpkg handles source packages so well then... Punching in 'apt-get -b source <whatever>' is not a whole lot harder than 'port install <whatever>' or whatever you prefer, is it? I know, I know... Don't feed the troll... Sorry.
"Live free or don't."
what else is new?
FYI, I am a little biased, but Debian is the distro that constantly gives me the least trouble.
And customise absolutely everything yourself.
There must be a thousand distros out there, so why not?
Everybody uses broad generalizations.
Give the guy a break. He just wanted to see Gentoo compile in less than 2 weeks; he had a cluster handy...
Censorship is telling a man he can't have a steak just because a baby can't chew it. --Mark Twain
While Ubuntu might be friendlier, and be more polished, I'd like to say that Debian is perfectly workable as a desktop os. (Started with a base install that didn't even appear to come with less, moved onto fluxbox when I wanted a gui, moved back to kde because I missed it).
It just takes a little more effort if you do something pointless like start out with just the min install.
Lenny is absolutely a desktop distro as well. In fact, pretty much any Debian testing has been suitable in that role for years.
what are you, a communist? **ducks**
Yep, Debian base install has more, not less. They do have vim, though ;)
No tyrant thrives when every subject says no.
Um, your unfamiliarity with Debian is painfully showing. apt-get update doesn't destroy servers. apt-get upgrade might, if you're running testing or unstable. I'd recommend you use neither for production servers, and stick with stable.
And consistency? Like how the entire Debian repository is cross checked every day to ensure consistency?
I'm also intrigued by your reference to updates destroying servers. Do you get this behaviour with Red Hat? Makes me glad I'm not using it, then.
And managing an arbitrary number of nodes under Debian is easy with clusterssh.
No tyrant thrives when every subject says no.
yea, vim-tiny.
sent from my slashdot browser.
I agree. I used to run some clusters for the UCLA Chemistry department and the only real customizations we did was to install a custom kernel in Redhat 9 to handle the huge amount of memory we had installed. And even that wasn't in all the clusters. But yeah, the code the clusters was actually running was pretty much always compiled by hand.
Three Ubuntu for the Elven-kings under the sky,
Seven Damn Small Linux for the Dwarf-lords in their halls of stone,
Nine Linspire for Mortal Men doomed to die,
One Debian for the Dark Lord on his dark throne
In the Land of Mordor where the Shadows lie.
One Debian to rule them all, One slocate to find them,
One distro to bring them all and in the darkness bind them
In the Land of Debian where the apt repositories lie.
http://www.debian.org/misc/children-distros
Moderation Total: -1 Troll, +3 Goat
The only way I could get Spamassassin 3.2.4 in the form of a deb was from Ubuntu so I built the source DEBs (there were some dependencies) on a machine that I use as a build host and testing playground. That Spamassassin instance has been chugging along without a hitch for a couple of months now on an Etch server. There is no way I'm to going install Ubuntu binary debs on a running server but dpkg-buildpackage really DOES make it all one big happy family. I've also built Sid source DEBs for my Ubuntu desktops.
This is one of the things I like most about Debian derived distros. The packages are highly compatible at the source level. My experiences a few years back rebuilding SRPMs between say RedHat and Mandrake weren't nearly as smooth.
There may or may not be that much performance gain from compiling from source, but depending on what your sysadmins cost, having the ability to run and have your system be automatically updated could end up being more cost effective than additional support even if that means that you have to buy more/bigger hardware.
Personally I use the package management system for all of the core subsystems/libs and only compile from source my main applications (apache, php, postgres, and mod_perl). There may be a point when I find it necessary to compile my own kernels and do other optimizations, but for me this is the ideal scenario.
When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
I know Linux and Debian are very configurable, but I'd be very interested in knowing how they configured a numerical weather prediction model to predict tsunamis. I mean what are the chances that Australian PCMag headline writers have their heads up their asses?
GTS was until recently largely an X.25 PSTN. I learned X.25 helping maintain message-switching software at a military weather forecasting center; we were a subsidiary node of GTS in that capacity.
I know that many nodes in the GTS have gone to FTP or TCP socket streaming over the Internet (or VLANs running under the Internet). Old-sk00l by 'net standards, but Très moderne in the WMO timescale.
Welcome to the Panopticon. Used to be a prison, now it's your home.
The article says that their preference on workstations is Ubuntu which is "basically Debian-based." Ubuntu isn't just Debian-based, it's entirely dependent upon Debian for it's continued development.
Dinomite.net
So, x64 processors beat out the MIPS 10000 from 1997? Go figure.
I wouldn't want X installed on a cluster or server. I cringe every time I use a server-room KVM and end up on a Linux server that has X running...WHY?! FOR THE LOVE OF GOD, WHY?!
:-)
That being said, I've had very few problems getting X running in Debian. At least 80% of that is researching before I buy a video card... If X doesn't support the card/chipset, I don't care if it's the latest and greatest card out there with 2G of FTL SDRAM, 2 quintillion colors, and able to support 300 fps @ 16000x9000 resolution.
"The urge to save humanity is almost always a false front for the urge to rule." --H.L. Mencken
the code the clusters was actually running was pretty much always compiled by hand.
You really oughta use a compiler for that.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
Debian's not a desktop distribution? After a dozen years, NOW someone tells me? Geeze. Now I've got to install something else on all of my desktop machines.
Warning: This signature may offend some viewers.
Agreed
I use debian as desktop system too.. and a bunch of other stuff.. can't beat the other systems I tried (Suse, RH, Gentoo) on stability and reliability.. of course it is only useful people capable of solving problems at times..
Wouldn't install it on the pc of my grandmother anyway, but It really does the job for me..
Can't help the people who want the newest unstable features all the time.. it's their problem.. and in uniqe situations, there are always possibilities to get a new package working..
Agreed... I use Debian 4 on two low end machines for the purpose of running my web servers, SSH, webmin, TeamSpeak, ... Prior to using Debian I had Gentoo running. It ran fine for my purposes but ran into some issues when updating some packages. Took me too long to fix issues when updating broke things so I made the decision to go with Debian. I'm glad I made that choice because its been very stable and easy to manage. I leave it up for months at a time and are on a UPC.
For main use I use Kubuntu or Ubuntu with the GDM disabled which means I just use a text login. I Used to use Gentoo prior to that but took forever to install new packages and was pretty easy to screw things up when emerging some things. I learned a lot when I used Gentoo but Kubuntu is what I prefer at this moment in time. The only real annoyance with Kubuntu/Ubuntu is that when booting the text login comes up then other services start after that. Debian does it correct and starts all the services in the default runlevel before comming to the text login prompt. You know what I mean?
Actually it is quite possible to get the best of both worlds - or even better - at least if the cluster nodes are reasonably identical. Just use one box for building the binary packages and have a local repository. Set all the cluster nodes to only use your own repository, and have them update automatically with a cron job.
On your build box, you update as normally from standard repositories, but those packages you want to build yourself, you set a pin for, so they don't update.
When you build a package, you can use the source package and apply patches and options to suit your environment, or you could even build a package from scratch. (Since you have a cluster, distcc might be useful when building.)
When everything works put the packages in the repository.
In case you require all nodes to have the same version, it might be smarter to have a script which handles the updating.
That is like saying that a lion isn't a mammal! It is a cat!
Semi-automatic amateur armchair Australian philosopher; conjecture ready at any moment...
Vista who?
Exactly! MetPX is a tcp/ip only switch. It implement WMO manual 386 tcp/ip sockets, as well as file exchanges over ftp & sftp. It was written to accomplish a transition away from proprietary mainframe stuff exchanging X.25 with the GTS. It also does AFTN (Aviation Fixed Telecommunications Network) over tcp/ip, in contrast to traditional X.25. It is used for the Canadian gateway between GTS and AFTN in Canada, as well as the GTS node itself.
Many think of the GTS as an X.25 network, but X.25 is going away. All of the commercial
switch vendors, as well as MetPX, support WMO sockets at a minimum. File based exchanges are
the new frontier. This software is such a niche application, that there isn't a lot of ''community'' that will be interested. It's kind of a vertical market thing. So it hasn't exactly taken the world by storm.
I agree, it's great for servers but pathetic on desktops. I have the one of the worlds most common integrated graphics chipsets from Intel; every Debian based distro I have used cannot configure any kind of widescreen monitor display with this chipset. It's not just me, all the people I have spoken to on seven different forums can't get it to work either.
I just hope that the Philipine weather stations can find standard ratio monitors in the shops because otherwise their maps are going to be a real funny shape (in most countries in South America you can't buy standard aspect monitors anymore).
I'm using dual-head with widescreen on one monitor with Ubuntu with no trouble. I had to change some settings in the config for X I believe, but that was about it.
Actually, Debian is intended for servers and runs on more architectures than any other L inux distro.
Fixed it.
I had to send six different xorg configurations to Ubuntu and the post is still on the bug reports without a solution. It's receiving about twelve hits a day with possible solutions or further bugs related to this so if you've got it sorted could you post on their forums cos a lot of people would like the solution.
I noticed that the issue is with Intel 82945g chipsets, Debian based distros, LG/Samsung/AOC/Soyo & CTX monitors though all those with Dell or Viewsonic seem to have corrected the problem but their solutions don't work for anyone else. Unfortunately you can only buy LG or Samsung within 1000 miles of Ecuador.
Debian runs perfectly as a desktop anything else is just useless FUD. I have ran Debian unstable as me desktop for the last 4 years, and now on my Eeepc. Why the hell do think 70% of other distros are based on Debian ? It truly is the Universal OS.
Sarge was released in not quite 2 yrs, Etch is due to be released in Sept, Stable is fine on a server, but most people that run debian on their desktop use testing or Sid.
Debian Sid LXDE Firefox 3.6.4
GNU/Linux and Firefox, surfing the internet safely.
It just takes a little more effort if you do something pointless like start out with just the min install. If you want to use Debian on Desktop then DPN'T make just base install. Base install is for the servers. You can blame only yourself if you make just base install. When you start Debian installer it clearly asks what you want to install. Select desktop environment and then Debian installer will install it. Ut really can't be so difficult. Starting from Debian Etch there is even new graphical point'n'click intaller.
So tell me, how do you really feel?
Did try it with Potato, Woody, and Sarge.
So yes, you're right, I don't know a damn thing about Debian or Debian on the desktop.
Seriously, Debian can be used on the desktop. So can Slack or just about any distro out there, but Debian was designed for "set it up and forget it" server usage.
By the same token, my 1986 Mercedes 560SL convertible has a kick-ass engine and I can use it to haul trailers full of wood or bricks, but that's not what it's designed for. There are word processors written in Java, but they never work as well as ones in something like C or C++. I have next to me an 8 CD set of the last season of a hard-to-find radio show. I can use those CDs as coasters or frisbees if I want. It's not the best use of my 560SL, Java, or my CDs, but I can use them as such.
The same is true with Debian. It's designed for server use. That doesn't mean it can't or won't work on the desktop, but the intent all along has been more focused on servers. If you think otherwise, hang out on the Debian-Users list for a few weeks and see what most of the questions and topics deal with. The vast majority are not about how to get a game to work or other desktop questions. Most topics are more applicable for server or workhorse situations.
But then, I wouldn't know a damned thing about Debian. After all, just because I've been experimenting with it since Potato and have been using it on my computers for years, including on laptops, doesn't mean I know what I'm talking about and it's quite clear, from your brilliant logic, the plethora of facts you use to back up every statement you make, and your focus on pure logic instead of any emotional and reactionary statements, that you know everything about Debian.
It may work on a desktop, but if you have been using it for 10 years, then you know the focus has been elsewhere and only recently has there been a focus on creating something that's easier to install.
I'm also making a bit of a distinction here you may not follow. There's a difference between using a distro on the desktop and a distro being intended for a desktop. But that's a whole different topic and I'd rather discuss it with someone who is more interested in dealing with facts than spending several sentences calling me a liar because they disagree with me.
We ended up using OpenMPI and distcc, and we were so much happier!