Slashdot Mirror

← Back to Stories (view on slashdot.org)

Linux To Power NWS's Storm Prediction System

Posted by timothy on from the mind-the-ups-batteries-please dept.

Mr. Plow writes "The National Weather Service is migrating to Linux-based IBM workstations and has purchased 900 IBM computers and 160 servers to do so." He includes links to coverage at Forbes (a Reuters wire service story there), Government Computer News, and Computerworld.

41 of 154 comments (clear)

  1. Of course... by kinnell · · Score: 5, Funny

    With windows all you see is blue

    --
    If I seem short sighted, it is because I stand on the shoulders of midgets
    1. Re:Of course... by Ske · · Score: 4, Funny

      and with IBM it is BIG blue...

  2. linux to power.... by mrsev · · Score: 4, Funny

    ... I use electricity for power. Should I change to this "linux". Maybe linux is more secure.

  3. One Problem... by JamesD_UK · · Score: 5, Funny

    Who wants reports favourable to penguins?

  4. Bravo by enodev · · Score: 3, Interesting
    Now I'll now 185 seconds earlier that a flash just struck my home :-).

    The previous systems contained Hewlett-Packard PA-RISC processors with clock speeds ranging from 100 to 120 MHz. Although those specifications aren't directly comparable to the 2.4-GHz clock speeds of the dual Intel Pentium 4 Xeon processors, the new workstations, which have 2G of RAM each, run the various AWIPS applications noticeably faster than their predecessors, Piercy said.

    For instance, one AWIPS app that used to take more than 60 seconds to start up now loads in 18 seconds, he said.

    That's a increase of about 48 times of computing power and the app just loads 3 times that fast? Something has to be seriously wrong with this setup!
    1. Re:Bravo by kinnell · · Score: 4, Insightful
      That's a increase of about 48 times of computing power and the app just loads 3 times that fast? Something has to be seriously wrong with this setup!

      Startup times are affected more by disk access times than processor speed. Disk access times don't increase anywhere near as fast as processor clock frequencies.

      --
      If I seem short sighted, it is because I stand on the shoulders of midgets
    2. Re:Bravo by Silvers · · Score: 2, Informative

      The poster above about the disk times in application load performance is right now, however also...

      You make the mistake of assuming a 1:1 ratio of CPI (clock cycles/instruction) between each of the two systems, compounded by a difference in the number of instructions needed to complete a certain task, followed by the amount of parallel pipes in each processor... etc etc etc

    3. Re:Bravo by AmVidia+HQ · · Score: 3, Insightful

      load times is a wrong benchmark to pick, it doesn't show proccessing power well. Load times are mostly disk-bound.

      Load times more than 3x faster IS a LOT faster. And no doubt they will have many many times faster actual data crunching speed.

      --
      VIVA1023.com | Political Fashion.
    4. Re:Bravo by ocelotbob · · Score: 2, Insightful
      Read that quote again:
      ...Although those specifications aren't directly comparable to the 2.4-GHz clock speeds of the dual Intel Pentium 4 Xeon processors...

      The old machines were RISC boxes which were pretty damn quick on a Mhz per Mhz basis. While I wouldn't classify them as fast, they're definitely not 24 times slower. Given the clock cycle efficiency of the processors, I'd imagine that all things considered, the new boxes probably have at best ~20-30 times the raw computing power. Now, add things like disk speeds, memory speeds, bus speeds, and other things that haven't increased at the same rate as processor speeds, and things start making more sense. Add in the fact that startup of applications is probably a fairly serial process, a dual processor machine probably won't help that much in that portion of execution time metrics.

      --

      Marxism is the opiate of dumbasses

    5. Re:Bravo by Xenolith · · Score: 2, Informative
      There are still old HP systems running the database and application servers. That is part of the reason. These systems will be replaced shortly as well with x86 architecture. Hopefully with up to date and better optimized database software.

      Assuming it is the application I am thinking of... it was more like 5 minutes to start up, not 60 seconds. Yes, the HP systems were ancient (relatively).

      --

      Journal
  5. Interesting by frankthechicken · · Score: 4, Interesting

    Interesting, according to this

    Linux first appeared on the NWS scene in 1995 in the form of Slackware

    And in comparison to their previous HP-UX systems:-

    . . .the switch has helped increase data processing speed.

    "The performance that we measure with our benchmark has increased by over 100% since we completed phase one of our Linux migration," happily boasts Piercy.

    1. Re:Interesting by 4of12 · · Score: 4, Interesting

      That's the key issue.

      The purchase price of high performance RISC machines has always been O(US$10^4) and the acquisition cycle a period of several years.

      While you can still get high performance RISC workstations for those prices, it doesn't make any sense when comparing price/performance.

      In 1993, PC's significantly underperformed RISC machines, so paying 5-8 times the prices was worth it.

      In 2003, PC's perform equitably to the best RISC workstations: you need a really good reason not get a really great Linux workstation for $6-8K instead of high-end RISC hardware for $20-100K.

      --
      "Provided by the management for your protection."
  6. So what's the forecast? by trystanu · · Score: 3, Interesting

    Unsettled weather for the duration of the migration?

    Are forecasts really getting better as computer power increases? I know that Metrology is always in need of more computer cycles to model the weather, but have forecasts actually improved with this power? Are there any success statistics around?

    1. Re:So what's the forecast? by SpaceLifeForm · · Score: 4, Funny

      They keep saying that the 5 day forecast is just as in-accurate as before, but now, they know this fact with more accuracy than ever before.

      --
      You are being MICROattacked, from various angles, in a SOFT manner.
    2. Re:So what's the forecast? by frankthechicken · · Score: 3, Informative

      Well, since Edward Lorenz and his discovery of chaos, many of the current weather prediction systems are chaotic, and thus the forecast becomes less accurate over a longer time period.

      Like many chaotic equations, the systems of weather patterns follow a `normal' path for a period of time before becoming chaotic. So this method is useful in predicting specific weather patterns for as long as 5 days in advance. After this point though the chaotic nature of the systems become apparent, with different scenario runs of similar data leading to completely different results.

      So, more advanced computer power may lead to a better understanding of the chaotic equations which drive our weather and other environmental factors, though I for one remain sceptical, there are just too many factors to take into account for anything more concrete than the weather might possibly be . .

    3. Re:So what's the forecast? by Xenolith · · Score: 5, Informative

      Well, to throw one stat out at you. In 1987, the NWS had a probability of detection score for tornadoes of 25%. This past year it was nearly 80%. Now given, part of that improvement is better trained employees and an excellent skywarn/spotter network. But yes, bigger and better computers are helping to improve the NWS mission.

      --

      Journal
    4. Re:So what's the forecast? by Anonymous Coward · · Score: 2, Informative
      In 1987, the NWS had a probability of detection score for tornadoes of 25%. This past year it was nearly 80%. Now given, part of that improvement is better trained employees and an excellent skywarn/spotter network.

      *cough*cough* You sure that wouldn't have something to do with NWS fielding the WSR-88D (aka NexRAD) doppler radars? there is a boatload of information one can extract from that data. One of the particulars is the tornado vortex signature (TVS) which usually forms anywhere from 15 to 30 minutes in advance of a tornado. One of the nice things about TVS is that the system can detect it and then notify the severe weather guru who can issue a warning.

  7. We have are workstations already by Xenolith · · Score: 5, Informative

    I'm a NWS forecaster. We have our new workstations... they are pretty darn nifty. They are dual Xeons (2.4 ghz) running RedHat 7.2. These are much improved over the old HPs, which were J200 (overview). The bad news is that the database and application servers are still the old HPs. These are scheduled to be replaced in the near future as well.

    --

    Journal
    1. Re:We have are workstations already by tacocat · · Score: 2, Insightful

      Did you have to fight for getting Linux onto your desktop or was there the cost motivation to push it along?

      My impression is that any beaurocratic organization would be loathe to consider anything but Microsoft on the grounds that selecting Microsoft is safe. Examples of this would be trying to use Linux in large companies.

    2. Re:We have are workstations already by Xenolith · · Score: 2, Insightful

      I'm a meterologist, not computer dude. But my guess is that since the software was already written in a unix environment (HPUX), choosing Linux made it the logical choice.

      --

      Journal
    3. Re:We have are workstations already by girouette · · Score: 4, Insightful

      The computing context is similar where I work (Meteorological Service of Canada). There is a big difference between the forecaster's (technical) desktop and the clerical desktop. Our technical desktop never belonged to MS WIndows. Cost remains a concern, obviously, which is why we are moving away from proprietary Unix.

      For what it's worth, our next-generation workstation is going to be java-based. (Joint effort with a handful of European countries led by Germany.) http://www.dwd.de/de/Technik/Projekte/NinJo/ (German language)

      Dunno about the NWS, but for our clerical desktop to move away from Microsoft would be nothing short of miraculous.

  8. Windows by RevMike · · Score: 4, Insightful

    Of course... With windows all you see is blue

    Of course, the NWS is using Linux to replace HP-UX workstations.

  9. Dear NWS by Anonymous Coward · · Score: 2, Interesting

    Dear NWS,

    As I understand, you've recently upgraded your workstations to some slick IBM machines running Linux. Congrats. Now, about your old workstations. I'd be willing to take the off your hands, I won't even charge anything. I just want to see them go to a good home, not some dumpster behind your building -- ya know, [insert their address here].

    Thank you,
    Me.

    Fortress of Insanity

    1. Re:Dear NWS by Xenolith · · Score: 2, Interesting

      Yep, we have some 90 lb. paperweights at our office. The OS has been wiped from the hard drive. Is their a linux flavor that will work on a HP J200?

      --

      Journal
    2. Re:Dear NWS by Anonymous Coward · · Score: 2, Informative
      I work at NWS and they are decommissioning the HPUX boxes by the dozens/hundreds in many different locations around the county.

      They will be surplused by the General Services Adminstration like any other old piece of government hardware. See ifyou can find your local depot/auction location.

  10. Linux replacing Unix : just standard evolution by slb · · Score: 4, Interesting

    What would have been far more interesting is a Windows to Linux migration with the same improvement in performance and TCO.

    --
    http://www.transparency.org
  11. Maybe this will help Linux's respectability by Maestro4k · · Score: 4, Interesting

    Having someone big and respected like the NWS using Linux and announcing it publically should help Linux's respectability quotient in the US a lot. I'm glad to see this happening, both because it increases Linux's usage out there, and because it'll save us money (IIRC, the NWS is taxpayer funded), and probably lead to more accurate forecasts. It seems every time the computers they use get faster/more powerful, the forecasts get better. :)

    1. Re:Maybe this will help Linux's respectability by Stile+65 · · Score: 2, Informative

      NASA and the Department of Energy use Linux, and have for years. They use Linux Beowulf clusters for computational fluid dynamics. Most of them have an SCI backbone for inter-node communications. Although, I guess nowadays for CFD you might as well write multithreaded code that's compatible with OpenMosix and, instead of spending a ton of $ on a cluster, run the simulations on your employees' workstations.

      Oh, and you might as well not bother imagining a Beowulf cluster of those babies... it's been done.

      --
      I claim first use of "Error No. 0B" - or "No. 0B error." It'll be the new ID 10T!
  12. Important to mention... by Anonymous Coward · · Score: 5, Informative

    That the migration is from HP UX boxes to Linux, not from Windows as some people might assume.

  13. My weather man well, frog by snkmoorthy · · Score: 3, Funny

    My frog, actually croakes and stares at the open window, whenever it is going to rain, otherwise he just stares at the TV - although I have started noticing recently, that he doesn't care much about rain if Sex in the City is on.

    --Sig here--

  14. lies by n0k14 · · Score: 4, Interesting

    Almost every day I read about a department, agency, company or even country switching to linux. I rarely read about switching from linux back to windows. Where is this inconceivebly high implementation and training cost for linux that micosoft keeps whining about?

  15. Practical Outcome of This Change? by BallPeenHammer · · Score: 2, Interesting
    I'm curious about the expected practical outcome of this change. Presumably, they would be using the same prediction routines, but on faster boxes.

    Would this mean that they would get the same predictions, just a little faster?

    Would more capable machines mean that they could run some more-complex versions of the prediction routines they run now? Say, with more grid points, or smaller time divisions?

    Are the current prediction routines OS-dependent, so that they'll have to be ported to the new Linux OS? Is that easy or hard?

    What effect does the new Linux OS have on future application development? Are the existing development tools HP-UX oriented? Does that mean they would need a new tool set to do their development?

    --
    The Law of Falling Bodies
    1. Re:Practical Outcome of This Change? by Anonymous Coward · · Score: 2, Informative

      AWIPS is used mostly for severe weather forecasting. The 42 seconds saved, as quoted in the article, is enough to get a Tornado Warning out.

      This is not about trying to predict if it's going to be nice out next weekend.

  16. Re:all you see is blue by Technician · · Score: 2, Funny

    Actualy I see a storm brewing from the Redmond area. It could be a big one.

    --
    The truth shall set you free!
  17. Why not using distributed.net... by Ummite · · Score: 4, Interesting

    Why not using free systems like distributed.net, wich is 1) more powerfull and 2) free. People would be glad to help predict better weather, since actually they can't predict correctly 24h in advance ....

  18. Re:powered by linux.... by Anonymous Coward · · Score: 2, Insightful

    they paid ibm for them and ibm is going to develop for linux as long as they have customers willing to purchase it.

    So, your point while valid for small minded people, failed to extend to the long term and missed the obvious benefit to the Linux Community in supporting one of our more powerful corporate supporters.

    Not to mention that when they need some extra functionality such as better desktops...they will be able to pay for it.

    Lastly, its a governement agency, and you should be happy that your tax dollars aren't being wasted on proprietary software. Once they train enough sys admins at NWV they can cut IBM support out of the picture and support themselves...thus saving tax people some money.

  19. Re:powered by linux.... by Orp · · Score: 5, Insightful

    I, for one, give a shit. I am a meteorology professor who has been using Linux for over a decade. Having the NWS transition over to Linux means it is possible for me to run the same software that NWS operational forecasters are using. In fact, that's already possible - I have a copy of the AWIPS software on a CD-ROM sitting in my office, ready to install. This, in turn, opens the door for research possibilities and just as importantly, allows me to expose students to the kinds of software they will be using should they opt to work for the NWS (many do).

    I visited the Grand Rapids NWS office a month ago and most of the workstations were already running Linux. The SOO (Science Operations Officer) seemed to be pretty happy with this. Why shouldn't he - hardware and software costs go down, machines are faster, and the OS is something that most scientists are using anyway.

    Leigh Orf

    --
    A squid eating dough in a polyethylene bag is fast and bulbous, got me?
  20. Re:Let's laugh at the sheep by gmack · · Score: 2, Insightful

    First, as I repeatedly say, this isnt an organization choosing "Linux". This is an org choosing IBM, who is intelligently exploiting a free product in order to sell their product. Im shocked more companies arent jumping on the OSS bandwagon for the same reason- having people make your programs for you, for FREE? Amazing!

    IBM is probably getting very good value for the money they spend on Linux but saying they get it for free is showing a complete lack of understanding on how IBM has been doing buisness for the past 3 years.

    Just off the top of my head.. they have three Linux development labs of their own staffed with people who have PHDs (not cheap) plus they contribute to OSDL.

  21. The times they are a'changing by Anonymous Coward · · Score: 2, Interesting

    I used to work as a Sys Admin for the NWS a a few years back where we used AWIPS(the forecast folks were in another building) - and even then the HP workstations were costly and slow. AWIPS was a closed system and we didn't directly do any development on them but because NWS had hardware contracts with HP we had to usually purchase their equipment. The problem was that the HP workstations/servers were so expensive (especially compared to PCs) that the office I worked for couldn't afford to upgrade them - we had a lot of old, slow and just clunky HPUX systems. Several times I suggested to management that switching to Linux on Dell machines (easily available through Fed government contracts) for our internal development would save a bundle of money and mean better equipment but was always dismissed with the claim that Linux was a toy system. Lo and behind, somebody in the NWS has finally seen the light! (Although, to be fair, the NWS AWIPS developers were investigating Linux long before management ever knew what was going on ...).

  22. NWS = Not Work Safe? by Teflonatron · · Score: 2, Funny

    Anybody else think 'not work safe' when they read the title?

  23. Re:Let's laugh at the sheep by Lxy · · Score: 2, Informative

    First, as I repeatedly say, this isnt an organization choosing "Linux". This is an org choosing IBM, who is intelligently exploiting a free product in order to sell their product.

    You can't be farther from the truth. In fact, some of these workstations are Dell. I've seen them with my own eyes, and in fact ran top on one of them *drool*.

    The NWS's AWIPS product has always run on HP-UX. A couple years ago they had started looking into linux because they'd get a free OS for cheap hardware, a definite advantage. They've been running linux and HP-UX side by side for a good year now, but it wasn't until a few weeks ago that AWIPS finally supported linux. Once the support came through for AWIPS on linux, they abandoned HP-UX.

    This was a move a long time coming, and I'm glad they went through with it. So no, they are not sheep, and you are an idiot.

    --

    There is no reasonable defense against an idiot with an agenda
    :wq