Speaking of legally downloading large amounts of data per day, let me just give a quick run-down of things I as a meteorology student do regularly. Note that none of the things I am listing here are part of my education per se, but simply things that I do on my own time.
Run a weather model on my machine 4 times/day, initialized off of the NCEP Eta model (which it also uses for boundary conditions). I use the lower-resolution grids and omit every other model output time, and it's still ~700MB/day (over 20GB/month).
Download archived Level II Nexrad data for examination (usually to perform ad hoc, informal case studies). Data amounts for a single day of interest for a single radar typically run between 300-500MB compressed.
Download model output data for examination. Using the Eta model as an example, a single "tile" (covering only a few states in the U.S., for example) for a single time is around 1.5MB (which might not sound like much, but it adds up really fast when trying to look around at various parts of the model domain for various times).
View and download for processing a wide variety of constantly updating real-time weather data (bandwidth usage varies widely based on current conditions).
Run Gentoo.
All of this is in addition to the standard things that other people do, such as download ISOs/large programs, view movie files, browse the web, etc. I can't even tell you how glad I am that I no longer live in the dorms at my university, as they currently limit students to 500MB/day (after this is used up they throttle your connection so it slows to a crawl). How the students there can get anything cool done is beyond me, and this goes for people of other interests as well--bandwidth can be important in many other areas than just meteorology or the sciences as a whole!
That being said, it's unrealistic for us to believe that we have some kind of God-given right to infinite bandwidth. Bandwidth is limited, and perhaps a reasonable, and let me stress reasonable surcharge for excess bandwidth usage would be a good thing, in that it would force us to use our bandwidth more judiciously. It seems as though Napster, Gnutella, etc., while they initially fueled the explosion of broadband, are now its worst enemy as they tend to be very wasteful of resources, though they certainly have legitimate uses. Oh, the irony...
"Would you support a bill that would allow taxpayers to voluntarily contribute money to NASA?"
Is this a joke? What would that accomplish? Does anyone really expect, come tax time, Americans to open their checkbooks and start shelling out the money for NASA, let alone at a rate which would add up to billions of dollars? You must be kidding me...
I think not, a friend of mine misspelled "public" in a history paper once. The spell-checker didn't seem to mind, although his teacher didn't think it was too funny...;-)
Yeah, like I said I'm still a fan of FreeBSD. I just really, really wish that it had the filesystem support that Linux does...I really miss that when I'm working with FreeBSD machines. In particular, I heard something about someone working on journaling for UFS (has this come to fruition yet? haven't used FreeBSD in a while...)--this would be an awesome improvement to FreeBSD. Running fsck can really suck...;-)
Thanks for the info--it sounds pretty capable. I especially like the idea of a dialog-based interface for configuring stuff...sounds useful for those users who are not as familiar with the innards of an OS or don't have the time/inclination for that type of stuff. I've always thought Gentoo should have a tool like that.
How do the source and configuration tools work? I really am curious, it's just that there's no explanation of how anything works, how the install progresses, etc. I loved FreeBSD, then was wowed by Gentoo when I saw how superior (at least in my opinion) their system management tools are. It could be that Lunar Linux is even better still, but I have no way of comparing the two since there's no description that I can find of how Lunar Linux manages the preinstallation, installation, and postinstallation management aspects of the OS.
To synchronize your portage tree with the official Gentoo one:
# emerge rsync
To install GNOME:
# emerge gnome
To update all of your packages to new versions:
# emerge -u world
To remove all of the *old* versions of the packages updated by the last command, since they are protected until "cleaned" out:
# emerge -c world
To forcibly remove a package:
# emerge -C $PACKAGE_NAME
To "pretend" you're running a given command:
# emerge -p$ADDITIONAL_FLAGS $PACKAGE_NAME
**
There are far too many conveniences to list them all so far as emerge goes, but those are just a few. Other conveniences:
The portage system (emerge, ebuild, etc.), when installing files to/etc, does not by default overwrite any config files--so you'll never lose your favorite config file in/etc again. Instead, it write the file to the same directory, with the prefix like._cfg0000_ for instance.
Updating your default runlevel is amazingly easy:
# rc-update del telnetd default # rc-update add sshd default
Each rc-script has a "dependency" function which relates it to other rc-scripts (for instance, "checkroot" must run *before* all other rc-scripts; or the mail server courier must run *after* all its components).
The default Gentoo kernel is based off the Alan Cox tree, and thus includes support for things like xfs, jfs, grsecurity, etc. Very convenient, and I have to say that xfs rocks.;-)
The list of cool features like these just goes on and on...if you go to the Gentoo website there'll be plenty of info there.
I'm still a fan of FreeBSD, but I have to say that Gentoo's
Well, one could argue that Redhat is a "source-based" distribution since they publish all the source, but it would certainly be a chore to build & install Redhat from the source RPMs!
Anyone know how Lunar Linux works, what their philosophy behind it is, etc.? Their webpage is less than informative (other than to say that it is a "source distribution"--which can be taken a lot of different ways)...the closest thing I saw to "information" was a comment which said that it was "better than Gentoo in some ways", but never specified how or why.
Anyone used it and knows how it works, etc.? Those of us who've never heard of it would appreciate a little information before wiping our hard drives...
Gentoo would suit you very well...when I was first introduced to FreeBSD I just loved the ports system. Then I tried Gentoo and realized how much better the Gentoo scheme (known as "portage") is...not to mention the way they handle runlevels/init scripts, config files in/etc, and just about everything else. Hands down the most coherent, logical system I've ever seen.
Ok, that makes more sense--I hadn't considered faulty equipment (obviously you don't want stuff freezing onto the fuselage;-) ). Once you've got the stuff frozen on there, however, it is not at all surprising that it would fall off in a chunk--it was probably daylight and as the plane was descending the metal, likely being warmer than the rest of the surface (esp. considering evaporation/sublimation), rose above the melting point of the solid and caused the whole thing to come crashing off. Of course, the odds of it landing in someone's bathroom are still pretty slim...;-)
*laugh* Right...not sure what universe I was coming from with regards to that comment. Just a *little* bit of a difference between "constant acceleration" and "constant terminal velocity".;-)
> I thought that "terminal velocity" is the same > for all objects, no matter size/weight > (disregarding wind friction for size; and since > these are somewhat round...).
As you say, terminal velocity *is* the same for all objects in the absence of friction. However, friction is a pretty important factor, and cannot be disregarded.
Once you get more than 50-100km away from a NEXRAD system, the resolution is pretty bad--basically it's averaging everything over a roughly 1km^3 volume, and even more than that the farther away you go.
The ice structure is a good thought, since if these things exist there must be some way they stay up in the air while forming. Never heard of any atmospheric ice structure that would fit the bill, though, but who knows...
> What all of that basically amounts to is...well...that dog don't hunt. The only realistic > source of that kind of thing is water being dumped by airplanes at high enough altitudes so > that it has time to freeze before reaching the surface. A chunk of ice that big should be easy to > save long enough to be examined; they should give a few samples to NOAA, say, and let them inspect > it.
Agreed. I'd like to see some hard evidence of this given our present knowledge of precipitation formation and cloud physics (which admittedly is pretty limited, but certainly precludes events of these kinds).
The problem I have with the airplane theory, however, is that the instant that the water is ejected from the plane it will break up into thousands upon thousands of tiny drops which will instantly freeze at that altitude. A large mass of water would not freeze instantly due to its large heat capacity, yet at the same time it will not remain together due to the various forces yanking it apart (esp. friction).
So in other words our "megacryometeor" would (for it to form in the 4-9km AGL range) have to grow from a infinitesimally small nucleus (since allegedly the composition of these things is similar to the composition of rain) to the size of at least a basketball before it strikes the earth. Keep in mind that it can't simply start out as a mass of water the size of a basketball (as mentioned earlier), and thus must grow slowly enough to have all the water freeze/vapor condense onto it without losing it all, and yet somehow stay up in the air without the support of a strong upward current of air to balance out its terminal velocity, which will be on the order of 50 m/s (110mph) by the time it is the size of a baseball, let alone a basketball or more. Note that these upward currents of this magnitude occur *only* in the presence of strong thunderstorms.
The only thing I can think of that might remotely be able to do something like this is a very strong jet stream placed favorably next to a mountain range. You could then possibly have very strong vertical winds (this is very favorable for cloud formation, however) up to and even exceeding 50 m/s, and under *just* the right conditions you could probably grow a chunk of ice from nothing other than vapor over a long period of time...however we're talking about growth rates on the order of days and even weeks in the absence of clouds for a chunk of ice of any reasonable size. Thus even this highly idealized setup is not realistic, as it would have to persist steady-state for weeks.
In short, there could be some really weird process out there we haven't the foggiest notion about that is causing this, but I'll believe it when I see it...
True, I was referring only to the models themselves. Post-processing, though CPU-intensive, pales in comparison to actually running the model. Thus the post-processing software is not written in fortran. BTW, any idea when they're going to finish porting AWIPS to Linux?
I am well aware of the meaning of the name, and simply use caps for the sake of consistency. Thanks for pointing that out, though, as I'm sure there are many others who may be wondering just what the three letters ETA stand for besides the obvious...;-)
> Some time ago I found a big bug that when removed increased the rain in some satellite data by 30% ! Guess how it was handled ? "Put it back in !"
Yeah, some of the older models especially are terribly maintained and full of all kind of cruft and crap. I could be wrong, but I imagine the reason you were told to put the 30% bug back in is because that's the way the model has been for eons, and the forecasters that use the model likely know its biases in a qualitative sense. Also, if they're attempting to do any sort of long-term statistical analyses using the model, altering it in such a way would render those analyses useless, whereas leaving the bug in means that the comparisons can still be made, since they are all biased one way or another. Thus removing the bug, though obviously the correct thing to do from a programming standpoint, might actually cause more trouble and confusion than it's worth, especially if the model is to be retired soon.
> Most of the models were all written during the 60's & 70's...
Not true, most atmospheric models have been written since 1980, although some of them were an extension of earlier models written in the 60s/70s. The earliest models were written in the mid-60s, and development on them didn't really take off until the mid-late 70s and early 80s, when models began to be introduced in rapid succession. Even today many models are currently being developed anew (the WRF and ARPS, for instance, neither of which existed prior to the 90s--in fact, the 1.0 release of the WRF was in Fall 2000), and new ones are announced quite frequently. They are still all written in FORTRAN, however, as FORTRAN is the standard for large-scale scientific computing.
Speaking of meteorological programming, ALL the major atmospheric models are written in FORTRAN. The ETA, AVN, NGM, MM5, WRF, and scores of lesser-known models...all of them written in FORTRAN (most of them FORTRAN-90 now, but some of the older ones are FORTRAN-77). The MM5 & WRF may be found here and here. The source code to several others is readily available as well if you're so inclined, for instance the ETA and the ARPS. Anyone wanting to run them may do so fairly easily on a PC running Linux (any new PC will be able to run a fairly hi-res model real-time); I do so myself.
Though I agree that many things can be art, and that a great deal of art is found in the unlikeliest of places, what is the point of even having the term "art" if *anything* can be considered "art"? The word then becomes meaningless, and might as well be removed from the language.
Slashdot Mirror
- Run a weather model on my machine 4 times/day, initialized off of the NCEP Eta model (which it also uses for boundary conditions). I use the lower-resolution grids and omit every other model output time, and it's still ~700MB/day (over 20GB/month).
- Download archived Level II Nexrad data for examination (usually to perform ad hoc, informal case studies). Data amounts for a single day of interest for a single radar typically run between 300-500MB compressed.
- Download model output data for examination. Using the Eta model as an example, a single "tile" (covering only a few states in the U.S., for example) for a single time is around 1.5MB (which might not sound like much, but it adds up really fast when trying to look around at various parts of the model domain for various times).
- View and download for processing a wide variety of constantly updating real-time weather data (bandwidth usage varies widely based on current conditions).
- Run Gentoo.
All of this is in addition to the standard things that other people do, such as download ISOs/large programs, view movie files, browse the web, etc. I can't even tell you how glad I am that I no longer live in the dorms at my university, as they currently limit students to 500MB/day (after this is used up they throttle your connection so it slows to a crawl). How the students there can get anything cool done is beyond me, and this goes for people of other interests as well--bandwidth can be important in many other areas than just meteorology or the sciences as a whole!That being said, it's unrealistic for us to believe that we have some kind of God-given right to infinite bandwidth. Bandwidth is limited, and perhaps a reasonable, and let me stress reasonable surcharge for excess bandwidth usage would be a good thing, in that it would force us to use our bandwidth more judiciously. It seems as though Napster, Gnutella, etc., while they initially fueled the explosion of broadband, are now its worst enemy as they tend to be very wasteful of resources, though they certainly have legitimate uses. Oh, the irony...
"Would you support a bill that would allow taxpayers to voluntarily contribute money to NASA?"
Is this a joke? What would that accomplish? Does anyone really expect, come tax time, Americans to open their checkbooks and start shelling out the money for NASA, let alone at a rate which would add up to billions of dollars? You must be kidding me...
I think not, a friend of mine misspelled "public" in a history paper once. The spell-checker didn't seem to mind, although his teacher didn't think it was too funny... ;-)
Yeah, like I said I'm still a fan of FreeBSD. I just really, really wish that it had the filesystem support that Linux does...I really miss that when I'm working with FreeBSD machines. In particular, I heard something about someone working on journaling for UFS (has this come to fruition yet? haven't used FreeBSD in a while...)--this would be an awesome improvement to FreeBSD. Running fsck can really suck... ;-)
Thanks for the info--it sounds pretty capable. I especially like the idea of a dialog-based interface for configuring stuff...sounds useful for those users who are not as familiar with the innards of an OS or don't have the time/inclination for that type of stuff. I've always thought Gentoo should have a tool like that.
How do the source and configuration tools work? I really am curious, it's just that there's no explanation of how anything works, how the install progresses, etc. I loved FreeBSD, then was wowed by Gentoo when I saw how superior (at least in my opinion) their system management tools are. It could be that Lunar Linux is even better still, but I have no way of comparing the two since there's no description that I can find of how Lunar Linux manages the preinstallation, installation, and postinstallation management aspects of the OS.
To synchronize your portage tree with the official Gentoo one:
/etc, does not by default overwrite any config files--so you'll never lose your favorite config file in /etc again. Instead, it write the file to the same directory, with the prefix like ._cfg0000_ for instance.
;-)
# emerge rsync
To install GNOME:
# emerge gnome
To update all of your packages to new versions:
# emerge -u world
To remove all of the *old* versions of the packages updated by the last command, since they are protected until "cleaned" out:
# emerge -c world
To forcibly remove a package:
# emerge -C $PACKAGE_NAME
To "pretend" you're running a given command:
# emerge -p$ADDITIONAL_FLAGS $PACKAGE_NAME
**
There are far too many conveniences to list them all so far as emerge goes, but those are just a few. Other conveniences:
The portage system (emerge, ebuild, etc.), when installing files to
Updating your default runlevel is amazingly easy:
# rc-update del telnetd default
# rc-update add sshd default
Each rc-script has a "dependency" function which relates it to other rc-scripts (for instance, "checkroot" must run *before* all other rc-scripts; or the mail server courier must run *after* all its components).
The default Gentoo kernel is based off the Alan Cox tree, and thus includes support for things like xfs, jfs, grsecurity, etc. Very convenient, and I have to say that xfs rocks.
The list of cool features like these just goes on and on...if you go to the Gentoo website there'll be plenty of info there.
I'm still a fan of FreeBSD, but I have to say that Gentoo's
Well, one could argue that Redhat is a "source-based" distribution since they publish all the source, but it would certainly be a chore to build & install Redhat from the source RPMs!
Anyone know how Lunar Linux works, what their philosophy behind it is, etc.? Their webpage is less than informative (other than to say that it is a "source distribution"--which can be taken a lot of different ways)...the closest thing I saw to "information" was a comment which said that it was "better than Gentoo in some ways", but never specified how or why.
Anyone used it and knows how it works, etc.? Those of us who've never heard of it would appreciate a little information before wiping our hard drives...
Gentoo would suit you very well...when I was first introduced to FreeBSD I just loved the ports system. Then I tried Gentoo and realized how much better the Gentoo scheme (known as "portage") is...not to mention the way they handle runlevels/init scripts, config files in /etc, and just about everything else. Hands down the most coherent, logical system I've ever seen.
Ok, that makes more sense--I hadn't considered faulty equipment (obviously you don't want stuff freezing onto the fuselage ;-) ). Once you've got the stuff frozen on there, however, it is not at all surprising that it would fall off in a chunk--it was probably daylight and as the plane was descending the metal, likely being warmer than the rest of the surface (esp. considering evaporation/sublimation), rose above the melting point of the solid and caused the whole thing to come crashing off. Of course, the odds of it landing in someone's bathroom are still pretty slim... ;-)
*laugh* Right...not sure what universe I was coming from with regards to that comment. Just a *little* bit of a difference between "constant acceleration" and "constant terminal velocity". ;-)
> I thought that "terminal velocity" is the same
> for all objects, no matter size/weight
> (disregarding wind friction for size; and since
> these are somewhat round...).
As you say, terminal velocity *is* the same for all objects in the absence of friction. However, friction is a pretty important factor, and cannot be disregarded.
Once you get more than 50-100km away from a NEXRAD system, the resolution is pretty bad--basically it's averaging everything over a roughly 1km^3 volume, and even more than that the farther away you go.
The ice structure is a good thought, since if these things exist there must be some way they stay up in the air while forming. Never heard of any atmospheric ice structure that would fit the bill, though, but who knows...
> What all of that basically amounts to is...well...that dog don't hunt. The only realistic
> source of that kind of thing is water being dumped by airplanes at high enough altitudes so
> that it has time to freeze before reaching the surface. A chunk of ice that big should be easy to
> save long enough to be examined; they should give a few samples to NOAA, say, and let them inspect
> it.
Agreed. I'd like to see some hard evidence of this given our present knowledge of precipitation formation and cloud physics (which admittedly is pretty limited, but certainly precludes events of these kinds).
The problem I have with the airplane theory, however, is that the instant that the water is ejected from the plane it will break up into thousands upon thousands of tiny drops which will instantly freeze at that altitude. A large mass of water would not freeze instantly due to its large heat capacity, yet at the same time it will not remain together due to the various forces yanking it apart (esp. friction).
So in other words our "megacryometeor" would (for it to form in the 4-9km AGL range) have to grow from a infinitesimally small nucleus (since allegedly the composition of these things is similar to the composition of rain) to the size of at least a basketball before it strikes the earth. Keep in mind that it can't simply start out as a mass of water the size of a basketball (as mentioned earlier), and thus must grow slowly enough to have all the water freeze/vapor condense onto it without losing it all, and yet somehow stay up in the air without the support of a strong upward current of air to balance out its terminal velocity, which will be on the order of 50 m/s (110mph) by the time it is the size of a baseball, let alone a basketball or more. Note that these upward currents of this magnitude occur *only* in the presence of strong thunderstorms.
The only thing I can think of that might remotely be able to do something like this is a very strong jet stream placed favorably next to a mountain range. You could then possibly have very strong vertical winds (this is very favorable for cloud formation, however) up to and even exceeding 50 m/s, and under *just* the right conditions you could probably grow a chunk of ice from nothing other than vapor over a long period of time...however we're talking about growth rates on the order of days and even weeks in the absence of clouds for a chunk of ice of any reasonable size. Thus even this highly idealized setup is not realistic, as it would have to persist steady-state for weeks.
In short, there could be some really weird process out there we haven't the foggiest notion about that is causing this, but I'll believe it when I see it...
True, I was referring only to the models themselves. Post-processing, though CPU-intensive, pales in comparison to actually running the model. Thus the post-processing software is not written in fortran. BTW, any idea when they're going to finish porting AWIPS to Linux?
I am well aware of the meaning of the name, and simply use caps for the sake of consistency. Thanks for pointing that out, though, as I'm sure there are many others who may be wondering just what the three letters ETA stand for besides the obvious... ;-)
> Some time ago I found a big bug that when removed increased the rain in some satellite data by 30% ! Guess how it was handled ? "Put it back in !"
Yeah, some of the older models especially are terribly maintained and full of all kind of cruft and crap. I could be wrong, but I imagine the reason you were told to put the 30% bug back in is because that's the way the model has been for eons, and the forecasters that use the model likely know its biases in a qualitative sense. Also, if they're attempting to do any sort of long-term statistical analyses using the model, altering it in such a way would render those analyses useless, whereas leaving the bug in means that the comparisons can still be made, since they are all biased one way or another. Thus removing the bug, though obviously the correct thing to do from a programming standpoint, might actually cause more trouble and confusion than it's worth, especially if the model is to be retired soon.
Is there *that* much difference between DOS16 and Win32? ;-)
Unfortunately, watcom has only a f77 compiler, and it's for windows at that.
> Most of the models were all written during the 60's & 70's...
Not true, most atmospheric models have been written since 1980, although some of them were an extension of earlier models written in the 60s/70s. The earliest models were written in the mid-60s, and development on them didn't really take off until the mid-late 70s and early 80s, when models began to be introduced in rapid succession. Even today many models are currently being developed anew (the WRF and ARPS, for instance, neither of which existed prior to the 90s--in fact, the 1.0 release of the WRF was in Fall 2000), and new ones are announced quite frequently. They are still all written in FORTRAN, however, as FORTRAN is the standard for large-scale scientific computing.
CAM2 is only one of many atmospheric models using FORTRAN, as mentioned here.
Speaking of meteorological programming, ALL the major atmospheric models are written in FORTRAN. The ETA, AVN, NGM, MM5, WRF, and scores of lesser-known models...all of them written in FORTRAN (most of them FORTRAN-90 now, but some of the older ones are FORTRAN-77). The MM5 & WRF may be found here and here. The source code to several others is readily available as well if you're so inclined, for instance the ETA and the ARPS. Anyone wanting to run them may do so fairly easily on a PC running Linux (any new PC will be able to run a fairly hi-res model real-time); I do so myself.
Though I agree that many things can be art, and that a great deal of art is found in the unlikeliest of places, what is the point of even having the term "art" if *anything* can be considered "art"? The word then becomes meaningless, and might as well be removed from the language.
Why is this guy still moderated at 0???