16 Years of GPS Space Weather Data Made Publicly Available (sciencemag.org)

← Back to Stories (view on slashdot.org)

16 Years of GPS Space Weather Data Made Publicly Available (sciencemag.org)

Posted by EditorDavid on Saturday February 4, 2017 @09:34AM from the new-from-New-Mexico dept.

"It's not often that a scientific discipline gains a 23-satellite constellation overnight," reports Science magazine, describing 16 years worth of radiation measurements from GPS satellites finally released by Los Alamos National Lab. "Although billions of people globally use data from GPS satellites, they remain U.S. military assets." Scientists have long sought the data generated by sensors used to monitor the status of the satellites, which operate in the heavy radiation of medium-Earth orbit and can be vulnerable to solar storms. But few have been allowed to tap this resource... That attitude changed in October 2016, when the outgoing Obama administration issued an executive order aimed at preparing the country for extreme space weather. Such bursts in charged particles, originating in a solar flare or coronal mass ejection, could disable the electrical power grid or divert flights away from the Arctic, where radiation exposure is heightened. The GPS data, which dates from December 2000, fill a hole in studies of space weather, the complex interplay of Earth's magnetic field with bombarding radiation from cosmic rays and the sun.

52 comments

Min score:

Reason:

Sort:

Sure by Mikkeles · 2017-02-04 09:37 · Score: 2

Like how much weather is there in space?

--
Great minds think alike; fools seldom differ.
1. Re:Sure by Anonymous Coward · 2017-02-04 10:58 · Score: 0
  
  Ion storms.
2. Re:Sure by Anonymous Coward · 2017-02-04 11:23 · Score: 1
  
  How's the weather? See for yourself!
  https://soho.nascom.nasa.gov/spaceweather/
  http://spaceweathernews.com/
  https://www.spaceweatherlive.com/
  http://www.swpc.noaa.gov/communities/space-weather-enthusiasts
  http://www.swpc.noaa.gov/products/alerts-watches-and-warnings
  http://www.sws.bom.gov.au/Space_Weather
  http://spaceweather.gc.ca/index-en.php
3. Re:Sure by Anonymous Coward · 2017-02-04 20:50 · Score: 0
  
  Ever heard of "analogy"?
4. Re:Sure by Mikkeles · 2017-02-05 01:32 · Score: 1
  
  Q: If you call a tail a leg, how many legs does a sheep have?
  A: Four. Calling a tail a leg doesn't make it one.
  
  --
  Great minds think alike; fools seldom differ.
5. Re:Sure by Capsaicin · 2017-02-05 15:52 · Score: 1
  
  Q: If you call a tail a leg, how many legs does a sheep have?
  Let A = 1;
  A: Four. Calling a tail a leg doesn't make it one.
  A == 1;
  => F
  Really?!
  IF we call a sheep's tail a 'leg,' a sheep has One leg! ... Unless, of course, we also call other body part 'legs' (eg. those appendages which hold the sheep off the ground which, outside your conditional, we traditionally call 'legs')
  There is no property inherent to an object which binds it to the English language noun by which it is called. The meaning of 'leg,' is given by the calling of objects by that label. Thus if we call X, by the name Y, the name Y refers to X.
  
  --
  Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
Now, because by ChrisMaple · 2017-02-04 09:40 · Score: 1

It takes a while to falsify the data.

--
Contribute to civilization: ari.aynrand.org/donate
1. Re:Now, because by Lumpy · 2017-02-04 09:44 · Score: 2, Funny
  
  Because we all know there is no sun, all the light we get is from fearless leaders hair.
  Hail the orange one! for he is the light of our day! If you say otherwise it's HERESEY!
  
  --
  Do not look at laser with remaining good eye.
2. Re:Now, because by PopeRatzo · 2017-02-04 10:14 · Score: 5, Funny
  
  It takes a while to falsify the data.
  They also covered up the fact that the Bowling Green Massacre of 2011 was a false flag operation and thousands of Muslims in New Jersey were in the streets cheering.
  
  --
  You are welcome on my lawn.
3. Re:Now, because by Anonymous Coward · 2017-02-04 10:58 · Score: 0
  
  Trump will release data to Russia, and all of the liberals cheering Obama for his "openness" will suddenly turn into vicious savages.
  Latest labor reports indicate that 90 million Americans are STILL out of work. Trump will fix that.
  naw, because they're LOSERS.
4. Re:Now, because by BenBoy · 2017-02-04 12:28 · Score: 2
  
  Hail the orange one! for he is the light of our day! If you say otherwise it's HERESEY!
  Hail the orange one! for he is the light of our day! If you say otherwise it's HAIR-ESY! (FTFY)
5. Re:Now, because by Anonymous Coward · 2017-02-04 13:47 · Score: 0
  
  They've actually become quite good at it since the 1980's or so. Take a look at NASA temperature data. Completely falsified by James Hansen. Reference (redpill): https://www.youtube.com/watch?v=Gh-DNNIUjKU
6. Re:Now, because by Anonymous Coward · 2017-02-04 13:51 · Score: 2, Informative
  
  LOL
  However, the 90 million number is padded, since this number includes a lot of Americans who wouldn’t be expected to be working. Specifically:
  People age 16 to 17, who likely are in high school: 9 million
  People who are enrolled in either two- or four-year colleges: 21 million
  People age 65 and older, who have reached retirement age: 40 million people
  That means 20 million people are of normal working age, not in college and not working. That’s less than one-quarter the amount repeatedly cited in the blogosphere.
  http://www.politifact.com/truth-o-meter/statements/2013/jul/30/blog-posting/are-90-million-americans-not-working-or-looking-wo/
7. Re:Now, because by Ol+Olsoc · 2017-02-04 14:18 · Score: 1
  
  It takes a while to falsify the data.
  They also covered up the fact that the Bowling Green Massacre of 2011 was a false flag operation and thousands of Muslims in New Jersey were in the streets cheering.
  Well I for one, will not stand by while the countless familie killed by these evil people are not helped.
  I've started a GoFundMe and Patreon page to collect money to help these poor people. And if they don't claim it..........
  
  --
  The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
8. Re:Now, because by Anonymous Coward · 2017-02-05 01:26 · Score: 0
  
  Hairesy [hair-uh-see] noun
  1. The act of speaking or acting contrary to the wishes of our glorious leader, Ubermenschfuhrer Herr Drumpf.
  2. Making inferences regarding hand size.
9. Re:Now, because by laurencetux · 2017-02-05 08:48 · Score: 1
  
  okay so now add the folks that
  1 are in the "working" age range but have given up on being able to work
  2 have cooked records (or damaged themselves) to get on the disabled rolls
  3 add back 50% of the folks 65 or older that still need to work because they got err Shafted by their employers when it came time to pay out retirement
  4 also add back 25% of the kids (because they need to earn something to survive due to mom/dad not earning enough)
Get it now by Anonymous Coward · 2017-02-04 09:40 · Score: 0

Before Drumpf shuts this down!!
1. Re:Get it now by cusco · 2017-02-05 05:42 · Score: 1
  
  NASA did the same thing several times during the Shrub mAdministration. At one point they were told to turn off some aging space probe, Pioneer I think. Not "stop monitoring" but "turn it off so that no one else can monitor it" (not that anyone else has the equipment), and were openly annoyed that the ability to do so didn't exist. Repeatedly NASA was instructed to **destroy** data, which they did only after handing a copy over to other parties (generally the Planetary Society). In at least one case staff was instructed that it would be considered cause for dismissal to allow distribution of the data (the early Pioneer data tapes), which they defied.
  I'll never understand the conservative mindset. "If I can't make use of it then no one else will be allowed to." I just don't get it.
  
  --
  "Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
Needs Compressed Download by MatthiasF · 2017-02-04 10:04 · Score: 2

They present the data as 23 folders with over 100 ASCII at 5 MBs each. Downloading each one by one is annoying but not a show stopper, but why not compress?

I downloaded one of the files and used 7Zip to throw it into a Z archive. The size went from 5 MB down to 500 KB. I uncompressed to make sure it wasn't a fluke but was 100% accurate.

Shouldn't we expect a little more from the country's best and brightest? I mean, come on...
1. Re:Needs Compressed Download by MichaelSmith · 2017-02-04 10:10 · Score: 1
  
  wget FTW
  http compresses anyway.
  
  --
  http://michaelsmith.id.au
2. Re:Needs Compressed Download by MatthiasF · 2017-02-04 10:21 · Score: 2
  
  The website does not seem to be http gzipping the files. I think it's just a pass-through portal page for their FTP.
  
  But gzip would help. Just tested, it brought the files down to 600-700KB from 5-6 MBs.
  
  Still, it's over 8000 files. Could have compressed each directory to a single file and made it easier on everyone.
3. Re:Needs Compressed Download by mmell · 2017-02-04 10:32 · Score: 2
  
  Why compress? HTTP can already compress the data stream and reliably decompress it on the receiving end. If they compressed it, they'd have to hear from everyone that had difficulty decompressing the data (because of the compression algorithm chosen, the configuration of that algorithm, possible incompatibility caused by people using old or broken decompression clients, etc.).
  They made the data available. Now that you've downloaded one of the files (for some reason other than to load down their server and amuse yourself, I presume?), I'm sure you'll find the data so useful as to make compressing it yourself a welcome task rather than a burden.
4. Re:Needs Compressed Download by MichaelSmith · 2017-02-04 10:32 · Score: 1
  
  Do us all a favor. Tar, bzip2, and torrent.
  
  --
  http://michaelsmith.id.au
5. Re:Needs Compressed Download by MatthiasF · 2017-02-04 10:36 · Score: 1
  
  As I mentioned above, there is no HTTP compression and it's over 8000 files at over 60 GB uncompressed. Had they compressed it, I am certain it would be less than 1 GB based off the tests I mentioned.
  
  Imagine all the bandwidth being wasted. I think it's incredulous to release a huge treasure trove of data this big and not think of compressing.
6. Re:Needs Compressed Download by MatthiasF · 2017-02-04 10:39 · Score: 1
  
  At the rate it's going at the moment, it'll be done somewhere in the next 4 hours.
  
  When, as I mentioned, it could have been done downloading before I even posted the first comment above had they used any form of compression.
  
  But I'll throw in into an S3 bucket after it's all down and compressed for anyone interested.
7. Re:Needs Compressed Download by MichaelSmith · 2017-02-04 11:32 · Score: 1
  
  Thanks I'll check back in a couple of days ;)
  
  --
  http://michaelsmith.id.au
8. Re:Needs Compressed Download by Ol+Olsoc · 2017-02-04 14:21 · Score: 1
  
  Do us all a favor. Tar, bzip2, and torrent.
  Y'all would bitch and moan about free beer.
  
  --
  The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
9. Re:Needs Compressed Download by Ol+Olsoc · 2017-02-04 14:31 · Score: 1
  
  Imagine all the bandwidth being wasted. I think it's incredulous to release a huge treasure trove of data this big and not think of compressing.
  Good heavens - if it is such a hardship, don't download it.
  Regardless, I'm not certain if it is a good idea to download this or not. It's probably NASA trying to get as much of their data out there before they are cleansed of scientists who do not straighten up and fly right. Which is apparently what is happening with the AGW data as well. This stuff might be illegal soon.
  
  --
  The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
10. Re:Needs Compressed Download by MatthiasF · 2017-02-04 15:19 · Score: 1
  
  Finally done and posted links above, but here they are for your convenience.
  
  GPS+Energetic+Charged+Particle+2016-12-08+63GBs.zip.001
  
  GPS+Energetic+Charged+Particle+2016-12-08+63GBs.zip.002
11. Re:Needs Compressed Download by MatthiasF · 2017-02-04 15:28 · Score: 1
  
  Which is why I want to download it and archive it. If there is a threat that scientific data is going to be attacked and purged, we should all be saving a copy just in case.
12. Re:Needs Compressed Download by cusco · 2017-02-05 05:21 · Score: 1
  
  If it was PBR or Coors, sure.
  
  --
  "Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
13. Re:Needs Compressed Download by Ol+Olsoc · 2017-02-05 15:37 · Score: 1
  
  If it was PBR or Coors, sure.
  Sir! I said Beer, not PeeWaa!
  
  --
  The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
Finney faked his own death by Anonymous Coward · 2017-02-04 10:32 · Score: 0

before the ion pod was jettisoned.
why withhold it by EE101 · 2017-02-04 10:48 · Score: 0

Why was the data withheld for 17 years in the first place?
1. Re:why withhold it by Anonymous Coward · 2017-02-04 11:04 · Score: 0
  
  The data was compiled at LANL from observations taken by GPS satellites, which are US military assets. It's likely that much of the information was classified until late last year.
2. Re:why withhold it by cusco · 2017-02-05 05:25 · Score: 1
  
  It's the frelling military. If it wouldn't benefit whatever corporate board they're going to sit on after they retire the generals won't allow anything that might serve the public good to escape.
  
  --
  "Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
"Obama administration issued an executive order.." by turkeydance · 2017-02-04 11:31 · Score: 1

Mom! he started it!
Please please..... by Anonymous Coward · 2017-02-04 13:11 · Score: 0

Copy early, copy fast, share widely. Quick. You know who is just a few hours away from getting up from his nap, when he hears about this and someone tells him what do, it will be gone pronto.
1. Re:Please please..... by Anonymous Coward · 2017-02-04 20:17 · Score: 0
  
  Dump Trump. You know it makes sense.
  Soon the only truth in the USA will be (Apart from his Diktats) what you get at the supermarket checkouts.
  Make USA even dumber Again.
Compressed Download Available by MatthiasF · 2017-02-04 15:19 · Score: 2

I downloaded the entire dataset folder and compressed into a two-part ZIP for easy download and/or archiving to DVD.

Files are available on Amazon S3 here:

GPS+Energetic+Charged+Particle+2016-12-08+63GBs.zip.001

GPS+Energetic+Charged+Particle+2016-12-08+63GBs.zip.002

Keep in mind that the total folder is over 63 GBs and 8000 files.
1. Re:Compressed Download Available by ColdWetDog · 2017-02-05 05:35 · Score: 1
  
  Sure you did.
  How much do you want to wager that it now is ten thousand copies of Goats.exe?
  
  --
  Faster! Faster! Faster would be better!
2. Re:Compressed Download Available by MatthiasF · 2017-02-05 06:36 · Score: 1
  
  You would lose the wager. Because the files are on my personal AWS account's S3 bucket and can be easily traced back to me by law enforcement if it has anything illegal or harming inside.
  
  I get the joke but the zips are a serious effort to help get the data shared. The original source took almost 6 hours to download fully because did they not compress or compact the files in any way. I'm just trying to save others interested that effort (and their bandwidth budgets for people with caps).
3. Re:Compressed Download Available by N7DR · 2017-02-13 11:13 · Score: 1
  
  I downloaded the entire dataset folder and compressed into a two-part ZIP for easy download and/or archiving to DVD.
  And also available as a single xz-compressed tarfile: http://drevans.blog.enginehous.... (This file is about 1.3 GB smaller than the sum of the two zip files.)
4. Re:Compressed Download Available by MatthiasF · 2017-02-14 07:23 · Score: 1
  
  Nice. Thank you for posting.
This is great for time travelers by no1nose · 2017-02-04 17:22 · Score: 1

But what we could really use is the next 16 years of space weather.
Broaden your definition of "weather" by laurencetux · 2017-02-05 08:42 · Score: 1

underwater has its own kind of "weather" so why wouldn't above the atmosphere have "weather" (used as shorthand for a measure of the ambient conditions in X).
So now the raw data will not be available? by Anonymous Coward · 2017-02-05 08:45 · Score: 0

So now the raw data will not be available?
Why do you hate science?