Slashdot Mirror
How a Frozen Neutrino Observatory Grapples With Staggering Amounts of Data (vice.com)
citadrianne writes: Deep beneath the Antarctic ice sheet, sensors buried in a billion tons of ice—a cubic kilometer of frozen H2O—are searching for neutrinos. "We collect...one neutrino from the atmosphere every ~10 minutes that we sort of care about, and one neutrino per month that comes from an astrophysical sources that we care about a very great deal," researcher Nathan Whitehorn said. "Each particle interaction takes about 4 microseconds, so we have to sift through data to find the 50 microseconds a year of data we actually care about." Computing facilities manager Gonzalo Merino added, "If the filtered data from the Pole amounts to ~36TB/year, the processed data amounts to near 100TB/year." Because IceCube can't see satellites in geosynchronous orbit from the pole, internet coverage only lasts for six hours a day, Whitehorn explained. The raw data is stored on tape at the pole, and a 400-core cluster makes a first pass at the data to cut it down to around 100GB/day. A 4000-CPU dedicated local cluster crunches the numbers. Their storage system has to handle typical loads of "1-5GB/sec of sustained transfer levels, with thousands of connections in parallel," Merino explained.
7-zip. Works for me.
We had one of the professors who work on the project from F&M university give a talk on the project to our local astronomy club. The amount of work required to build that thing was amazing. They are using the Earth to filter out local sources of interference so that they can find true reactions caused by neutrinos. The Earth filters out other man-made particles. They can spot neutrinos from super novas coming through the Earth.
Perhaps they could buy a station wagon, load it up with tapes and send it with the next dogsled. (I kid.)
It's not like they are using real-time data from this thing - it's more like a traditional particle smashing experiment where most of the analysis is done months and years after the data is collected.
37Mbps via satellite is pretty impressive. I'd hate to pay that bill.
If there are 6 events every minute, and each last 4 microseconds, then that is 131,400 events to review per year. If you multiply all those microseconds and events you get 525,600 microseconds of data, or about .5 seconds worth of neutrinos to review per year. What the heck is this guy so upset about! They must get really bored down there in the Antarctic.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
It's the 400 core cpu cluster right at the south pole.
"We collect [...] one neutrino [...] every ~10 minutes," researcher Nathan Whitehorn said.
How did he pronounce "~"?
systemd is Roko's Basilisk.
Holy shit. This is awesome. The article contains virtually nothing beyond some large statistics and a picture of an ice hole and some cable trays.
What do they do for energy down there?
A tonne is the SI unit for 1000 kilograms.
A ton (US) is a funny unit of measure for 2000 lbs (907kg)
A ton (Imperial) is a funny unit of emasure for 2,240 lbs (1,016 kg)
Thus a tonne is about 1.1 tons (US), and 0.98 tons (Imperial)
A cubic kilometer of water is 1 billion (1E9) tonnes
But water expands when it is frozen by about 9%
So a cubic kilometer of ice would be about 1E9 tons (US)
Thus the statement in TFS
a billion tons of ice—a cubic kilometer of frozen H2O
while numerically about correct is a hell of a mess of mixed units.
I am Slashdot. Are you Slashdot as well?
If they are all Pentium I, God help us for we are all doomed.
Well, there's spam egg sausage and spam, that's not got much spam in it.
No. They get a neutrino every 4 microseconds. They get one neutrino of some interest every 10 minutes, so 1 out of 150M neutrino hits is of some interest. And the really interest stuff is 50 microseconds worth per year (12 or 13 neutrino hits out of 7,884×10)
Why are they processing it in place if the processed data requires more bandwidth and they have a transmission bandwidth problem?
What did I miss?
David
The original P5 architecture used between 8 and 17W.
Although somewhat notorious at the time for counting as the first major consumer CPU that "required" active cooling, you could still get away with a big-ass heat sink.
Funny, really, how it's taken us 22 years to get back to TDPs in a range that makes passive cooling once again practical.
You have all sorts of issues with your calculation. 131,400 events per year would be 1 every 4 minutes. 6 events per minute as you state would be 6 per minute * 60 minutes * 24 hours * 365 days equaling 3,153,600 events per year.
However that doesn't really matter because 6 events don't happen a minute. They state in the summary and article that they detect 1 neutrino about every ten minutes or 6 an hour. They only sort of care about those because those aren't the ones from astrophysical sources. The ones they really want occur about once a month.
So 1 event per month * 12 months * about 4 microseconds per event = about 50 microseconds...as was stated. Worst case adding in all the ones they care much about would add about .21024 seconds to the .000050 seconds they care about.
sorry, but you're both all fucked up. They don't know *when* an event has occurred unless they sort through the data. So, they sort through 1000 ms of data per second. That filters the data down to a survivable amount of data, which is then NRT processed on the south pole to determine if it has the signature of a cosmic sourve, which generates the text message that gets sent immediately. However, this is a "quick and dirty" analysis that is often wrong, so all relevant data from the sifting gets shot up every 6 hours or so when one of the TDRS birds comes into view.
Not having read the article or not knowing anything about how an event is detected... It rather sounds if CPUs are not the best tool for the job. FPGAs should be able to run data acquisition and filtering in real time, doing most of the heavy lifting. A single FPGA (rather large FPGA like the Virtex range from Xilinx) can do thousands of multiply accumulates in parallel. GPUs like the Tesla or similar may also be a better fit.
A picture is worth exactly 1024 words.
Aren't roioom temperature neutrinos good enough?
Let it go man, let it go.
Liberty - Security - Laziness - Pick any two.
They get stale too fast.
Clearly it's because frozen neutrinos have lower kinetic energy, and are thus easier to detect.
Silly me. Being accustomed to paying a ~$200 per month for unlimited 100Mbps, I find $2,200 per month to be rather pricey for internet access. The thought that my bill could go even higher for using the connection more is just more scary.
It seems to me that, in a place like Seattle, one could get faster, lower latency, higher bandwidth wireless internet access for FAR cheaper than what you describe. Is there some specific reason for your satellite choice or is IT simply not that bright at your place?
ISDN? People still use that shit?
Never underestimate the bandwidth of Antonov 225 fully loaded with BRDVD's.
It's just the awful latency.