How a Frozen Neutrino Observatory Grapples With Staggering Amounts of Data (vice.com)

citadrianne writes: Deep beneath the Antarctic ice sheet, sensors buried in a billion tons of ice—a cubic kilometer of frozen H2O—are searching for neutrinos. "We collect...one neutrino from the atmosphere every ~10 minutes that we sort of care about, and one neutrino per month that comes from an astrophysical sources that we care about a very great deal," researcher Nathan Whitehorn said. "Each particle interaction takes about 4 microseconds, so we have to sift through data to find the 50 microseconds a year of data we actually care about." Computing facilities manager Gonzalo Merino added, "If the filtered data from the Pole amounts to ~36TB/year, the processed data amounts to near 100TB/year." Because IceCube can't see satellites in geosynchronous orbit from the pole, internet coverage only lasts for six hours a day, Whitehorn explained. The raw data is stored on tape at the pole, and a 400-core cluster makes a first pass at the data to cut it down to around 100GB/day. A 4000-CPU dedicated local cluster crunches the numbers. Their storage system has to handle typical loads of "1-5GB/sec of sustained transfer levels, with thousands of connections in parallel," Merino explained.

Comments

Attended talk by one of the professors

[jfdavis668]

We had one of the professors who work on the project from F&M university give a talk on the project to our local astronomy club. The amount of work required to build that thing was amazing. They are using the Earth to filter out local sources of interference so that they can find true reactions caused by neutrinos. The Earth filters out other man-made particles. They can spot neutrinos from super novas coming through the Earth.

Re:Perhaps they could buy a station wagon and

[JoshuaZ]

Yes and no. There is some advantage to getting close to real time data: there's a a Supernova Early Warning System http://snews.bnl.gov/. This isn't a safety issue, but rather an astronomy issue.

Detectors like IceCube can be used to actually detect the neutrinos from a supernova before the supernova's light reaches Earth. This isn't due to the erroneous claim from a few years ago that neutrinos travel faster than light, but rather because when a supernova occurs, the light from the core of the star takes multiple hours to get out of the core because of all the mass in the way, while the neutrinos aren't slowed down by this almost at all. This means that the neutrinos effectively get a few hours head start on the light- since they are traveling so close to the speed of light, they get to keep almost all this head start by the time they reach Earth. In the case of SN 1987A https://en.wikipedia.org/wiki/SN_1987A, a supernova in 1987 which was close enough that we could detect the flood of neutrinos, the neutrinos did as predicted arrive a few hours before the light. This means we can if we detect a neutrino burst and can get its directional data (which IceCube can approximately do) then we can point our telescopes at a supernova *before the light arrives at Earth* which means we'll get to see the very beginning of the supernova and hopefully get a much better understanding.

In order to do this you have to do at least some of your processing in at least close to real time as you can. This is especially important because it isn't actually easy to figure out from the neutrino burst what direction the supernova is coming from, and IceCube is one of the few detectors which gets any good directional data at all, so if this happens we want to process the data rapidly enough to get a good idea of where to look.

Holy coincidental mixed unit mess batman ..

[OzPeter]

A tonne is the SI unit for 1000 kilograms.
A ton (US) is a funny unit of measure for 2000 lbs (907kg)
A ton (Imperial) is a funny unit of emasure for 2,240 lbs (1,016 kg)
Thus a tonne is about 1.1 tons (US), and 0.98 tons (Imperial)
A cubic kilometer of water is 1 billion (1E9) tonnes
But water expands when it is frozen by about 9%
So a cubic kilometer of ice would be about 1E9 tons (US)

Thus the statement in TFS

a billion tons of ice—a cubic kilometer of frozen H2O

while numerically about correct is a hell of a mess of mixed units.