New Zealand Joins Aussie Bid For Vast Radio Telescope Array
schliz writes "A radio telescope in New Zealand has joined five in Australia to challenge Southern Africa to host the international Square Kilometer Array (SKA) in 2012. The newly connected telescope in Warkworth, New Zealand (PDF), is connected to an Australian data processing facility via a 1 Gbps network. Each telescope reportedly produces up to 1 Tb of data per hour of observation. IBM expects the whole of the SKA to produce an exabyte of data per day."
Gotta love the creativity that goes into these names. Too bad they stopped with the previous naming pattern, I was waiting for the "Fucking Enormously Huge Array".
I work on millimeter- and submillmeter-wave frequency radio telescopes in Arizona that occasionally do VLBI runs at 1.3mm (230 GHz). We don't have anything like a 1Gbps data link; it's more like 10 Mbps. It's hard to get the phone company to install a fiber cable run up a mountain.
Our VLBI data are stored on hard disks at a rate of ~1 Gbps, then correlated later on some big computer back east. We have to wait days to learn if interference fringes were detected!
The determined Real Programmer can write Fortran programs in any language.
Think twice, New Zealand. If the aliens start beaming you back porn, then Australia will have to filter your radio telescopes.
Well, that's partially true. The online/realtime processing doesn't store the data. We have the data coming in at about 150 Gbit/s into the BlueGene supercomputer (34 TFlop), which does initial realtime processing and writes resulting files at about 50 Gbit/s on a roughly 1 Petabyte intermediate storage system for offline processing. From there it gets initial processing/calibration and a factor of 16 to 64 reduction in size on the offline processing cluster (about 200 8core machines). Also some inspection of the data is done for quality assessment, and sky images are made. If the quality is good, the resulting data and images are moved to out 4+ Petabyte long term archive, where further processing can also be done to achieve publication ready results.
On average we are expected to be producing about 20 TB/hour raw data, and about 700 GB/hour data that gets archived (life time of more than a week).
As I speak we are 2 weeks away from the system being opened by the Dutch Queen, and we are operating on about half the above numbers with about 25 antenna fields online. By the end of 2010 we should be operating at full strength. Also, those numbers are from the top of my head, so I might be off a bit here and there.
LOFAR is seen as a precursor to SKA, the first of a new generation of telescopes based much more on software and firmware and cheap off the shelve instead of expensive dishes and custom DSP hardware. After the various VLBI efforts (Including JIVE, which is housed in the same building as ASTRON and LOFAR), LOFAR now is the biggest telescope on Earth.
http://www.astron.nl/dailyimage/main.php?date=20100421
http://www.astron.nl/dailyimage/main.php?date=20100125
There are some rather nice images starting to come out of the system, if you're a radioastronomer, but most are under embargo until the opening. I can show you this older image:
http://www.astron.nl/dailyimage/main.php?date=20100208
When mentioning SKA precursors, it's also noteworthy to mention that the 14 25meter dish WSRT array will be upgraded with focal plane arrays over the next two years, and the EMBRACE test bed has almost finished building it's first three stations.
http://www.astron.nl/dailyimage/main.php?date=20070801
http://www.astron.nl/dailyimage/main.php?date=20091012
RogerWilco the Adventurous Janitor