Slashdot Mirror
SKA Telescope To Provide a Billion PCs Worth of Processing
Sharky2009 writes "IBM is researching an exaflop machine with the processing power of about one billion PCs. The machine will be used to help process the Exabyte of data per day expected to flow off the Square Kilometre Array (SKA) telescope project. The company is also researching solid state storage technology called 'racetrack memory' which is much faster and denser than flash and may hold the secret to storing the data from the SKA. The story also says that the SKA is unlikely to use grid computing or a cloud-based approach to processing the telescope data due to challenge in transferring so much data (about one thousand million 1Gb memory sticks each day)."
Could we get that in LoC's? Also, could we stick to the standard "one million thousands" unit, please?
A slashdotter who didn't build his own computer is like a Jedi who didn't build his own lightsaber.
Let's hear it for Reel Big Fish and the Pietasters! Is there a Reggae telescope?
Free Martian Whores!
SKA telescope? Madness!
Seriously, how is a PC a unit of processing ability? And one thousand million GB sticks is an Exabyte (hence the name). Perhaps you can just say 10^18 bytes. This is slashdot, not msnbc.
"We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
Hey, IBM, you dropped your telescope.
Someone should pick it up, pick it up, pick it up, pick it up.
In Capitalist America, bank robs you!
Come on... the moment IBM makes a computer with a billion cores, both Microsoft and Linux will be salivating at the change to get -something- to run on them. I mean, what's a GB sized array just to keep track of the CPUs. Pure insanity. Any real geek would love to tackle that.
This is my sig.
Anecdotal evidence is the best evidence!
I have an 800MB HDD that still works, and up until a couple of years ago was in constant use. It was only retired because the old workhorse of a machine it was in was finally replaced. That said, I have also worked with big farms of disks and know that failures happen, and the hard drive is the second least reliable part of most computers after the fans. Anything with moving parts is going to eventually fail, there's no way around it.
I read the internet for the articles.
...run Crysis at full resolution!?
First of all, no one would be using manual storage to transfer the data.
Just throw up some numbers that makes sense to us. Like 99,420.5393 gigabit/second.
Most large ISPs use OC-192 as the backbones of their infrastructure. You'd need more than 10,200 of those to handle that data load, and that's ignoring the overhead.
Or to put it into numbers that the RIAA can understand: 1.5707309 * 10^9 music CDs every single day.
At 15 pieces of music per CD and $80,000/song that's $1.88 * 10^15 dollars/day flowing through that network. That's 632 times larger than the US federal budget for 2008.
No wonder the music industry is in trouble!
Imagine a beowulf of those
omglol!
And I have an uncle who smoked a pack a day for 40 years and never got lung cancer.
Astronomy comes up with pretty pictures. Other areas not so much so. So what would you rather have you tax money go to... Pretty pictures. Or columns of crunched numbers.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Where are the obligatory "beowulfcluster" tags and jokes already?
Sheesh, the standards around here sure are slipping.
So if I did my math correctly they're saying if they did distributed computing they'd need to transfer data at a rate of 92.5 Tbps.
I'm assuming a 1 Gb memory stick is actually 1 GB though...
They're gonna need a LOT of pigeons.
Maybe they could construct a really fast computer to process the data in real time so they wouldn't have to store it all. They might even release a press release about it.
From the FAQ:
How far can this telescope see?
A: ONE STEP BEYOND!
nonsig. unsig. desig.
Some people might be interested in knowing where all this data comes from. There's a rule of thumb in astronomy that the angular resolution of your images is the wavelength of the radiation you're receive divided by the diameter of your telescope. Radio wavelengths are pretty long (up to tens of meters), so you need really big telescopes, which you get by scattering lots of little telescopes all over the place and then looking at the how the phase of the incoming radiation shifts based on location. So what you do is you sample the voltage of each antenna with 1 or 2 bits of resolution at the Nyquist frequency. So for 100Mhz radio waves you sample at 200MHz. That's 50MB/s for a single antenna. The SKA will likely have tens of thousands of little antennas scattered all over the place. So say 50MB/s times 20,000 antennas = 1TB/s = 100 petabytes/day, which is about what the summary says.
Now, it's not quite as bad as it looks. You don't have to pipe all this data to a central point to analyze it. You can take a small group of antennas and just look at the correlations between those, combine the data from that group and send the combined data to a second level of correlators, which takes data from a set of small groups, and so on, in a hierarchical fashion. You lose some information this way, but you get most of it, and the only wa to get all the information out of the data would be to bring it all to a central processing location so that data from all antennas could be compared to that of all other antennas, which is O(N^2) in the number of antennas and obviously infeasible for a telescope like the SKA. Even as it is, the system of hierarchical data collection is really pushing Moore's law, as the article shows.
average english football field of 110 x 67.5 meters (74,250,000 sq cm)
1,000,000,000 memory sticks of 4cm x 1.5cm each (6,000,000,000 sq cm)
About 80 football fields.
the SKA is unlikely to use ... a cloud-based approach
Well, duh. You can't see anything when it's cloudy.
Have you read my blog lately?