IBM Designing Superman Servers For World's Largest Telescope

Nerval's Lobster writes "How's this for a daunting task? By 2017, IBM must develop low-power microservers that can handle 10 times the traffic of today's Internet — and resist blowing desert sands, to boot. Sound impossible? Hopefully not. Those are the design parameters of the Square Kilometer Array (SKA) Project, the world's largest radio telescope, located in South Africa and Australia amid some of the world's most rugged terrain. It will be up to the SKA-specific business unit of South Africa's National Research Foundation, IBM, and ASTON (also known as the Netherlands Institute for Radio Astronomy) to jointly design the servers. Scientists from all three organizations will collaborate remotely and at the newly established ASTRON & IBM Center for Exascale Technology in Drenthe, the Netherlands. By peering into the furthest regions of space, the SKA project hopes to glimpse 'back in time,' where the radio waves from some of the earliest moments of the universe — before stars were formed — are still detectable. The hardware is powerful enough to pick up an airport radar on a planet 50 light-years away, according to the SKA team."

Comments

10x today's internet traffic

[Anonymous Coward]

can handle 10 times the traffic of today's Internet

Yeah, you can get something on the front page of slashdot if you use stupid, misleading metrics like this. Soulskill has his head buried in the sand.

Re:10x today's internet traffic

[c0lo]

can handle 10 times the traffic of today's Internet

Yeah, you can get something on the front page of slashdot if you use stupid, misleading metrics like this. Soulskill has his head buried in the sand.

A single computer, probably not.
Otherwise, the entire SKA will indeed produce 10 times the amount of data trafficking the today's internet.

Re:10x today's internet traffic

[Trepidity]

True, but that's getting pretty common in large-scale scientific applications these days. The LHC generates about 100 terabytes per second, for example. The numbers on the page you linked say SKA will generate "enough raw data to fill 15 million 64 GB iPods every day", which is actually an order of magnitude lower: 15 million * 64 GB = 960 PB per day. Divide that by 86400 seconds in a day, and you get about 11 TB/s.

Re:10x today's internet traffic

[c0lo]

True, but that's getting pretty common in large-scale scientific applications these days. The LHC generates about 100 terabytes per second, for example. The numbers on the page you linked say SKA will generate "enough raw data to fill 15 million 64 GB iPods every day", which is actually an order of magnitude lower: 15 million * 64 GB = 960 PB per day. Divide that by 86400 seconds in a day, and you get about 11 TB/s.

While LHC generates 10 times more data in a single experiment (usually scheduled months or years ahead), think that SKA will generate data each day every day.

Re:10x today's internet traffic

[Anonymous Coward]

The LHC only records about 25 PB a year though, as the raw data is heavily filtered by custom hardware before getting to the more off the shelf computers that record data for later use. SKA on the other hand, needs to hold on to raw data for a couple hours until a run is complete, requiring intermediate storage of data of about a PB an hour, which will then get reduced to about a 1-5 PB a day for longer term storage and analysis. The intermediate data will use conventional hardware for processing, but even ignoring that, the long term data, that which needs to be stored and distributed, will out pace LHC's year' production in about a eek. If you wanted a more apples-to-apples comparison to LHC's raw data collection, you would need to look more at the amount of raw data produced before filtered down to commodity computer hardware. And with a final goal of thousands of antennas collecting up to 30 GHz signals across nearly the full spectrum, that is a lot more than the 10 terabits/s LHC roughly generates, and the intermediate 1 PB/hr data for SKA is much more than LHC's intermediate ~ 1 TB/hr.

Re:10x today's internet traffic

[Trepidity]

Interesting numbers; thanks for the clarification! I agree that's a significantly more ambitious goal by some of those metrics. A PB/hour is indeed quite a lot of intermediate storage, and even the reduced 1-5 PB/day is more than any existing experiment.

I realize it's a lot to ask for popular science journalism, but that's one reason I'd like more specifics and precision in some of these stories. What do we mean by data being generated: where is it generated, how long is it stored for, what are its characteristics, etc. And ideally figures in units of bytes (or a multiple) rather than "iPods" or "internets" would be nice; it was sort of ridiculous that I had to multiply out what 15 million 64GB iPods hold to arrive at a real number (and even that number could be wrong if they really meant GiB rather than GB, though the actual iPod space is indeed in GB).

Re:10x today's internet traffic

[Anonymous Coward]

The reference to 15 million iPods seems to be a common PR quote used by a lot of descriptions of the project. That in the end is kind of part of the nature of pop-Sci, it is about making things relatable to those that have little understanding of the topic. And sometimes it needs to be short sound bites, as that is all the air time or attention span you get. Not to say there is not a need for more detailed information for those that want it. But if you search around, there are plenty of papers, presentations, and reports freely available online showing proposed processing setups, including more detailed estimates of data rates, etc., that should be easily understood by someone with some IT experience.

Re:10x today's internet traffic

[fyngyrz]

And with a final goal of thousands of antennas collecting up to 30 GHz signals across nearly the full spectrum

Hmmm. Sounds like a marvelous database for us SDR freaks to troll through, big chunks of spectrum at a time, eyes on the waterfall and spectral displays.

All ya need to do is create a server that will supply a file that is a chunk-o-spectrum as baseband IQ data, and you'll likely have a whole bunch of eyes on it for you. You'd certainly have mine!

Re:10x today's internet traffic

[Anonymous Coward]

I think they've used a low estimate for the SKA. SKA phase I consists of three parts: a single-pixel dish array in South Africa, a focal-plane-array dish array in Australia, and an aperture-array tile array in Australia. The second part, with the focal plane arrays, is about twice the size of the precursor instrument ASKAP. The data rate for ASKAP is:

(36 antennas) * (192 elements per antenna) * (384 MHz bandwidth) * (factor of 2 to get the Nyquist rate) * (1 byte/sample) = 5.3 TB/s

So data rate for one of three parts of SKA phase I will have a data rate of twice this, or about the 11 TB/s you calculated. SKA phase II will be a lot bigger, probably exceeding the LHC's 100 TB/s.

Re:10x today's internet traffic

[Anonymous Coward]

Arecibo produces quite a lot of data too. It's interesting, (and amazing considering when the project was started).

The article linked states this can detect an airport radar on a planet 50ly away. But realistically how much time would it take to find that result in the data retrieved. Would reading one scan be a Seti like project requiring years of signal processing on many computers?

Not that this thing won't be useful. I mean we'll get some really images out of it for sure.

Re:10x today's internet traffic

[ldobehardcore]

it is about making things relatable to those that have little understanding of the topic.

What's ironic is that 15 million iPods is no more relatable than saying 960 Petabytes. What do you imagine a pile of 15million iPods looks like? Or is it 15million retail boxes stacked in neat rectangular prisms? Or is it 15 million iPods laid end to end? Big numbers need to be expressed in ways that are easily divisible into realistic chunks people can hold in their heads. I have no idea what kind of volume this many iPods takes up, or how expensive it would be, I doubt anyone really does off the top of their head.

Re:10x today's internet traffic

[Anonymous Coward]

common

You keep using that word. I don't think it means what you think it means.

Re:10x today's internet traffic

[Anonymous Coward]

Look! Up in the sky! It's a bird... it's a plane... it's...

A telescope??

Re:10x today's internet traffic

[Anonymous Coward]

People know what a million is though. They know it is a big number, although they can probably think of bigger ones. You can find pictures of what a million of some things look like around (especially what a pile of a million dollars look like). They might not be able to know what a million iPods looks like exactly, but many people will have a rough idea of the number, as opposed to knowing what a petabyte is, (or alternatively a million gigabytes is beyond a stack of something else of similar size).

If you know of some perfect analogy that makes such numbers instantly and perfectly acceptable, I'm sure many researchers in different fields would like to hear what it is so they can use it to explain things. In the mean time, explaining such things frequently comes down to just several examples that gives a rough or brief image of what it may be referring to, while those who are able or want to more exactly to know more exactly what they mean can seek out the specifics.

26 petabytes?

[bloodhawk]

Is internet traffic really only 26 Petabytes a month, while that is a big number it sounds awefully low to me as the place I work does 15 Terabytes a month and they are little more than a miniscule pimple on face of the internet.

Re:26 petabytes?

[Anonymous Coward]

Hrm they say "today's" internet, so maybe it's 26 pedobytes a day???

Re:26 petabytes?

[iYk6]

Agreed. There are roughly 100 million internet enabled households in the United States. If each of these sent and received, on average, 1GB per month, that's 100 PB.

Re:26 petabytes?

[Anonymous Coward]

Wikipedia says 26 thousand Petabytes so maybe they missed by a bit!

Re:26 petabytes?

[Cammi]

Yep, i think they are WAGging. We do roughly 7 TB a day at work as well. I smell... BS in the article.

Re:26 petabytes?

[bloodhawk]

I would seriously doubt where I work even ranks in the top 10,000 internet content providers by volume, yet the article would suggest they would have to be one of the top providers for there numbers to be true.

Re:26 petabytes?

[c0lo]

that is a big number it sounds awefully low to me

Well, it is actually low. E.g. the entire cloud fits a single server on a cable modem (true, with lots of caching). You ask for citations? Here you go

Re:26 petabytes?

[Anonymous Coward]

only out by a factor of 1000, these articles are becoming much more accurate :-)

Re:26 petabytes?

[Anonymous Coward]

It would be allot more but ISP's in general cap the upstream bandwidth at 1mbit while 100mbit down. While flooding your basic tv service with more garbage channels. Because what do consumers want? More golf and "reality" tv channels. And there's no real competition to tell the fucking assholes to fuck off since they all have the same service and business structure reeking of cartel behavior.

Re:26 petabytes?

[Anonymous Coward]

I found his storage device.

Re:26 petabytes?

[ButchDeLoria]

A lot of people order Cheese Pizza on the Internet, so I'm surprised it would only be 26 pedobytes.

Re:26 petabytes?

[scheme]

Is internet traffic really only 26 Petabytes a month, while that is a big number it sounds awefully low to me as the place I work does 15 Terabytes a month and they are little more than a miniscule pimple on face of the internet.

That's just wrong. Open Science Grid transfers about 1.4PB a day and I seriously doubt OSG uses a significant fraction of the bandwidth on the net.

Re:26 petabytes?

[Anonymous Coward]

And of course the rest of the world does not count.

Planets 50 Light years away have airports?

[Paleolibertarian]

Wow! How is this possible given that the intensity of radio waves diminish at a factor equal to the square of the distance? That's some powerful radar or a darned big capture area of the antenna here on earth. How is it distinguishable from CBR.

Re:Planets 50 Light years away have airports?

[stanlyb]

Where? On the earth, in the empty space?

Re:Planets 50 Light years away have airports?

[Anonymous Coward]

You're assuming it's done real time. In practice, you listen for a few hours in one spot and use auto correlation.

Re:Planets 50 Light years away have airports?

[Anonymous Coward]

Well you see it is called the square kilometer array, so I would imagine its capture area here on earth is something like a square kilometer. I could be wrong.

Re:Planets 50 Light years away have airports?

[Anonymous Coward]

The CMB peaks around 200 GHz, while SKA is only going up to 10 GHz initially (and 30 GHz later). 99% of the power from the CMB is above 40 GHz.

Consider an example of a ~2 GHz radar, with 50 MHz bandwidth, 100 kW of power and 30 dB antenna gain. Within this band, CMB produces about 2e-13 W/m^2/Sr. For the SKA's rough estimated resolution of 0.1 arcseconds, this works out to 3e-26 W/m^2. The radar on the other hand produces about 4e-29 W/m^2 with those parameters. With only a factor of thousand difference, there are methods for picking out a time varying, repetitive signal from such a background.

It has both a lot of receiving area (~ square kilometer....) and large baselines for some components, up to 3000 km.

Re:Planets 50 Light years away have airports?

[Paleolibertarian]

A lot of great math there but it completely misses the point. A point source radiator, at that distance couldn't be distinguished from the radiation coming from the planets star. I don't care how much processing power you put on the various correlated antenna. The capture area BTW is not the longest baseline between array collectors. That would be the resolving power meaning the beam-width of the antenna array at 50 light years which would be far greater than the distance between the planet and its star. Capture area OTOH would be the sum of all the antennas dish or aperture area. Therefore the planet or any radiator on it's surface would be completely lost in the noise of the star not to mention the noise generated by the system itself. It isn't necessary for me to point out the noise created by the intervening 50 light years of sources which would also be captured by the array.Only with an antenna with a beam width infinitely small would the system be able to resolve the radar on the planet and only when the antenna was pointed directly at the radar source which is so impractical as to be impossible. I studied antennas when I was in high-school 45 years ago when I was in electronics class. It's called signal to noise ratio and believe me there's a lot of noise. Just on logic alone I have proven the absurdity of the statement. You can do the math yourself since you seem to be so good at it.

Re:Planets 50 Light years away have airports?

[Anonymous Coward]

You *do* know we can detect signals well below the noise floor, yes?

Re:Planets 50 Light years away have airports?

[Anonymous Coward]

That would be the resolving power meaning the beam-width of the antenna array at 50 light years which would be far greater than the distance between the planet and its star.

Conservative estimates of the angular resolution put it at a tenth of an arcsecond. At 50 light years, that resolution gives about 1.5 AU. More specific estimates of the angular resolution in certain bands, after phase two of SKA are in the 0.04 down to less than 0.01 arcseconds. So, yes, resolving the planet from the star is within the realm of possibility, and at a distance of 10 light years that is almost the resolving power to distinguish the Earth-Moon distance.

It isn't necessary for me to point out the noise created by the intervening 50 light years of sources which would also be captured by the array.

Actually, yes it is necessary for you to point out what noise is created in the median in between.

The interstellar medium is quite thin, including optically thin short of distances typically much larger than 50 light years. While temperatures of 100-200 K typical of local interstellar dust would suggest maybe a hundred times brighter blackbody power at the same 2 GHz used above, that is only true if it were optically thick. That requires distances on the order of a couple thousand light years for stuff similar to our part of the galaxy for visible and IR wavelengths (or immediate area around the sun, for a few tens of light years, is actually less dense compared to the other parts of the galaxy the same distance from the center..., and the above ignores that, so is rather conservative). The effect drops off with lower frequency though, and by the time you get to microwaves, the effect of the interstellar medium in our part of the woods is pretty minimal. This is why we have made such great observations of the CMB, which comes from outside the galaxy, not just 50 light years away.

The only place that the interstellar medium would interfere is at specific, well known transition lines, or at higher frequencies (a lot of the local measurements in the couple to hundreds of light years range is done with UV as a result, or with lines specific to hydrogen or molecules).

The capture area BTW is not the longest baseline between array collectors.

And the post you are replying to seems to quite clearly spell that out, that the effective area of the telescope is about a square kilometer, which is quite different from the largest baselines of a couple thousand kilometers.

I studied antennas when I was in high-school 45 years ago when I was in electronics class. It's called signal to noise ratio and believe me there's a lot of noise. Just on logic alone I have proven the absurdity of the statement.

This really doesn't have much to do with antenna design, but understanding the magnitudes of relevant radio sources, which doesn't seem to be covered frequently in high school electronics classes...Your "logic" not only contradicts basic math showing the plausibility of such statements, but also goes against previous measurements already done.

Re:Planets 50 Light years away have airports?

[flyingfsck]

...and the SKA low noise receivers are super cooled.

Re:Planets 50 Light years away have airports?

[tibit]

This should be +5 informative. Well done.

Re:Planets 50 Light years away have airports?

[Paleolibertarian]

The sensitivity and thermal noise of a receiver has absolutely nothing to do with the fact that a mythical airport radar on a planet orbiting a star at 50 light years distance, that cannot be resolved as a separate source by any known optical or synthetic aperture will be completely overwhelmed by the output of the planet's star. A wildly inaccurate (because I'm to lazy to do the math) analogy would be comparing the output of a microwave oven with the output of 1 billion blast furnaces. While the EM spectrum of the blast furnaces would not be in the gigahertz range there is still some present. Likewise with a star there is EM "noise" that extends well down into the gigahertz range so having a highly selective filter won't help.

The antenna will essentially "see" a blob of radiation coming from the star and while the airport radar would be in there somewhere it couldn't be picked out at all.

OK so lets assume for the sake of argument that we're NOT talking about an airport radar on a planet orbiting a star but instead we're talking about a radar aboard a space ship which is far enough from any other source that it COULD be resolved by the antenna. Is the amount of radiation that has traveled 50 light years enough to overcome the CBR? That's questionable as well.

I'm merely pointing out the absurdity of the original statement and my perhaps faulty assumptions.

Don't worry

[Anonymous Coward]

they'll just do what they did with NCSA and decide they can't make money doing it and walk away.

Why?

[Titan1080]

Why on Earth did they choose to put part of the telescope in Africa? Don't they realize how unstable South Africa is becoming? The natives are killing whites in unprecedented numbers and it looks like, longish term, that the country is going to devolve much like other African nations that suddenly decide they don't need help from 'whitey'. I think it was a very bad choice, and probably one they made out of political correctness, which is absolutely ludicrous.

Re:Why?

[PlastikMissle]

From the Wikipedia entry:

"Suitable sites for the SKA telescope need to be in unpopulated areas with guaranteed very low levels of man-made radio interference. Four sites were initially proposed in South Africa, Australia, Argentina and China.[16] After considerable site evaluation surveys, Argentina and China were dropped and the other two sites were shortlisted (with New Zealand joining the Australian bid, and 8 other African countries joining the South African bid):"

Re:Why?

[Anonymous Coward]

There was some press about it before they were making the final decision of Africa vs Australia (it rated a few articles in the Australian press, positioning the upcoming decision outcome as the key article point). And then the result was - "well, let's do both".

Re:Why?

[Paleolibertarian]

Government masters who by their nature violate ZAP are equally evil no matter the color of their skin.

Re:Why?

[Anonymous Coward]

When you have removed your whitey hoody you can see the reasons for selecting a selection of remote areas, low of interference even from the white washing powder movement.

Re:Why?

[lexsird]

Why not build a monster one in space instead? Think space elevator please.

Re:Why?

[Anonymous Coward]

Uh, that makes as much sense as asking Santa Claus.

Re:Why?

[Anonymous Coward]

Yep, muggers and murders are equally evil, and there is no need to distinguish between the two in when dealing with them, or when trying prevent the problems that lead to their existence.

worldwide blackout imminent?

[rjejr]

Is this the machine that lets us see our lives 9 months in the future?

sensitive

[planckscale]

The impressive part of the blurb to me was the ability to detect something like an airport radar on a planet 50 light-years away. With that sensitivity I would think this could go a long way towards SETI, nevermind background radiation.

Re:sensitive

[femtobyte]

One big obstacle is, as with SETI, not merely gathering super-sensitive data, but processing all the data to identify E.T.'s air traffic control in trillions of other (natural) radio sources. Just because you're sensitive enough to tell whether a signal is present or absent *when you know exactly what to look for* doesn't mean you'll be able to identify previously unknown signals.

LOFAR - interferometric array

[Frans Faase]

ASTRON is the organisation that is also running LOFAR, which is basically a smaller version of SKA in a different frequency range. It is an interferometric array which requires a central system to process all the signals into one result. LOFAR is using a lot of dedicated hardware and a IBM Blue Gene/L supercomputer for this purpose. Because all the signals are digitized at the receivers, this result is a very large stream of data, which are processed (but not stored) by a pipe-line of processors, each combining more and more signals, into one final image.

Re:LOFAR - interferometric array

[Anonymous Coward]

LOFAR, which is basically a smaller version of SKA in a different frequency range

The SKA has components operating at different frequencies. The low-frequency component will be pretty-much like a larger version of LOFAR. (Or of the MWA, a similar Australian telescope.)

Re:LOFAR - interferometric array

[broekema]

The Blue Gene/L was replaced by a Blue Gene/P in 2008. It is interesting to see that, at the beginning of the project, the most powerful supercomputer in Europe (for abour a month or so) was required. Now, only a few years later, more work can be done using only a few servers with accelerators. The SKA will ofcourse be a whole different challenge, which requires a completely different way of thinking about computing.

Bah.

[Black Parrot]

I'll pick a Batman server over a Superman server every time.

Re:Bah.

[ButchDeLoria]

A Goku server would totally kick a Superman server's ass!

Re:Bah.

[captjc]

Yeah, but you couldn't install Linux on it because as we all know Batman hates the Penguin!

The submission included the magic word.

[Anonymous Coward]

"Australia."

That's all it takes these days.

“Resist blowing desert sands”

[LMariachi]

So they’ll be putting the servers indoors then?

Re:“Resist blowing desert sands”

[Anonymous Coward]

most likely underground in containers acting as Faraday cages

Superman?

[wonkey_monkey]

Since when did "Superman" become a good adjective for describing powerful computer systems? I mean, if you must appeal to the hardcore comic book reading geek, wouldn't Brainiac be a better choice?

How many?

[dutchwhizzman]

How many olympic sized swimming pools can be filled with the servers?

Here is an geek info on SKA.

[hamster_nz]

This link is a really interesting info on some of the SKA signal processing.

The SAK's power budget is 58MW for signal processing - this is such a high running cost that by spending 30 Million Euro on developing a few custom ASICs to halve that power usage will pay off in 9 months!

Re:Here is an geek info on SKA.

[tibit]

Very informative. While it's a cool project, in a way I'm fine only dealing with distributed signal processing that can power a dozen nodes through one PoE ethernet drop :)

Apollo hardware on moon

[nelsonteixeira777]

Maybe then we'll be able to see Apollo mission hardware on the moon :|

http://thedroidcity.com/

[Anonymous Coward]

Hi ,
Please let me know how can i make my own compiler. !!