Sloan Digital Sky Survey

Swannie writes: "There's a story in today's Chicago Tribune about a joint project that Fermi Lab is taking on with Rensselaer Polytechnic Institute in New York. The goal is to produce a 3D map of the universe using a really big digital camera, and a really creative way to add "depth" to the image. The article has some decent technical details for a newspaper, including a pretty picture." Update: 03/12 15:44 GMT by M : The blurb is in error. A particular scientist from Rensselaer is mentioned in the article, but Rensselaer isn't part of the project as an institution.

Comments

What is a true map?

[phunhippy]

I think it would be really cool if when they take all this data they are collecting, they produced a 3-d image of the COSMOS and a 3-d image of the cosmos with every star's location shifted to show its theoretical place today... or in the case of billion light year stars.... nothing if they are burned out by now.... that woould truly be an intersting map to look at :)

Asking for funding...

[flipflapflopflup]

[Scientists] "Right, we'd like some funding to build a map of the universe"

[Investor] "Sounds good. How are you going to go about it?"

"Well, we are going to get a really big camera..."

"click......"

"hard to say why people should study astronomy"

[da5idnetlimit.com]

"It's hard to say why people should study astronomy," said Gunn. "But in the scheme of human intellect, it is important to know where we came from and what's likely to be in store for us."

Oh, you mean this is a $85 million Horoscope Machine....

What about this one ?

[kraf]

I've been playing around with this prog, it has quite a big 3d map in it.

Re:What about this one ?

[Mortenson]

You should go take a look at the Celestia project on Sourceforge. It is very well made. And free

http://celestia.sourceforge.net

All that Data and no-one to look at it?

[fruey]

So with all the data that few have seen, and few practical business applications, it seems to raise the question as to why are they mapping the universe.

Because it's cool, OK... and because some day the data will be useful, viewable, etc? It will be a map for space travel?

Each tape and a backup copy are sent overnight to Fermilab, where they are transferred to a host of Linux servers. Stoughton said the amount of data is small compared with Fermilab's other projects but is the largest capacity project ever assembled in astronomy.

Cool. They are using the penguin...!

"It's hard to say why people should study astronomy," said Gunn. "But in the scheme of human intellect, it is important to know where we came from and what's likely to be in store for us."

Well, it is interesting to know all about that. But collecting data that can't be used... tough cookie.

In general, these kind of projects get funded by curious people who can't use the data. Loads of data written to disks is not ever looked at, and this article raises that question. This is the discussion which interests me, quite apart from the greatness of some liquid nitrogen cooled super telescope with so many megapixels that at any kind of CRT resolution, for example, we would be decimating 99% of the data in order to get something reasonable to look at.

Re:All that Data and no-one to look at it?

[davecl]

SDSS has been running for quite some time, and has already produced some spectacular results. These include the most distant quasars known, and the largest most homogeneous quasar survey yet (in fact I'm working with some of this data at this very moment).

Whilst much of the data is not eyeballed by a person it is still processed, so there isn't a vast amount of data sitting on disk being ignored. Yes, people can look at the images, but there are too manmy of them and the human eye and mind are too inconsistent to be able to select items of interest in a coherent way. That's why we have computers! Particle physics operates in much the same way - you get a vast number of reactions in an accelarator and only a few of these are of interest, so you use computer-based filters to just select these. In much the same way with the SDSS images, most of them are of blank sky or of stars and galaxies that aren't of interest to the project. They're still cataloged and characterised though, and put into lists to be scanned by later data miners, while the specific goals of SDSS are dealt with at higher priority.

Excellent! Finally SDSS Gets Public!

[hondo_san]

Howdy all. I've been following the project online for over a year. The cool part is that this is sort of a googlebot for the heavens. See: http://www.astronomy.com/content/dynamic/articles/ 000/000/000/502vwthx.asp

Impossible

[Czarnian]

A true map (100% correct - however see my comment below) cannot be currently made, the science behind the calculations needed to make such a map is uncertain (dark matter, universal expansion speeds, unpredictable effects effects of undiscovered objects - black holes, etc, astrophysics is evolving all the time) and you'd a Beowulf cluster of processors the size of a galaxy to do the math.

In fact, even if we had all the science needed to make the calculations and the equipment to do so, a true map is theoretically impossible, based on the Uncertainty Principle it is impossible to determine with 100% accuracy the state of even an atom, let alone a universe.

Re:Impossible

[Gary Yngve]

Oh, give me a break. The original poster wanted a 3-d map of the universe. By your argument, we don't even have true maps of Earth.

A map with "approximate" positions of the stars/galaxies would be plenty good. Oh no, the sun is off its true center by the diameter of an apple? I am wetting my pants! I can even see the difference of the sun's placement on my super-duper 1e10x1e10 resolution monitor (that provides the ultimate resolution for viewing porn: not only do I see all her parts, I see all the parts of the gonorrhea bacteria that she has).

The real interesting question is how does one navigate [in a user interface] around such a sparse map?

Re:Impossible

[FreeUser]

A true map (100% correct - however see my comment below) cannot be currently made, the science behind the calculations needed to make such a map is uncertain (dark matter, universal expansion speeds, unpredictable effects effects of undiscovered objects - black holes, etc, astrophysics is evolving all the time) and you'd a Beowulf cluster of processors the size of a galaxy to do the math.

While the last portion of your comment (needing a cluster the size of a galaxy to do the math) isn't really true, you are correct in your assertion that we lack the scientific knowledge, and likely the computational power, to make an accurate map of our galaxy as it is today. In other words, if we invented superluminal propulsion today we'd have to make our maps the old fashioned way ... by going there and surveying the space a la Star Trek.

However,

In fact, even if we had all the science needed to make the calculations and the equipment to do so, a true map is theoretically impossible, based on the Uncertainty Principle it is impossible to determine with 100% accuracy the state of even an atom, let alone a universe.

The uncertainty principle only applies to subatomic scales at the quantum level, not to macro objects like planets, stars, and galaxies, or even smaller objects like grains of sand. Calculating the location of every star and planet, if we had the scientific knowledge and computational capacity to do so, is perfectly possible in theory ... just not practical given the current state of the art (and our current ignorance, as you pointed out before, with respect to undiscovered objects, dark matter, etc.).

What fascinates me...

[petis]

.. is the creative process behind projects of this magnitude. Where do people get these ideas? Is the scenario below how it happened?

- Hey, I just got this great idea!
- Uhu, what's that?
- Let's make a 3D map of the entire universe!
- Sounds cool, let's get some funding!

Or is it more like:
- Hmmm, this problem would be easily solved if we had a 3D map of the universe..
- Sounds cool, let's get some funding for that project!

Or the third alternative:
- Hmmm, we need some funding!
- Sounds cool, let's make a 3D map of the entire universe!
- Great, that will keep us running for a couple of years!

:)

Re:What fascinates me...

[TMB]

That second one's pretty accurate. :-)= Though it's more than just one problem that they had in mind, the main motivator is understanding galaxy formation and evolution.

Of course, you forgot the third line... ;-)

- Hmmm, the Sloan Foundation will only give us the money if we name the survey after them. Fine by me!

[TMB]

Forgive me, but... `news'??

[Cally]

I know this is a lame sorta thing to say but really, the Sloan DSS has been running for... what, 3? 4? years now? Interesting, I grant you, but hardly news.

Now the 2df galaxy cluster mapping project which are giving us maps of our galaxy's position out to about 1B light-years -- /that's/ interesting AND news. hell,

Now in stock: 0.1Gpixel Web Cam

[ArcSecond]

Each detector is rated at 4 megapixels, giving the 30-CCD array a whopping 120-megapixels (120 million pixels) sensitivity. The higher the number of pixels, the greater the resolution.

Great!! Where can I get one? Does it have USB 2.0 or FireWire? And how many hi-res shots can I fit in memory? Does it take SmartMedia or CompactFlash?

I just hope that I can find a big enough LCD to view these pix at 100%...

Re:Now in stock: 0.1Gpixel Web Cam

[zer0vector]

The quality of astronomy grade CCD chips is so far beyond anything you've seen in commercial digital cameras, and so much more expensive. I've used a 2048x2048 chip, cost on the order of 50k, and keep in mind when they sell it to you you don't get anything but the chip, you have to build the rest yourself. Oh yeah, I wouldn't be waiting around to make a webcam, the single readout time for the full chip was about 2 minutes. Luckily this particular chip was recently upgraded to quad readout.

Good application to all that data

[hyrdra]

I can think of a good application for this data.

Why doesn't the SDSS code up a distributed program like SETI to help in the analyzation of all that data to find something unique or unknown. Convert the pictures to 2D FFT's and have a set of known astronomical element FFT's and then do constant comparisons against this set to see if there is anything "unknown". I'm sure it would be more complex than this, but this is how visual image recognition works so I assume it could be tailored for this application.

I would certainly download an run an application that looks for new things in astronomy. I'm sure others would as well. Somehow it's slightly less frivilous than what SETI is doing and we stand to gain more in a quicker amount of time.

That way, when we do actually find something that looks interesting SETI would know where to point that big antenna...

Re:Good application to all that data

[grid geek]

Why doesn't the SDSS code up a distributed program like SETI to help in the analyzation of all that data to find something unique or unknown

There are several reasons why this hasn't been done.

• The amount of data is fairly trivial compared to the Particle Physics data (AstroGrid, EU DataGrid, Grid Forum
• Trust. As in lack there of. SETI had problems with people returning bogus results in the past and had to spend time (and several articles) on improving security and getting the same unit processed by several seperate users.
• Lack of man power / time.
• Dependability - e.g. an astronomer wants to run an analysis now! Is there any certainty that there will be x amount of processing power available? That your computer will still be on the network in 5 minutes time? That the unit will get processed in a reasonable real time?

It's ok for seti to send out a unit and not get it back for a week, or a month but not for things which have to be done right now. More importantly seti units are discrete, they dont depend on other units for the results.
Physics data can & therefore any machine would have to have an always on connection to communicate with other machines. Bye Bye to your bandwidth.

Re:Good application to all that data

[john_ee]

Automated object identification is already a standard part of any sky survey data pipeline. Among other things, this is used by surveys like SDSS to not only find and identify objects, but to tag the ones that are galaxies for later redshift measurement. (That's where the "3D" part comes in - redshifts of galaxies. Measuring radial ditances to closer objects - say stars in our own galaxy - is a pretty involved bootstrapping process where distance is derived from a ladder of indirect indicators.)

As far as grid computing goes, the US NVO (National Virtual Observatory) will implement a grid computing model. The NVO will be a unified portal that will federate about 50 digital sky surveys. It will have a database of billions of objects with hundreds of characteristics each, and rich relational structure spanning those hundreds of dimensions. They'll use grid computing to perform astronomers' queries on those data. That's really the main reason why these sky surveys are being done - to allow astronomers to do their work by data mining. A "map" of the universe will be a byproduct.

Re:Good application to all that data

[hyrdra]

Yeah, heh, too early in the morning.

They need our help

[Scorchio]

From the article : "Surprisingly, with all the money and time spent in the quest for a road map of the celestial past, "most of the pictures have never been looked at," Stoughton said. Stoughton said that because of the immense amount of information seeing any part of it would take a lifetime"

What they need is SETI-like distributed software than farms the pictures out to us to look at, and we'd get through them in a week or two. Or stick them all on a website - www.AmIAMinorAstronomicalAnomaly.com - with user rankings. Job done.

Tricky part

[JJ]

The tricky part of this mapping is that any images recorded represent only a single snapshot of events. 15 billion year old quasars may be long gone by now, but we can't tell. We have very recent images of close stars, somewhat recent images of most of our galaxy and ancient images of most distant galaxies. Any map produced is really just a reflection of this snapshot of images from the past.

Insider's view

[srhuston]

Since I'm the sysadmin at Princeton's astrophysics department, perhaps I can shed a few more links for the picture-hungry (and the information hungry):

• SDSS image gallery
  
• SDSS Project Book
• Astro's webpage

Re:Distance from earth

[(void*)]

The redshift of an object can be measured really accurately to about 1 in a million. The important thing about the SDSS is that you can get good photometric, not spectrographic redshifts.

really big

[trb]

The goal is to produce a 3D map of the universe using a really big digital camera...

Of course they need a really big camera, they're making a map of the whole universe!

Ouch

[NiftyNews]

A sky survey? Man, that sounds like a rough job...

"Excuse me Mr. Cloud, but what do you think about the war in Afghanistan? Hello? Cloud? Okay, how about chocolate milk, do you like it? Hey, I'm talking to you! Where are you going? C'mon, answer my question! Dammit..."

Can this be used for torture?

[clmensch]

I don't remember the name of it, but it seems that this could be used for a torture chamber described in the Hitchhiker's Guide to the Galaxy series! The gist of it was that the victim entered a chamber where the entire Universe was projected all around him in stunning detail...and the sheer unthinkable magnitude of it all made the victim feel so minor and insignificant that it killed them.

I remember reading it and thinking that's not only frickin' hilarious, but it's COOL!

liquid nitrogen

[Alsee]

cooled by liquid nitrogen to keep moisture out

Excuse me? Somehow I think there are easier ways to keep moisture out LOL.

-

Distributed Human Processing

[MagikSlinger]

Wouldn't it be cool if...

We use a big cluster of *nices to serve the data up to everyone on the Internet. Every amatuer astronomer on the web can then visit the site once a day or so and cruise the catalog. All those eyeballs looking for stuff will do something.

Specific projects I can think of are:

• Near-Earth Asteroid/Comet spotting