Slashdot Mirror
Sloan Digital Sky Survey
Swannie writes: "There's a story in today's Chicago Tribune about a joint project that Fermi Lab is taking on with Rensselaer Polytechnic Institute in New York. The goal is to produce a 3D map of the universe using a really big digital camera, and a really creative way to add "depth" to the image. The article has some decent technical details for a newspaper, including a pretty picture." Update: 03/12 15:44 GMT by M : The blurb is in error. A particular scientist from Rensselaer is mentioned in the article, but Rensselaer isn't part of the project as an institution.
That is cool.
fp
I think it would be really cool if when they take all this data they are collecting, they produced a 3-d image of the COSMOS and a 3-d image of the cosmos with every star's location shifted to show its theoretical place today... or in the case of billion light year stars.... nothing if they are burned out by now.... that woould truly be an intersting map to look at :)
The universe is going to be an interesting
place when you can just fly around.
The field guide to the universe is slowly taking shape at the Fermi National Accelerator Laboratory near Batavia.
Hundreds of gigabytes of digital pictures are delivered weekly to Fermilab for compilation that, over the next three years, will deliver the most comprehensive map of the hundreds of millions of galaxies that make up what astronomer Carl Sagan called the "cosmos."
Like the human genome project used microscopes to look deep within us, the Sloan Digital Sky Survey uses telescopes to look toward the universe to discover our place.
The Sloan survey's results are as immense as the cosmos.
The main telescope, in the mountains of south- central New Mexico, is picking up light from 14 billion years ago--a time when the universe was just 6 percent of its current age. Focusing on one-quarter of the sky in what is known as the northern galactic hemisphere, the Sloan survey (www.sdss.org) has compiled enough data on the heavens in the last 36 months to fill 1,000 laptop computers, each with a 10 gigabyte hard drive, and it's just 35 percent of the way through its mission.
"This is what we call industrial astronomy," said Chris Stoughton, the head of data processing and distribution at Fermilab. "In the old days you'd take 10 pictures a night and write a paper" on the findings. "This was one-by-one, hand astronomy.
"Sloan produces 600 pictures through five different filters every hour. That's industrial astronomy, and enough to keep an astronomer busy for a year."
And while Sloan's work may not affect our daily lives, its discoveries are shaking up the world of astronomy. In addition to locating quasars at the beginning of time (about 15 billion years ago), Sloan scientists have identified new star structures that could alter how we view the galaxy and a new class of celestial objects called brown dwarfs.
Scientists from Fermilab and Rensselaer Polytechnic Institute in New York recently presented their findings on the discovery of new star structures in the halo of the Milky Way. Isolated from the 5 million stars logged so far by the Sloan study, the new stars appear to be clumped in a "puffier" configuration rather than what we're used to seeing as a flatter, spherical look of the galaxy.
Before the latest Sloan findings, the first and only "cool" brown dwarf star, which has properties of a planet and a star, was discovered in 1995. They're called "free floating" structures because they don't orbit a star or planet. The most recent discoveries are a mere 30 light-years away.
"They are still so new to astronomy that they require a new vocabulary," said astronomer Tom Geballe of the Gemini observatory in Hawaii. "The name `methane dwarf' has emerged, because of the dramatic presence of bands of methane in their spectra. Methane is characteristic of giant planets, like Jupiter, but it never appears in normal stars--they are much too hot--or even in most brown dwarfs."
World's most powerful camera
While there are many areas of interest to the non-astronomer or astrophysicist, the camera and the information it gathers are probably most interesting to the technologist.
The digital camera built for the Sloan Survey is the world's most powerful, according to Jim Gunn, professor of astrophysics at Princeton University, a Sloan project scientist and builder of the camera. The digital camera detects 7 out of 10 photons (particles of light without mass) hitting the lens, netting an efficiency approaching 70 percent. The efficiency of a standard film camera is about one-tenth of 1 percent, Gunn explained.
What makes the Sloan camera so efficient is its use of charge-coupled devices. These light-sensitive squares detect the intensity of incoming photons and convert light into digital signals. Each detector is rated at 4 megapixels, giving the 30-CCD array a whopping 120-megapixels (120 million pixels) sensitivity. The higher the number of pixels, the greater the resolution.
Gunn said the array of CCDs took four years to accumulate partially because of exacting specifications that meant as few as 1 of 3 CCDs shipped from the manufacturer were accepted. "They're very expensive and we have only a few spares," Gunn said.
The CCDs were half of the telescope's $5 million cost, paid for by the Japanese government's Monbukagakusho scholarship program. The entire survey will cost around $85 million, with the largest private donation of around $20 million coming from the Alfred P. Sloan Foundation.
Another $42 million came from Fermilab through the U.S. Department of Energy, the National Aeronautics and Space Administration and the National Science Foundation. Other underwriters include Johns Hopkins University, the University of Chicago, the University of Washington, New Mexico State University, the U.S. Naval Observatory and Germany's Max Planck Institutes.
The 700-pound camera, which took seven years to build, is a meter-long, vacuum-sealed cylinder cooled by liquid nitrogen to keep moisture out. In addition to the two mirrors found in a reflector telescope, two lenses focus incoming light to the 30 CCDs that make up the camera. Two spectrographs are mounted nearby.
A fifth of the CCDs are receptors for ultraviolet light, and another three-fifths captures green, red and near-infrared light. The remaining CCDs concentrate on getting what's called far-infrared light, and that's where amazing things happen.
That "far" category reaches back almost to the beginning of time.
Overall, the Sloan telescope picks up light from an immense number of galaxies and quasars, estimated by the astronomers at 100 million. They range in age from as young as two-tenths of the age of the universe to events that occurred just a billion years after the universe was formed and whose light is just now reaching the camera.
When a night of observation is over, the millions of bytes of data are written to 20 gigabyte data storage tapes in a protocol known as flexible image transport system. The findings of five CCDs are recorded on one tape.
Each tape and a backup copy are sent overnight to Fermilab, where they are transferred to a host of Linux servers. Stoughton said the amount of data is small compared with Fermilab's other projects but is the largest capacity project ever assembled in astronomy.
Most images go unseen
Surprisingly, with all the money and time spent in the quest for a road map of the celestial past, "most of the pictures have never been looked at," Stoughton said. Stoughton said that because of the immense amount of information seeing any part of it would take a lifetime.
Instead, investigators are writing programs to look for specifics like "low surface brightness galaxies," which could be described as wide--not bright--galaxies.
So with all the data that few have seen, and few practical business applications, it seems to raise the question as to why are they mapping the universe.
"It's hard to say why people should study astronomy," said Gunn. "But in the scheme of human intellect, it is important to know where we came from and what's likely to be in store for us."
Copyright © 2002, Chicago Tribune
[Scientists] "Right, we'd like some funding to build a map of the universe"
[Investor] "Sounds good. How are you going to go about it?"
"Well, we are going to get a really big camera..."
"click......"
"It's hard to say why people should study astronomy," said Gunn. "But in the scheme of human intellect, it is important to know where we came from and what's likely to be in store for us."
Oh, you mean this is a $85 million Horoscope Machine....
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
...a 3D map of URANUS !!!
[snigger, chortle]
I've been playing around with this prog, it has quite a big 3d map in it.
So with all the data that few have seen, and few practical business applications, it seems to raise the question as to why are they mapping the universe.
Because it's cool, OK... and because some day the data will be useful, viewable, etc? It will be a map for space travel?
Each tape and a backup copy are sent overnight to Fermilab, where they are transferred to a host of Linux servers. Stoughton said the amount of data is small compared with Fermilab's other projects but is the largest capacity project ever assembled in astronomy.
Cool. They are using the penguin...!
"It's hard to say why people should study astronomy," said Gunn. "But in the scheme of human intellect, it is important to know where we came from and what's likely to be in store for us."
Well, it is interesting to know all about that. But collecting data that can't be used... tough cookie.
In general, these kind of projects get funded by curious people who can't use the data. Loads of data written to disks is not ever looked at, and this article raises that question. This is the discussion which interests me, quite apart from the greatness of some liquid nitrogen cooled super telescope with so many megapixels that at any kind of CRT resolution, for example, we would be decimating 99% of the data in order to get something reasonable to look at.
Conversion Rate Optimisation French / English consultant
Really creative, yes. Pity we don't get to hear about it. Or am I overlooking something?
Howdy all. I've been following the project online for over a year. The cool part is that this is sort of a googlebot for the heavens. See: http://www.astronomy.com/content/dynamic/articles/ 000/000/000/502vwthx.asp
And ruin his day!
Who run Barter Town?
In fact, even if we had all the science needed to make the calculations and the equipment to do so, a true map is theoretically impossible, based on the Uncertainty Principle it is impossible to determine with 100% accuracy the state of even an atom, let alone a universe.
Hey, fuck you you arrogant prick. What do I care who employs your fat ass? Why are you making light of an important journalistic tradition... namely pointing out situations where your affiliations might bias your writing? WHY DO YOU SMELL LIKE FISH STICKS?!?
So, yes. I am in a box. Now I will put you in a box. A large box, all duct taped up. Then I will let you and your box become damp with the blessed rainwater. THEN I WILL KICK THIS BOX UNTIL MY BOOT IMPACTS YOUR FAT FACE!!! All doe-eyed and sad from the face kicking, you might pause to weep. I would then redouble my efforts AND KICK YOUR BLUBBER PUSS WITH BOTH FEET!
Learn this lesson well! Use what I have taught for Good, not Evil!!!
.. is the creative process behind projects of this magnitude. Where do people get these ideas? Is the scenario below how it happened?
- Hey, I just got this great idea!
- Uhu, what's that?
- Let's make a 3D map of the entire universe!
- Sounds cool, let's get some funding!
Or is it more like:
- Hmmm, this problem would be easily solved if we had a 3D map of the universe..
- Sounds cool, let's get some funding for that project!
Or the third alternative:
- Hmmm, we need some funding!
- Sounds cool, let's make a 3D map of the entire universe!
- Great, that will keep us running for a couple of years!
:)
This is a straight paste from an old email.
"
I promised that I would tell you how to visualise a four dimensional object.
Imagine a picture in a book that shows a scene viewed through an arch. The picture is two dimensional (2D) but it creates the illusion of what the scene would look like viewed in 3D. If we want to find what is around the corner then we need a different picture viewed from a different angle. If we have enough pictures then we can create a movie of what it would look like to walk past the arch and view the scene as we walked (or flew because they are pictures we are not limited by such mundane things as gravity).
We can use a similar technique to visualise a 4D object. By the way, dimensions come in different sorts. Our three spatial dimensions are bidirectional but time is monodirectional and we can only travel through time in one direction. Those tiny rolled up dimensions that are produced by string theory are just little pockets that could make a grain of salt vanish from existence (as we knew it, it would still exist where we had no way of seeing or measuring it) . My fourth dimension is quite clearly a bidrectional spatial dimension.
Imagine a cube on a pedestal. The cube is perfect and all one colour and is four dimensional. Start with the 3D cube and imagine if it went back the same distance as the length of its sides and the other cubes stretched out behind it like cream on milk. You have just seen the image through the arch. Move around a little and the cubes stretch behind the one on the pedestal no matter which angle you view it from, even from directly above or below.
In reality the 'other' cubes would be directly behind the 3D cube and so invisible but I was only after a way to visualise or perhaps conceptualise the fourth dimension. When I consider objects that are different colours and shapes along their oopth (I have invented this word, length, width, depth and now we have oopth). When objects change through their oopth then they become much more interesting and much harder to visualise. Once you can do it then the relationship of time to space is immediate and necessary. A suitable object to conceptualise is a person changing from a baby to a child to an adult to a public servant.
This is all rather hard and you can forget it if you like but it will change your view of everything if you can master it.
"
Cuiusvis hominis est errare; nullius nisi insipientis in errore perseverare.
Now the 2df galaxy cluster mapping project which are giving us maps of our galaxy's position out to about 1B light-years -- /that's/ interesting AND news. hell,
"None are more hopelessly enslaved than those who falsely believe they are free." -- Goethe
If that's a word... ;)
DataSquid.net, a little about me.
Great!! Where can I get one? Does it have USB 2.0 or FireWire? And how many hi-res shots can I fit in memory? Does it take SmartMedia or CompactFlash?
I just hope that I can find a big enough LCD to view these pix at 100%...
I've got a bad attitude and karma to burn. Go ahead. Mod me down.
I can think of a good application for this data.
Why doesn't the SDSS code up a distributed program like SETI to help in the analyzation of all that data to find something unique or unknown. Convert the pictures to 2D FFT's and have a set of known astronomical element FFT's and then do constant comparisons against this set to see if there is anything "unknown". I'm sure it would be more complex than this, but this is how visual image recognition works so I assume it could be tailored for this application.
I would certainly download an run an application that looks for new things in astronomy. I'm sure others would as well. Somehow it's slightly less frivilous than what SETI is doing and we stand to gain more in a quicker amount of time.
That way, when we do actually find something that looks interesting SETI would know where to point that big antenna...
"I'll just chip in a bit for RedHat: I actually have that installed on my university machine." - Linus, '95
From the article : "Surprisingly, with all the money and time spent in the quest for a road map of the celestial past, "most of the pictures have never been looked at," Stoughton said. Stoughton said that because of the immense amount of information seeing any part of it would take a lifetime"
What they need is SETI-like distributed software than farms the pictures out to us to look at, and we'd get through them in a week or two. Or stick them all on a website - www.AmIAMinorAstronomicalAnomaly.com - with user rankings. Job done.
'A really creative way to add "depth" to the image'
Well, this superlative piqued my interest. Unfortunately I then read the linked article (yes I do realize that isn't the done thing). The "really creative" way?
Red shift
I don't mean to undermine the goals of the project, which are clearly noble. But the top level comment is rather tabloidy.
I do have a serious question. What kind of accuracy do you get from this data? I understand latitude and longitude (or psi and phi) can be given to a tiny fraction of an arcsecond, but how about distance from earth? +-10%?
A flame may be fleeting but obscurity is forever.
The tricky part of this mapping is that any images recorded represent only a single snapshot of events. 15 billion year old quasars may be long gone by now, but we can't tell. We have very recent images of close stars, somewhat recent images of most of our galaxy and ancient images of most distant galaxies. Any map produced is really just a reflection of this snapshot of images from the past.
So long and thanks for all the fish . . . !!!
Three dits, four dits, two dits, dah!
Radio, radio, rah rah rah!
RPI doesn't get a lot of funding. NSF Awards Rensselaer $10 Million For a Nanotechnology Center; "a $450,000 grant from the National Science Foundation, Gyorgy Korniss, assistant professor of physics, will use a computational technique called Parallel Discrete-Event Simulation to model large-scale systems, where events occur randomly in space and time"; " A $700,000 effort by the Kauffman Center to understand how entrepreneurship can serve as a catalyst to revitalize former industrial areas" all adding up to their 50 million in grants for fiscal year 2001. This of course does not included the $360 million 'anonymous' donation to "galvanize the Institute's plan to more than double its research activity and its graduate enrollment in the next five years by creating new programs in biotechnology and information technology as well as to undertake a number of additional strategic initiatives." Yumm, additional strategic initiatives. - Sorry. This is why RPI needs funding.
I thought the sequence was baby, child, public servant, adult. Truly being an adult requires maturity and behaving properly, not mere age.
You said
"Time is an observational reference, not a measured dimension. We don't move forward in time, we experience infinitely small discrete moments, which do not necessarily sync with someone else's experienced moments."
Can you extrapolate on that a little. I am seriously interested.
Cuiusvis hominis est errare; nullius nisi insipientis in errore perseverare.
If one wants a true 3D map of the universe they should be seeking the fractal formula that describes it, IMNSHO. Anything less is a fundamentally innacurate map.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
A sky survey? Man, that sounds like a rough job...
"Excuse me Mr. Cloud, but what do you think about the war in Afghanistan? Hello? Cloud? Okay, how about chocolate milk, do you like it? Hey, I'm talking to you! Where are you going? C'mon, answer my question! Dammit..."
------
Today's Top Deals
This people has so much data, that It will take a lot of time to make that map.
OverLord
I don't remember the name of it, but it seems that this could be used for a torture chamber described in the Hitchhiker's Guide to the Galaxy series! The gist of it was that the victim entered a chamber where the entire Universe was projected all around him in stunning detail...and the sheer unthinkable magnitude of it all made the victim feel so minor and insignificant that it killed them.
I remember reading it and thinking that's not only frickin' hilarious, but it's COOL!
There is no gravity...the earth just sucks.
Now, call me a little biased, but SDSS was designed by Alex Szalay at Johns Hopkins. I see NO MENTION of him OR of Hopkins anywhere in the article. What, are we just a bunch of doctors?
-RHW
it's not renn-suh-leer, it's RPI.
I didn't see it mentioned, but astronomers always complain about our inability to track dangerous objects(like we could only track ten percent of the sky). It seems that with a few iterations of this project, when a few complete pictures have been taken, on could begin to plot the course of the objects that move with simple algorithms.
Asking why a project such as this is worthwile is like asking why where decoding DNA.
--
What is the sound of this sentence?
cooled by liquid nitrogen to keep moisture out
Excuse me? Somehow I think there are easier ways to keep moisture out LOL.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
Urgh, can't remember my stinking password...
- 10 4.pdf
This paper on public access to the SDSS is also of interest:
ftp://ftp.research.microsoft.com/pub/tr/tr-2001
It contains details on how the public site has been implemented, how the data are stored and retrieved, what kind of data are stored, and so on.
On a side note, please notice the url... SDSS is also a Microsoft Research project, and runs SQL Server on the backend. The Linux servers are not the whole story here =)
No, not your MySQL database for your CD collections, I mean the real databases on top of terabytes of data. In fact, the Sky Survey is working with Microsoft on such a database, so people who study astronomy can actually query this tremendous amount of data. Check out this SIGMOD paper (http://dblp.uni-trier.de/db/conf/sigmod/SzalayGKT 00.html)
Wouldn't it be cool if...
We use a big cluster of *nices to serve the data up to everyone on the Internet. Every amatuer astronomer on the web can then visit the site once a day or so and cruise the catalog. All those eyeballs looking for stuff will do something.
Specific projects I can think of are:
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
Slooooooooooan
Discover magazine November 2001 had the Sloan Digital Sky Survey as it's cover article (online at //www.discover.com) It did a much better job of explaining the 3d process and the special nature of the telescope. The impression that most of the pictures are not looked at is VERY misleading. Each image is scanned by software to identify 500 or so "most interesting" light sources. The coordinates are then transmitted to a machine shop. There they make a custom aluminum plate with 500 small holes corresponding to the selected sources. The plate is returned to the telescope and later reattached with fiber optic cables attached to each of the holes. The telescope is repositioned to where it was when the original image was taken. Then the light from each of the selected sources can be run through a spectrograph. This allows them to determine the distance to the star or galaxy based on the red shift. It is this very accurate 3rd dimension over a huge number of sources that makes this survey so unique and exciting.
kstars is fun, too. Sometime, I'd like to take one of these planetarium progams and find a way to keep an updated star map as a desktop background.
Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.