National Virtual Observatory

scubacuda writes "According to this Technology Review article, U.S. astronomers (compliments of a $10M grant from the National Science Foundation) are building a National Virtual Observatory to make accessible terabytes of astrononomical data to a web browser. One interesting challenge is how the scientists are going to query so many *different* distributed databases (which they're leaving in their respective places to avoiding clogging network bandwidth)."

Comments

P2P as an alternative

[DirtyJ]

That's a pretty interesting idea, but I don't think it's applicable to the Virtual Observatory. What is being discussed here is creating a central engine which can seamlessly access multiple large databases which are served out of different locations. These are databases which are frequently all-sky surveys conducted by one group and stored in one central location - not necessarily in small sections on multiple persons' hard drives.

The P2P idea is interesting in that it could apply to individually collected small data sets. Here's how observational astronomy has traditionally worked:

Astronomer writes a proposal to do some research using a specific telescope(s)

Proposal gets accepted after peer review

Astronomer travels to observatory to spend many of his own nights collecting data

Astronomer takes the time to reduce and analyze his own data

Astronomer writes a paper(s) saying, "Hey - look what I did!"

(Sometimes) astronomer writes a proposal for further funding based on the merits of this work

This procedure is inefficient in that you sometimes get multiple people who are not working together, doing the same project on different telescopes. If I collect a bunch of data in one part of the sky, try to use it but don't actually get around to finishing and publishing a paper, and then archive it locally, nobody in the world knows that the data exists. So now if someone else wants to do the same project, they go to the telescope and recollect the same data. In other words, there's no central log of who's done what when it comes to individual observing.

P2P could be useful to remedy this. The problem is that astronomers tend to be very proprietary about their data. Sometimes research and publishing can be very competitive, and you don't want to give the competition an edge when it could mean that they publish a paper on a particular topic before you and reap the rewards, or get funding when you don't. So I think that most astronomers would share their data openly in a P2P network only after they were completely finished using it, and some would never do so.

The difference with the data sets being accessed by the proposed Virtual Observatory is that the people who create those sets typically get their funding with a stipulation that the data be publically accessible some time after the work is finished. They're not allowed to keep it proprietary even if they'd prefer to do so for competition reasons.