National Virtual Observatory

scubacuda writes "According to this Technology Review article, U.S. astronomers (compliments of a $10M grant from the National Science Foundation) are building a National Virtual Observatory to make accessible terabytes of astrononomical data to a web browser. One interesting challenge is how the scientists are going to query so many *different* distributed databases (which they're leaving in their respective places to avoiding clogging network bandwidth)."

Comments

National Virtual Observatory?

[Rhinobird]

No, no. It should be renamed the National Space Wallpaper Archive.

Web Browser

[LittleBigScript]

I tried to look at the universe through my web browser but all I saw was this prompt that told me I should update my web browser to the latest version in order to see the universe.
So will the universe be viewable in the next point release or is it several years away.

Is it possible to look at the universe with, say, lynx?
Or if that is not possible with javascript turned off?

Re:Web Browser

[ghostlibrary]

Is it possible to look at the universe with, say, lynx?

I know this was a joke, but that's actually a topic debated by webmasters at GSFC. In theory, all NASA web pages should be accessible, e.g. all browsers, readers for the blind, etc.

For images, this means descriptive image 'alt' tags. For links, it means including a link description. But what to do for data?

It's kinda subtle. The best answer is 'give data informative tags that can be domain-specific.' "Image 5b" is useless, saying "DI Peg data, X-ray wavelengths, reduced, FITS format" is good but tedious for whomever makes the page, giving a spec like 'ASCA dataset1, DI Peg, FITS, reduced' is something that could likely be automatically generated and fits the bill.

But the issue of folks using non-visual browsers is pretty real. Besides lynx and browsers for the blind, there's also data hunting scripts and programs that need to figure out what is on a page, and so it's a problem worth solving.

The problem of data interfaces and the layman

[JayBonci]

From what the article reads, it seems to be a very ambitious and interesting project. Very rarely do you see people trying to get together to spread information out to the web in such a fashion. The major problem in my (and I can imagine in their) mind is of format? How can you accomodate the mythical layman's and his or her inherent lack of skill, and still have it be available for advanced researchers to make use of.

It seems that there is simply going to be a huge amount of data-cross referenced and collated. From the second page of the article, it seems to include pictoral data. I also hear talk of XML being thrown around, which is a good start, but there's a lot that goes into that transition. Are they looking to set the layman bar at "your novice astronomer", "the third grade science report", or "grad student". Where is this information really being targeted at the sub-obscure level.

While I don't want to trivialize their massive IT effort, it seems that a lot of this is going to come down to the end user of the data. Their sample study using this information isn't trivial stuff, and does seem to set the aforementioned bar at somewhere in the undergrad-graduate level. Perhaps that is the nature of the data (I'm not that familiar with it). There's an XML schema, some request examples, and other framework stuff already in place to view by potential client writers.

I'm glad to see XML being done the right way (by collaboration with its end users), and those pictures /numbers being available for public research. Maybe someone will throw together an inverse Terraserver or something with Whiz-bang true-layman appeal. Until then, the geeks bow at the effort, because man, space is BIG.

Anyone closer to the project know of any simplification efforts?

--jaybonci

Re:The problem of data interfaces and the layman

[KjetilK]

IAAABDTTALA (I Am An Astronomer, But Don't Take This As Legal Advice), and I doubt that they are actually aiming this at the layman. What they are doing is opening it up to everyone, and everyone is free to use it and learn how to use it, but really, you expect mainly professional astronomers to use it.

There are lots of databases that follows this philosophy allready, the NASA Astrophysics Data System, the Digitized Sky Survey, not to speak of the larger arxiv.org. You can all grab whatever you like from there.

That being said, there are a number of amateur astronomers who are extremely dedicated and are willing to obtain the skill needed to use such a system, even if there is a tough learning curve. These can be considered "laymen", but they are actually very good at what they do. That's the kind of "laymen" you would expect to use it. Not Joe Sixpack, but the people who are dedicated enough to learn how to use it.

Re:How much will this data get re-analyzed?

[ghostlibrary]

A lot of astronomy data is looked at by its principal investigator (PI) for something specific. Really, data has 5 'lives'.

1) The original proposal by the PI, e.g. 'looking for cornonal emissions from DI Peg, an Algol-type system'. Sort of the pass/fail of the research world.

2) Survey. Someone decides to do a survey study among existing data, e.g. "Light curves from all Algol-type systems".

3) Unexpected. Someone finds a new thing to look for, sometimes due to better theoretical understanding. "Coronal sources should be iron-enhanced, so let's reanalyze DI Peg, specifically looking for iron lines."

4) Data-mining. Searching an archive for a given property. "Looking for all sources with X-ray emission above a given threshold... hey, DI Peg matched!"

5) Grad students. Doing their thesis on a topic, use archival data to support. "Dissertation on coronal systems, using data from DI Peg and others".

So data is often used beyond its initial acquisition!

P2P as an alternative

[DirtyJ]

That's a pretty interesting idea, but I don't think it's applicable to the Virtual Observatory. What is being discussed here is creating a central engine which can seamlessly access multiple large databases which are served out of different locations. These are databases which are frequently all-sky surveys conducted by one group and stored in one central location - not necessarily in small sections on multiple persons' hard drives.

The P2P idea is interesting in that it could apply to individually collected small data sets. Here's how observational astronomy has traditionally worked:

Astronomer writes a proposal to do some research using a specific telescope(s)

Proposal gets accepted after peer review

Astronomer travels to observatory to spend many of his own nights collecting data

Astronomer takes the time to reduce and analyze his own data

Astronomer writes a paper(s) saying, "Hey - look what I did!"

(Sometimes) astronomer writes a proposal for further funding based on the merits of this work

This procedure is inefficient in that you sometimes get multiple people who are not working together, doing the same project on different telescopes. If I collect a bunch of data in one part of the sky, try to use it but don't actually get around to finishing and publishing a paper, and then archive it locally, nobody in the world knows that the data exists. So now if someone else wants to do the same project, they go to the telescope and recollect the same data. In other words, there's no central log of who's done what when it comes to individual observing.

P2P could be useful to remedy this. The problem is that astronomers tend to be very proprietary about their data. Sometimes research and publishing can be very competitive, and you don't want to give the competition an edge when it could mean that they publish a paper on a particular topic before you and reap the rewards, or get funding when you don't. So I think that most astronomers would share their data openly in a P2P network only after they were completely finished using it, and some would never do so.

The difference with the data sets being accessed by the proposed Virtual Observatory is that the people who create those sets typically get their funding with a stipulation that the data be publically accessible some time after the work is finished. They're not allowed to keep it proprietary even if they'd prefer to do so for competition reasons.