Slashdot Mirror
Digitizing 100 Years of Astronomical Data
Maximum Prophet writes to mention that a collection of glass plates containing astronomical information from the late 19th century through the mid-1980s is being considered for digitization. "The accumulated result weighs heavily on its keepers on Observatory Hill, just up Garden Street from Harvard Square: more than half a million images constituting humanity's only record of a century's worth of sky. 'Besides being 25 percent of the world's total of astronomical photographic plates, this is the only collection that covers both hemispheres,' said Alison Doane, curator of a glass database occupying three floors, two of them subterranean, connected by corkscrew stairs. It weighs 165 tons and contains more than a petabyte of data. The scary thing is that there is no backup." I'm sure that anyone with a spare $5 million or so would be welcomed with open arms.
165 tons of glass plates?
Sounds like a typical lunch clean-up after Rosie O'Donnel.
Sorry. I'm truly sorry.
Best Windows Freeware
... and call it Alta Vista?
now there is some irony.
I am very small, utmostly microscopic.
You are watching stardata
Cancel/Allow?
...all the alien porn I've been missing!
I'll form my OWN solar system! With blackjack! And hookers!
* Astronomy Knowledge Base
* Astronomy resources on the web
Also, this post will push down the presence of my post on artificial meats on the profile page, and I think it somewhat important to keep the link available.
I'd think they'd hit it.
"I'm sure that anyone with a spare $5 million or so would be welcomed with open arms."
I'll do it. I've saved all the money I didn't spend by downloading entertainment, and I'm willing to give to a more worthy cause.
That's about 6e6 Bytes per gram. Digitizing that data means lots of redundancy while preserving the total mass of this collection.
Google provides views of the Earth, Moon and Mars, why not stars? If the information was made available for them to deliver to their users, they might be interested.
Who else felt the pang of irony considering what happened at Harvard last year with the then president saying something about women and science and math?
I prefer Flambe as apposed flamebait.
This seems like a great opportunity for either corporate sponsorship, or a grass-roots donation drive. In all honesty, $5 million isn't a whole lot of money for the likes of any real corporation, and it probably wouldn't be that hard to raise it through small donations from individuals. Espectially if you could ascribe names to some or all of it. How would it feel to be able to personally identify which plates you paid to have scanned? (this image of the Crab Nebula brought to you by John Smith) I'm surprised Paul Allen or Richard Branson aren't all over this like stink on shit.
I'm sure that a company like google would be MORE than willing to fund a project archiving these. The positive press, proliferation of their intended "do no evil/good guy/just another bunch of geeks" image, having their name on a major scientific project would easily be worth the investment.
NewslilySocial News. No lolcats allowed.
Of course, as long as they can keep mildew at bay, odds are that the plates will long outlast any digital record. Of course it always makes sense to keep a backup, not to mention the value of an instantly-retrievable library.
for Google. Man if I had a spare 5 million I'd be all over that, I love data.
Si vis pacem, para bellum! For evil to succeed good men need only do nothing!
I spearheaded a "digital backup" of around 90 filing cabinets of papers at my last job by spending $1500 on a decent scanner setup and paid a pair of temps to scan everything in. They simply scanned the documents, named the scan the identifier at the top of the page (invoice number or recordID number) and then went to the next one.
It took 2 years and way WAY WAY less than $5,000,000 to do it. granted these are delicate glass plates but hiring a pair of competent people or astronomy students to do it would not take 5 million dollars unless someone was trying to squeeze a summer home and a new farrari out of the deal.
A decent lighted copy table and a cheapish 20 megapixel hasselblad digital camera with the right lens will do the trick nicely.
Do not look at laser with remaining good eye.
Glass photographic plates, especially from silver emulsion, are analog at extremely fine granularity. Effectively molecular, depending on how flat the glass surface was settled from its molten liquid state. The features of its silver oxide crystals, laid in place by individual photons arriving from vastly distant stars, could be meaningful at less than a nanometer. Especially when measuring extremely subtle influences, like the gravity from one distant star bending the light of another distant star, measured across a century in which those stars lost gravitational mass, for comparison.
There is a practically infinite amount of data on each of those plates, limited by our precision in measuring them. It's a smaller degree of infinity than that of the sky. But the original infinite sky is lost. While the plates' lesser infinities are impossible to replace, and all we'll get to use to look back across all the billions of years we saw in a long century of them.
--
make install -not war
a thousand minibuses full of magtape.
That's what she said!
Maybe we'll get some data on possibly extra solarplanets from this?
My web domain.
When I completed my Astronomy masters access to publicly available data from various sources (most notably NASA data made free to the public) was a real boon. It meant we could do analysis on actual real data instead of artificial or sanitized textbook material. A couple of the students built on this to do some original research. (Sadly that's not the way I went, as my time was more limited).
There are also lots of amateurs out there running a wide variety of very specialized packages to do everything from discovering asteroids to keeping tabs on the brightness of stars and watching for supernovae.
These posts express my own personal views, not those of my employer
From here: http://www.hno.harvard.edu/guide/finance/index.htm l,
This:
Harvard University's endowment, valued at $25.9 billion at the end of FY 2005, is a collection of more than 10,800 separate funds established over the years to provide scholarships; to maintain libraries, museums, and other collections; to support teaching and research activities; and to provide ongoing support for a wide variety of other activities. The great majority of these funds carry some type of restriction.
I think they can scare up the change.
If they manage to standarize a century of these plates, it would significantly extend the time range of data to digitally extrapolate and detect objects previously missed. Just to speak of mapping our own cosmic backyard, a significant amount of slow moving, previously undetected Kuiper Belt Objects, for example, would more easily pop into view. Surely a bunch of comets, too.
Clyde Tombaugh captured Pluto several times during his three decades long hunt for the elusive Planet X, but failed to put the pieces together. If he had had digital technology, he would have shaved off at least a decade of effort. So imagine all the extremely useful raw data still stored in those plates.
Lil' Thindime, lilting a lacrimose lament, krashes the kwaint konfines of Kokonino Kounty
A simple way to fund this would be to sell the scanned plates as scientific history artifacts/souvenirs. I bet you could not only sell them to universities worldwide but also to cosmologists, scientists and astronomy fans in general.
I mean heck, $10 a plate for 500K plates gets us to $5M. I'd pay that without even knowing what I was getting. Up it to $25, $50 or $100 and I'm probably still interested.
=======
Science -- Sealed, Delivered.
Luckily glass isn't a liquid so they won't distort.
No sig today...
"This seems like a great opportunity for either corporate sponsorship, or a grass-roots donation drive. In all honesty, $5 million isn't a whole lot of money for the likes of any real corporation, and it probably wouldn't be that hard to raise it through small donations from individuals."
I doubt there's a million geeks on the entire planet.
that's astronomical!
There is more to this than simply scanning a flat image. The emulsion on these plates is a three dimensional medium, and different data can be extracted depending on your focal depth into the the emulsion. I believe David Malin did much pioneering work on this kind of thing, including the use of different layers for unsharp masking.
There will be information in the plates that is not yet part of human knowledge, and a simple scan of one focal plane is not going to get it all.
Certainly it is worth taking backup images of these plates in any way we know how, but we should remain aware that, as of today, no technology exists that will make exact duplicates of them, so great care should always be taken to preserve the originals.
Microfiche has a short life span. When I was working at the Royal Greenwich Observatory they'd done some research and discounted that as a feasible option. Something like 25 years if you're lucky? The glass plates in the RGO were from the same period as these American ones, and in equally reasonable condition (in most cases.... the problem was there as well... we were transferring them to acid free paper sleeves).
That answer astounded me as in our own project the point was to make the data public as efficiently as possible. I mean, their funding is public, so why not their data ? I can understand holding onto it until you have a paper published, but after than it should be required in the funding statement. I don't know if this is typical of the field of astronomy, but I've searched high-res sky images in the past without finding anything systematic except some specific projects such the Sloan Sky survey (which are just coordinates) or the odd marketing Hubble shot.
Non-Linux Penguins ?
Seriously, let Google index not only that collection, but any stellar image information and launch GoogleSky.
What's in a sig?
I don't have US $5 Million to offer, but how about a suggestion. Set up a mechanism to have individuals/entities sponsor (pay for) the digitization of individual plates. The sponsor gets a public credit for being the sponsor (perhaps displayed around the exterior perimeter of the digital image also). I think the astronomy fans would help. Perhaps organization like Sky and Telescope magazine or Astronomy magazine would sponsor groups of plates. Perhaps schools could be induced to have the kids collect to sponsor a plate for their class/grade. Once the public excitement is largely over, then see if a white knight will finish the job.
Step 1: Ask 15 to 20 major companies to each sponsor a "scanning trailer". They'd get their name and logo all over it and be part of the on-going story and never-ending literature, etc.
Step 2: Build-out a tractor trailer per sponsor to include everything needed to do scanning of archived materials (books, papers, photos, glass photo plates, etc.). Power source, scanners (many per trailer), etc.
Step 3: Drive the swarm of scanning trucks to the parking lots of an archive in need of backup.
Step 4: Connect the truck's network output to the archive's network to store the scanned data.
Step 5: Get local volunteers to work with the "full-time professional" in each truck to retreive (a little at a time), scan, and return the materials to the archive.
The plan would be to drive this swarm around the country, full time, and do this kind of work whereever it is needed.
Error:
You can be sure that all those PETA-bytes are vegetarian!
Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
Cost of storage? Free!!! They should get a few gmail accounts and store the scans there. Occasionally mail them between accounts for redundancy. 8-)
Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
Not sure what's meant exactly by it being the "only collection to cover both hemispheres". The Digitized Sky Survey covers the whole sky and it's been online for 12 years.
Exit, pursued by a bear.
I don't think it would be hard to find some company to pay $5m, if they could keep the rights to the images, and pull a Westlaw type of scam. I am sure Harvard-Smithsonian isn't going to fall for this. They want to keep these images for the public, which makes it difficult for anyone to build a business model on and therefore difficult to get funding for. How would Google make money on this? Google adwords for a particular star? Or perhaps on google maps - "coffee near Barnard's star"? I am not saying that Google won't do this, just that it's not as simple of a decision for them as you might think. There's really no way to prove it's value to Google stockholders.
I am sure that the glass plates aren't going to be thrown away when this is done. They'll just be moved away from the very expensive Cambridge real estate on which they currently sit, and the space will be reused for storing more astronomers.
If that's all you found, you didn't look hard enough. Sloan serves imaging and spectral data, and all of Hubble's science data (for example) has been available from three different data centers since 1992. (This is data we're talking about, not pretty pictures.) In fact, all NASA-funded missions are required to archive their data, and NSF is (finally) getting into the act. I don't know what ESA requires, but I know they're building a large archive. And just about every large ground-based project in development has a significant archival component.
I'd say your previous employer's attitude ("our funding, our data") is the exception nowadays. Even privately funded projects are looking at archives, if only to connect to the VO.
Exit, pursued by a bear.
If you were a professional astronomer I'd say it sounds like you'd be better off finding a different organization to work for.
Try looking at cited sources on published papers for starters. http://arxiv.org/ will give you plenty of pre-publications. Here too http://sesame.stsci.edu/library.html
I'm well out of touch but here's what you get just from Google:
Skyview is a must. Images in any wavelength (multiple instruments)
http://skyview.gsfc.nasa.gov/
Learn about the FITS data format. Not just pretty pictures by any means.
http://fits.gsfc.nasa.gov/
Chandra data
http://cxc.harvard.edu/cda/public.html
You want Hubble data? (and software to process it)
http://archive.stsci.edu/
More software to process astro data:
http://www.stsci.edu/resources/
SOHO use to publish their images in real time and if you want data...and apparently still do.
http://sohowww.nascom.nasa.gov/data/data.html
There's lots more out there if you look at the major space instrument's web pages. I'm sure some of it is paid and a lot of it is held back for a year or so, but there's a LOT out there.
These posts express my own personal views, not those of my employer