Slashdot Mirror
Digitizing 100 Years of Astronomical Data
Maximum Prophet writes to mention that a collection of glass plates containing astronomical information from the late 19th century through the mid-1980s is being considered for digitization. "The accumulated result weighs heavily on its keepers on Observatory Hill, just up Garden Street from Harvard Square: more than half a million images constituting humanity's only record of a century's worth of sky. 'Besides being 25 percent of the world's total of astronomical photographic plates, this is the only collection that covers both hemispheres,' said Alison Doane, curator of a glass database occupying three floors, two of them subterranean, connected by corkscrew stairs. It weighs 165 tons and contains more than a petabyte of data. The scary thing is that there is no backup." I'm sure that anyone with a spare $5 million or so would be welcomed with open arms.
165 tons of glass plates?
Sounds like a typical lunch clean-up after Rosie O'Donnel.
Sorry. I'm truly sorry.
Best Windows Freeware
now there is some irony.
I am very small, utmostly microscopic.
Google provides views of the Earth, Moon and Mars, why not stars? If the information was made available for them to deliver to their users, they might be interested.
This seems like a great opportunity for either corporate sponsorship, or a grass-roots donation drive. In all honesty, $5 million isn't a whole lot of money for the likes of any real corporation, and it probably wouldn't be that hard to raise it through small donations from individuals. Espectially if you could ascribe names to some or all of it. How would it feel to be able to personally identify which plates you paid to have scanned? (this image of the Crab Nebula brought to you by John Smith) I'm surprised Paul Allen or Richard Branson aren't all over this like stink on shit.
I'm sure that a company like google would be MORE than willing to fund a project archiving these. The positive press, proliferation of their intended "do no evil/good guy/just another bunch of geeks" image, having their name on a major scientific project would easily be worth the investment.
NewslilySocial News. No lolcats allowed.
Glass photographic plates, especially from silver emulsion, are analog at extremely fine granularity. Effectively molecular, depending on how flat the glass surface was settled from its molten liquid state. The features of its silver oxide crystals, laid in place by individual photons arriving from vastly distant stars, could be meaningful at less than a nanometer. Especially when measuring extremely subtle influences, like the gravity from one distant star bending the light of another distant star, measured across a century in which those stars lost gravitational mass, for comparison.
There is a practically infinite amount of data on each of those plates, limited by our precision in measuring them. It's a smaller degree of infinity than that of the sky. But the original infinite sky is lost. While the plates' lesser infinities are impossible to replace, and all we'll get to use to look back across all the billions of years we saw in a long century of them.
--
make install -not war
a thousand minibuses full of magtape.
Those plates as well as being old and delicate contain a LOT more information then a piece of paper. Considering that something less then 1/4 the size of the period on the end of the sentence is important your scanning at a much higher resolution.
Si vis pacem, para bellum! For evil to succeed good men need only do nothing!
How about we make a backup of the backup on glass plates...
Ack! Put down that knife!
That's what she said!
Google might do it just because it would be un-evil, and worth quite a few brownie points with scientists around the globe, not to mention that it would be cool archive to search.
The higher the technology, the sharper that two-edged sword.
> I spearheaded a "digital backup" of around 90 filing cabinets of papers ...
> It took 2 years and way WAY WAY less than $5,000,000 to do it
500,000 plates. Over 2 years, assuming 50 wks/yr means just 5000 plates need be scanned per week. 1000 plates per day. 125 plates per hour. And this is large, fragile glass with really high data density, so you have to be a) careful in handling and b) use slow high-res scanning.
Let's take a guess that it takes only 10 minutes per plate (to fetch, tag, load, scan, and return). So we need only 20 people to scan 125 plates/hour.
Well, assume 20 scanning people and 1 IT guy handling the sysadmin work for the petabyte storage. Also one scientist/manager. Take a low intern/grad student $35k, 1 sysadmin at $65k, 1 PM/sci at $85K. All x2.5 for overhead, for 2 years. That's $4.25 mil in salaries.
There's also buying a redundant petabyte and all the necessary gear. I'm amazed they figure $5mil can do it.
A.
It depends on the number of pounds in a ton, but if it's short tons then
165 short tons = 149,685,482 grams
1e15 / 149,685,482 = 6,680,674 bytes per gram
A quick check of amazon turns up a 1TB drive which weights 2.4 pounds.
That's 1,089 grams which is 918,592,757 bytes per gram.
Unless I've messed up my math, it looks like hard drives store 137 times more information per gram. That's not as large a multiple as I had imagined though. The whole thing should still be between 1 and 2 tons when put on hard drives.
Cow Cube
When I completed my Astronomy masters access to publicly available data from various sources (most notably NASA data made free to the public) was a real boon. It meant we could do analysis on actual real data instead of artificial or sanitized textbook material. A couple of the students built on this to do some original research. (Sadly that's not the way I went, as my time was more limited).
There are also lots of amateurs out there running a wide variety of very specialized packages to do everything from discovering asteroids to keeping tabs on the brightness of stars and watching for supernovae.
These posts express my own personal views, not those of my employer
From here: http://www.hno.harvard.edu/guide/finance/index.htm l,
This:
Harvard University's endowment, valued at $25.9 billion at the end of FY 2005, is a collection of more than 10,800 separate funds established over the years to provide scholarships; to maintain libraries, museums, and other collections; to support teaching and research activities; and to provide ongoing support for a wide variety of other activities. The great majority of these funds carry some type of restriction.
I think they can scare up the change.
Google Universe (beta)
Searchable in Lat/Lon/time/intensity
that would be awesome...
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
If they manage to standarize a century of these plates, it would significantly extend the time range of data to digitally extrapolate and detect objects previously missed. Just to speak of mapping our own cosmic backyard, a significant amount of slow moving, previously undetected Kuiper Belt Objects, for example, would more easily pop into view. Surely a bunch of comets, too.
Clyde Tombaugh captured Pluto several times during his three decades long hunt for the elusive Planet X, but failed to put the pieces together. If he had had digital technology, he would have shaved off at least a decade of effort. So imagine all the extremely useful raw data still stored in those plates.
Lil' Thindime, lilting a lacrimose lament, krashes the kwaint konfines of Kokonino Kounty
Luckily glass isn't a liquid so they won't distort.
No sig today...
Holy crap dude, you just won the asshat of the year prize. Do you have any idea of the magnitude, delicacy, or importance of the data you're talking about? To say nothing of the needed precision when scanning.
"I scanzord 90 filing cabinets of paper into teh computerz"
You know what, I used to launch model rockets. Its really easy to make stuff go up. Just buy the kit, attach a little engine and off it goes. $30 easy! Freakin NASA I bet they're spending all of our tax dollars on pr0n.
"cheapish 20megapixel camera" - Ever hear of the Hubble? I hear people like it for more than those weird nebulae pictures. I guess we should have just given one of those astronuts a Nikkon and let him go to town. Much cheaper.
And I guess we should use lossy compression, its just empty space out there right? I bet we could get the infinite sky down to a couple hundred GB. (JPEG, its for astronomy too!)
that's astronomical!
There is more to this than simply scanning a flat image. The emulsion on these plates is a three dimensional medium, and different data can be extracted depending on your focal depth into the the emulsion. I believe David Malin did much pioneering work on this kind of thing, including the use of different layers for unsharp masking.
There will be information in the plates that is not yet part of human knowledge, and a simple scan of one focal plane is not going to get it all.
Certainly it is worth taking backup images of these plates in any way we know how, but we should remain aware that, as of today, no technology exists that will make exact duplicates of them, so great care should always be taken to preserve the originals.
You need specialized scanning machines for astronomy. Office equipment doesn't do the job.
My colleagues in the UK had such a scanner. It was ~7 tonnes of metal, glass and electronics (heavy so as to be very stable), lived in its own building and needed several clever people to keep it running. Building one of these (or cloning one you already have so as to work faster) could cost a big chunk of the $5M.
The scanner I knew took ~ 30 minutes to scan a plate. For the harvard collection, choose between one scanner (which they may not have; otherwise why did they wait until now to start the project?) and a long project with big sallary bill, or multiple scanners, at extra capital cost, and less money for people.
Seriously, let Google index not only that collection, but any stellar image information and launch GoogleSky.
What's in a sig?