Slashdot Mirror
Avoiding a Digital Dark Age
al0ha writes to recommend a worthwhile piece up at American Scientist on the problems of archiving and data preservation in an age where all data are stored digitally. "It seems unavoidable that most of the data in our future will be digital, so it behooves us to understand how to manage and preserve digital data so we can avoid what some have called the 'digital dark age.' This is the idea — or fear! — that if we cannot learn to explicitly save our digital data, we will lose that data and, with it, the record that future generations might use to remember and understand us. ... Unlike the many venerable institutions that have for centuries refined their techniques for preserving analog data on clay, stone, ceramic or paper, we have no corresponding reservoir of historical wisdom to teach us how to save our digital data. That does not mean there is nothing to learn from the past, only that we must work a little harder to find it."
Our landfills will provide all the info they need.
For justice, we must go to Don Corleone
The main way ancient writing reached us is because someone copied it. Lots of copies. Sometimes translated into another language and back, for example, a lot of Greek learning went into Arabic and came back out into Latin or Greek. With all the copy protection and encryption on our media today, can we ever copy the data and be able to decipher it again?
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
It is indeed a big problem. The problem was illustrated recently when Yahoo suddenly pulled the plug on Geocities, wiping out a vast cultural archive that went back to the early days of the internet, a lot of valuable information was lost as a result of that. Yahoo's blatant arrogance caused me to refuse to ever use any of their products again. Geocities was actually a fairly nice service, often people criticised it because of the ads, but how do you pay to continue to offer a free service. The loss of geocities was a perfect example of the need for a permenant store or online archive of information, personal websites and so on that can be maintained as a cultural legacy and informational resource.
I think that many people are failing to appreciate the longevity of information preservation
that cloud computing (more specifically, redundant, geographically distributed network storage) can bring.
If we get the protocols right, and insist on open standards for data interchange, we can obtain
properties such as:
Data bundles that know how to move themselves to more recently commissioned, and/or more
reliable hosts.
Data bundles that know how to check in with copies of themselves, to make sure there are enough of
them alive, and that they are adequately geographically distributed, at every given moment.
If not, then more baby copies of the same data would be produced and stored elsewhere automatically.
There are other issues to longevity of course, like maintenance of software that understands different
versions of data etc. Not trivial but very doable.
How long an individual disk or SSD or stone tablet lasts is COMPLETELY IRRELEVANT to
the prospects for information longevity, given the network, and new levels of automated distribution
that will take place on it going forward.
Where are we going and why are we in a handbasket?
Oh god, why doesn't Gmail have a print all function!?
When you're afraid to download music illegally in your own home, then the terrorists have won!
The Domesday Book was commisioned in December 1085 by King William (aka William the Conqueror, who invaded ngland in 1066). The first draft was completed in August 1086 and contained records for 13,418 settlements in the English counties south of the rivers Ribble and Tees (the border with Scotland at the time). It is a detailed statement of lands held by he king and by his tenants and of the resources that went with those lands. It records which manors rightfully belonged to which estates, thus ending years of confusion resulting from the gradual and sometimes violent dispossession of the Anglo-Saxons by their Norman conquerors.
In 1986, at a cost of £2.5 million, the UK compiled the contents of the Domesday Book into electronic form that was stored on laserdiscs. The information stored on the laserdiscs, which is the equivalent of several sets of encyclopedias, is now unreadable because the equipment needed to read the discs is no longer available. Meanwhile the original book is still readable after more than 900 years.
In my own quest to preserve my digital photos, I've created multiple backups on hard disk including a remote backup which gets updated every few months. I use different disks created by different manufacturers and buy new disks every couple of years (but do not throw away old copies).
I've recently come across another aspect that isn't addressed by the article. Data that is in use in an online copy can be modified (including corrupted).There is no point in copying/propagating data if the data you are copying is damaged. Typically this has happened when I've tried DAM software like Lightroom which will modify the original file despite claiming to be non-destructive I have no proof that photos were re-encoded or quality was reduced but I do know original files were altered, and I want an original unaltered file preserved
Most people when they backup files do very little verification to ensure the files they are copying today are the same files that were created 5 or 10 years ago. They rely too much on backup software to do this for them, with no attention paid to what's happened to the data between copies. To keep this under control I've started putting checksums on all my photo files, which I check when I create a fresh copy.
Of course where my photos are captured in a proprietary format I copy to an open or at least well documented format (typically jpg, sometimes also tif). This is done as soon as I transfer the photos, which are not removed from the camera card until i have 2 additional copies. So I shouldn't have the same issues that the author had assuming jpg can still be read throughout my lifetime.
--
Sammy
These posts express my own personal views, not those of my employer
Google does one better: Gmail Paper
Many of the laws that overly stymie information flow (DMCA etc.), I think, are just a knee jerk reaction in the way printing presses were suppressed, and controlled until everyone realised the benefits of having them opened up.
Barbarians have always burned down libraries. No reason to think they'd stop doing that just because they wear ties these days.
If all else fails, immortality can always be assured by spectacular error.