Slashdot Mirror
Lockheed Chosen For Electronic Records Archives
TrentL writes "How will we be able to read 1990's email messages in the year 2090? Will GIF files still be accessible in 2105? The US National Archives - tasked with preserving records "for the life of the republic" - has chosen Lockheed Martin to solve exactly this problem. Lockheed was awarded the $308M Electronic Records Archives contract after a year-long design competition. Full Disclosure: I worked on Lockheed's demo team."
It's not just the government that needs this. Since we're funding this effort with our taxpayer dollars, I'm hopeful that some of the results from this work will lead to the availability of tools us normal folks can use to make sure our precious data can be preserved and passed down from one generation to the next.
Are you against the National Archives? This program enables the National Archives, into which we've already sunk billions over the centuries, to continue to be (even more) useful in the Information Age. That's our information. Why should we throw it away now?
I'm curious, did you have any criticism for the $300M "bridge to nowhere" in Alaska when it was reported in the new budget this year? And where are you on the $200B+ we're spending in Iraq?
--
make install -not war
For a start, they should stop using stupid proprietary formats like Real Video (the Press Conference Video on their website is only available for Real Player).
I remember reading an article about the archival of scientific research; many researches involved in the discovery of DNA's structure didn't keep their (hand-written) notes, but they were later recovered by others who saw the value of such significant documents to future generations.
Nowadays, of course, we can just trash something by clicking the delete button, and one the hard drive's formatted, it's gone. This does make me wonder how much historical and scientific information will be lost to future generations simply because of this ease of deletion.
First off, you do not seem to know (or do not remember) that NASA is losing all sorts of data. They have 2 problems. Just 40 years ago, they were storing data on Tape Drives. The tapes are decaying so the data is disappearing. In addition, the formats are disappearing. Back then, all the specs were written down, and yet, the formats are hard to find in mountains of data.
SO now, forward a hundrew years. Just 15 years ago, I was working with CDs that would last 100 years (50 bucks a pop). Now, ppl seem to assume that the current disk will last that long. They will not. The old disks were made out of thin gold sheets in plastics. They are now some plastic in plastic. These CDs/DVDs will last less than 10 years (and probably closer to 5). In addition, the tape drives and hard disks are storing million time more data than what was in tape in the 60s. That is the storage density is WAY up. So now, as a small pox shows up, it will affect millions x more data, making recovery very difficult.
I prefer the "u" in honour as it seems to be missing these days.
This has a fundamental chicken and egg problem: So you store the information, you also need to store the format of that information. So then how do you read "format of the information" document? What format is *that* in?
Latin, videlicet.
But seriously the problem in records is not going to be collecting the data, but turning it into knowledge. Meaning that humans in the future are likely to seriously misinterpret or be unaware of the intended meanings and social and political contexts of the preserved data.
This is not a technology problem.
They ought to make sure that real professional historians are there.
Don't knock Walt Disney. The man was a genius, and the pioneer of modern animated films. The corporate Disney that we know today should not diminish the work of one of the 20th century's greatest imaginative minds.
Technically I don't see any problem with storing 100PB of data in the next decade, and keep it safe from natural disasters. But how about unnatural disasters, such as an evil administration changing the entire archive to reflect better on itself or protect itself from criminal prosecution? Copies of the archive packages need to be suitable dispersed in multiple jurisdictions or even shot into space in order to make this kind of data destruction infeasible.
The IDE-raid doesn't solve all the problems. One of the big ups for tape backup is that you can take it off site. So you either have to copy the raid and move it off site or setup a second set of systems off site.
Either way tapes still hold some value in offsite storage.
For ordinary backup requirements, where data only has to be retrievable for a few months or years, disks can be useful. Under these conditions, the mean time between failures of the backup drives is at least as good as that of the production drives.
Archival backup, however, depends on an extremely low rate of failure over a very long time. The ideal backup medium is not only stable but can also be read using simple methods, so that failure of mechanism will not make the data irretrievable. For that purpose, disk drives aren't good candidates.
Parity: What to do when the weekend comes.
Yeah I've got to agree with this. If pages of the bible made it this long on paper (and commodore emulator geeks are going to be around forever by the looks of it), I have my doubts that machines are going to be having trouble interpreting code for reading ascii or utf8. Please...if the data's that important then the people who own it should upgrade it to the latest format (if the old is 'suddenly'about to become totally obselete).
All these people whinging about about how cd's won't last - I'm pretty confident that if I bother to hold on to the cdroms in my draw, provided they're kept in their cases/good condition they'll be just as playable (on the same hardware) in 100 years.
Frankly I hope (probably all) of the stuff in my e-mail isn't around in 100 years. What's this obsession with holding on to temporary/garbage data? If it's important, back it up, look after it/upgrade the format if you must (that's what people did with out of print lp recordings). $308 million, jeez.
"You know you don't act like a scientist, you're more like a game show host." Dana Barret
I have been saying for years that the DoD should make an initiative to move towards open standards for this exact reason. The document retention requirements they have are incredible, and yet nearly all the documents generated are saved in proprietary formats. Now with the OASIS (OpenDoc) format solidifying and there is more than one implementation of it, they wouldn't even have to define a standard for word processing or spreadsheets.
Obviously, open standards are not a panacea. There are countless standards created by the military that never really spread farther than that, and therefore the support for them is limited (and thus companies that do support it can charge a pretty penny for it). And with open standards, at it is much easier to write an implementation if you need to. Compare this to MS Word, which is a pain to reverse engineer now, just imagine having to do so in the distant future, when it is not as widespread. And of course, for the very long term, nothing is more certain (and more inconvenient) than printing everything out and storing it in a warehouse, which is what is done now. But the longer that can be postponed, the more money can be saved.
As an added bonus, just imagine the competition that would spring up in the word processor market, if the DoD mandated that all new word processor documents generated internally or by contractors be in OASIS format, starting 5-10 years from now. Microsoft would have to support it (and well) or throw away a huge number of Office sales. The DoD would no longer be locked into a single vendor, saving them money upfront in addition to the money they saved on document retention.
Until then, the best plan is likely to convert as much as possible to a few standards like PDF, which is what I expect will happen here.
I'm trying to find out where in our Constitution does the Federal Government find an enumerated power to pay for this.
Wow, you can access the Constitution? I mean it was written in 1776. That's a long time ago. Good thing somebody thought to save it!
We're saving lots of data, because 1) lots of it is important and 2) we have very little perspective on it yet. In 200 years we might very well have a very different idea of what was important today.
So close and yet so far from the world's perfect ID number
People are severely over complicating this problem.
While making things last as long as possible is a good thing, you can't plan for 100 years down the road. You have NO idea what will truly happen.
There needs to be a system to move data from older mediums to newer mediums every few years as they become available. Multiple copies, with verification. Checking. Double checking.
Complex data backup solutions and the use of lossless formats has not, for example, kept the critical Pioneer space probe data available, after less than 30 years (http://www.planetary.org/news/2005/pioneer_anomal y_faq.html)
How in the fucks sake do you expect this to last 100+ years? Don't use lossy compression? How is that a solution?
Take Windows Bitmap image format. It's not lossy. That doesn't mean that we won't forget how to display the damn thing...
Raid 5? What problem do you think you're solving? Keeping the data around, or making the data accesible for (as the OP makes clear is the LoC's responsiblity) as long as the United States exists?
Humans will be extinct in 100 years. How many of us think we can really last for 100 more years before we have another couple world wars?
If humans do exist in 100 years, I can guarentee that this economy wont.
This has been discussed before. The sheer volume of data that would have to be copied every few years is HUGE. How long would it take you to transfer a stadium full of CDs onto DVDs? How much would that cost?
There's good reason people are looking for digital technologies that are as inherently stable, long-lasting, and reliable as writing on paper.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant