Lockheed Chosen For Electronic Records Archives

TrentL writes "How will we be able to read 1990's email messages in the year 2090? Will GIF files still be accessible in 2105? The US National Archives - tasked with preserving records "for the life of the republic" - has chosen Lockheed Martin to solve exactly this problem. Lockheed was awarded the $308M Electronic Records Archives contract after a year-long design competition. Full Disclosure: I worked on Lockheed's demo team."

Comments

Why not?

[Poromenos1]

Analog media couldn't be restored because the machines that read it broke (couldn't they make new ones?) but as long as the specs exist, I don't see why they won't be able to read the digital data (assuming we still use two bits in the future).

Re:Why not?

[WindBourne]

First off, you do not seem to know (or do not remember) that NASA is losing all sorts of data. They have 2 problems. Just 40 years ago, they were storing data on Tape Drives. The tapes are decaying so the data is disappearing. In addition, the formats are disappearing. Back then, all the specs were written down, and yet, the formats are hard to find in mountains of data.

SO now, forward a hundrew years. Just 15 years ago, I was working with CDs that would last 100 years (50 bucks a pop). Now, ppl seem to assume that the current disk will last that long. They will not. The old disks were made out of thin gold sheets in plastics. They are now some plastic in plastic. These CDs/DVDs will last less than 10 years (and probably closer to 5). In addition, the tape drives and hard disks are storing million time more data than what was in tape in the 60s. That is the storage density is WAY up. So now, as a small pox shows up, it will affect millions x more data, making recovery very difficult.

Re:Why not?

[tabkey12]

The specs could easily be lost over a long period of time, and it's very hard to reverse engineer algorithms from scratch (given that in 100 years, newer and more optimal algorithms than, LZW will be used). It's predicted that the only image format that will still be around in 100 years is ppm, simply because it only takes about half an hour to implement from scratch!

Re:Why not?

[RiotXIX]

Yeah I've got to agree with this. If pages of the bible made it this long on paper (and commodore emulator geeks are going to be around forever by the looks of it), I have my doubts that machines are going to be having trouble interpreting code for reading ascii or utf8. Please...if the data's that important then the people who own it should upgrade it to the latest format (if the old is 'suddenly'about to become totally obselete).
All these people whinging about about how cd's won't last - I'm pretty confident that if I bother to hold on to the cdroms in my draw, provided they're kept in their cases/good condition they'll be just as playable (on the same hardware) in 100 years.
Frankly I hope (probably all) of the stuff in my e-mail isn't around in 100 years. What's this obsession with holding on to temporary/garbage data? If it's important, back it up, look after it/upgrade the format if you must (that's what people did with out of print lp recordings). $308 million, jeez.

Re:Why not?

[TrentL]

All these people whinging about about how cd's won't last - I'm pretty confident that if I bother to hold on to the cdroms in my draw, provided they're kept in their cases/good condition they'll be just as playable (on the same hardware) in 100 years. Frankly I hope (probably all) of the stuff in my e-mail isn't around in 100 years.

The amount of data we are talking about is HUGE. There is no way humans could manually upgrade the data. It would be a technical and policy nightmare. As for preserving emails, the email messages of the executive branch contain much historical significance.

Re:Why not?

[wo1verin3]

People are severely over complicating this problem.

While making things last as long as possible is a good thing, you can't plan for 100 years down the road. You have NO idea what will truly happen.

There needs to be a system to move data from older mediums to newer mediums every few years as they become available. Multiple copies, with verification. Checking. Double checking.

Re:Why not?

[drsmithy]

They are now some plastic in plastic. These CDs/DVDs will last less than 10 years (and probably closer to 5).

Even that's pretty generous IMHO. In my experience, recent blank CDs (and DVDs) are lucky to make out 18 months, and many of mine are delaminating or corroding after only 12. I've now gotten into the habit of burning two copies of everything I "archive", and re-burning them every 12 months. Thus far I've had errors, but never errors in the same place on each copy.

Contrast this to the good old "Kodak Gold" CDs I was burning onto back in 1996, almost all of which are still readable with 0% errors...

Re:Why not?

[jcr]

A couple of years back, I worked for a company that was building medical records-keeping systems for a Workman's Comp insurance company. We had a statutory requirement for the records to persist for fifty years, I think it was.

Our plan allowed for perishable media. The data was kept on CDs in cold (not cryogenic, but cold) storage, and they were scheduled to be read in, and burned to new disks every couple of years.

-jcr

Re:Why not?

[Frank T. Lofaro Jr.]

In 100 years, there will STILL be many who would not have switched away from GIF. :/

Re:Why not?

[2Bits]

As for preserving emails, the email messages of the executive branch contain much historical significance.

Blah, if that has so much historical significance, you just need to post it to the newsgroups, and it will be preserved for as long as internet exists.

Re:Why not?

[WindBourne]

Why do you say that and what do you predicte it on?

As it is, those tapes were suppose to last 100 years. And you can bet on it, they were certain that the formats would be around.

Keep in mind that in 1987, I was working at Monforts Beef (now conagra). I saw that in ~15 years, we were going to run into issues with our 2 letter datestamps. When I started pushing that we needed to solve it as we did maintence, their response was that it was not needed. They felt certain that all of the code would be re-written LONG before then.

Of course, as luck would have it, in mid-1999, I was teaching a Java class, and one of my students was from Monforts. She informed me that they were desperate to get all the code re-written and tested before the year-end. It turned out that very little bit of code changes had happened in 15 years. It turned out that things were actually far worse (more programs still using the same structures). IOW, they were so certain that it would change and it did not.

Now, you are certain that it will not change in 100 years. Yet, I would bet that by 2050, GIF will at best a dieing lib, but still kicking. But by 2075, nearly all apps will have removed it, just due to the fact that no maintainer will want it. Now is the time to solve it.

Re:Why not?

[HiThere]

Just switch to xpm. It may take a bit more space, but when formatted correctly it can be read with the naked eye (if you're willing to tolerate a bit of image degradation). And if you included some pictures of natural phenomena, e.g. a rainbow, a stellar map, and true color photos of Mars, Venus, Jupiter, and Saturn I think the coloring could be deciphered even if the specs were lost.

Re:Why not?

[AstrumPreliator]

I completely disagree. While the PPM format is simple it will still be horrible to reverse engineer in the year 2105. Think about it, they view it all in binary. The first thing you should see is the header (which is ASCII). The first field is Pn where n is a number, then any type of whitespace, then the width, then whitespace, then height, then whitespace, then the scale, then whitespace, then data. Don't forget that anywhere inbetween Pn and the scale can have a comment started by # and ended by a linefeed. Now if they even have the ASCII character set in 2105 it might be possible to reverse engineer the format. However, by 2105 we could have quantum computers or something way different. Trying to figure out when a header ends and the data begins would be a nightmare.
 
And I actually just got done writing a PPM reader for my C++ class. It took more than 30 minutes to write and I had the specs for the format.

Re:Why not?

[goodcow]

What brand(s) are you using that are failing so quickly? I would assume, or at least hope, that more expensive media like Verbatim isn't going to fail for decades.

Re:Why not?

[bdcrazy]

Took me about 45 minutes back when i was working on a image toolkit for kettering medical center.

Messing with 25MB ppm images of brains and hearts and stuff is kinda fun.

Re:Why not?

[Idealius]

I believe you're arguing a different point than he.

You're referring to a hypothetical situation where the TXT format is lost.

Your wiki source betrays you! The fact the ppm's data is arranged pictorially means it's fairly obvious we'll easily decode it in the future, with image dimensions to boot.

Re:Why not?

[dieman]

Mitsumi gold cd's are the only way to go. We buy 500-packs of them. Way more durable *and* they last way longer.

Re:Why not?

[hjf]

ok I don't know about nasa, pal. but I can honestly say I once found an 8-track player, dated 1970-something. It worked, all it needed was a new belt. I got it and it worked.
So I went looking for a tape and found one at some pawn shop. the tape was about 20cm from the ceiling, and judging by the amount of dirt that was on it, it had been there for a LONG time. I don't remember what the music was, but the tape was dated 1960-something. I popped it on the player and it played. great. Once and again.

Now, if I made a 25 year old piece of junk work with just a new belt, and a 35 year old tape played on it, why would NASA have problems restoring the data from old tapes? if the tapes were really damaged beyond reading but they REALLY wanted to read them, they WOULD HAVE already.

Re:Why not?

[patio11]

53 years after SpamAssassin 17.2 and the Comprehensive Death Penalty for Electronic Terrorism Bill lick unsolicited bulk email for good, historians will be baffled at why Dubya was SO interested in a substance called "v1agr@".

Re:Why not?

[stonecypher]

Actually, you seem to have missed the point of the parent post. I tend to agree with him, but I'll phrase it differently.

Yes, physical media is a serious problem. However, TFA is not talking about physical media. TFA is talking about encodings. There's no reason why email shouldn't be readable in 2090, provided we take the time to progressively archive it on modernized media. Now that backups are automated, this isn't nearly as big a problem as when we're dealing with storage devices that Kirk would have had.

GIF is not going to disappear. The idea that we'll have trouble unwrapping LZW in a hundred years is just nonsense. You want to maintain your data? Fire up your backup program, and replace those eighty cent DVDs once every five years.

It's just not that hard.

Re:Why not?

[masklinn]

How 'bout paying a few more bucks for quality CDs/DVDs instead of buying the cheapest shit that has started decaying at a hellish pace before you even opened the shrink wrap?

Cheaper is not always cheaper in the end, cowboy

Re:Why not?

[AvitarX]

The principals of LZW will not be lost. We will still use data compression of some type (more spae will always be better) and LZW will be part of it at least as a background, a here is something anyone can understand and visualize, even if it is useless (or almost useless). We still teach the bouble sort, it was one of the first non hello world programs at class, I know technically it is fast when data is expected to be sorted already, but it wasn't taught as such, it was the first looping algorythm taught in stractured programming, and I think it was the first one taught in object oriented programming too (didn't take it).

The only way I see somethign like LZW as a probkem is if binary computing is completly eliminated and there is confusion as to what to do wiht all these ones and zeroes. I personally think media will be a far harder problem to cope with than formatting. Look at the alphabet from 200 years ago, it is a lot alike, ASCII will remain intact too, as will conventions for deviding things, numbers too will be the same (endianess may change, but it is not too hard to figure out).

Texture of the data will show when the header ends and the data begins, and ASCII would take less than a half hour to decode from a single text document (hell I can do a cryptic byword in that amount of time, and it is a 2 sentance sample).

I bet a smart computer archiologist could decode the format in under a week worse case positive future (obviously if thee is a nuclear war and he is trying to read a floppy with a microscope it aint gonna work).

Re:Why not?

[FireFury03]

How 'bout paying a few more bucks for quality CDs/DVDs instead of buying the cheapest shit that has started decaying at a hellish pace before you even opened the shrink wrap?

The problem is identifying _which_ brands are durable. I've not seen any independent tests that compares the life of the different brands that are on the market. And before you start saying that clearly the more expensive discs are better, this is clearly not true - I have many TDK discs (one of the higher cost brands at the time) which are degrading. I'm also not at all convinced that the discs that claim to last 100 years really will last that long.

I wish someone would come up with some _cheap_ write-once solid state storage, preferably in an existing format like Compact Flash or something, which could be used as "electronic film" for storing important long-lived data such as photos, etc. for guaranteed long periods of time (I'm talking 100 - 200 years here). I just don't trust *any* of the current technologies to store my data for a significant amount of time... maybe an option would be to construct such devices in an internal RAID style configuration for redundency too...

Re:Why not?

[evilviper]

There needs to be a system to move data from older mediums to newer mediums every few years as they become available. Multiple copies, with verification. Checking. Double checking.

This has been discussed before. The sheer volume of data that would have to be copied every few years is HUGE. How long would it take you to transfer a stadium full of CDs onto DVDs? How much would that cost?

There's good reason people are looking for digital technologies that are as inherently stable, long-lasting, and reliable as writing on paper.

Re:Why not?

[evilviper]

In my experience, recent blank CDs (and DVDs) are lucky to make out 18 months, and many of mine are delaminating or corroding after only 12.

Sounds like you're either getting the very cheapest "free after rebate" CD-Rs you can find, or you are doing a very poor job of handling those CDs properly.

Sticking with major brands (eg. Memorex, and TDK) I've got dozens of CDs recorded 5 years ago, that are still doing fine. Of course, I've always handled CDs quite well. I store them in a closed CD cabinet, put them in decent jewel cases, etc. I also make 2 copies of anything I want to keep, but so far I haven't needed the backup copies even once.

I'm sure I will eventually, but the point is that your "18 months" assesment is so far off the mark, it's ridiculous.

Re:Why not?

[Anonymous Coward]

I don't really think you can say that... I won't even get into reasons why.

Plus, do we really have today a complete historical archive of USENET? Hasn't a small, early portion of it been lost or at least not yet recovered? It could happen again, even if it seems unlikely right now... -- tmegapscm

Re:Why not?

[evilviper]

I have many TDK discs (one of the higher cost brands at the time) which are degrading.

Either your mishandling them, or you were unlucky enough to buy them in the "dark ages" when TDK (shortly) switched manufacturers, before discovering the crap they were getting, and switching back.

I wish someone would come up with some _cheap_ write-once solid state storage,

This isn't magic. You can't sprinkle fairy dust on cheap crap technologies, and magically convert them into high-end equipment. If you want some really long-lasting data storage, then you have to spend money to get it. Write-once MO technology might be an option, but it's not cheap, and not common enough to expect the hardware to be around in 200 years. You can always build a new drive for it, but it goes back to that "cheap" part.

Re:Why not?

[FireFury03]

Either your mishandling them, or you were unlucky enough to buy them in the "dark ages" when TDK (shortly) switched manufacturers, before discovering the crap they were getting, and switching back.

Rubbish - they are stored in normal dry room temperature conditions in jewl cases and were bought over a period of several years from a number of different suppliers.

This isn't magic. You can't sprinkle fairy dust on cheap crap technologies, and magically convert them into high-end equipment.

With development effort and popularity, technologies become cheap - just look at how cheap Flash memory is these days. IMHO most "normal people" who are doing digital photography would want a write-once long term media to store their photos on. ATM I suspect most of them are storing the data on a un-backed up hard drive or CD without any thought as to the safety of that data (when they come to look at their old photos in 15 years time I expect a lot of people will be shocked to see their CDs are unrecoverable).

Re:Why not?

[evilviper]

Rubbish - they are stored in normal dry room temperature conditions in jewl cases and were bought over a period of several years from a number of different suppliers.

Well then, something "magic" is happing to your discs that is not happening to anyone else on earth. I have dozens of 5+ year-old TDK CD-Rs that are ALL still doing just fine.

With development effort and popularity, technologies become cheap - just look at how cheap Flash memory is these days.

Technologies become cheap mostly because companies start selling products with much cheaper components. Writable CD/DVDs have become so incredibly cheap mainly because so many cheap junk CD-Rs are on the market. Meanwhile, the good CD-Rs from good brands really haven't gotten any cheaper over the past few years.

Re:Why not?

[Deideldorfer]

The information you consider junk may be very useful to future archaeologists. Think about how significant it is today when we find some scrap of parchment with an ancient Egyptian's grocery list. Archaeologists would would LOVE to have access to an actual conversation between two common people from 1000 years ago.
Such a record would reveal a lot about how the language was being used, and about what people were interested in.

Re:Why not?

[BRonsk]

I'm pretty confident that if I bother to hold on to the cdroms in my draw, provided they're kept in their cases/good condition they'll be just as playable (on the same hardware) in 100 years

You're up for a big surprise in a 100 years... Oh wait...

Hmm

[yurivish]

So it just stores the data? Or is it also required to be able to view all the types of media that will be put into it?

GIF?

[crimethinker]

Of course we'll be able to read GIF files! By then, all the stupid patents should have expired (pending action by the House of Misrepresentatives, of course).

We're just lucky that Walt didn't dream up LZW compression while he was working on Steamboat Mickey, or we'd have patents lasting for the author's life plus 90 years!

-paul

Re:GIF?

[ValourX]

Don't knock Walt Disney. The man was a genius, and the pioneer of modern animated films. The corporate Disney that we know today should not diminish the work of one of the 20th century's greatest imaginative minds.

Re:GIF?

[sleighb0y]

Correct me if I'm wrong, but the GIF patent boat has sunk. Save IBM's never-enforced LZW patent, GIFs are free.

http://en.wikipedia.org/wiki/GIF#Unisys_and_LZW_pa tent_enforcement

Re:GIF?

[bloo9298]

The corporate Disney that we know today should not diminish the work of one of the 20th century's greatest imaginative minds.

I agree, Walt was much more evil than corporate Disney. Credit where it's due.

Re:GIF?

[kzinti]

I agree, Walt was much more evil than corporate Disney.

Indeed. The esteemed authority Dr. Hibbert agrees: "Well, only one in two million people has what we call the 'evil gene'. Hitler had it, Walt Disney had it, and Freddy Quimby has it." You just can't argue with the Simpsons.

Re:GIF?

[timeOday]

But I'll betcha 50 cents gnu still has their little "why we don't use gif" page...

Yup!

Re:GIF?

[evilviper]

Don't knock Walt Disney. The man was a genius,

...and a lemming murder.

http://www.snopes.com/disney/films/lemmings.htm

Chick and Egg problem

[Manip]

This has a fundamental chicken and egg problem: So you store the information, you also need to store the format of that information. So then how do you read "format of the information" document? What format is *that* in?

You see; whatever format you used for anything has to be documented and you can't use paper because it won't last as long ... Do you carve it into stone?

Worse still you need some computer science grads to write up exactly the format down to how long a char is and the bit/byte order. It is a extremely difficult task even if you don't take into consideration finding a storage medium that will last that long. :-(

Re:Chick and Egg problem

[Tyrdium]

Do you carve it into stone?

No, but long-lifespan microfiche could work (AFAIK). I suppose data density is lower compared to digital, but you could undoubtedly improve it with that kind of money.

Re:Chick and Egg problem

[HybridST]

You see; whatever format you used for anything has to be documented and you can't use paper because it won't last as long ... Do you carve it into stone?

As for the limitations of paper storage, there must be some other medium that could be used...

Re:Chick and Egg problem

[Surt]

Let me see if I can save them some work:

1) Carve one copy of the ascii table in metal (choosing metal as less brittle than stone).

2) Store the description of the file format in english ascii, and give it a unique identifier, also in english-ascii.

3) Store the files as whatever binary you want, alongside a pointer to #2. The pointer should be english-ascii as well.

If you're real paranoid, you store a copy of webster's english dictionary on metal (only a few thousand metal pages to print, should be pretty low cost), and you store another copy of the same dictionary digitally as the first file in the archive.

You store your digital archive on the 3 most popular digital media available at any given time, and you store 3 or more copies on each media as your level of paranoia requires. When the popular media change, you perform a conversion to the more popular media.

This stuff is not rocket science, but it does take money, which is really all NASA lacked, or they would have done the same thing. It wasn't that they didn't see the problem coming, nor that it was impossible to move their data to a new media before the old one expired, but that they couldn't afford to do it. My Dad worked for JPL, their major subcontractor on many of the science missions, and he and others were certainly aware of the issues, and had proposed solutions to these problems, but there wasn't enough funding.

Re:Chick and Egg problem

[El Cabri]

This has a fundamental chicken and egg problem: So you store the information, you also need to store the format of that information. So then how do you read "format of the information" document? What format is *that* in?

It looks like you'd enjoy reading Godel, Escher, Bach by D. Hofstadter. The whole book's about dealing with the philosophical implications of this problem.

Re:Chick and Egg problem

[joshv]

This has a fundamental chicken and egg problem: So you store the information, you also need to store the format of that information. So then how do you read "format of the information" document? What format is *that* in?

You print it on archival quality paper. If the paper is infrequently accessed it can last hundreds of years. You retranscribe the documents and make copies every once in awhile.

I'd be more worried that over a long enough period of time we'd forget the language the documents are written in.

Re:Chick and Egg problem

[Easy2RememberNick]

Make it format-less, and make the media also the player.

Comment removed

[account_deleted]

Comment removed based on user account deletion

I want it too

[spblat]

It's not just the government that needs this. Since we're funding this effort with our taxpayer dollars, I'm hopeful that some of the results from this work will lead to the availability of tools us normal folks can use to make sure our precious data can be preserved and passed down from one generation to the next.

Re:I want it too

[Professor_UNIX]

Since we're funding this effort with our taxpayer dollars, I'm hopeful that some of the results from this work will lead to the availability of tools us normal folks can use to make sure our precious data can be preserved and passed down from one generation to the next.

It'll probably be some $50 million database system that runs on Microsoft Windows 2003 Server and requires Oracle along with a mish-mash of Visual Basic .NET applications to accept data input and display it. I mean hell, we'll still be running Windows in 2090 so it only makes sense to stick with standards.

Re:I want it too

[blitz487]

The easiest way to ensure data will never go away is to publish it on a web site or post it in a newsgroup. Ever tried to delete a message that found its way into Google's results?

Re:I want it too

[TrentL]

It'll probably be some $50 million database system that runs on Microsoft Windows 2003 Server and requires Oracle along with a mish-mash of Visual Basic .NET applications to accept data input and display it. I mean hell, we'll still be running Windows in 2090 so it only makes sense to stick with standards.

I worked on Lockheed's demo and sat through their entire presentation to the NARA staff. What you describe could not be further from the truth. The design team was well aware of the value of open standards, and avoided vendor lock-in like the plague.

Re:I want it too

[Anonymous Coward]

I worked on Lockheed's demo and sat through their entire presentation to the NARA staff. What you describe could not be further from the truth. The design team was well aware of the value of open standards, and avoided vendor lock-in like the plague.

You must have worked for a different Lockheed than the one I'm used to that is a major contractor for a certain government space agency. That Lockheed Martin is a Microsoft whore.

IDE Raid..

[markass530]

Not sure where I read it, but there was an article I read about using good old cheap IDE Raid as a tape replacement. Some guy did it on a large scale for university, and a (relativly low cost). Considering the low cost per GB, and easy scalability, why not?

Re:IDE Raid..

[Tsiangkun]

What happens to your data if someone deletes it, and your "backup system" is a RAID array ?

What happens to your data if someone deletes it, and your backup is on tapes ?

One protects against hardware failures, one protects against users and hardware failures.

Re:IDE Raid..

[ipjohnson]

The IDE-raid doesn't solve all the problems. One of the big ups for tape backup is that you can take it off site. So you either have to copy the raid and move it off site or setup a second set of systems off site.

Either way tapes still hold some value in offsite storage.

Re:IDE Raid..

[starfishsystems]

At current prices and density, disks often work out to be cheaper per byte than tape.

For ordinary backup requirements, where data only has to be retrievable for a few months or years, disks can be useful. Under these conditions, the mean time between failures of the backup drives is at least as good as that of the production drives.

Archival backup, however, depends on an extremely low rate of failure over a very long time. The ideal backup medium is not only stable but can also be read using simple methods, so that failure of mechanism will not make the data irretrievable. For that purpose, disk drives aren't good candidates.

Re:IDE Raid..

[mrm677]

Better have a solution for off-site and incremental backup. Accidentally deleted that file from your IDE RAID array last week and now need it to save your job? Oops! Fire destroyed your building but now the IRS decides to audit your files from 2002? Oops!

Re:IDE Raid..

[Anonymous Coward]

Let me see if I can recall what I have read on /.

Raid isn't a backup solution.
If the data isn't in/on 3 diffrent location then it isn't secure.

Re:IDE Raid..

[Fweeky]

"What happens to your data if someone deletes it, and your "backup system" is a RAID array ?"

I restore it from my collection of incremental dumps from said backup RAID array to the live array.

Re:IDE Raid..

[bajo77]

Not sure where I read it, but there was an article I read about using good old cheap IDE Raid as a tape replacement. Some guy did it on a large scale for university, and a (relativly low cost). Considering the low cost per GB, and easy scalability, why not?

How long do you have to pay for electricity before tapes are more cost-effective?

Re:IDE Raid..

[evilviper]

The ideal backup medium is not only stable but can also be read using simple methods, so that failure of mechanism will not make the data irretrievable. For that purpose, disk drives aren't good candidates.

Using that metric, tapes aren't a good candidate either. Subject to time, the tape on which the data is stored will harden, break, and turn into dust.

A medium that is RIGID rather than needing to be pereptually flexible, is inherently better for the long-term. (ie. I'd trust glass CDs over plastic ones)

Hard drives will shatter if subjected to extreme forces, where tapes will not, but tapes will fall apart due to their very nature. Tapes are also completely vulnerable to the smallest ammouts of static or magnetic fields, where-as hard drives have built-in metal sheilding, and simply require stronger forces to do damage to the data.

In the long-term, I'd trust hard drives much more. Their electronics may fail completely, their cases may become corroded, they may almost completely fall apart, but with a clean-room and a normal equipment, you can still completely recover the data relatively easily. Good luck recovering the data from a tape after 200 years, when it is breaking into billions of fingernail-sized pieces.

mbox & gif formats are so complex

[winkydink]

and so poorly documented. I can see why we'd spend hundreds of millions making sure we preserve the formats.

Personally, I'd be more worried about proprietary formats.

Offsite storage and What type of Media?

[acherrington]

Hmmm... Offsite storage for the United States... Satellite Launch?

Seriously though, what medium would work the best for this? At this point I think that hard drives cost just as much as Ultrium tapes, for just as much storage. Seeing as tapes die so quickly, you may as well back them up onto true hard drives, then just let them sit for a few years. After ten to twenty years, carry it forward to the next big storage medium.

Re:Offsite storage and What type of Media?

[WindBourne]

As strange as it sounds, it would be good to send to Mars, once we start sending man there. It would be nice to have a collection of all that knowledge on a different planet.

That way, if earth really does get in the way of a new highway, well, it will not hurt the mice.

Re:Offsite storage and What type of Media?

[ScrewMaster]

Every five years, maybe. Hard drives aren't particularly stable long term, any more than tape. I had a 5 Mb. (yes, megabyte) Corvus hard disk that I bought back in 1983 or so ... I tried hooking it up a couple years ago and it was, well, blank. Worked fine after I reformatted it, but it had just erased itself sitting in the closet. And the problem is only getting worse as data density increases.

But, yeah. Every fews years I move everything that was previously on the mirrored array in my server to whatever the current tech is. That usually buys me ... a few more years until I fill that up. For me, it's just a minor inconvenience but for an entire nation? Maintaining all this old data is going to be an absolutely horrendous task, long term, since we simply don't have anything remotely permanent enough. So you're right: it will be a continuous process of transcribing information to newer forms of media (which will itself incur data loss or corruption.)

And that's something for the folks that research this stuff to think about: there'd be a market for a truly reliable storage medium, as measured in decades or centuries. Problem is always the same though: by the time you find out that it's not as permanent as the manufacturer promised, you're in deep Bantha poodoo. Maybe we need to start micro-etching important data onto, say, gold plates. Being a noble metal it won't corrode, and it would certainly outlast a CD or a hard disk. Be a good use for all that gold bullion in the Federal Reserve.

Re:Unconstitutional, unnecessary, and unacceptable

[Stevyn]

Alright, but why are you posting after the fact on slashdot? Maybe your efforts would have been more effective if you had started this campaign a year ago to the people who make these decisions.

Re:Unconstitutional, unnecessary, and unacceptable

[Doc Ruby]

Are you against the National Archives? This program enables the National Archives, into which we've already sunk billions over the centuries, to continue to be (even more) useful in the Information Age. That's our information. Why should we throw it away now?

I'm curious, did you have any criticism for the $300M "bridge to nowhere" in Alaska when it was reported in the new budget this year? And where are you on the $200B+ we're spending in Iraq?

Re:Unconstitutional, unnecessary, and unacceptable

[dada21]

Doesn't matter on slashdot, as any anti-social program post gets modded flamebait lately.

Nonetheless, I believe the "Why?" question should be asked before AND after the fact, continuously. I don't believe we should just roll over when the majority says its ok to tax-and-spend. I've met many people through slashdot who have come to agree with the non-authoritarian positions I've stated in the past, and I love the debates I usually get before I get modded down.

That's what /. is for -- honest and open debate from all 3 sides of an issue.

Real Video

[wildzer0]

For a start, they should stop using stupid proprietary formats like Real Video (the Press Conference Video on their website is only available for Real Player).

Re:Real Video

[Anonymous Coward]

For their purposes, Real video is possibly the best option. No matter how much we love to hate it, there are many more users on Windows that don't have access to, or are aware of open-source formats. Real Player, however is available for free on the top three architectures, Windows, Mac, and Linux; making it the easiest option to solve their problem.

I sincerely doubt they would like to cut out 95 or more percent of their audience just to please a rather small portion of the market who demands the use of open-source codecs.

Re:Real Video

[SheeEttin]

Yeah, proprietary formats are [Buffering...]

Re:Real Video

[Anonymous Coward]

Yeah! What are they thinking? Everyone knows that Windows Media will be around forever, and it's universal!

Re:Real Video

[Martigan80]

Hey stop that! Don't bring common sense here!

Do you think that a government chained to MS will say "Lets use a better documented and open standard?" Nope that open standard doesn't "pay the bills"

Re:Real Video

[Anonymous Coward]

Their LMPeople website (for accessing employee utilities) only works in IE as well... :-(

Re:Real Video

[theOnlyTPC]

Their LMPeople website (for accessing employee utilities) only works in IE as well... :-(

Tweet! I'm throwing the BS flag.

LMPeople works just fine in Firefox.

Will we want to archive what we can?

[gunpowda]

This is a topic that's been raised quite a lot recently. Firstly, would we even want to read emails from 1990 in the future? Unlike, say, Byron's letters, that give us lucid insights and useful historical detail, most of modern day e-mail- and IM-based communication is mostly functional and lifeless.

I remember reading an article about the archival of scientific research; many researches involved in the discovery of DNA's structure didn't keep their (hand-written) notes, but they were later recovered by others who saw the value of such significant documents to future generations.

Nowadays, of course, we can just trash something by clicking the delete button, and one the hard drive's formatted, it's gone. This does make me wonder how much historical and scientific information will be lost to future generations simply because of this ease of deletion.

Re:Will we want to archive what we can?

[martalli]

When my great, great, great....grandfather came to the States ~1720, he left very little records. His "last will and testament" is still available to be read in the local county records. The language is touching (probably written by some long-dead lawyer given that my ancestor didn't speak English and couldn't spell his own last name), but it is a real interesting view into our family's history.

Granted, people didn't set out to write blogs on their offtime in 1720, so what they might have left seems pretty important. Believe me, some of this prattle we're producing is going to be read by our kids. All my confused politics of the early 90's is right there on usenet ("Google groups" for you young-uns!) for my kids to look through in a few years.

Re:Truth be told.

[op12]

While it's nice that his memory looks attractive in photographs, it'd probably be more useful if it were photographic :)

The US National Archives

[isotope23]

tasked with preserving records "for the life of the republic"

Task completed......

Re:The US National Archives

[lakin]

The US is a federal republic.

Re:The US National Archives

[Anonymous Coward]

Though not necessarily - Lockheed's other "job" is to ensure the their own fatty DoD contracts keep a-comin'! And if those contracts don't keep bringing home the bacon, it sure would be an awful shame if some emails or other documents were inadvertently 'leaked' during the archival process.

Of course, Lockheed didn't get to where it is today by pissing away 35 billion in revenue (that was for last year alone) for one measly tape-backup contract. Depending upon your point of view, this is blackmail or redundancy (the good kind) at its finest!

Not that you'll see this, as I can almost feel it being modded down while I preview!

Re:The US National Archives

[HiThere]

Well, I guess since the title of our official head of state is "President", you are basically right. I'm a bit skeptical of designating the states as "a number of self-governing regions (often themselves referred to as 'states')", but one could certainly make an argument that it's true.

Re:The US National Archives

[jcr]

Task completed...

In 1865...

-jcr

Re:The US National Archives

[Anonymous Coward]

All powers not listed in the Constitution as being powers of the Federal Government were reserved as rights of the states to wield.

Technically the Federal Government has no business in areas such as Education as that was a power not granted them in the Constitution.

Re:The US National Archives

[HiThere]

Actually, as opposed to legally, however the Feds do whatever they want, and justify it as affecting in some obscure way interstate trade. Or come up with some other absurd justification that the Supreme Court with reverently validate as wholly and holy unquestionable.

Legally were are indeed a Federal Republic. It's the practice that is questionable.

What about Resolution?

[ackthpt]

Will GIF files be accessible???

In 2105: "Yessir, your documents are available in GIF format, unfortunately the 1024 by 768 resolution is almost unviewable on today's 1,048,576 by 786,432 document readers."

Re:What about Resolution?

[gatzke]

Funny, but displays are not going by Moore's law.

Old high res Sun monitors from a deacade (or more) ago still work just fine at 1600x1200. That was super high res back then, when most were lucky to get 800x600.

Now, Apple has a 2500x1600 display that is pretty sweet, and IBM has the highest density monitor at something like 4000x3000.

Now if I could just get a 2100x1080 projector for something reasonable so I can watch 1080i on a wall... Sony has one, but it costs thousands...

Re:What about Resolution?

[Anonymous Coward]

Except no one will be using a display with a 1.33:1 ratio by then.

Re:What about Resolution?

[zakezuke]

In 2105: "Yessir, your documents are available in GIF format, unfortunately the 1024 by 768 resolution is almost unviewable on today's 1,048,576 by 786,432 document readers."

Well... think of 8mm film. It's grainy, low quality and generally a poor medium. But you can still watch it. The best example of this is the stock suspention bridge footage. Granted this was shot on 16mm, but given the fact that this is the only copy available it's still seen today, and this was shot 65 years ago. It's in black and white, looks very good, and the quality leaves much to be desired, but never the less it's a piece of history archived for all time.

Photogenic memory.

[Poromenos1]

Yes, that's true, I've seen pictures of his memory. Looks quite nice :p

I hope they archive...

[electricsalmon]

...all the 1990's pr0n! We need to keep that in a repository for the benefit of mankind for generations to come!

Re:I hope they archive...

[kazoosandinstruments]

...all the 1990's pr0n! We need to keep that in a repository for the benefit of mankind for generations to come!

Intentional? Perhaps. Hilarious? Certainly!

Re:I hope they archive...

[Anonymous Coward]

Their standards of beauty may well drift too far from ours for them to find our pr0n "useful".

Re:I hope they archive...

[PunkOfLinux]

That's where the other 50 million dollars is going -- to buy the porn

Oh God, no...

[Rorschach1]

Is this how they plan to bury the records forever?

It sometimes amazes me that LMCO manages to build planes that actually fly. But then I have to remember that the people designing the planes apparently aren't the ones designing their software.

If they build aircraft the way they build software, their planes would make these look like this.

Re:Oh God, no...

[drxenos]

As a Software Engineer who works for LM and is proud of what I do, you don't know your ass from a hole in the ground. As a typical Slashdotter, you run your mouth without even know what the Hell your are talking about, boy.

Re:Oh God, no...

[Rorschach1]

Maybe it's not fair to say all of LMCO's software projects are screwed up, but go check out the RSA IIA stuff. I've had quite a bit of exposure to parts of it. It's years behind schedule, large portions are still unusable, it's way overbudget, and there's been massive turnover in the programming staff.

I know some of the people who are still there, and they're good people. But they got stuck with a poorly conceived project that seems to have been pretty much doomed 8 years ago. As far as I can tell, they gave up long ago on delivering anything useful or innovative, and for years have only been interested in patching enough holes to get the Air Force to sign off on it.

from the US government?

[zogger]

good luck! What's theirs is theirs! What's yours is theirs! from the PR:

"The system's "initial operating capability" should be available during Fiscal Year 2007. Weinstein noted that "the system's architecture makes it flexible enough to accommodate evolving policy change," including the importance of "providing public access while protecting privacy and sensitive information.""..HAHAHAHAHA! Anything even *remotely* important or interesting, paid for by tax payers or not, sorry, "terrorism, security", yada yada yada.

Momentous Task Indeed

[Nerd Systems]

Lockheed Martin is going to have fun with this one... preserving records for that length of time will be a considerable task... and hopefully they will figure out a way that will succesfully archive records forever...

Just look back at how much technology has changed in the past 10 years. We had 5.25" Floppy drives used back in those times, and 3.5" floppies were used as well, and CD burners were just starting to come available at the speedy rates of 1-2x, not to mention hard drives were so small compared to the 500gb drives we have today... and Windows 95 was just released, wonderful system based on FAT architecture... not NTFS like we have today...

Computer technology is increasing at such a rapid rate these days. I can only imagine how it will be in 10 years, much less 100 years from now. I am sure by then that clock speed will be in hundreds of gigahertz, memory in the terabytes, and storage in the petabyte range... if not even higher... who knows...

I also wonder, if in 2090, will their CD-ROM equivalent even exist to read this storage library? They may have long ago abandoned CD-ROMs for being too slow, and if data is stored in this format, how will it be read? Also, as hard drives get larger and larger, am sure the IDE, SCSI, and SATA drives of today will not be readable by the BIOS of tomorrow... much less have connectors to fit...

This is a huge undertaking... good luck Lockheed Martin...

Re:Momentous Task Indeed

[wo1verin3]

Whats to stop them from transfering from one medium to another every year (or 5, or 10). Make 5 copies to the new medium and then verify.

Rinse and repeat from CD to DVD to DVD+DL to BD to ???

As long as someone is actively responsible for this....

Re:Momentous Task Indeed

[dunng808]

... and hopefully they will figure out a way that will succesfully archive records forever...

Simple. Just tell my wife it was something I did.

Re:Momentous Task Indeed

[deathy_epl ccs]

I am sure by then that clock speed will be in hundreds of gigahertz, memory in the terabytes, and storage in the petabyte range... if not even higher...

Actually, according to Moore's Law, it should be quite a bit higher.

Let's see... current processor is, to keep it simple, 2GHz... about 1GB of memory and about 100GB of disk.

100 years means 50 doublings of that, so... carry the 1...

2,251,799,813,685,248 GHz boxes with 1,125,899,906,842 TB of memory and about 100 times that in disk.

Of coz, this assumes Moore's Law holds, the world doesn't get blown up, go to hell, the ants haven't taken over, etc... but it does show that 100 years from now we don't even have frame of reference for what the difference will really be like.

Which was what you said anyway, but I like crunching numbers.

Re:Momentous Task Indeed

[cosmol]

Not to mention it would be a good idea to at least read the data periodically to make sure it is still there and passes checksum.

Keeping data alive forever will require effort forever. It would be poor form indeed to toss your data into safe, just assuming that you will be able to read it X hundreds of years down the line.

There is no eternal data format or medium, but the data itself can be preserved eternally. People don't seem to "get" this.

Come on...

[Anonymous Coward]

Papyrus. It's simple and cheap. Hell, etch it into a clay tablet... or glass or something.

Re:Truth be told.

[Petrushka]

Photogenic memory? What an intriguing concept. Something like this, perhaps?

Oh that answer is obvious.

[mbkennel]

This has a fundamental chicken and egg problem: So you store the information, you also need to store the format of that information. So then how do you read "format of the information" document? What format is *that* in?

Latin, videlicet.

But seriously the problem in records is not going to be collecting the data, but turning it into knowledge. Meaning that humans in the future are likely to seriously misinterpret or be unaware of the intended meanings and social and political contexts of the preserved data.

This is not a technology problem.

They ought to make sure that real professional historians are there.

Re:Oh that answer is obvious.

[Anonymous Coward]

I can only imagine what future historians will think of when they find millions of messages about
"L0w C0$7 v!@gRa, 2 uR D0oR"

Re:Oh that answer is obvious.

[Anonymous Coward]

...it is probably some kind of a fertility cult. The encoding systems used in the cult messages may be evidence of systematic religious repression by the secular authorities.

These messages present evidence of widespread following of the goddess "V14gR4" (also known as V!@gRa in Egypt), and it is believed that a separate class of priestesses would send their images, "photographs", to the followers of the cult through the primitive internets of the time.

Software is easy. Re:Chick and Egg problem

[hypnagogue]

This is not nearly as difficult as you make it seem: implement the parser in a standardized language. The formal specification of the standardized language can then be included with the source of the parser.

Getting code to run on later architectures is not usually very difficult. I am fairly comfortable with the proposition of porting any code to any future architecture -- the "emulator scene" testifies to the viability of this strategy. The biggest problem to be solved is reading storage media for which no hardware exists.

For example, how do I get to my college research stored on AmigaDos floppies? Tragically, the easiest solution is to try to get my Amiga running again, and then move the data over a serial cable with kermit. I'm awfully glad I have kermit on that computer, because I don't think I'd be able to find any 2400 baud Amiga BBSes around to download it.

Priorities.

[9mm Censor]

goatse and tubgirl shall be archived, in all their digital glory for the ages to see.

Re:Truth be told.

[Deflatamouse!]

Would love to meet this Mr. Martin...

maybe you meant photographic memory...

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

I modded you up :)

Google?

[dustinbarbour]

Did Google compete for this contract? They're the ones with the largest infrastructure for such a project and the brains to give us a really slick interface to it all. Not to mention that they could probably have faster response times than archive.org which totally fuckin' blows.

Re:Google?

[Surt]

your sig link is out of date

Re:Google?

[TrentL]

The two companies that were "down selected" to compete in the Analysis & Design phase were Lockheed Martin and Harris Corporation. I don't recall what companies participated in the initial competition...I doubt Google was involved.

Lockheed partners include BearingPoint Inc., McLean VA; Fenestra Technologies Corp., Germantown, MD; FileTek Inc., Rockville, MD; History Associates Inc., Rockville, MD; EDS Corp., Plano, TX; Image Fortress Corp., Westford, MA; Metier Ltd., Washington, DC; Science Applications International Corp. (SAIC), San Diego, CA; and Tessella Inc., Newton, MA.

Re:Google?

[Tontoman]

I worked on the project as well, with Verity, Inc. We worked with Lockheed to make a search GUI for the Electronic Records. (I don't know why we weren't given acknowlegement as one of the Lockheed's partners.) I understand this amount of data, when ERA is fully operational, will be many times larger than the data (the Internet) that Google tries to index.

not fair

[ac1djazz]

there should have at least been some kind of public contest where other companys can prove themselves worthy of this contract. -acidjazz http://www.litebay.org/ - the most popular movies,music,games,software,etc.

Re:not fair

[TrentL]

There was a public competition. Lockheed Martin and Harris Corporation were the two companies who won the initial competition and were given money for analysis and design. Lockheed won.

The Bridge has a name you insensitive clod

[Phelan]

The Bridges name is the Don Young Bridge, after the Alaska Congressman that had the forsight to find a prime piece of real estate where he can build a Manhattan to the west.

Re:The Bridge has a name you insensitive clod

[nitehawk214]

Yeah the Bridge to Nowhere is in Pittsburgh.

What is a formsfor ?

[zrq]

I was curious to see if the plans included making any software developed for the project OpenSource.
While looking through the documentation http://www.archives.gov/era/about/documentation.ht ml/
I found a link to the project requirements : http://www.archives.gov/era/about/requirements.csv /
Which contains the following line :

ERA2.6.3,The system shall check online formsfor correctness,,

I know, one typo in one line in several hundred, but why that line ?

Re:What is a formsfor ?

[isbhod]

to show that irony is a cruel, yet funny, mistress ;)

Protect against the 1984 "memory hole"?

[Anonymous Coward]

Technically I don't see any problem with storing 100PB of data in the next decade, and keep it safe from natural disasters. But how about unnatural disasters, such as an evil administration changing the entire archive to reflect better on itself or protect itself from criminal prosecution? Copies of the archive packages need to be suitable dispersed in multiple jurisdictions or even shot into space in order to make this kind of data destruction infeasible.

Re:Protect against the 1984 "memory hole"?

[TrentL]

This is a real concern, and one that Lockheed spent a lot of time working on. Another issue is authenticity: what's to stop someone in the year 2050 from inserting some new records and claiming they were from 2005? These are problems that currently exist in the paper world, and they will exist in the digital world as well.

Re:Protect against the 1984 "memory hole"?

[Anonymous Coward]

yes, because the tracking of (and retrieval of) information which has been secured, encrypted, and then launched into space will be a luxury afforded to, oh, everyone in the future.

Re:Protect against the 1984 "memory hole"?

[zaguar]

Another issue is authenticity: what's to stop someone in the year 2050 from inserting some new records and claiming they were from 2005? Hashing the data? It's already widely used.

I know that MD5 has it's issues (recent discoveries of collisions etc.) but SHA-1 and other methods should be enough to ensure authenticity. A 128 bit hash has a minuscle size compared to the data, and is probably secure enough - but maybe not. Maybe that's why Lockheed has 308 million - to solve these problems.

Re:Unconstitutional, unnecessary, and unacceptable

[Shut the fuck up!]

Provided you are not lying, you just reversed your moderation.

Standards are good

[Anonymous Coward]

Now let's get the Federal, State, and local governments to create standards-based web sites as policy. (Remember FEMA and the US Copyright Office.)

http://narnia.dnsalias.org/freegovernment/

Next, let's go for standards for electronic office documents (word processor, spreadsheet). I doubt Lockheed Martin's system here is going to understand every obscure format---say Microsoft Word 5?

In the year 2525!

[MrLint]

"How will we be able to read 1990's email messages in the year 2090?

Mebbe if we kept email as plain text we wouldnt have to ask this question. Im fond of mbox myself. However any sufficiently documented format will only leave us with storage media issues.

Re:In the year 2525!

[TrentL]

Mebbe if we kept email as plain text we wouldnt have to ask this question.

We don't have a choice. Not only do we have to keep track of the text of the message, but we also have to archive the attachments and the arrangement (i.e. if this email is just one message in a dozen-email exchange, that has to be preserved).

Re:Truth be told.

[VAXcat]

Photographic memory? You mean one that fades over time and undergoes color shifts?

Re:Unconstitutional, unnecessary, and unacceptable

[dada21]

The National Archives are important, except so much that SHOULD be in the Archives is not, for whatever (illegal) reason:

http://www.lewrockwell.com/orig/stinnett1.html

http://www.lewrockwell.com/pilger/pilger17.html

http://www.lewrockwell.com/rogers/rogers40.html

And some of those items took decades to make it. If they're going to keep certain government information in the Archives, make it available immediately.

I'm against the $300M bridge to nowhere -- I believe in privatized roads funded by local businesses, industry, and homeowners divisions.

I'm against ALL unconstitutional wars (every one since WWII has been unconstitutional).

Cue Stealth (TM) jokes here

[Spy der Mann]

"The Electronic Records Archives. By the same man who gave us the Stealth(TM) aircraft".

Hhmm...

Privacy issues

[Anonymous Coward]

The US Government and its agencies will have a perfect tool of mine for your interests, associates, lovers, sexual orientation etc. Make a join with Google's database, and they got 50% of your life covered. Enough to blackmail or nail you over and over.

I can make this work...

[Linker3000]

1) Give me the $308M.

2) I'll bank the cash and cream off some interest to keep me in luxury - let's say I can get 5% interest PA, that's $15.4M a year so I'll take, say, $5M/year for myself and my efforts.

3) I'll Spend up to $10M/year maintaining a secure storage facility and purchasing 3 units of every storage device that comes to market together with a range of media, host systems and documentation on acid-free archive paper.

4) There will be an annual charge for subscribers to the facility.

5) Profit!!

What a waste of money!

[Excelcia]

What a waste of money. Massechussets is doing it for free. All you need to do is make sure that all your document file format standards are free and open. After that, it doesn't matter if they exist in 2090 or not. If they don't exist, it's because no one has really needed the documents. If some historian comes along and does need the document, then a document import filter can be easily created because the document's file format was based on a free and open standard.

Suddenly the volume of documents you need to maintain drops from all of them, to just the documents that describe the file format standards.

Re:What a waste of money!

[Vombatus]

What a waste of money. Massechussets is doing it for free. All you need to do is make sure that all your document file format standards are free and open.

Free?

What are you going to store these open file format documents on?

You are going to need a damn big server of some sort.

You are going to need some other infrastructure as well - to prove that the record is still reliable and authentic (at least).

You will also have to refresh your media regularly.

And many more things you will have to have in place.

That will not come cheap.

Re:What a waste of money!

[Excelcia]

It only doesn't come cheap because people have been convinced that it can't come cheap. For example, why is huge server infrastructure needed? This is archiving, not serving. Archiving, by definition, is where you have data that is important enough to keep but where the immediate need is low enough that the burden for providing it is on the looker. If someone wants the data, they can look up the location and find it in room full of DVDs. There are lots of off-the-shelf storage solutions with shelf-lives to 2090 and beyond. Assurance of authenticity? Good grief, again, this is archiving, not real-time bank transactions. The data is in the hands of one party - who the heck needs to be assured of its authenticity? Anyway, an assurance of authenticity is only as good as the word of the assurer. What's the word (digital or not) of some assurer 50 years dead going to be going to be worth in 2090? In short, you don't need to award a third of a billion dollars for this sort of nonsense. Hire some pimply, fresh-out-of-college nerd for 30k a year and you'll get a solution that makes a lot more sense.

Re:What a waste of money!

[Vombatus]

This is archiving, not serving. Archiving, by definition, is where you have data that is important enough to keep but where the immediate need is low enough that the burden for providing it is on the looker.

The definition of archiving in the technology world is different to definition used by recordkeepers and archivists.

If someone wants the data, they can look up the location and find it in room full of DVDs

Do you really want to store terabytes of data on DVDs?

There are lots of off-the-shelf storage solutions with shelf-lives to 2090 and beyond.

Archival institutions around the world would be most interested in these off-the-shelf solutions. Can you provide a reference? Are they just the software and hardware, or do they provide the rest of the system as well?

The leading archival instutions around the world are all building their own solutions (and collaborating), which would tend to suggest that there is nothing suitable available off-the-shelf. The data is in the hands of one party - who the heck needs to be assured of its authenticity?

Surely there are enough conspiracy theories about the governance of the USA without adding to the mix by not having authentic records. There are recordkeeping mechanisms available to ensure that records are pretty damn authentic - why not use them?

IFF-ILBM

[Zobeid]

This example of format obsolescence just popped into my head. Back when Commodore-Amiga was a going concern, the IFF-ILBM graphics format was pretty widely used. It was a nearly universal standard on Amiga.

A fair number of artists and video producers used Amigas. One of Amiga's advantages was that practically all the graphics programs used ILBM format, which meant you could easily feed the output from one application into another, and then into another. It was good, and it wasn't all that many years ago.

Just trying finding a program on Mac OS X or Windows today that can read IFF-ILBM files! Go on, try it! Photoshop, for one, doesn't have a clue about them. The best you can hope is to find some obscure freeware IFF-to-PNG converter that someone has hacked together.

Another example: It's getting harder to find apps that play "tracker module" music, and the programs that are available tend to be awkward and unreliable. Everything went to MP3, and mod music was quickly forgotten.

So if the idea of today's commonplace formats becoming unknown in the future sounds far-fetched at all. . . It's not.

Re:IFF-ILBM

[emd]

Boy, you're right, that was hard

Re:IFF-ILBM

[emd]

And of course, for Windows: http://www.irfanview.com/main_formats.htm

Re:IFF-ILBM

[fudg3tunn3l]

XMPlay plays Modules well enough... and IFF Pro does a good job of conversion... the tools will always be there, you just gotta find them

Re:IFF-ILBM

[rubypossum]

Hey, someone else has had this problem too. Fortunately, free software to the rescue. and this plugin works nicely.

This is where the true support for these formats will remain - open source. If you want support, you have the freedom to write it yourself.

Of course, if memory serves me right non-free Paint Shop Pro still has IFF support as well. Hmmm. This page seems to say so. I seem to remember Photoshop having IFF support, but that was 3.0 or 4.0ish many moons ago onna MacOS classic box.

Re:IFF-ILBM

[JaumPaw]

Tracker module? hard? Are you joking or what?

The first that comes in my mind is CheeseTracker .

Take a quick look up in wikipedia and you'll find others too.

Re:IFF-ILBM

[Anonymous Coward]

From a few minutes' test on my dual G4 Mac, it's apparent that Photoshop CS2 and Photoshop 7 will not open IFF files, though Photoshop 6 used to do so (and does under Mac OS Classic, though PS6 is behaving badly here).

However, Graphic Converter 4.4 does a fine job opening IFFs on OS X Panther, and I'd imagine that apps like this and like Equilibrium's DeBabelIzer will continue to be used in future to open legacy formats.

BTW, I believe Graphic Converter was bundled with my Mac for free - standard equipment, at least at one time. Hardly meets the test for obscurity.

Interesting: I wonder if there's a market for software that will scan your disk for aging file formats (from a known list) and offer to batch convert them into something modern?

D

Re:IFF-ILBM

[Anonymous Coward]

> Just trying finding a program on Mac OS X or Windows today that can read IFF-ILBM files! Go on, try it! Photoshop, for one, doesn't have a clue about them.

We'll photoshop supported it for a long time, but since nobody uses IFF anymore Adobe made the IFF fileimport filter optional. All you have to do to use it, is to copy 'IFF Format.8BI' from CD (goodies/optional plugins/) to PS plugins dir, and you're done.

Re:IFF-ILBM

[petermgreen]

there is a big difference between becoming obscure (e.g. you need to get hold of specialist utilities to handle them) and becoming totally unknown.

i never used amiga so i don't know about IFF-ILBM but a google search turns up lots of hits including a gimp plugin, Some tracker formats are deffinately supported by winamp and i'm pretty sure the format specs are out there.

and there are emulators out there for almost any old system you can think of (though i admit getting the roms can be tricky).

Re:IFF-ILBM

[Anonymous Coward]

GraphicConverter

Re:IFF-ILBM

[moosesocks]

Another example: It's getting harder to find apps that play "tracker module" music, and the programs that are available tend to be awkward and unreliable. Everything went to MP3, and mod music was quickly forgotten.

Winamp does, and there's various ports and frontends of mikmod for virtually every OS imaginable. On the other hand, tracker modules were a very cool form of music, and it's a shame to see them begin to die after the community refused to adopt aa format that supported mp3-encoded samples....

Re:IFF-ILBM

[FlynnMP3]

http://www.polyview.com/

Windows only, but your example inferred windows only programs. Also I know it supports amiga IFF format all bit depths, cause I pointed out that that 1 and 2 bit depth clut was wonky and the author fixed it in the very next release. OF course the program (when purchased) will export to PNG format.

Tracker Module music is not dead, not is it in any danger of being dead. Heck there are professional musicians that still use tracker style composing. http://www.skale.org/ is a free tracker that supports almost all the old formats. It is real cool to tinker around with, and incredibly powerful. Plus it works in Windows and (though I've never tried it) Linux.

Cheers!

Re:Truth be told.

[Basehart]

"Would love to meet this Mr. Martin..."

He's good buddies with Jethro Tull.

How about Clay?

[Narmer_the_King]

YES! Finally a job after all those years studying Akkadian! Clay tablets are some of the most durable media I know. At least they have a proven record. Vast numbers of documents illustrating the fascinating world of accounting, esp. Sumerian sheep and goat transactions, is available thanks to the scribal choice of clay (combined with hot arid conditions). Will soon Lockheed HR soon be seeking 8-10 years of prior "Cuneiform/Pictographic" scribal experience? I can also read omens in the entrails of an ox. That can come in handy.

Resize?

[Poromenos1]

Yes, resize it.

Step 1) Generate more documents in open formats

[pavon]

I have been saying for years that the DoD should make an initiative to move towards open standards for this exact reason. The document retention requirements they have are incredible, and yet nearly all the documents generated are saved in proprietary formats. Now with the OASIS (OpenDoc) format solidifying and there is more than one implementation of it, they wouldn't even have to define a standard for word processing or spreadsheets.

Obviously, open standards are not a panacea. There are countless standards created by the military that never really spread farther than that, and therefore the support for them is limited (and thus companies that do support it can charge a pretty penny for it). And with open standards, at it is much easier to write an implementation if you need to. Compare this to MS Word, which is a pain to reverse engineer now, just imagine having to do so in the distant future, when it is not as widespread. And of course, for the very long term, nothing is more certain (and more inconvenient) than printing everything out and storing it in a warehouse, which is what is done now. But the longer that can be postponed, the more money can be saved.

As an added bonus, just imagine the competition that would spring up in the word processor market, if the DoD mandated that all new word processor documents generated internally or by contractors be in OASIS format, starting 5-10 years from now. Microsoft would have to support it (and well) or throw away a huge number of Office sales. The DoD would no longer be locked into a single vendor, saving them money upfront in addition to the money they saved on document retention.

Until then, the best plan is likely to convert as much as possible to a few standards like PDF, which is what I expect will happen here.

Re:Unconstitutional, unnecessary, and unacceptable

[aengblom]

I'm trying to find out where in our Constitution does the Federal Government find an enumerated power to pay for this.

Wow, you can access the Constitution? I mean it was written in 1776. That's a long time ago. Good thing somebody thought to save it!

We're saving lots of data, because 1) lots of it is important and 2) we have very little perspective on it yet. In 200 years we might very well have a very different idea of what was important today.

Re:Unconstitutional, unnecessary, and unacceptable

[dada21]

When it comes to constitutionality, it doesn't matter IF we need it, it doesn't matter WHY we need it, and it doesn't matter if the MAJORITY thinks we need it. The federal government has certain enumerated powers, and the rest is left to the States and the people.

Why can't private companies do it? $300M in every pork barrel program adds up quickly, and government just inflates more currency into creation whenever things get tight.

Our economy is heading to the gutter fast once the housing bubble bursts, and all these projects are part of the problem.

Drm

[panxerox]

Of course they need a national digital archive since the death of the general purpose computer has been already decided non DRM'd content at some time in the future it will no longer be accessable. We must protect the corporate entities by making sure that only government has access to non DRM'd content.

Re:Drm

[martalli]

I sure hope that in 2090 they can defeat our puny drms! After all, dvd encryption was defeated in just a few years! Alas, the way copyrights are going these days, most of these copyrights will probably stay in effect until the original owners could care less, losing the information forever, like many of the old films of the 30's and 40's.

Unnecessary.

[Digemedi]

the works of william shakespeare didn't start out their star studded lives as computer code. They went through a lengthy evolution through at least several iterations of printing presses before they made their way to a digital medium.

Data evolves. If theres one thing that google searches have shown it is that the demand of information is always changing, people arent going to want the same chicken noodle soup that they had fifty years ago because we make better chicken noodle soup now.

If the census 2000 information really deserves saving it will be copied hundreds of times into different isolated servers by individuals or companies who use the information. it will be ported to a new format, it will evolve.

-----

Re:Unnecessary.

[SheeEttin]

we make better chicken noodle soup now

Personally, I liked the chicken soup of fifty years ago.

Re:Unnecessary.

[Trevahaha]

This goes with some scholar's definition of information - Information is no longer information if it stops circulating/being used. But this is really flawed, as you state that information changes. No one may care about the census data now, but 100 years from now, it could be very useful. I don't want my census data from 2000 meshed with my census data from 2100. And as the responder states, we may want the chicken soup from 50 years ago... without all the additives and chemicals added.

Re:Unnecessary.

[TrentL]

If the census 2000 information really deserves saving it will be copied hundreds of times into different isolated servers by individuals or companies who use the information.

FYI, the same division of Lockheed that is doing the archives did the 2000 census. In fact, the demo for the 2010 census occured in the same room the just a few weeks before we did our Archives demo.

XML Duh!

[synesis]

Save the data in XML and save the format in XML Schema and/or Relax NG. Please send your check for $308M to synesis...
Seriously, for $308M what hairbrained scheme is Lockheed going to dream up to justify the price tag.

Great - Lockheed Martin. Now there is an idea

[Ada_Rules]

Great. Now we can be sure that the only way to read any of these documents will be with IE 4.0 since they will prohibit even the sound of the word open source (sorry PDF. See clause 11) and given that they are a defense contractor you can bet they will lock in to some proprietary SW version that is 4 versions older than what is current.

Re:Great - Lockheed Martin. Now there is an idea

[Minna Kirai]

the word open source (sorry PDF. See clause 11)

All that says is that if you sign up to sell Lockheed some software, they're paying for something they'll own, not a Free thing you downloaded and repackage. That's obligatory if they intend to resell the project, because it would be unethical to secretly mix GPL into your proprietary projects without giving customers the source.

Incidently, Lockheed uses ADA in several of their projects, including the GNU gnat compiler.

Re:Great - Lockheed Martin. Now there is an idea

[drxenos]

Really? Is that why, as a Lockheed employee, I use GCC and GNAT everyday? Or why I using Mozilla on my office computer? Oh, I get it, it's the Linux machines we have in the labs. NO, it must be the whitepaper I was tasked by my manager to write on why we should CONTINUE to use open source software.

Re:Great - Lockheed Martin. Now there is an idea

[MrBadbar]

I'm a software engineer with LMTSS (Transportation and Security Solutions - the business unit that just won NARA). I don't know all the licensing details of this, but I know guys down the hall who are quite happily writing kernel modules for 2.6 to support proprietary hardware used in air traffic control systems we run on Linux.

To be sure though, not all is rosy and open source at LM... I'm currently working on a project that's being developed in a weird combination of C++ and Ada on AIX, using lots of wacky, expensive, definitely closed-source tools. No gnat or gcc for me... it's all Visual Age and PowerAda (which have the annoying property of producing binaries generally impossible to debug with gdb). I really need to look into moving to other projects...

Re:Great - Lockheed Martin. Now there is an idea

[Ada_Rules]

No. It does not just say you can't sell them repackaged open source software.

Read clause d

Unless SELLER has obtained LOCKHEED MARTIN's prior written consent, which LOCKHEED MARTIN may withhold in its sole discretion, SELLER shall not use in connection with this Contract, or deliver to LOCKHEED MARTIN, any Prohibited Software.

So, if you win a contract with Lockheed Martin you can not use GCC (for example) unless they approve it. (SELLER Shall not use in connection with this Contract). Granted I suspect they would propably approve it but clauses that are this strong have a (a hate this overused term) "Chilling Effect" in that most engineers are not going to want to take the time to spend 2 months arguing with lawyers and management that they should be allowed to use the best tool for the job. They will just cave and use some proprietary tool that has a license clause that says "No use in DoD contracts" which of course will sail through because no one will read it (It is proprietary so it must be good)...

It certainly is not prohibited by the LM policy.

And yes, I am aware they use Ada. Does not mean I will cut them that much slack. I agree they need a policy like this. I just think this one goes a bit too far in some areas and the wrong direction in others.

This was clearly written by someone that got freaked out by a Microsoft or SCO buddy. They ban several specific liceses, then the GPL then any license that is GPL compatible... Granted it has the "not limited to" clause but banning GPL compatible free licenses without banning non-GPL compatible free licenses makes no sense whatsoever (So they want the BSD advertising clause imposed on them and really really don't want the version of the license that imposes less on them....)

Re:Great - Lockheed Martin. Now there is an idea

[Minna Kirai]

So, if you win a contract with Lockheed Martin you can not use GCC (for example) unless they approve it.

You are drastically overinterpreting the plausible scope of "use in connection with". Prehaps their lawyers were a little vague, but the only sane interpretation (the only one consistent with the rest of that contract text) is to mean "as part of the deliverables".

Given that the whole contract is only applicable in to suppliers who "represent" themselves as authors of the software, it makes perfect sense. The customer of a work-for-hire will be the owner of the eventual copyright, so it's her discretion as to what Open Source licenses it might be supplied under.

Tax dollars at work: 300M to solve a 100k problem

[lanimreT]

100k is an estimate of the national archive's Network Administrator's sallery.

Seriously. I'd do this for free. Many people probably already have.

step 1. Massive raid 5 array with a mirror array off site and pleanty of hot spares.

step 2. Fix date stamping on whatever image format, sound format, text format you decide on.

step 3. Don't use lossy compression.

No offense to the poster but it's not rocket science to create a compression free data format with proper date formatting built in. This sounds like yet another way that the alledgedly conservative Bush administration is simply finding ways to line its friend's pockets with our tax dollars.

I'm sure that Lockheed came up with a much more complicated solution to justify their large check but I fail to see how this is a 300 M dollar problem.

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

>Our economy is heading to the gutter fast once the housing bubble bursts, and all these projects are part of the problem.

Wow, maybe I'll be able to afford a house in Denver after that happens....

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

Centuries? NARA has only been around for a few decades, and has only been its own agency since 1984.

Microfiche

[Anonymous Coward]

Believe it or not alot of companies that have tasked with data retention of 100 years+ have adopted microfiche as their standard for data.You don't need to store any programs or anything just dump all the data out onto microfiche and you have a format that will in theory indefinitly

Re:Microfiche

[TrentL]

Microfiches (sp?) aren't accessible to a grandmother sitting at home browsing the NARA website and doing genealogy research. Also, I'm sure there are some librarians in New Orleans who can give you explicit details on the problems of storing microfiche which hasn't been backed up.

Re:Microfiche

[the_2nd_coming]

we are talking about ARCHIVING.

that means that the data is stored in a less accessible format for long life. That does not mean that you cannot keep the data in a "liquid" state as well for easy consumption, but in case of a catastrophic data loss, the archive could be used to rebuild the database.

Re:Unconstitutional, unnecessary, and unacceptable

[phoenix.bam!]

blah blah blah the federal government is supposed to provide for the general welfare of the nation, and i'd say important information about this nation is part of it's welfare.

Pfah.

[Grendel Drago]

You're not posting "any anti-social program post". While you may not, in fact, be a Randroid, your post is indistinguishable from one written by a Randroid.

Happy to help!

--grendel drago

Re:Pfah.

[dada21]

I am definitely NOT a Randroid by any means.

While Rand had some interesting ideas, so did Clinton and Bush at times.

I don't believe in the use of force by anyone against anyone else except in the direct defense of one's property or person. To me, taxation is force, and I just can't understand why laws are necessary at the federal level.

I'm not advocating shutting down ALL government (yet), just the federal level. It is my firm belief that the State-Countries that will be left will be MUCH more competitive for their constituents, and continued disassembly of lower level governments may not even be necessary.

Sure, my dream world is one without regulations and taxes, but I know it won't happen. There are so many ways of living below the radar and through the black markets (we all do with certain parts of our daily lives) and I've been walked on more by government than ever by thieves or conmen. Most of you are in the same boat, but just refuse to see it.

I guess I should be part of the system, keep voting for more and more stupid pork projects such as this. Enough pork will likely bankrupt the country, and I'll be safe with my gold and my property and my gun. After complete collapse, maybe then the people that survive will realize what a mistake they made.

Re:Pfah.

[Anonymous Coward]

Give Me Liberty

Re:Tax dollars at work: 300M to solve a 100k probl

[Philip K Dickhead]

Solved right here! In my 100,000,000,000 volume, blue-leather, hard-bound "All the Internet - 2005 Edition". Act now, and we'll throw in this FREE remarkable magnafier - which makes type seem AMAZINGLY larger.

Offer void where prohibited, tax and shipping not included.

Lockhead - Martin data entry...

[zenneth]

I worked with them for a while, as a data entry person back in the early 90's. Basically, we were responsible for keying in a parcel's 5-9 digit Zip code after it had been scanned into the system. By scanned, I mean the front of the package or envelope showing the send-to and return addresses was presented on a monochrome display, which allowed the person operating the terminal to enter the zip codes for the parcels. Then you'd hit a key and move to the next one, and so on and so on.

The bizarre thing is that I found out a few of the invididuals would "pad" their PPM (Parcels Per Minute) by typing in zipcodes they were familiar with instead of reading what was on the display, just to enter a dozen or so really quickly. It didn't happen often, but it helped them keep up the pace and "clear" the system queue more quickly, thus gaining them and their workmates an early break. However, I've no idea what damage may have occurred by their lax attitudes, and I really don't want to know now.

Which brings me to my point (I think): how can we be certain the data they're entering is one-hundred percent accurate, regardless if the medium lasts a century?

Re:Lockhead - Martin data entry...

[Anonymous Coward]

This was also a problem for Royal Mail in the UK, where a similar system was called video-coding. You got to hit a key before moving to the next one?!? You were lucky. Our letters used to come past on a conveyor system.

UK slashdotters may remember that letters used to arrive with two lines of grey dots on the front of the envelope. You still get them from certain backwards regions of the country :-) The dots encoded the two parts of the Post Code (e.g. one line would encode W12 and the other would encode 8QT, if you were writing to W12 8QT). Once an envelope was video-coded, it passed through most of the automated sorting pipeline, with machines reading the video-code.

The trouble was, it could be extremely tiring work. Video-coders needed to take a 15 minute break every two hours (double the normal rule). It's difficult to judge where most errors came from: laziness, fatigue, bad typing, bad eyesight... Bad hand-writing on the envelopes certainly did.

Consequently, my office in Leeds (LS3 Post Code) would receive a few dozen letters every day addressed to Slough (SL3), Liverpool (L53, L58) and even the wrong part of the city (LS8). Once the video-code was applied, the letter was doomed to go to the wrong place.

Imagine the confusion that innaccurate copying of CD-ROMs might cause. A byte here, a byte there...

What? Not Haliburton?

[kvn]

They must not have a digital archives division... ...yet.

so rescale it

[petermgreen]

the results may not be pretty by the standards of 2105 but they will certainly be readable.

just as old archive film looks pretty shitty by modern TV standards but you can still see whats going on just fine.

Is lockheed really doing anything?

[duckpoopy]

Or are they just subcontracting it out, as usual.

Chinese whispers

[oliverthered]

I have code on a modern HDD that I typed into a BBC computer 15+ years ago fro ma magazine.

I took it off of tape, via the BBC and a serial lead, I have all my chickens and all my eggs. So long as you move to a newer form of media before the old one perishes then your going to be OK.

I think it's a Chinese whisper problem not a chicken and egg problem, what happens when inaccuracies are introduces

e.g. Someone writes a file in a odd charset, nobody notices that the charset is different from ASCI when they convert the file into unicode. In 20 years time will someone notice that the file has been converted badly or will they think it's corruption? What happens when there are lots of tiny conversion errors like this.

Re:Unconstitutional, unnecessary, and unacceptable

[deathy_epl ccs]

Regarding this $300,000,000 "bridge to nowhere," it is obvious that you (and most of the world) only know half of the story.

The piece that you are missing is that it is significantly cheaper to fix the damn thing than it is to tear it down.

Just think how many other things folks have a knee-jerk reaction to that they know nothing about.

Re:Unconstitutional, unnecessary, and unacceptable

[YrWrstNtmr]

Why can't private companies do it?

I'm pretty sure LockheedMartin IS a private company.

Oh, did you mean you want them to front the cost, too? And then who pays them for their services of preserving the National Archives. I expect that would be the government.

Re:Tax dollars at work: 300M to solve a 100k probl

[erikharrison]

Complex data backup solutions and the use of lossless formats has not, for example, kept the critical Pioneer space probe data available, after less than 30 years (http://www.planetary.org/news/2005/pioneer_anomal y_faq.html)

How in the fucks sake do you expect this to last 100+ years? Don't use lossy compression? How is that a solution?

Take Windows Bitmap image format. It's not lossy. That doesn't mean that we won't forget how to display the damn thing...

Raid 5? What problem do you think you're solving? Keeping the data around, or making the data accesible for (as the OP makes clear is the LoC's responsiblity) as long as the United States exists?

Re:Unconstitutional, unnecessary, and unacceptable

[deathy_epl ccs]

Great! Anything we can do to speed the collapse seems like a good thing to me, because we sure as hell ain't going to fix the political climate.

Of course, I'm a known nihilist, so take my opinion with a grain of salt. Or don't. What do I care?

Porn, porn, porn

[rawwa.venoise]

Will they store my favorite porn stuff?

$308M?? MySpace sold for far more than that....

[v3xt0r]

and it WILL NOT last too long (now that morons @ fox are running it).

This type of project requires a continuous life-cycle of evolving, or else it is as pointless as archiving film-slides like the FBI was doing years ago (and probably still uses when they have too).

Rule #1 - bits visible under a microscope

[davidwr]

Rule #1 - with just a high-powered optical microscope, you should be able to see the data, preferably in a human-readable form.

Rule #0 - you can still do so 100 or better yet 10,000 years from now.

Back in the days of punch-cards, you COULD see the bits.

I've heard stories, perhaps apocryphal, of taking 60s-era computer tape, exposing it to certain chemicals, and being able to visually see the bits.

Why is this important? One day our civilization will die out and after a time another will rise in its place. We owe it to their anthropologists to make it easy on them.

Re:Rule #1 - bits visible under a microscope

[Anonymous Coward]

Yeah, check out the nickel disc produced by the Long Now Foundation's Rosetta Project.

http://www.rosettaproject.org/live

http://www.rosettaproject.org/live/diskdiagram

http://www.rosettaproject.org/live/disk

Re:Unconstitutional, unnecessary, and unacceptable

[Doc Ruby]

We agree on all of that you just mentioned, except the roads which seem to me a perfect "natural monopoly": government territory.

But since you see the value in the National Archives, why don't you see the value in digitizing it for archive and access by the people who own it?

Personally, I expect that Lockheed will bungle the job. They're not experts in the archive business, no matter how much of their own they do, or how much they "demonstrated" to the government. They're in the "blow things up" business.

Really, they're in the "get government contracts" (corporate welfare) business, which is how I expect they were chosen for this project. And why I don't expect them to pull it off, putting our own archives at risk the same way Bush put the Iraqi "archives" (7000 years of it) at such risk that it's now destroyed.

But that's why Lockheed is suspicious. The project itself has merit, and $308M seems worth achieving its goals, especially if it's really $30M for a decade (or even the $60M a decade it will turn out to be).

Already here

[slashflood]

In Germany, we already do that: Zentrale Bergungsort Bundesrepublik Deutschland (German, but pictures). Here is a short description in english. All the documents are kept on microfilms, but I don't know what they do with audio/video material.

Re:Already here

[Ph33r th3 g(O)at]

Seig Heil, mein network Nazi Obergruppenstraumbfuhrer.

Re:Step 1) Generate more documents in open formats

[TrentL]

I'm not going to go into Lockheed's plan in detail, but you may be happy to know that my teammates were well aware of the OASIS standards and their value.

A lot of people in these comments keep saying, "I can solve this problem for $10k! Convert everyone to Open Office!" That's all well and good, but people, we are already DECADES behind on this problem. Whether you like it or not, there's a boatload of Word95/Excel/BMP/etc files out there (and worse).

I can fix this.

[sbaker]

Hmmm - I'd better email myself the GIF spec - maybe along with some source code to read it with - and a C compiler to compile the source with. Ah WTF - I'll just email myself the Linux sources. ...but seriously...there won't be any problem with reading GIF if anyone actually wants to - the file format is documented all over the place and in 100 years, if there are still GIF files on some kind of readable media - then the odds are very good that those documents will be easy to find. Programming a GIF reader (or a reader for almost any documented file format) is easy - presuming you are sufficiently motivated. A historian who is interested in 100 year old documents shouldn't have any problems getting them converted to whatever format is needed.

The HUGE concern is the undocumented, encrypted or (worse) DRM'ed files. Reading those in 100 years may be exceedingly difficult.

We can read documents written in heiroglyphs around 2000 years ago. The only problem is with languages for which no translations *ever* existed.

Survival and longevity of antique media are a much bigger problem.

Re:I can fix this.

[evilviper]

Programming a GIF reader (or a reader for almost any documented file format) is easy - presuming you are sufficiently motivated.

You are jumping to the conclusion that C will still be common in 100 years. Since it was only even invented 35 years ago, that's very hard to predict. Remember the Y2K problems? Old mainframe languages haven't been around much longer than C, and even now, they've almost been relegated to complete obscurity. When all the guys that know the language die off, it may be very hard to learn the language from scratch from the few documents that remain.

We might not even be using binary systems anymore. The most basic computers came about only ~70 years ago. Who's to say they will be exactly the same in 100 years time?

Sure, displaying GIFs to a screen may be fairly easy, but how do you export viewable GIFs to a neural implant?

Survival and longevity of antique media are a much bigger problem.

A bigger problem, definately, but that's not to say the former isn't an issue.

Re:Unconstitutional, unnecessary, and unacceptable

[servognome]

I'm trying to find out where in our Constitution does the Federal Government find an enumerated power to pay for this

"To make all Laws which shall be necessary and proper for carrying into Execution the foregoing Powers and all other Powers vested by this Constitution in the Government of the United States, or in any Department or Officer thereof."

You need to keep official records of things like: court rulings, legislation, federal expenditures, etc.

Different project

[Anonymous Coward]

They ought to make sure that real professional historians are there.

Your point is very valid, as far as correctly interpreting or getting meaning out of the data. However, this project is for archiving the raw data.

Someone now should fund the project for historians to create historical knowldege using this project's raw data output as it's input.

Re:Unconstitutional, unnecessary, and unacceptable

[dada21]

Roads are indeed one of the hardest points for me to debate with even myself in terms of "who provides?"

As a multi-business owner, I can tell you that I'd rather be paying for the roads that come to my store from the highway, and I'd rather co-op it with other businesses by finding the BEST builder and maintainer of the roads. Today, we all pay hidden gas and other taxes for use of the roads, but the costs are crazy (I should know, I've worked for a highway contractor).

If we could should the average taxpayer what we pay in FEDERAL taxes per mile of driving, they'd be outraged.

I'll grant you the roads, this time, as a government responsibility, but how about we leave it up to the towns or maybe the precincts, rather than the State or Federal governments?

A Life On The Toilet: The Scott Lockwood Story

[Anonymous Coward]

Coming soon to a bookstore near you! Reserve your copy online at lockwoodtoilet.com.

Hi Trent

[Catamaran]

I went browsing through some of your older comments and I came across this one which contains this broken link! Get it? archives.gov -> broken link? Ha ha ... heh.

Re:Hi Trent

[TrentL]

NARA recently re-organized some of their website. Info on the ERA project should now be available here.

dspace.org

[Broadcatch]

HP and MIT have been working on this same issue with the DSpace Project.

$308M would sure go far if doned to this open source federation!

Re:dspace.org

[Thu25245]

DSpace solves only a small portion of the archiving problem.

Where does the metadata come from? And more importantly, what happens when the bitstreams are dumped into the repository? You'll end up with a bunch of well-organized, but completely unreadable, binary data. Unless you convert everything into XML, which is probably a $300 million project right there.

Re:Unconstitutional, unnecessary, and unacceptable

[rstultz]

I have no idea what you're talking about. I pretty sure that the original poster was talking about the Alaskan bridge to Gravina Island. It is a $300 million bridge connecting a town of 8,000 to an island of 50. The island currently has no bridge, it has no paved roads for that matter. It is served by a ferry. Residents are quoting as saying they like their ferry, and didn't ask for a bridge. This is the project that the nation got up in arms about. What project are you talking about? Or are you just making things up? And as for how I know? Well I spent the five minutes to look it up. Looked to see what the project was called in the transportation bill and then found the actual information. Research, there's no substitute.

Ryan Stultz

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

* Nitpick: U.S. Constitution was written in 1787; ratified in '89.

1776 was when the Declaration of Independance was written. Don't forget about the Articles of Confederation between the two.

An Arms Dealer to Guard the Memory Hole!

[monk]

The articles were light (to the point of vacuum) on details about the approach proposed by the company.

From the article: "The system's architecture makes it flexible enough to accommodate evolving policy change," including the importance of "providing public access while protecting privacy and sensitive information." From the sound of that I'm betting its some wonky and ridiculous XML format infected with a sadly pathetic little DRM imp.
The fact is that I can read anything if I have a copy of the software that originally viewed/created it and the machine (or an emulation of the machine) on which the software ran. Adding one more format to the mix just means we have to emulate one more machine and keep track of one more piece of software and all the doubtlessly expensive effort which will be spent in conversion is wasted.

It's great to see the National Archives working on this but I would rather see the tax money farmed out in challenge grants to organizations like the
Long Now that have a chance in Hell of delivering something useful than pouring money into yet another defense company to ensure that whatever technology we use to store records can be properly sanitized and locked away according to the whims of government and "changing policy."
The biggest issue facing us right now is that most of the music, words and images created by our civilization are illegal to preserve. Ridiculous copyright extensions have ensured that the huge mass of data for which no rights owner can be found will simply rot instead of being digitized and stored.

A software emulator can ensure that historic file formats are readable in the future, but Big Media would rather squeeze our history to death before it letting go of the rights.

This is like 1000 fires at the Library in Alexandria. Future generations will curse us for every scrap of information we allow to rot while we squabble.

god, do what everybody else dose

[LWolenczak]

print the files to tiff format, then extract the text and metadata, and toss that into a searchable database, and then keep the orig file around for good measure

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

I wasn't the one that modded him down in the first place :) it was -1 when I got to it, and I made it 0 :) Now it is at 2...I am happy

Re:Unconstitutional, unnecessary, and unacceptable

[jcr]

Will we need old information in digital format?

It's not just old information. The Archives have a great deal of material from the second Bush administration already, and the rate of data accumulation is increasing steadily.

The reason to go digital is to reduce storage costs. Paper is a very low-density medium and it needs to be kept in climate-controlled warehouses if you don't want it to rot away.

-jcr

Re:Software is easy. Re:Chick and Egg problem

[bjbyrne]

I have always tried to get my data off old technology and move it to new technology before unpluging the old system. I think as newer technology comes out, designers are making it easier to move data. Now days since everything connects to eachother or the internet it is a piece of cake to accomplish this. Every time I move to a new system with a harddrive 5 to 1o times larger, I always have room to completely back up all the old data. For my life time, I am not worried.

Re:Unconstitutional, unnecessary, and unacceptable

[glitch23]

We might even be able to read between the lines 100 years from now and insert text like "separation of church and state" into various documents where it never existed before in order for some people to get their way about things and of course by that time no one would be around from this time to contradict the meaning.

Re:Tax dollars at work: 300M to solve a 100k probl

[alodien]

RTFA: "The search for a solution began seven years ago"

If I remember correctly, seven years ago was before Bush's presidency began.

Re:Software is easy. Re:Chick and Egg problem

[HiThere]

You mean code everything in MIX? It seems the most stable widely spread interpreted environment around. I suppose it could be done in p-code, that's pretty stable, and it used to be widely spread, so with a bit of work it could be so again.

The jvm on the other hand keeps changing every few years or so, ditto the Python interpreter. Parrot isn't ready and the C# interpreter has no track record (but since it comes from MS, I'd bet on it changing more often than the jvm + the Python interpreter combined).

Re:Truth be told.

[bjbyrne]

That must be similar to my pornographic memory.

Really Full Disclosure

[Anonymous Coward]

I worked on Lockheed's demo team and like to wear women's underwear.

Walt named names...

[Anonymous Coward]

Walt's testimony to the House Committee on Un-American Activities, 24 October, 1947

Not such a long term problem

[Crashmarik]

The republic isn't in the best of health lately. There are far too few people in the country that even understand what the republic was and why it was important. The education system seems purely concerned with the faults of the republic and the men that created it. The current media runs cowering away from anything good done in the nation.

How many people do you know that can say what the separation of powers between the federal government, state governments and local governments is meant to be. How many people do you know that understand why highway and education programs are little more than subversions of the constitution by the federal government. What is the commerce clause actually meant to regulate and what is it being used for.

So...

[sootman]

...Lockheed's getting $308M to keep a roomful of Pentium 200s running for eternity? :-)

Saving data for THOUSANDS of years

[Artifice_Eternity]

Check out the micro-etched data disks used by the Rosetta Project. Their goal is to create a long-lasting archive of the basic elements of 1,000 different languages. The storage medium they're using involves etching readable words on to metal disks. The words are not readable by the naked eye, but all you need to read them is a decent optical microscope -- no special hardware or software.

The Rosetta Project's customized "Rosetta Disk" adds another clever innovation: naked-eye-readable words around the edge of the disk get smaller as they spiral inward, making it clear to anyone who might find this disk in the future that there is more information to be read at greater magnifications.

Re:Unconstitutional, unnecessary, and unacceptable

[SiliconCactus]

"Research, there's no substitute."

Dig a little further. That island has the regional airport for that section of Alaska. Ferry service in that region is out a large fraction of the time due to fog.

I still don't think the bridge is justified but you are leaving out some relevant facts

Porn for life...

[Arimus]

Oh great, now future generations can be guaranteed the chance to see late 20th/early 21st century porn... ;)

Linux

[Anonymous Coward]

They should just use Linux. It might not support wireless networking or motherboard RAID properly, but it certainly supports all manner of hardware from 30 years ago that most users can't even remember, and archaic file formats are positively encouraged, half of them are part of the OS.

Re:Linux

[Anonymous Coward]

Agreed. And not knowing whether my 64bit cpu will segfault during compliation in a new kernel version, just adds the spice of life!!

Re:Unconstitutional, unnecessary, and unacceptable

[drxenos]

Only on slashdot would such an ignorant post be considered "insightful."

Rock...

[monkeyboy87]

Rock! nothing beats good old rock....

Re:Rock...

[TrentL]

Rock! nothing beats good old rock....

Sadly, neither IE6 nor Firefox have fully implemented a Rock Rendering Engine.

there's lost works too

[Anonymous Coward]

same with Aristotle and a lot of the greeks. Just because most of the stuff made it 500 years more or less intact doesn't mean that there wasn't stuff that was forgotten.

Just because it falls out of use for some years isn't a good reason to let it become forgotten.

Defense contractor?

[bergeron76]

Why did they pick a Defense contractor for this and why was it open only to Harris and Lockheed, and not other technology companies?

Also, is the implication here that the historical archives of our country (I'm in the US) are now controlled by the department of Homeland Security?

Re:Defense contractor?

[Anonymous Coward]

Working for the government is kind of painful (bureaucracy), which is why most private sector companies don't do this sort of thing.

Likewise, gov't contractors are not well suited for commercial pursuits, so they find work wherever in the gov't they can.

Not familiar with this contract, but generally there are multiple layers of proposals and cuts. I've never heard of a company denied the chance to compete (if there was a competition), but I'm not sure how that works exactly.

Generally contractors try for contracts they have a good chance of winning and that will provide a lot of money (and jobs too, usually). Not to mention having resources to work on the bids and proposals.

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

Yeesh, can't believe this is the only comment pointing out that booboo. Makes me wonder if most nerds are guilty of a poor knowledge of American history. :-P

Re:Unconstitutional, unnecessary, and unacceptable

[timeOday]

When it comes to constitutionality, it doesn't matter IF we need it, it doesn't matter WHY we need it, and it doesn't matter if the MAJORITY thinks we need it. The federal government has certain enumerated powers, and the rest is left to the States and the people.

Ha, that's a good one. Look, for better or worse, the idea of limited federal powers died with the civil rights movement and the New Deal. I don't think you can make much of an argument simply by pointing out that today's federal govt. is not particularly constitutional, it's so obvious I can only conclude that nobody cares.

Re:Unconstitutional, unnecessary, and unacceptable

[KarmaMB84]

If they were so unconstitutional, perhaps the supreme court wouldn't have refused to even hear challenges against them...

A complete waste of money.

[elucido]

Humans will be extinct in 100 years. How many of us think we can really last for 100 more years before we have another couple world wars?

If humans do exist in 100 years, I can guarentee that this economy wont.

Re:A complete waste of money.

[AussieVamp2]

If everyone will be extinct, no need to save money either?

Uh huh...

[skintigh2]

So, the company that printed a word doc, deleted the file, and then scanned the 100-odd page doc as IMAGES and handed it to my wife to make some changes, is going to preserve our info until the end of time.

The same company that backed up F-22 radar system designs to an obsolete tape, and then threw away every device in the company that could read the tape, is saving our data.

What could possibly go wrong?

Re:Unconstitutional, unnecessary, and unacceptable

[daigu]

While I sympathize with your libertarian critique of government spending, I do take issue with your free market fundamentalism. Free markets can be good at efficient allocation of resources. However, research is notoriously inefficient.

The more you outsource research to companies, the more incentive you have to offer them in terms of monopolistic protections such as copyright laws and patents. Whereas if the government sponsors this kind of work, it ultimately can be used for the benefit of society - by making the innovation available to the citizenry faster than when technology is developed by private industry given the monopolisitc protections they would need to be viable.

The Internet is the quintessential example of how this works. Would we be using Slashdot right now if we used your preferred method of outsourcing this work to private companies that "do it better"? I'd wager we would not. We would either have one company dominate the market a la Mircosoft or we would have many companies with only large corporations being able to pay the cost involved in all the duplication of effort and monopolisitc licensing fees.

Also, government monopoly is one way to deal with the inefficienies of free markets and can drive adoption rates. You wouldn't want 20 sets of telephone poles in your neighborhood for 20 different telephone companies. Being able to concentrate infrastucture enables you to distribute the technology to a wider population, faster. Of course, monopolies also have their inefficiencies. They have their place - just as free markets do.

On an unrelated note, The Constitution says in Article 1, Section 8 - Powers of Congress:

"The Congress shall have Power To lay and collect Taxes, Duties, Imposts and Excises, to pay the Debts and provide for the common Defence and general Welfare of the United States; but all Duties, Imposts and Excises shall be uniform throughout the United States;...To borrow money on the credit of the United States;..."

If you believe, as I do, that the history of the Republic is important for its future and its current Welfare, then it doesn't seem too much of a stretch that this covers the taxes and spending of it on a project like this one.

Yeah ...

[Anonymous Coward]

Go write some a few satellite software suites, then code a few embedded targeting systems for fighters, then write the software for a few air traffic management systems ...

Then, after you've written those five or ten million lines of reliable code, you will have written about 1/100th of the reliable operational code we probably have, and you can tell us how our software sucks.

for the life of the republic

[Anonymous Coward]

The jokes sort of write themselves, don't they.

Rosetta Project

[moosesocks]

Many /.-ers would be interested in the Rosetta Project which aims to preserve many world languages using an extremely failsafe medium. defintiely a cool read -- check it out.

sure, it may not be terribly convenient, but it's certainly going to be readable 100 to 1000 years from now (by which point we should have adequate OCR to complete the task of reading the disc automatically)

Re:Rosetta Project

[Anonymous Coward]

I have actually searched for this, to use on my own data.. like photos of family and such.. I lost a bunch of images due to a harddrive crash, 2 years of memories gone for ever. Resetta Project looks nice, but abit expencive for the normal user. Why not a small device that could burn binary data into steelplates.. would be nice.. aslong the reader are available later on (or easy to reproduce).. even punchcards could be acceptable for me =)

See you in court!

[adam31]

"Electronic Record Archive"???

Yeah, I suppose that's for "personal-use only"? Dude, they are so sued.

budget your time

[Anonymous Coward]

don't spend too much time on archiving the porn, i got that covered.

photogenic

[eweaver]

His memory looks good in pictures? Excellent! I'm sure it has something to do with homeland security.

Oh, brilliant....

[Anonymous Coward]

We're going sign a data archiving contract to the tune of $300 million with a company that has repeatedly demonstrated it can't find its own ass with both hands? Did anyone even look at their record with DOD contracts?

Simply amazing. I wonder if they'll be archiving the job offer letter from Lockheed that the government contracting officer probably has in his back pocket right now.

On the other hand, I am a little surprised, did Haliburton forget to bid or something?

Sheesh, could you be any less specific?

[Grendel Drago]

Since you were involved, could you let us in on what some of the strategies are/were? There are a lot of people pulling wild guesses out of their asses; it'd be nice to know what they actually came up with.

--grendel drago

Re:Unconstitutional, unnecessary, and unacceptable

[Jacked]

Now, I don't know what I'm talking about, since I didn't do any research. Having said that, it would seem to me to be far cheaper and more sensible to build a new airport on the mainland, where the majority of the people are, rather than a bridge to the island.

But, hey, that's just me.

Re:Software is easy. Re:Chick and Egg problem

[mrchaotica]

No, more like code everything in ANSI C (which, as far as I can tell, is the most standard, portable language currently in existence). Either that, or some other language with a formal ANSI or ISO specification (maybe something garbage-collected? I don't think C# is widely supported enough yet, not to mention the issues with non-Free libraries...).

Re:Step 1) Generate more documents in open formats

[pavon]

That's all well and good, but people, we are already DECADES behind on this problem. Whether you like it or not, there's a boatload of Word95/Excel/BMP/etc files out there (and worse).

Tell me about it. It would take more fingers than I have just to count all the proprietary formats we have for engineering drawings around here :) Sorry if I came across as trivializing your work, I definitely didn't mean to do so. Good luck, and hopefully the job will get easier in the future and not worse.

Re:Why not? FOUND! Gold Disk 'Kodak' CDRs!

[iamcf13]

Contrast this to the good old "Kodak Gold" CDs I was burning onto back in 1996, almost all of which are still readable with 0% errors...


American Digital

Mitsui MAM-A CDRs

Kodak probably 'rebranded' the MAM-As and sold them under the Kodak name years and years ago.

I miss the Kodak gold disks and still have a few around brand new and ready to burn.

As far as I know, the Kodak CDRs I've burnt are still 100% error free after having some of the burnt CDRs for well over 5 years. I simply stored them in a box out of direct sunlight and did what I could to keep them safe from extreme temperatures....

Long Term Archival - The Long Now Foundation

[Anyd Blek]

Some of the goals of this project parallel those of the Long Now Foundation: http://www.longnow.org/ Seems like they should have been able to bid, if they did not.

Re:Unconstitutional, unnecessary, and unacceptable

[Doc Ruby]

Most roads are city, county or state projects, for design, production, maintenance and financing - though the feds subsidize even state highways, and often local ones. Interstates require larger scale planning, though the Federal planning is usually influenced by local politicians and contractors, almost always for patronage and other favoritism, rather than design optimization. That corruption usually is worse when local bosses are in control, though this year's Congressional Highway Bill is one of the greatest travesties since Robert Moses remade New York in his own twisted image.

Re:Unconstitutional, unnecessary, and unacceptable

[Doc Ruby]

Yeah, what exactly is your response to the comment replying to yours, with the facts about the $300M bridge to nowhere? What is this "knowledge" you have, that justifies spending $300M on a bridge serving 8050 Alaskans? Isn't it really just lying, to cover up some project that you have some obscured interest in? Don Young, is that you?

Re:Unconstitutional, unnecessary, and unacceptable

[Doc Ruby]

The National Archives is composed of many materials we've accumulated, often at great cost in collection and curation, over the life of our Republic.

Can I have some space??

[mlush]

Trying to keep the family electronic archives requires active maintainance (basically copying to a new redundant array every few years). I'd love to have access to a well backed electronic safe deposit that didn't require my maintainance, on whom I could rely on to provide at least 100 years worth of archiving

Re:Unconstitutional, unnecessary, and unacceptable

[Anonymous Coward]

-1 stupidest comment ever moderated up on slashdot

Re:Tax dollars at work: 300M to solve a 100k probl

[dangitman]

Seriously. I'd do this for free. Many people probably already have. step 1. Massive raid 5 array with a mirror array off site and pleanty of hot spares. step 2. Fix date stamping on whatever image format, sound format, text format you decide on. step 3. Don't use lossy compression.

Yeah, all that really sounds like it will be free. Whatever you say, boss.

What a waste of money

[Anonymous Coward]

I can tell you right now that when the Mechas of the future finish off the carbon-based units infesting the Earth, the only thing left will be a little bit of porn and a little bit of plastic. Thats right folks that's gonna be our lagacy.

(and just to make sure I do my part, i am now going to go take a picture of my dick, place the photo in a ziplock and bury it out back.)

Have a nice day!

IrfanView

[Crouty]

IrfanView is an excellent free Windows program that makes converting large numbers of image files a breeze and of course it can convert from IFF. Plus it's my favorite image viewer.

Re:Software is easy. Re:Chick and Egg problem

[smallpaul]

This is not nearly as difficult as you make it seem: implement the parser in a standardized language. The formal specification of the standardized language can then be included with the source of the parser.

That's just weird. Now you have to interpret the document format that the language specification is written in, write a compiler for the language, run the compiler for the language on the documents and then interpret the output of the parser. Alternately you could document the document format that the content is in and let later consumers write parsers in whatever programming language they feel is best.

I'm not against providing implementations of parsers as documentation for the document format. They might still have a C or Python compiler around and it could be handy. But it doesn't make sense to focus on documenting the parser implementation language rather than the content format language.

Re:IFF-ILBM (Interchange File Format)

[Anonymous Coward]

That's funny. As an example of OBSCURITY, IFF is a poor pick. It's an expansion of the Macintosh resource concept that, like XML, allows for a hierarchical organization of data, where if you simply can't understand (or use) one of the internal data chunks, you can more or less safely skip over it (since each chunk has a length attached to it).

And, if you really need to understand it, I can probably spare one of my old Addison-Wesley Amiga guides that details IFF and its various implementations (it was also used for word processors, musical composition and multimedia presentations). ILBM is simply the Interleaved BitMap format.

The Microsoft WAV file format, is, in fact, a slightly perverted form of IFF.

Now if your Windows apps can't READ IFF, I'd take it up with the vendor.

Mmmm, Irony!

[Anonymous Coward]

Pretty funny, considering Lockheed just instituted a corporate policy that *all* employee emails be deleted after 90 days.

No joke.

Re:Chick and Egg problem - XML Schema

[Anonymous Coward]

XML Schema is in XML. XML is a text format where the tags indicate the form and the attributes and values provide the content.

Not to say that XML is a self-explanatory panacea - Microsoft is already hard at work trying to ensure that never happens with their patented XML document format which doubtless will be full of the indosyncrasies and obscurities for which the DOC file format is infamous.

But don't make life more difficult than it needs to be.

Check their recent work on the Census project

[mekkab]

Individuals input is periodically checked for accuracy. As more mistakes are found, you get less and less work (as in, you get fired).

When a data entry clerk is let go, all of their previous work is re-worked.

Re:Check their recent work on the Census project

[zenneth]

That makes no sense. How can you redo someone's previous work that has no real method for correction?

Re:Check their recent work on the Census project

[mekkab]

They are typing in pieces of paper. You associate their user id with each piece of paper the process. You double check their work against those pieces of paper. If they screw up too much, ALL the paper they EVER touched goes from the "DONE" pile back into the "TO BE DONE" pile.

It makes perfect sense.

Re:Chick and Egg problem - the egg

[ebvwfbw]

The answer of course if the egg. Eggs have been around millions of years longer than the chicken.

Format Accuracy

[Anonymous Coward]

This actually brings up a very good point relating to TFA and preserving accuracy: Mikmod seems to be the standard library for tracked music, with Modplug libraries a close second. The problem is that neither of them is entirely faithful to some formats, Impulse Tracker files in particular. It gets the point across OK but there are sometimes very obvious differences between the songs as played in Impulse Tracker and one of these other players.

The only software I know which is meant for modern computers that does a 100% accurate job is XMPlay http://www.un4seen.com/ which is sadly windows-only. Though there is an XMMS plugin.

Good Luck Guys! (You'll need it)

[Ranten_N_Raven]

I was on the team for another major aerospace firm that considered bidding on this work. It is a huge, huge, gigantically awesome problem. The expected amount of data to be archived and indexed and protected is mind boggling -- many terabytes the first year, then it really grows!

We decided to not bid on it. Too much technical risk.

Re:Unconstitutional, unnecessary, and unacceptable

[MrCopilot]

Thank You, Chief Justice Gonzalez/Roberts.

BTW.. Only the ACTUAL Supreme Court can decide what is Unconstitutional.
Unacceptable.. Hmmm Lockhead wouldn't be my first choice either, especially if the goal is to keep costs down.
Unnecessary, Come on man. We lose information on a daily basis. We've already paid for it. You want it to decay on magnetic tape? NASA Photos, Log data, NOAA, ARMY, Nuke teting, all of this should be preserved and made available. I for one don't want to have to load the Reels to access that info anymore. Might as well ask my kids to load the punch cards for census data.

long-term storage

[radu124]

Digipress had some interesting solution for long-term storage called "eon disk".

http://www.mosarca.com/CDINFO/CENTURY.htm

also you could consider storing information in the DNA of some insect or fish. Assuming it won't become extinct you'll only have to catch one in order to get your information back.

The advantage is you don't have to make the backup copies yourself.