Software Archaeology

Plug1 writes "Salon (day pass needed) has an article about preserving software for historical purposes. It discusses source code archiving, and the effect the DMCA is having on attempts to catalog and analyze legacy code. It will be a shame if in the future a wealth of information is locked away because knoweldge of the underlying technology is lost."

Comments

Please understand...

[Creepy Crawler]

That the DMCA DOES NOT APPLY outside the USA. However, hardware Digital Restriction Management DOES.

I really dont want strong crypto keeping out of stuff that I OWN, or My CONTENT.

I'td be a neat experiemnt to create a Linux driver that emulates TCPA chips so that stupid software thinks you're auth'ed.

Re:Please understand...

[Creepy Crawler]

It's a signature/encryption mechanism. Wait till MS requires it ON. They could even make it so the whole fucking partition is encrypted by a software key that YOU CANT GET.

And once MS requires it, how's Linux going to fit in there? I'd figure that MS TCPA computers would have to be signed to even speak to other MS machines. We cant have traffic going out of the network that isnt validated for internal traffic.

Re:Please understand...

[Lazar Dobrescu]

This is not the only problem the article addresses though. As it is now, there are already tons of old file formats for which the software needed to read it is nearly impossible(or totally impossible) to find. Documents written in those file formats could contain useful, or at the least interesting content, but we can't get to that content.

We are talking here about file formats 30 years old, or even less. Try to imagine what will happen in 200 years. Most of our history will be written to electronic media, and for people that will live in 200 years, the file format used for that media will very probably be undecipherable.

What is the solution? Some say that we need to convert all documents in a more recent file format every x years. That will really become a pain in the ass as the number of archives go higher and higher.

Another trick could be to describe in whole the file format used and attach that description to every file. That, of course, brings up the problem of what file format to use for that description... (will even plain ascii files still exist in 200 years? Maybe not, but I think it is reasonnable to expect that people will at least still have an idea of how to read them...)

Comparing this to the problem faced for dead languages gives a good idea of the repercussions... There is already countless documents written in very old ages that we cannot decipher because the language used to write it is loss. People are working all their lives trying to understand a dead language. But with computers, we're not talking about something that happened 4000 years ago, but 30 years ago... That means that in the course of your lifetime, You could see obsolete file formats 3 times!

Someone will need to find a solution for this, and preferably before the problem happens for real...

Re:Please understand...

[Ominous Coward]

What we need to do is have a large book that describes all of the file formats. ASCII encoding, JPEG encoding, etc.

The real worry I'd have is how someone will be able to get the stuff off of the media if the directory and interface standards change. Will their advanced computers even be able to read the disk to see that goatse.jpg is on that disk? Even if they had the algorithm to decode the image, they might not see it's there.

Re:Please understand...

[Sique]

The rosetta stone contained the message in Ancient Greek (a dead but widely known language at the time of deciphration), Coptian and Hieroglyphic. Even though Coptian was at least known to some specialists and people able to read Ancient Greek were abundant at the time (and still are), it took about 25years to decipher the hieroglyphic texts.

And this was with a language which itself was very easy mapped to the letters (every consonant mapped to a letter, vowels omitted).

The rules which encode a file may be much more complicated. Look just at the most common compression methods (Run Length for instance), how they just add another layer above the already encoded contents. And they remove something very important for deciphration, the redundancy, out of the data. Then the subjects that are stored in files are much more diverse. We have not only language, we have music and graphics, 3D data and cryptographic certificates, configuration files and program binaries.

Just to be able to know what the file is about and thus have an idea how to get started can prove to be more complicated than any deciphration from archaeologic texts.

Re:Please understand...

[DiscoDave_25]

It's not just the file format that will be the problem (although MS aren't helping in that respect) but simply ensuring that the media that the file is written on can be read. Physical media degrade and the hardware to read them become obselete. An example of this was the BBCs Doomsday disk which contained a huge amount of information (for those days) on a laser disk that is today virtually unreadable. Thankfully this has been recently transferred onto DVD before ALL the readers died but just because someone can understand HOW to read a file doesn't mean they'll be able to access it in the first place.

Re:Please understand...

[4of12]

an archaeologist of tomorrow can figure out ascii.

To be sure.

And will they be able to figure out PowerPoint?

And how about Secure PointPoint 2005 with automatic DocuSafe technology that incorporates encryption with a public key that is automatically downloaded over the network from microsoft.com after your VISA card number has been authenticated with citibank.com?

No, tomorrow's archaeologists will miss out on the whole indecipherable morass that is today's data formats.

Documents and presentations will look indistinguishable from random noise.

And, honestly, a lot of what gets attached in those formats looks that way already to me in 2003.

Re:full article text, no pass required

[mozumder]

You know, it really isn't fair-use to repost an entire article from another website site.

Re:Explain the Pyramids?

[KalvinB]

There's also the problem of grave robbers and that whole burning of the great library thing.

The Egyptians could very well have written down the instructions for building them. There have been numerous opportunities for that information to be have been destroyed. Or they may have viewed their construction as too sacred and only passed down information on a need to know basis.

Our problem is that we charge for rocks and lack the motivation. We just assume we couldn't build such things as they did but never really bother to try.

Ben

Another red herring from salon?

[poptones]

In one part of the article they mention losing "structure" of programs and talk about source code, then they talk about "losing" old code like the original DOS - for which, so far as I know, there is no publically available archive of source code. So too of Lotus 123, another piece of code mentioned in the article. this is just more fatalistic nonsense people spew when criticising the DMCA. Yeah, it's a bad law, but this nonsense about "losing old works" is just that.

If you have the source code for something then you have no cause to fear the DMCA, since you don't need to decrypt it. And if you don't have the source code, where is the value? Is there really any value in running lotus 123 for the Apple//? Perhaps if you have an Apple//, but so what? You cannot "fly over the code" from any height (as was mentioned in the article) because you don't have any code to fly over. You have an executable, and the "structure" there is quite different than looking at source code.

If you want source code for DOS, hit freedos.org and download it. It's not Microsoft's source, but so what? It does the very same job and, in many cases, it's superior to the original. Works that have value will be replicated and emulated; works thta have no value simply have no value - where is the need (or logic) in "preserving" them?

Formats not the problem

[Mr_Silver]

I don't think the format issue is that big a problem. A large number of closed formats have been reverse engineered to a point where you can extrapolate the pertinant information. Your biggest problem is availability of the hardware.

Take the Doomsday Project (in the UK) as an example. An Acorm Archimedies lazerdisc full of content relating to life in the 20th century. The problem came when they wanted to get the data off .. and couldn't easily find a compatible lazerdisc reader.

Of course, the format of the data is an issue. But if you can't get the data off the media, then the format of it isn't going to matter in the slightest.

Re:full article text, no pass required

[Andrew Leonard]

At least with jay-walking, no matter how many times you do it, the road will still be there. But if you post the full text of Salon stories without either subscribing or getting the FREE day-pass, eventually we will no longer be able to pay fine writers like Sam Williams and Rachel Chalmers to write the stories that Slashdot readers like to read.

Re:Knuth is only one foundation that won't be lost

[Kaa]

The most fundamental concept in computer science is logic, not algorithms (or worse programming languages). If a 'programmer' hasn't written a program in a low level language like C or assembler, the hiring manager should beware. Without hands-on experience with the fundamentals of computer science that person is lacking at the most basic level, regardless of whether he knows 1 language or 50 languages. He is handicapped.

Bullshit.

"Computer science is about computers in the same way astronomy is about telescopes" --Edsgar Dijkstra

Programming isn't about knowing how to twiddle bits in registers or even how to leverage strengths of a particular processor.

Programming is about dealing with complex problems which can be solved by manipulation of information. I would say the the quality a programmer needs most of all is not logic or math, but just the ability to hold and manipulate large and complicated structures inside his head. And no, it doesn't have anything to do with assembler, low-level languages, ALUs, bits, etc. etc.

Re:full article text, no pass required

[Andrew Leonard]

You watch an ad to get a day pass. Advertisers pay to sponsor the daypass. The more people use the daypass, the more valuable that sponsorship, and the more we can charge for it.

Mandatory source code deposit

[Animats]

This is a good argument for mandatory source code deposit. To get a copyright on code, you should have to deposit a copy of the source with the Library of Congress. The Library of Congress has the authority to require this, but currently they only require a printout of the first ten and last ten pages, because they didn't want to store all the paper. That should change.

Re:Other technologies go obsolete too, So what?

[jafac]

That same SciAm article mentioned the impending loss of archived data from NASA, data collected from satellites launched at a cost of hundreds of millions of dollars.

Some of this data is useless, today. In the future, someone might find it useful. Do we allow this data to degrade, and then possibly launch a new satellite to collect new data (if that's even possible, in some cases, it's not - how do you gather climate data from the 1970's?).

The main problem is the tape backup companies no longer support the old tape drives, and new tape drives don't support the old tapes and tape formats.

Funny thing is, 5 years ago, I was there with everyone else saying that we should put this data on CD ROM, because that format will never, ever, ever go away. Now, I'm not so sure - if they ever straighten out the DVD standard, I can see a future, 10 years from now, when you won't be able to buy a new device that can read a CD ROM.

Anyone have a spare 8" floppy drive?

[spun]

I have some old AutoCAD 3 files from high school, a hopelessly optimistic design for an automatic vacuum cleaner, if I recall.

My dad still has a program he wrote on punch cards someplace.

That's the trouble, isn't it? Even if the data survives, the hardware to read it might not.