Slashdot Mirror
Examples of Obsolete File Formats?
reedk writes "I was having a discussion with my boss about long-term archives, and we got on the topic of older files becoming un-readable by newer versions of software. Not only are those old Ami pro files unreadable by today's common word processors, but I have heard that newer version of Office can't consistently open very old versions of Office documents. With the increasing retention periods being forced by current and coming regulations, this could become a problem of compliance in the future. We want to pursue this topic, but to build support for it internally, I am looking for examples of older file formats that are no longer readable by newer version of the same software or due to the market death of the product. If true, this would lend a lot of force behind moving to products that have an open file format. Can Slashdot readers come up with examples of this, or ways they have had to get around these kinds of problems?"
For this same reason I usually suggest to people that with very long term backups (assuming the backups actually survive) try to save your data in non propriatery forms. I am not trying to make a closed source vs open source argument, however if you want to save a large batch of word documents that you will not need to access in the near future try to convert them to plaintext where you can. Not fullproof, and not applicable for the majority of situations, but there are a few things that we can assume will not happen in the near future: 1) ascii will probably not die, so plaintext is often a good idea, 2) many of the more common image formats will probably be supported in one form or another (gif, jpg), you know stuff like that.
AppleWorks had no idea what to do with AppleWorks documents - assuming you can get a mac to read an Apple ][ floppy in the first place...
For that matter, is there anything that can read VisiCalc files?
Flame ON!
Don't call me back. Give me a call back. Bye. So yeah. But bye our, well, but alright we are on a shirt this chill.
Well, if it is an open format, nothing is stopping someone from writing something to read it and convert it to something "modern". If it is a closed format, and no longer in use, then the owner really should open it up. Would it be possible to setup an escrow of (closed) file formats - automatic open if the company goes defunct or individual dies.
Also, if you know what the end result data is supposed to look like, would it be possible to start "decompiling" it? Works with binary executables (sometimes)...
Don't blame me, I voted for Kodos
Formats that are worth using for old (and sometimes new) documents:
* RTF (quite universal)
* PDF (somewhat universal, will always have the same formatting)
* Plaintext (never becomes unreadable unless the file's character set ceases to exist somehow)
'Yes, firefox is indeed greater than women. Can women block pops up for you? No. Can Firefox show you naked women? Yes.'
To start, they used open records requests to get the details of people who recently voted, and details of those who recently died.
The goal was to find people who continued to vote after they died, which may sound funny, but is still happening.
The data the government data gave us was on magnetic reels. The data on the reels was stored in a fixed-width EBCDIC format. Talk about a dead format!
It turned out the local college still had a working magnetic reel reader, and was able to help me get the data out of EBCDIC into ASCII, but the project was cancelled anyway.
Well, yes and no. Let's say Ami Pro file format were fully documented. (I have no idea whether it is or isn't.) At what point would it be worthwhile for your company to actually write a file converter? I can certainly imagine a situation where it might be a cost-effective thing to do, but it's not the kind of thing that anyplace I've ever worked does routinely.
And from a retention point of view, I don't know if you _want_ whatever scumbag lawyer is subpoenaeing documents from you to be able to demand that you write him a converter. I'd rather be able to say "Here are our VisiCalc files. Enjoy!"
What I'm listening to now on Pandora...
A Boss I used to have that worked on many DARPA sponsored projects used to have to archive ALL data related to those projects. In order to this, not only did we have to archive the data itself, we had to archive a PC with all the pertinent software necessary to view/compile/manipulate that data including workstations, servers, you name it. Of course the government standard may be over kill for many companies.....
It was, frankly, awful. Someone had clearly designed it as some kind of "One size fits all" type thing, except that as it was text based it didn't really work that well. Typically graphics, for example, had to be represented by a block that contained a filename: yep, graphics, sound, anything more complicated than a word or a number had to be put in a separate file. Neither my collegues nor I could understand why anyone would try to put so much effort into making it look hierarchical and extensible, and then not include support for data that isn't well represented as text. Hell, most of the files on our PCs can't easily be represented efficiently or usefully as text.
It was also remarkably inefficient. To give you some idea, when we converted it into plain text files in a more efficient form, the files were typically 60-70% smaller. I've always found gzip a good indicator of the efficiency of a file format - usually, plain text compresses to about 30% of the original size. In this case, it was frequently 10%.
Absolutely horrible format. I hope I never have to work with it again.
You are not alone. This is not normal. None of this is normal.
While the file format itself may be a standard wrapper, there's many codecs out there that are obsolete and that only ever had proprietary drivers written for early MS Windows versions, for example.
It first depends on what you want to achieve with them, do you want them to be read only, or do you wish to edit them as well in the future? They may not be too much of an issue but something to think about.
For images, I would look at the past to see what file formats were around before the internet was mainstream, circa 1995. I remember Paintbrush PCX as a file format, but haven't since a file in that format since then. TGAs and TIFFs were around and still are today, that might be one possibility. You also have SVG formats, and that being an XML file format, allows you to convert it to another format in the future.
As for text documents, one definite possibility is XML. You can convert to many other formats from XML (HTML, PDF, RTF, etc.) Another possibility is RTF and plain text, though you might lose some of the more advance features. You might even have to extend the XML to deal with anything special in your files. Latex or Tex might be another solution since it's still around, though I have no experience with it, beyond being awware of them.
I would also recommend keeping a copy of the original software you used at the time, in case you need to get access to the files with a program that actually created. This way, you still have some sort of access. If that means you need to keep a copy of the original O/S as well, so be it.
But even so, the other day I got a shock, seeing how quickly the door closes.
A professor at the university where I work turned up with his original doctoral thesis from 1989 on disk. 3" disk, to be exact - the format that famously lost out to the ubiquitous 3.5" disk. He had written it on the Amstrad PCW 8256, a weird British CP/M machine from the mid 80s. No matter, I have several of these rotting in my loft!
But they don't boot. At this point you brace yourself for the long haul. The drive belts used to perish on those models, but look! There are loads of drive belts in the Maplin Electronics catalogue. You just need to order the right size.
No problem! You carefully dismantle the drive and dig out the belt. You broke it? No problem! Just makes it easier to measure. You can only measure the circumference, whereas Maplin only quotes the diameter? No problem! You are about to use Pi for the first and last time in your entire life! Order one that's slightly too big, and one that's slightly too small, just to feel safe.
When the belt arrives, you fit it. You carefully re-assemble the drive. You insert that CP/M boot disk that you carefully prepared in 1987, the one with the custom PROFILE.SUB that copies important utilities to RAMDISK. You power up and it boots! You feel young again.
Now your try your Locoscript boot disk - remember, Locoscript did not run under CP/M - it was an entire little operating system unto itself. It works, and when you swap disks (f7) you can read the Prof's work! It's yesterday once more! Shoo-bee-doo-lang-lang!
At this point I got lucky - I had the LOCOLINK package including the special Amstrad Bus PC parallel port link cable, so I was able to go Locosript PCW -> Locoscript PC -> Wordstar 3.3 -> Wordperfect 5.1 -> Winword. Those nice chaps at Ansible could have shortened that trip by a step or two.
In the absence of the proprietary LOCOLINK cable I could also have gone Locoscript 1 PCW -> Locoscript 2 PCW -> ASCII on PCW -> ASCII on PC via Kermit -> Winword. But I'd have lost all his bolds and underlines.
Now I got a fine bottle of Metaxa Greek Brandy out of this exchange, so I'm not exactly complaining. But I was shocked to realise that his files were younger than my eldest child, and she's got two years of school ahead of her.
In the absence of any credible international initiative to create a reliable permanent archive format, I'd say print it to acid-free paper, multiple copies in separate places, and hope for the best, like Cassiodorus.
XML! Open-source! Standards-compliant! Rag-doll physics! (Oh wait, wrong buzzword-bank...)
Sig Appended to the end of comments you post. 120 chars.
I keep double clicking on these ".mpg.avi.jpg.Donkey Bukkake Porn.wmv.exe" files and nothing happens!
Maybe I should start using Windows?
I contract out as a technical writer. For my primary client, I strongly encouraged and then delivered a plaintext solution that uses plaintext files stored repositoried in CVS, using the reStructured Text markup conventions processed through Docutils; and an XSL:FO template that is used by XEP to render the DocutilsXML to PDF. An autobuild system updates our documentation on a nightly basis.
This system has worked superlatively. In addition to creating a documentation solution that will forevermore be accessible without special software, our authors can focus entirely on content without concern for layout and visual appearance, our customers get a reasonably open file format (PDF) that looks as good on-screen as it does in print. It's win-win all around, by my reckoning.
--
Don't like it? Respond with words, not karma.
Before you can worry about reading individual files, you'll need to get them off the backup media.
Assuming that you've got some hardware that can physically read whatever it is, what about the backup software?
For example:
http://support.microsoft.com/?kbid=305381, complete with quote "this behavior is by design".
No, don't laugh.
Realplayer 10 doesn't support Realplayer 2 "out of the box". It will happily connect to Real to download said codec if you want - although obviously this assumes that Real will always be with us.
Any MS Word ships with only one version of Equation Editor; it was 1.0 in Word 2, 2.0 in Word 6, and probably 3.0 or higher now. It means you cannot edit your old equations after switching to a newer version. Therefore most of those who tried to use Word for writing scientific papers left Word after version 6 came out, now only biologists and like still use it because they don't need no bloody math.
Classic example: sharing MS Word files with other word processors. The problem isn't getting at the data in .DOC format (not an easy problem, but one that was solved years ago). The problem is rendering Word formatting using the conventions of other word processors. As anybody who's tried to import complex Word documents into Open Office will testify, that's a problem that's a long way from being solved -- if it ever is.
I've been working on a project for an organization that has a bunch of certificates created in Adobe Illustrator 6. The files are saved in EPS format, which belongs to Adobe, but is very well documented. So accessing the files should be a snap, right? Wrong. I have Adobe Illustrator 11 (better known as Illustrator CS), which uses completely different conventions for creating an EPS file. It can read the old files OK -- but it horribly mungs the formatting. Somebody's going to have to sit down and undo all that munging, which will be a day or two of work. Then we can make the simple change (inserting a new signature), that's the only change we want to make!
So true openness has more to it than knowing what all the bits and bytes do. It's making sure that all the different design teams for different products that use the format (or the same product at different times!) are on the same page when it comes to the fine details.
I don't understand where you're going there unless the spec allows embeded raster images?
It's straightforward to make an <img /> or <object /> element that contains raster image data. Look up the data: URL scheme.
1)Two copies of archival quality hardcopy stored off site printed using an OCR font.
2)Two copies of archival quality media stored off site saved as RTF as well as the working format.
3) Regular on site archives.
4) Regular on site backups.
At least one off site facility should be a secure storage facility. The other should be accessible 24/7/365, therefore it should be on company property. Each site has paper and media. Archive quarterly.
However, mostly it sounds like you need to hire a real Technical Writer and some competent IT people. This is 101 stuff.
I'd guess that CD-R is here to stay, given that it shows 0 signs of becoming unsupported on newly manufactured HW.
20 years ago, you could have said the same thing about a 3.5" floppy. When the iMac first came out in, what, 98, it was widely denigrated for not having a floppy. It's now getting increasingly harder to get floppy drives on PCs, and I wouldn't be surprised at all if they were special-order in another 5 years. In 10 years, your .sig file will be larger than the contents of a 1.4 MB floppy, so why would anyone include them on new hardware?
I think the only thing to do about data like this is to keep in on a fileserver, and then move the data as the server gets older. As long as it talks tcp/ip, you'll probably be able to get it off--that's one standard that's not going away for a long time, and will be backwards compatible when it does.
--
$tar -xvf
Some might argue that Quattro Pro is still alive (they're still releasing new versions), but its default spreadsheet format is entirely unsupported by the rest of the world. Every time someone cracks their file format, they make a new one. WB1, WB2, WB3, and now QPW. QPW is already 8 years old and still few have figured it out well enough to even extract data from it. If Corel Office dies, many old spreadsheets will slip into oblivion unless converted manually (open, save as, close, and repeat for each of your 500+ spreadsheets).
Now, playing viv files on windows is a pain, you have to install the archaic vivo player, which was designed for windows 95 or so. Also after years of searching, noone makes an app to convert them to mpg, sans some commercial screen capturing programs that I wouldn't touch. MPlayer plays the files, and Im pretty sure its a simple command to output it into an MPG.
Ever since I've been penguiny, I've wanted to do that - before the MPlayer team decides to depricate vivo support from the latest versions.
With the SDK, which you can download for free, you get full reference of the file formats of WP, Presentation and QuattroPro.
The problem is rather that nobody is interested in creating the conversion filters. For WordPerfect, there is now libwpd, which was built with the aforementioned reference. For QuattroPro, there isn't enough interest.
A secondary problem is that Corel Office programs have, for most of their programs, more powerful/flexible/numerous features than their competition, which can make conversion clumsy.
"I'm never quite so stupid as when I'm being smart" (Linus van Pelt)
...with embedded images and such that were created by Geoworks Ensemble back in the early 1990's, and converting them to another format has proven to be a bit of a pain due to the lack of good export filters in GeoWrite or its successors, and also due to the fact that nobody else seems to be able to read GeoWrite files.
Thankfully, I can still get the PC/GEOS environment to work on various PCs at home, but at some point that won't be an option.
Mainframe/UNIX Bit Twiddler and long time Windows/Linux Hobbyist.
The Theorem Theorem: If If, Then Then.
I work in an aerospace division of a very large corporation. I was talking to a design engineer about the FAA's data retention requirements - he said in most cases, it's the life of the product, plus a little cushion. For us, that's about 40 years. In addition to preserving data, you have to be able to recreate the analysis - so if you used a visicalc spreadsheet to perform an analysis, you have be able to do it again. (I think this is more an "in case we get sued" requirement than an FAA one). I was joking about 40 years being a long time when the coworker said, "Just be glad you don't work for the medical division. They have to keep their design data for the lifespan of the patient. For a neonatal ultrasound product, that's effectively one hundred years!"
Envy my 5 digit Slashdot User ID!
Looks like one of these
Unusual things. Harder case than a 3.25" disk, and slightly rectangular. Only ever really used on Amstrad machines.