Preserving Old Research Notes and Documents?

twistedcubic asks: "I have several thousand 8.5 x 11 inch dead tree pages of notes and research that takes up too much storage space. I would like to have all these notes scanned into PDF files (for example) so I can recycle the pages and reclaim storage space. Does anyone know of a store that provides this service, or an inexpensive machine that will do the job in a reasonable amount of time?"

Comments

In a few months time...

[Anonymous Coward]

In a few months time... coming as a duplicated story post...

"I have several thousand PDF files taking up too much disk storage space. I would like to have all these files printed on to 8.5 x 11 inch dead tree pages of notes so I can delete the files, empty the recycle bin and reclaim storage space. Does anyone know of a store that provides this service, or an inexpensive machine that will do the job in a reasonable amount of time?"

For future reference, I suggest a printer.

--BladeMelbourne

Easy

[ptaff]

10-year old nephew and a scanner.

Re:Not the ideal solution, but a start..

[NanoGator]

Sorry to reply to my own post, but I felt bad about the unhelpfulness of my previous comment. I headed over to Visioneer's site (www.visioneer.com) and found a few scanners that handle like 25 pages at a time. The more you spend, the faster it scans. Sorry, I cannot personally recommend a scanner in particular. Never had one like this.

Good luck!

Buy a scanner with an ADF

[zhiwenchong]

ADF (Automatic Document Feeder) scanners are fairly pricey (good ones are in the US$400 - US$1000 range, but you can get a cheapie Brother MFC-3240C All-In-One (C$140) that has a 20-page document feeder and then get a slave (e.g. some grad student) to feed in your pages for you.

My Brother MFC-2340C scanner comes with the PaperPort application, which generates PDFs and supports double-sided scanning even though the scanner doesn't support it. (You just flip over the whole stack once you've scanned one side, and start scanning the other side. Paperport knows how to automatically reconcile the pages.)

If you have Acrobat Professional, you can do a Paper Capture(TM) which is basically doing an OCR on the PDF and then storing the recognized words as "keywords" so that the PDF is searchable via Spotlight or other indexing mechanisms.

A document scanner is indeed a very useful piece of equipment -- I use it to scan notes and scrap paper containing rough ideas, often with lots of mathematics. Sometimes writing stuff on paper is just easier than typing in LaTeX...

The eminent computer scientist Edsger Dijkstra also liked to write stuff using pen and paper. His digitized works, called EWDs (after his initials, Edsger Wybe Dijkstra) are available here:
http://www.cs.utexas.edu/users/EWD/

Re:Buy a scanner with an ADF

[fm6]

That would be the best method, but I would seriously question the wisdom of PDF files. Although they represent documents fairly well, the format is too proprietary and too variable to be safe. You want the baseline documents to be in a format you can read at ANY time in the future, not just three weeks down the road.

I'm not a big fan of PDF, at least as it's commonly used. (It's essential for prepress applications, but it's most commonly used for online document sharing, an application for which it sucks.) So I hate to disagree with a fellow PDF-hater. But your arguments against using it are nonsense.

Technically, yes, PDF is a proprietary format. a well-documented, widely licensed format. Really, it's just Postscript with a few organizational elements. Both Postscript and PDF have many third-part implementations, including one that's available under the GPL.

With the merge of Adobe and Macromedia, the constant toying with DRM schemes, the allowing of unsafe code in current Adobe formats, etc, make format choice as vital as scanner choice.

I don't see what the merger with Macromedia has to do with anything. DRM would be an issue if Adobe was the only source for PDF software -- but it's not.

A good example of this was the use of Laserdisks for the 1980's survey of Britain to commemorate the Domesday Book. The Domesday Project is now unusable on anything but a very small number of machines, because they weren't adequately careful.

Hindsight is all very well -- but what format would you have chosen? Floppy disks would have been too expensive, CDs didn't exist yet. If it had been up to me, I would have chosen 9-track mag tape -- and I would have been wrong. (I still have a 9-track tape containing a backup of my student files, and no way to read it!) In any case, that mistake had to do with a choice of hardware. It's a lot easier to recreate old software than old hardware.

I'll skip past all your other hardware examples (papyrus???) and skip to...

In other words, don't digitize (or file) for the sake of doing so....

What, you think this is some kind of whim? If these documents are at all important, he has to bring them online. As long as they exist only in dead tree form, they are awkward to access, expensive to store, and run the risk of being lost in day-to-day use, to say nothing of the odd natural disaster.

Re:Buy a scanner with an ADF

[sribe]

That would be the best method, but I would seriously question the wisdom of PDF files. Although they represent documents fairly well, the format is too proprietary and too variable to be safe. You want the baseline documents to be in a format you can read at ANY time in the future, not just three weeks down the road.

Bull. PDF is completely open and is not going away. To get the specs you merely have to download them for free from Adobe's web site. There are multiple open-source implementations of PDF readers. Although Adobe is adding features all the time, the basic format that would be used for storing scanned images has been stable and forward-compatible for years and years. There are multiple court systems which have designated PDF as the format for filing, storing, and archiving court records. There is work on an official national standard for long-term archiving of records in PDF format. (PDF-A, specifies things like: the PDF must embed the fonts used, and so on, to ensure that it will be portable across OS's and decades.)

...the constant toying with DRM schemes...

A flaming example of a red herring. Your scanner software is not going to create a PDF with any DRM unless you tell it to. And some future version of your PDF reader is not going to suddenly refuse to read non-DRM'd files.

The "silver" alumin(i)um CDs are much less durable than the "gold" disks, but both will fail in the space of decades even if kept well.

Most "gold" CDs are merely "silver" CDs with a gold-colored label on the top. It's not even clear that the gold vs aluminum reflective layer is a real issue. But the dye type does matter, hugely.

OCR probably not the way to go

[Nutria]

OCR is no match for eyeballs. You'd spend so much time editing it for slight errors, it wouldn't be worth your time.

Are the notes graphics-heavy (i.e., scientific/engineering)?

If not, give it to a typing service. Once you show them how much "stuff" you have, I'm sure they'll give you a discount. They might even agree to use OpenOffice2 (because it handles huge documents well, the files are small, and it has an excellent PDF exporter).

You'd still have to scan in the pictures/drawing/graphs, and place them appropriately, which will take time.

Also, there are firms that specialize it digitizing paper documents (mostly forms and regularized documents for businesses). Depending on the amount of hand-writing & graphics, it might not be appropriate, though.

All in all, no matter how you do it, the project will

• take a long time
  
• cost a lot of money

imDex

[cstew]

Disclaimer: I used to work for this company as a coop student.

I would contact PRG Schultz as they have done this for large clients in the past. Hey have a program called imDex which is pretty slick. Basically, it's a searchable, cross-indexable database, so you'll have OCR'd text, along with TIFF's or PDF's of the documents. If you would like more information, let me know.

Re:What are you going to store them on?

[aminorex]

Not unless the notebooks in question were made of acid-free archival paper. I've seen cheap paper falling apart in 5 years, irrecoverable in 10. Phase-change media, like CD-RW, will easily outlast my children.

Re:What are you going to store them on?

[cfavader]

The matter of the fact is, documents on papers are not nearly as available as electronic copies. Hell, you could let thousands of people read all those documents at once for just a tiny amount of money in bandwidth costs (unless you have a university host it for free, which I'm sure they will). For most of us, this accessability is easily worth keeping a backup of the data, even if it also requires us to store it on new mediums as time goes on (i.e. switch from floppies to cdrs to dvdrs to whatever every 5-10 years).

hylafax

[np_bernstein]

go buy a modem, and grab an old fax machine, then fax the documents to yourself. You should be able to fax a decent number of pages at a time and can walk away and leave it running. these will be saved as multi-page tiffs which while not pdfs and searchable at least solve part of your problem.