How Would You Archive Mounds of Genealogy Data?

dexter riley asks: "Hello, all. My mother, a librarian, historian and genealogist for over twenty years, died about a year ago. She left a huge amount of genealogy information, culled from books, magazines, and the internet, mostly in the form of typewritten, photocopied, and printed pages. My main goals are: Preservation - converting the documents into a compact format that can be easily copied and transferred to others; and Indexing - making it possible for someone else to easily find the documents referring to a particular person, family, place, or document type (like land, marriage, military, birth or death records). To this end, I would like to convert her work into a format that can be stored digitally and scanned for keywords, to make it easier for others to use this information for their genealogy projects later on. What tools do you recommend for handling a project of this size?" " I'd estimate there are at least 10,000 pages of documents in all. Much of it is organized by binder into family groups, but a lot of it is unorganized, loose paper. Besides being an irreplaceable resource for any future genealogists in my family, there are other researchers working on related lines that may find some part of this data useful. At the very least, I would like the satisfaction of keeping some part of her work from being lost for a few years more.

Here's a general list of things that I've determined I would need:

• Scanners: What flatbed scanners would you recommend for fast, high-resolution scanning of documents?
• Image formats: What lossless image formats would you scan your original documents into?
• OCR software: Although OCR is not perfect, would you recommend using it to allow keyword searching to the original document? If so, which software would you suggest?
• Document Indexing: In addition to OCR, are there other tools (document tags?) that you would use to help classify and organize images and other digital documents?
• File organization software: Ultimately, many thousands of text and image files will be generated. Since I don't want to just convert a paper mess into a digital mess, what tools would you use to organize related image and text files?

Did I miss anything in the above list? Any suggestions you all might have would be hugely welcomed."

Comments

That is SOME work...

[McSnarf]

1. Check the Document Management Continuum !
http://www.archivebuilders.com/whitepapers/index.h tml
2. Get two reasonable scanners that work with whatever software you choose. One with a document feeder (can be monochrome). Modern office MFPs work fine. The other one is a cheap flat bed scanner with color for anything the big one won't process.
3. Doc prep and Indexing will take much longer than the scanning - and unlike OCR, are a lot of manual labour. Expect a couple of weeks, minimum, especially if you have't got an indexing scheme in place.
4. Use TIFF G4 and PDF (OCRed text over the images).
5. Profit.

OCR software: ABBYY

[mindaktiviti]

ABBYY FineReader is great OCR software. However it's not free, and is for Windows only I believe.

Mormoms can help

[HowlinMad]

The Mormon faith believes in tracing humans back to Adam and Eve. They have a hug geneaology library in Salt Lake City. There are several programs available that you can pu the information in, and submit it to them. They will keep it forever, and other can research it as well.

Re:Mormoms can help

[HowlinMad]

You would be very surprised. My mother is big into this stuff. You can put in any kind of information about a person. They want it. When looking for ancestors, that kind of information is very handy, as it tells the story of where they where, went, etc. That help you to uncover where to look next. You can include it all, picture, land purchases, you name it.

Mormons?

[bill_mcgonigle]

Maybe it's best to consider outsourcing it. Groups like the Mormons do this kind of work as one of their missions.

This link may or may not be useful.

Re:Family history organization

[Anonymous Coward]

This is a good idea as long as you don't mind them baptising all your dead relatives.

Fort Wayne Library Genealogy

[anhyzer_mush]

The Fort Wayne, Indiana library has an amazing genealogy section. From the website:

The Fred J. Reynolds Historical Genealogy Department of the Allen County Public Library was organized in 1961 by the library director for whom it was named. The department's renowned collection contains more than 300,000 printed volumes and 314,000 items of microfilm and microfiche. This collection grows daily through department purchases and donations from appreciative genealogists and historians. Because of the collection's size and continuous growth, the information in the following holdings summary will necessarily be brief and representative in nature.

Perhaps they would be interested in her collection. At the very least, they may be willing to help you with your project. Here's the website:

http://www.acpl.lib.in.us/genealogy/

First things first

[GCP]

Others are suggesting answers to your scanning needs, and your mother may have already done this, but just in case:

The most important thing you can do with a pile of data is turn it into useful information. I know that's what you're trying to do. In the case of genealogical data, the most important is not just an electronic version of paper sources, but something built from those sources: an electronic family tree with facts (birth date/place, death date/place/cause, graduations, marriage date/place, immigration, military service, etc.) attached to each person and SOURCES attached to each fact, and all of it exported to GEDCOM format.

For each source, good software will let you include transcripts (not scans but transcribed excerpts that aren't too big) along with the bibliographic reference that tells others where they could find the information for themselves.

I probably have as much of this sort of data as you have, but most of it has been "processed" by extraction. That leaves the rest as backup that I seldom need to refer to. Since 99% of the information (by VALUE, not by bits) has been extracted into a very useful form, the rest is referred to so rarely that paper is just fine (as long as there is another copy stored elsewhere).

I'm not saying that having electronic copies of all of the original paper sources wouldn't be better. It would, but mostly for ease of backup. It's just that the most valuable thing to do (in my opinion) is the extraction and assembly of a really rich information structure (family tree, facts, sources, transcriptions of sections of sources) in a standard interchange format (GEDCOM).

After doing so, the additional value in having electronic, searchable copies of the original documents gets much smaller, so if I were you I'd make sure the information extraction project was done first before embarking on the scan/OCR/indexing project. After you do the former, you may decide that the latter isn't worth the effort.

GEDCOM

[D.A. Zollinger]

A lot of what you are looking for has already been figured out by others who do not want to be locked into a single format, vendor, or system. There are many geneology programs out there, such as Gramps or Brothers Keeper who use a standardized file format, GEDCOM, that allows information stored in those specific programs to be transferred to other programs. This allows for easy upgrades in software, as well as the possibility of moving from one package to another as the information can be archived to the GEDCOM file, and then read in again once the new software has been installed.

As for hardware and other software, I would suggest you use what is familiar to you, or is compatible to the software package you have chosen.