Slashdot Mirror
Digitizing and Geocoding Old Maps?
alobar72 writes "I have quite a few old maps (several hundreds; 100+ years old, some are already damaged – so time is not on my side). What I want to do is to digitize them and to apply geo-coordinates to them so I can use them as overlays for openstreetmap data or such. Obviously I cannot put those maps onto my €80 scanner and go. Some of them are really large (1.5m x 1.5m roughly, I believe) and they need to be treated with great care because the paper is partly damaged. So firstly I need a method or service provider that can do the digitizing without damaging them. Secondly I need a hint what the best method is to apply geo coordinates to those maps then. The maps are old and landscape and places have changed, it maybe difficult to identify exact spots. So: are there any experiences or tips I could use?"
Why not use a handheld scanner and some stitching software?
I suggest you contact the restoration experts in major museums for (1) advice about preservation, and (2) how they go about their own digitizing projects. I read a fascinating article about the digitization of many medieval parchments, but I don't recall the particular museum involved now.
[
I have a map of the U.S. - its actual size. The legend says "1 mile = 1 mile".
People ask me where I live and I say, "E4".
It must have been something you assimilated. . . .
If so, give USGS a call. They may well be interested in helping you with this and obtaining data from the maps. I can't say for sure, of course, but this is the sort of thing they do. When it comes to map data for the US, they are the go to guys. Call them up, tell them what you've got and what you want to do, see if they can put you in touch with someone in their agency who'd be interested in helping.
I'd suggest appealing to Google or the brothers that did tapestries for the Met. What are these maps of? Is there a society for the place that they cover where you could appeal for funds under the pretense that you publicly release the maps?
Assuming all those avenues are exhausted, let's look at some cheap and dirty DIY methods. I'm assuming you've got a MP digital camera. There are sub $100 ten megapixel cameras out there but don't get anything with a fancy digital zoom. Next you'll need mosaicking software or if you're into software, you can try your own implementation of the KLT algorithm.
First off, practice all of this on layed out newspapers while developing your preferred methodology.
Your cheapest and most haphazard option is going to be lay the maps flat on the floor and cut a length of string with a washer on it (two to three feet?). Try to use brightly diffused lighting so that is normalized in the mosaics with no shots of your shadow over the maps. Now this is backbreaking but hold the camera flat over the map with the string extended in front of it so you can keep the distance to the map consistent. Don't angle the camer as this will slightly distort that tile and hinder the mosaicking. Put plastic bags on your feet if you need to walk on the maps. Take a picture, move a few feet in a grid style, take another picture. Rinse, wash, repeat until you have images covering all of the map. Collect the images and put them on the computer and verify the mosiacking works before preparing the map for storage forever.
A better method would be similar but to construct a large wooden rectangular box with plexiglass as a top so that you can fit this structure over the largest of the maps. Then cut holes in the plexiglass so that you can set your camera at a plane level to the surface of the map into the plexiglass. You might want to put an adapter on your camer that allows the lens and flash to be free of obstruction. You could make the tiles more uniform and save your back some work but you need to build and buy the materials for the structure. I think this is more time consuming but your best bet and will allow you to gather more images with less distortion.
Above all, remember to save the original images! It's probable that later better algorithms will be developed to normalize the images, remove distortions, light problems, shadows and increase clarity on your overlapping sections. If you do the plexiglass route, you could manufacture it so that every bit of the map is photographed three or four times.
Not professional, not flawless but cheap and dirty. Hope this helps.
As for the geocoding, what are the maps of? You should actually check out the feature extraction of the KLT algorithm and consider using that methodology for syncing these up with maps. That will require human intervention though to identify the features, I'm sure.
My work here is dung.
I actually have a strange fascination with old maps myself, and regularly crawl the web for all kinds of antique maps. One overwhelming commonality I have noticed is that even recent maps can often be wildly wrong. So for example, an 1600ish map of Europe will be so wildly inaccurate that you would only be able to pick one point on that map to apply geolocation specific coordinates, the rest would not match up. So, I know I didn't answer your question, but I just think that unless they are accurate maps, it would be a very hard challenge.
"It's ok, I'm completely secure as long as my iron is off"
You might want to find the local university cartography or geography department. They will probably already have a method of doing this, or at least could point you to someone who does. Here's an example: http://mapmaker.rutgers.edu/ and their historical maps: http://mapmaker.rutgers.edu/MAPS.html
-molo
Using your sig line to advertise for friends is lame.
Really, history majors will love this stuff. Giving them maps and a concept of Google maps overlays for real time comparisons to modern maps will likely be a capstone project for some undergrad.
A few years ago while working for the State of Wisconsin's Board of Commissioners of Public Lands we worked with the University of Wisconsin: Madison to get all of the original land plat maps of the state digitized, indexed and search-able. Same type of deal, huge maps on really old paper that had to be vault kept with humidity and temperature controls.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Focus on the preservation of the imagery first, obviously, because once that's gone it's gone forever.
The cheapest option is a large-megapixel digicam known for good image quality. SLR would probably be a good bet. You can take multiple images and stitch them together without too much trouble, so you can get reasonably fine detail with a little work even with a $200 consumer camera. Or, alternatively, hire a professional photographer and have him/her take really high resolution photos of the maps. The advantage of this approach is that you don't have to take the maps anywhere or do anything special with them. Just lay them out on a low table or the floor and align a camera over them, and take heavily-overlapping shots.
Large-format scanners might cost some serious coin even to use for a one-time project like this, but would probably yield better results with less effort.
You might check with local companies that deal in maps and cartography, they might be able to recommend ways of saving the imagery, and some might even offer to help out if the maps may be of commercial interest (they might even share the proceeds with you in addition to giving you high-res digital images).
But I'd say if the maps are truly delicate, your first focus should be to take the highest-resolution images you can of them now, even if it's multiple images per map that need to be stitched. That way, you have *something* preserved in case one or more of the maps is destroyed or deteriorates further before you can preserve it.
If there are particularly interesting features of the map, use the MACRO feature on your camera - most stitching programs can integrate images at different scales and preserve a lot of detail. I used the "Hugin" pano toolkit (free) to stitch together about 100 random photos I took at the top of the Eiffel Tower into an impressive contiguous 360 panoramic shot, and it was literally a "here are the pictures, figure it out" process. The pictures were all taken at different zoom levels, different angles, and all sorts of issues, yet it looked like a Google Street View 360 image. This was 5 years ago, I can't imagine how much better the technology is today.
The geolocation shouldn't be all that hard - it's a matter of choosing a few points on the map and identifying their coordinates accurately. Of course, if there are few/no reference points it gets a lot harder. http://www.openstreetmap.org/ is a good starting point to a group that does free, open-source mapping. They or some of their related sites might possibly have a tool that does what you want. Also, a professional cartographer may be able to help you out as well.
"This post contains words, known to the State of California to cause thought. Wash brain thoroughly after reading."
There has to be somewhere that does this.
Where I work we have a medium format scan back we use (it is essentially a 100 mega-pixel camera that scans across over about 15 minutes to get an image).
We are in Wilmington, DE, which is a fairly small city. I know there is a similar device in DC, which is not too far. We went to the Library of congress and actually scanned maps there as a test before purchasing it.
It actually allows for some interesting capture as one can adjust the lighting in different ways to get details a flatbed may not (we replaced a 2'x3' flatbed with it).
It could probably do the 1.5m one in two scans, and most smaller ones in one scan.
Look around your area for a place that offers art-reproduction for the service.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
Been there, fought lighting and camera distortion for hours, only to get bad quality (relative to a scanner)
Lay the maps out on a uniform surface, take the lid off a nice scanner and turn it upside down and move it place to place. Use rather big (1-2inch) overlaps, because the edges of the scanner sometimes are incorrect. You can make a batch process to crop the edges off in photoshop / gimp.
Most important is to lock down scanner settings so nothing is auto, or you will have colorcontrastluminosity differences between sections of your map.
Stitching these together requires 0 effort in any modern photographic editing software.
This is cheap, gives the best results and is the only way to get good quality without spending a fortune or damaging the documents.
Another possibility would be a really high resolution digital camera. My wife (historical linguist) has dealt some with manuscripts, and that was their method of digitizing them for further study. OTOH, she's not a museum curator or archivist; they probably have even better methods. If you want to do it right, talk to a curator or archivist of some sort. They deal with much more fragile and much more valuable documents on a regular basis.
I don't have any good ideas to contribute about the geocoding, unfortunately.
SIGSEGV caught, terminating
wait... not that kind of sig.
First, ask on the OpenStreetMap mailing lists. There's lots of us who've done this kind of stuff before, and we'd be really pleased to help. I collected, scanned and rectified the Ordnance Survey's New Popular Edition - a complete set of England and Wales maps from the '50s, now out of copyright. It's all available in OpenStreetMap as a background layer and loads of people use it for adding rural roads, rivers, placenamese etc. Others are scanning other old Ordnance Survey series right now. Seriously, we love this kind of stuff. (#osm on OFTC can help too.)
Secondly, GDAL is definitely your friend. It's the most amazing set of command-line tools for rectifying and reprojecting data. gdalwarp and gdal_translate are probably the two you'll use most.
If you are archiving things with a 15 mp slr you are missing ALOT of detail that the prints have. I agree that prints stuck to glass are a challenge, but taking a picture with a current slr is a last ditch option.
Take photographs of them and put them on the internet. Google will automagically index them and add them to their street maps in real time.
Like the beaver, it's just Dam one thing after another
Hi there, I am a spatial guy so thought my 0.02 may be worth something. I am not too sure about digitising them, maybe a print shop or as suggested in other posts you could talk to your local university geography department or a government mapping agency
Once they are digital though you need to georeference them. As mentioned in the title of my post, it is easiest to use GIS to do this and you can use QGIS with relative ease. Install it using osgeo4w on windows or the ubuntu ppa for qgis. Alternatively if you have a license then use ArcGIS. If you have a map of the underlying roads for the maps you are digitising then what you do is find points on the roads and match them to points on the scanned images, this provides data for a transformation and will shift the map onto your coordinates.
like phosphorescent desert buttons singing one familiar song
I spent a summer doing this in grad school for the Vegetation Type Mapper project at UC Berkeley. I'm not going to lie to you--it was a ton of work. But the results were cool. The site has all the old maps georeferenced, plus ways to download them.
Needless to say, the library was involved in the project, as was a giant scanner. We relied on ERDAS Imagine software to georeference the old maps to current USGS base maps. There was also a lot of accuracy assessment involved to make sure we minimized error in the georeferencing process. Probably one of the trickiest parts was making sure the old landmark you were using as a control point had not substantially changed in the intervening decades.
My professor and her colleagues published a paper detailing the project.
Per Square Mile, a blog about density
Others seem to be describing some good solutions to getting the map scanned, so here's how to geocode and rectify the image using the open source Grass GIS software:
Step 0:
- You need to have a location already created in grass, with some contemporary data in it (physical features, roads etc where there's some concurrence with the map you're trying to geocode). The projection you've used doesn't matter much - a later step is going to be rectifying (ie distorting) the scanned map to match the projection of the digital map. The created location does need be at least as large as the scanned map (ie if the map is everything in a 5 km radius of some town, the grass location also needs to encompass at least a 5 km radius of the same town).
Step 1:
- Come up with a list of features/points which exist on both maps. Depending on the scale of the map, this could be intersections of specific roads, locations of towns, peaks of mountains etc. You're going to need an absolute minimum of five points for the rectification process to have any chance of working; more than fifteen is much better. Try and select points which are unlikely to have moved over time (coastline or river features for example). In grass, mouse over each point and record the coordinates.
Step 2: import the scan
In grass, do: r.in.gdal input=[path to scanned file] out=[Mapname] location=templocation
Quit grass
Step 3: target, point, rectify
Open grass, but this time in the 'templocation' you created in step 2
i.target group=groupMapname location=[modern map location name] mapset=PERMANENT
i.group group=groupMapname in=Mapname
d.mon start=x0
i.points groupMapname
d.mon will open a window; i.points will display the scan in it. Select the mapname in the dialog that appears, then one by one select each of the points you've identified as having concurrence with the modern map. In the terminal window, enter the coordinates for the point taken from the list you created in step 1. When done marking points, click 'quit'.
i.rectify -a group=groupMapname extension=_1 order=1
Depending on the size of your map and your processor speed, this bit may take a while. When done, quit grass.
Step 4: admire output
Open grass in the modern location. The scanned map will be available as a raster layer for display. The scan will have been rectified so the map matches the projection of the modern map layers - ie you'll be able to see what's moved and changed, and what exists now that didn't then etc. There's other grass commands which will help you convert features of interest (rivers, roads, contour lines, whatever) into vectors if you really want.
If all this seems too hard, have a look at qgis - also open source mapping software; it's more gui-oriented and I know it has a georectifying plugin. I've just never used it.
Good luck.
Saw a great presentation at OSCON a few years back about the massive digitzation effort undertaken by David Rumsey (http://en.wikipedia.org/wiki/David_Rumsey): See http://conferences.oreillynet.com/cs/where2005/view/e_spkr/1867 -- Drew a well-deserved standing ovation.
In the course of the talk, I think he said that he'd scanned the first 10,000 maps (though even 1,000 sounds ridiculous -- maybe it was 1,000) before hiring assistance.
Of course, he had more money to play with, so he probably had a pretty big scanner ;)
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
Contact a local Licensed Land Surveyor. We are in the business of coordinating maps and making sure they are properly referenced. We also know the difference between NAD83 and NGVD29. This and the other coordinate system conversions and the proper use of scale factor in SPCS (State Plane Coordinate Systems) is something we do every day. Plus, most of us are really into local history and could possibly show you some other really neat uses for that data. Historic societies are always looking for ways to map past events. When speaking with a Surveyor, we can usually know what the practice for a given time period was. There are three different lengths for a foot that I have come across working. International Foot (not used in surveying, but sometimes engineers use the wrong foot), US Survey Foot (standard) and the Philly Foot. Philadelphia has a different set of standards for how a long a foot is, depending on what part of the city you are in and what you are trying to do. This is not something most historians would accurately pick up. Surveyors will. We also know who was the good and not so studious Surveyors in the area and what tricks each used to mark corners, turning points and reference markers. A local Surveyor in the area the map is of would be very interested in helping you with your work. He/she may have already done the heavy lifting for you. We have to trace maps back as far as possible, so sometimes (I am in New Jersey) we have to go all the way back to the Proprietors to get maps so that we can run lines that control our current work.
Long story short, if it deals with cartography or local surveying, seek a professional Surveyor.
One Token Ring to Rule them All, One Search Engine to Find Them, One WAN to bring them in, and TCP/IP Bind them...
I have done this for a grant-funded historical map digitization project at a university library. We used a $40k large-format scanner (from Betterlight) which can scan the whole item laid out flat. Trying to stitch together camera images will result in distortion across the image—if you didn't need to distort it, you wouldn't need special software to do it; you could just line the pictures up.
But even once you have image files, there's about zero chance you can just replace Google Maps' tiles with your own and expect geotagged stuff to line up where it should. If you have a finite number of places of interest, you could manually locate them on each map and then try to distort each map to align, but if you expect arbitrary geolocations to need to be right, give up. Non-satellite/GPS-based maps are examples of practical cartography, not theoretical. They will be even less perfect than you think, no matter how professional they appear. Or do what we did: keep the geotag display on Google's maps, but show your historical map of the same general region side-by-side and allow the user to calculate the precise correlation in his own brain.
For best results generally you would use a SLR with Macro lens. This type of lens generally provides the flattest field at reasonable cost even when it is not used in macro mode.
Then you've got architectural lenses, but those cost an arm and a leg and a foot - but then again, renting is always an option...
Just a pedantic little thing -- as a geomorphology instructor, I can tell you that rivers and coastlines are very, very likely to have changed. Check out pretty much any river mouth in Victoria, Australia, or any island off Maine, US in google earth vs google maps satellite mode for examples of how much they can change inside of just a few years. If something catastrophic has happened (big storm, big earthquake...), huge changes can shift the coastline inside of hours.
If you're going to use geomorphic features for your geocoding, find out what's most stable in your region (keyword search academic journals for geomorphology + your location + change and see what doesn't pop up, or ask a local university geo prof). Vegetated topography can be pretty stable over decades, especially if you only need relative shapes.
I can't speak on how to get them digitalized, but once you do, look to the web to get others to help geocode and get them into shape for overlays. Put them online in a liberal CC license and invite other people to use them. Given the popularity of google maps and the community that's grown up making mashups and apps, I'd be willing to bet there already existing communities of people good at, and interested in, doing this.
If I had this project, I would start with digital SLR with telephoto macro lens.
Lay the map on the floor and use a mat frame that has some weight to hold the section to be photographed down.
Weight should be such that it holds the map down but doesn't press so hard that it damages the map. The idea of the frame is that it delimitates that area that you are photographing so that you have a reference to the next spot to photograph (should overlap a little). In addition, you have a reference for squareness of the picture should the camera not be perfect parallel to the map - in which case you can use a program such as Photoshop to correct the perspective (at least the versions that I have used it is in the "crop" tool - there is an option for perspective)
no macro lens.
What the OP needs is called a large format copy stand.
You want a lens that is "normal" to your film size, thus 50mm on 35mm film as an example.
for an APSc size sensor in a digital camera I think the normal lens would be ~30mm.
I don't know if you want color information or not but honestly either way I would shoot film, then scan the result.
For B&W, shoot Technical Pan film as an ISO of 6, develop in technidol developer (enlist advanced photo class people at a UNI to help with this). For color use Fuji Velvia at 50.
The photo class people will have the copy stands and appropriate cover glass / filters to get the contrast you want as well.
While I have not done maps, I have done large hand drawn artwork this way and the result is vastly better than you would get from directly shooting on a digital (IMHO).
Cheers,
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
We deal with old data with weird and sometimes unidentified projections fairly often. You will need at least some landmarks, though. The larger the scale, the more you will need. The more complex the projection, the more landmarks you'll need. With really old maps that are hand drawn and don't match a distance scale (pre-17th century) you're probably out of luck even if it's a local street map.
Astronomical software to deal with converting between projections is typically open source, but the learning curve is steep. Don't even think about using commercial image editing software. Even if you think it might be doing things right, you'll never be sure unless you're writing your own plugins.
Support SETI@home
Stitching these together requires 0 effort in any modern photographic editing software.
I actually wrote a little piece of software to automate the stitching process. Just feed the script a bunch of scanned images and it'll align and stitch the images for you. Never got around to make a proper OSS project for it or advertise it, though.
It's here: http://mairas.net/wiki/Mapstitch
No, a tilt-shift would be the lens to go for. Architectural photography favours this type of lens because you can adjust it to match the plane of the subject, and it would be pretty easy to do this for the very easily identified image plane of the map under glass.
Distortion-free in one easy step.
More of a worry, really is getting all sections of each image evenly lit without reflections. Not as easy as you'd first assume.
http://slurl.com/secondlife/Rumsey%20Maps%201/133/247/56/?title=David%20Rumsey%20Maps&msg=David%20Rumsey%20map%20collection
See also http://www.davidrumsey.com/
There's no sense in being precise when you don't even know what you're talking about. -- John von Neumann
His point is that it won't - It's a discussion that regularly comes up on photography forums. Color accuracy is generally considered better (with the right film) and I've heard resolution figures of 30 or 40 MP for 35mm film. This does assume you scan the negatives properly with a wet drum scanner, an expensive and complex piece of equipment.
I shoot digital myself, but I'm planning to get a hold of a film body to do some comparisons. I don't have a scanner, nor can I develop, so then it becomes a question of how good is my local photo shop.
http://www.cs.ubc.ca/~heidrich/Papers/EG.04.pdf
"Design of an Inexpensive Very High Resolution Scan Camera System".
500 megapixels for $1600 versus scanning-back cameras that cost tens of thousands. Make the error correction a little bit simpler by rotating the scanning back ninety degrees for four 4-color shots (16 exposures), overlaying them, and taking the median pixel values, at the expense of some resolution.
There are four considerable challenges here, though, not just one, and this project could stall on any of them:
Image capture
Projection determination & georeferencing
Digitizing features & establishing topology
Geocoding
People in Soviet Russia, however, appear to be afflicted with amusing juxtapositions of the aforementioned situation