Slashdot Mirror
Software To Flatten a Photographed Book?
davidy writes "I have photographed some pages of a book for reading on my PDA. This is much faster than scanning and I don't have to carry the heavy books. However, the photographed books are not as nice: curved, skewed, and shadowed, as opposed to the much flatter, cleaner scanned books. I have searched for software that can flatten the pages for better reading on the PDA. So far I have come across Unpaper and Scan Tailor. Unpaper doesn't seem to have a windows GUI, and Scan Tailor doesn't unskew well. I remember reading about Google's technique of converting books to e-books with a camera and a laser overlay. Is there any home user software that can do a similar job without the need for a laser overlay or other sophisticated (and patented) technology?"
Snapter is a bit cumbersome but that's what it does.
.sig withheld by request
after photographing your book you have these huge image files that are barely readable, and now you want to spend MORE time trying to make em legible, wouldn`t it just be faster to scan em?, after OCR they would be much smaller and you could edit em/annotate to your hearts delight too.
seems like you made a problem looking for a solution rather than just scanning em in the first place
Get a thick, heavy piece of glass and lay it atop the pages to flatten them out before you photograph them. Use ambient light and avoid the flash.
Seriously, have you ever compared the time photographing a book vs. scanning it? The fastest scanners run like photocopiers. With a book, all you need is to set up a decent or ghetto rig for the camera and turn the pages. Until now, I've been shooting with a DSLR at the same lighting/camera settings for each shot, and applying a batch transform process followed by a universal levels setting, finishing up with a PDF assembly. But I'll report back on how Snapter works on the same files.
At version 0.9.6, perhaps Scan Tailor is 96% of what you want and it's F/OSS. If you *politely* contact the author(s) and lay out your concerns perhaps you can get what you need AND help make a project better. Worth a try.
These posts express my own personal views, not those of my employer
There are perfectly good machines that will do this for you.
They have suction systems to turn the pages of the book, and hold the book partially open so that the pages are more or less flat. There's one camera for each page, and the software that comes with the system deals with the curving, and obviously gets the lighting right to avoid shadowing etc.
OK, so maybe these machines aren't exactly cheap ... ... but at least one publisher is using them to photograph books (ones that are out of copyright, obviously) and put them into a print-on-demand programme. The deal is that if you suggest a book to the publisher, and they take you up on your suggestion, you get a free copy.
Cut the binding off and run it through a duplex scanner with a document feeder.
Not everyone has 5-10mm thick peices of book sized glass lying around and it can be hard to take that sort of thing about the place in case of requiring to photo a book.
There is software called Book restorer that does this removes curves 'geometrical correction' etc but it's pricy.
i've tried un paper and it's pretty decent for what it does but it does have some limitations and it's not the most convenient to use.
Deskewing, cropping, filling, etc etc are all easily done and I've even written imagemagick batch scripts in windows to do these things. The major trick is the curve removal.
There's various ways you can determine the curve from a scanned image. If you have the edge of the page, you can calculate the movement required to straighten that, and then apply it to the whole image. You can use text based curve removal, similar to well known deskew algorithms for text, but takes into account different parts of the text may be 'more' skewed. i.e. rather than a rotational deskew a 'sliced' deskew. This needs to be done from the top to the middle and the bottom to the middle.
If you have a good 'shape' of the page, and know the true size of the page, you can use a kind of morph operator to morph the corners back to th eright position and hope the image follows.
Using a Greyscale/colour source will work better than a black and white source image in general.
the other option is if the scanned / photoed page is actually of reasonaly good quality but if just a bit squint, then OCR it to a PDF and generate a new document using the OCR text, which will be pin sharp accurate, compress a lot better and be easier to use, although may not be ideal if there are too many errors.
Command line!, it looks like you revel in illegal activities...
"The likes of Facebook and WhatsApp are free to those whose privacy is of zero value."
I borrowed a simple 5 Mega Pixel camera from a friend 3 years ago and it had a built-in flat picture taking mode. It was one of the common brands probably a cybershot (though not really sure).
There's this nifty new program that just came out that's called Photography Shoppe. It lets you adjust the skew on images, slong with a whole bunch of other things. It's one to watch out for. It'll be really popular some day!
I scanned my books in so I did not have the page curve problem of davidy, but when I loaded the images up through iPhoto to my iPod they were compressed further and unreadable.
Anyone have any suggestions on what I did wrong?
Sorry for the question hijack, thanks!
I remember reading about Google's technique of converting books to e-books
My suggestion is that you look at some of the Google books that are on-line. I have, and they show the problems that you mention and more, curved pages, dark areas, and even text that is distorted and harder to read than most captchas. Whatever you have read (and yea, I remember reading it too), it doesn't seem to actually be viable in practice. Sure, photographs are easier than scanning, particularly if you do it fast and cheap, but the result is poorer. If you can scan the book without damaging them I suggest you go back and do that.
I'm an American. I love this country and the freedoms that we used to have.
What about Unpaper in a VM so that it has a GUI?
Learn to use the command line.
I'm surprised that Google doesn't do this, it would be SO much faster than scanning each page one at a time.
Another option, see if Amazon sells the book in digital format. Sometimes a few bucks saves a world of headache.
Now if these are expensive textbooks or reference books, or don't belong to you, the above may not apply, just my first thought on the subject.
Digital cameras from Casio (Exilim serie) have a dedicated to take picture of sheets of paper, whiteboards and visit cards. It detects the content boundaries, crops and unskew it. You could also save time (and money since time = money) by looking for and buying the electronic version of the book you want to read.
Sig (appended to the end of comments you post, 120 chars)
If you have ~$300 to drop on the project, Make has plans for a nice book scanner: http://blog.makezine.com/archive/2009/04/how-to_book_scanner_on_the_cheap.html It seems to hold the pages at an angle so there's little-to-no distortion on the page.
This comment is fully compliant with RFC 527.
By the time you've spend all this time digitizing, couldn't you have just read the book?
Take a look at decapod-project.org for a complete system. Note that software dewarping is quite a hard problem, but it is part of decapod.
You could consider Prizmodo. It includes OCR should you desire for $40.
http://hugin.sourceforge.net/tutorials/scans/en.shtml Here's a tutorial to stitch scans together the slight curve of the page is minimized where the scan joins. Might be what you are looking for.
This might be what you're after
You're welcome.
What is this PDA of which you speak?
What is even sadder is that in 2009, you HAVE to resort to the CLI for anything if you prefer not to. CLI is often faster, granted. I develop under linux every day and could not imagine not having a CLI, but if I prefer not to type to run a program dealing with images, I should not need to. I would expect typing for a GUI, but I should not have to type to resize and distort images. The fact that you complain about the windows jockeys who want the GUI is exactly the reason Linux is an OS for hobbyists, and not a mainstream desktop alternative.
I happen to like Linux, but GUI's are convenient -- particularly for images, as you said. Personally, I'd like it if a few more of the Windows configs were easily editable plain text. But you are also right, to *need* to use a text edit or a command line is different than choosing to do so.
Really, if you are doing this for yourself and have no intention of selling your product, then you are free to use their method all you want. Patent's protect the original inventor from companies who would profit from that invention. If you are not profiting (in ANY way except knowledge) then you are free to build your own system. There are several instances of this, for example the "balancing scooters" that resemble the Segway. There are even plans and software available on the internet for building your own. Bill
It's my Sig and you can't have it. Mine! All Mine!
Wouldn't OCR (http://en.wikipedia.org/wiki/Optical_character_recognition) be a much more practical option for this?
You could easily put it on your PDA, etc. and wouldn't have to worry about geometrically correct pages or reducing flashes.
Unpaper may work for you if you're not afraid to deal with a CLI.
There's no harm in giving it a look. Assuming it's properly designed I can see it being quite elegant.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Some guy posted a great instructables on building your own high speed book scanner, purposely designed to rapidly photograph book pages without curves. He even includes a software stream that OCRs the contents and sticks them into PDFs.
It's been quite popular -- so much so that he's created an online forum at http://www.diybookscanner.org/ dedicated to discussions from DIY book scanners all over the place, where they talk about builds, parts, and software.
I've been very tempted to build one myself just to avoid carrying heavy books around in my backpack.
John
Does that include the time needed to now fix the artefacts that scanning doesn't get you?
If you have a scanner, then why don't you just use that? And if you do not have a scanner, why even bother with the speed comparison and not settle for "I don't have a scanner"?
There's a reason that scanning takes time compared to just pointing a camera at a book and snapping a picture. You've now found one of those reasons. Congratulations.
Now you just have to find out if the up front time savings are greater than the post processing time costs, and since you're going to spend time reading through Slashdot and trying out the suggestions, I'm going to say "no, the savings aren't bigger". As someone else said, even Google's scanned/photographed books have issues, so don't be surprised when you do as well.
I want a solution to my problem that lets me be incredibly lazy, so I don't have to scan the books in (which i know will work VERY well if I just take the time to do it)...
I want software that will do it. For free.
Can I do it without a camera, too?
Actually I'd like it if there were some way I could get paid for using the software.
Can i just put my iPhone/PDA on the book and have it all sucked in via osmosis?
and then have the book read back to me w/ Morgan Freeman as the narrator?
Is there software that will turn the pages for me too? Oh wait - Morgan is going to narrate for me, that's right...
(sigh)
I know I'm probably getting modded Troll for this one; but there isn't always always an easy (magic!) software solution for every little thing. Sometimes you still have to put the work in if you want quality.
Politics will sooner or later make fools of everybody... - Dick Armey
Try a heavy piece of non-glare glass
i never built one of these, but if you have time, this might be your answer.
http://www.instructables.com/id/DIY-High-Speed-Book-Scanner-from-Trash-and-Cheap-C/
I went out and got one of the Planon Pen Scanners. Depending on what you want, they can go all the way to 600 dpi in color. Check out their web site. You can also get bargains on refurbished earlier models if you only need B&W and up to 200 dpi. http://www.planonstore.com/SearchResults.asp
Of course, neither his digital camera nor his PDA use any patented technology.
I don't know; for many purposes ImageMagick is perfect for my needs. Unlike the GIMP (as of 2.6.7), it supports 16-bit channels in images, which are useful for grayscale work on my side (256 grays is too limiting). It's also a lot faster for doing work over a lot of images in sequence.
Also, those scanners are VERY expensive. Using a camera would be MUCH better, if the problems can be solved. It doesn't matter how big the camera images are, since the ABBYY FineReader PDF-making software we use OCRs the image and makes searchable PDFs.
The Fujitsu fi-6230 Sheet-Fed and Flatbed Scanner gets good reviews and the flatbed scanner is fast, but it costs $1,200, and the sheet-fed and flatbed scanners are weirdly and unnecessarily connected.
Less expensive Fujitsu scanners lack TWAIN or ISIS driver support. Fujitsu uses proprietary drivers for the less expensive scanners, meaning that it can make them obsolete for some future operating system merely by not providing drivers.
The Fujitsu ScanSnap S1500 Instant PDF Sheet-Fed Scanner is excellent for what it does, we have one, but it doesn't do books, of course.
Not as fun as figuring out some massive kluge to do the job, but if it's a book that you can easily find used copies of just cut the binding off or remove the pages with a razor blade, and photograph them flat.
Some of the ebook-torrent scans use that method. It destroys the book but makes for cleanly readable scans.
Hugins certainly can do what's required. But OTOH, it can be painfully slow, so beware. A few seconds of attention when taking the pictures can save hours of struggle with any software later in the post-processing.
I would think you could correct for many of these issues by laying down several inches of foam, putting the book in place, and then pressing it down with a sheet of glass. Maybe with one more piece of foam to prop up the short side when you are at the beginning or end of the book. You know, kind of like what a scanner does...
Of course, you'll still have to deal with lens effects like trapezoidal issues, or skew, but these I imagine are much easier to deal with than curl.
Personally, I've switched to mostly reading Gutenberg books, there's a lot of good stuff there. I've finished Adventures of Sherlock Holmes, Around The World in 80 Days, and am working on A Study in Scarlet. They are books I can get in the format I want, at 3am when I'm looking for a new book to read before going to bed. I realize you may need books like textbooks that aren't available there, but if you have the option...
Sean
Load your raster files into OpenGL, apply them as a surface, then warp the surface until it appears flat to you. This is nothing more than what Google is doing automatically with lasers.
Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
For any given book it is not that hard to make a thin transparent sheet with lines on it. Lay out the lines in such a way that you can put the sheet on a page and the lines will not overlay the text. Since the sheet will conform to the page, you just need to have software that straightens out the lines and thus straightens out the text. The sheet needn't have a lot of lines, just enough to let the software work. It is easy enough to remove the lines in the final image.
The best resource I've found for book scanning is (use Google translate). In particular, the ScanKromsator software does pretty much everything, and does it well.
>Is there any home user software that can do a similar job without the
>need for a laser overlay or other sophisticated (and patented)
>technology?"
I generally try to help out instead of criticizing others, but...
Google almost always uses existing technoloy when it fits the bill. The fact they they spend the time/effort/money to develop their book scanning technology means they had to... so no, there is nothing acceptable on the market, much less for free.
I do the same but use the pages for references. Sometimes I up-load pages to EVERNOTE for searching on words, like indexing. THE ANSWER TO YOUR QUESTION IS THE LENS SHAPE. The lens is concave and gives you a sloping-up contrast on the edges. In a software program one will write a logarithm to flatten the lens. However, if you roll-up the page's sides before taking the picture it will work too.
do these programs have the "FIND" option? like on the MicroSoft programs??? that would be awesome! I really love "FIND" options on the programs;)
... even contracts are scanned and the original destroyed as legal has deemed that a PDF scan of a signed document is as legally binding and secure as the actual paper.
Wow, you have a dumb legal department.
It is "as legally binding" only if it can be used to coerce the other party into admitting that they signed the document. A wise but immoral signer could take the opportunity to say they signed something else and that you must have manipulated the file. There's nothing you can do to prove that you didn't. At best you can only show that it is unlikely that you did. I hope they keep the physical paper for any truly important contracts they have.
I won't join Slashcott. OTOH, If Beta goes live, I just won't be back until it's fixed. Sorry Dice.
No need. At a quick glance, Scan Tailor is programmed in QT3 (a superset of C++, used by KDE). This is a multi-platform environment, making it very easy to fix something on all supported platforms at once. If unskew doesn't work well, then that should be addressed in both versions. Fixing the Linux version will fix the Windows version too (unless he's relying on platform specific libraries in addition to QT).
I won't join Slashcott. OTOH, If Beta goes live, I just won't be back until it's fixed. Sorry Dice.
I don't know of any software that will do this as well as just biting the bullet and scanning it. If you have a scanner, it's a no-brainer. If you have access to an advanced copier, you can scan the pages to a PDF file. Used to do this all the time.
The link was very interesting, but gave no clue as to how you take the perfect image of a book with a crease down the middle (which is the original starting point of the question), and then get an image with no crease. They demonstrated a result with no crease, that was very convincing - but I didn't see just how they got rid of it. Simple cropping of each side?
"There is more worth loving than we have strength to love." - Brian Jay Stanley
For some cases, this might be suitable:
i) take the book into a small print shop and have them chop off the spine with a paper guillotine (about $2)
ii) scan the pages through a high-capacity feeder (or just scan individually at home)
iii) re-bind the book with a plastic coil (about $2)
I do steps i) and iii) with all my music books so they lay flat while I'm trying to read them.. Might do it with some textbooks for a course I'm taking next year, too.
I'm sorry but this question is just ridiculous, or if not actually ridiculous then wasteful.
Let me see if I understand this correctly: The author has photographed a selection of pages from a book that he wants to have available to read at his leisure. He doesn't want to carry the book around because it's too heavy, so what he wants to do, because the photographs aren't as pretty as scanned copies but scanned copies have artifacts of the scanning process, is to develop a home rig to scan the pages he's interested and run them through some piece of software to straighten them out before loading them onto a pda so that they're available whenever and wherever he wants them.
Right?
Here's my issue: This is a problem that's been solved by a library, a photocopier and a pocketful of nickels for 30 years - no matter how efficiently he manages to get his whole basement book scanning setup working (which, let's be honest here, is going to be a mess of trouble and a timesink of not insignificant proportions) the benefit (PDA-stored documents) can't possibly be outweighed by the hours he'll end up spending researching and implementing a system that can't compete with a manila folder with a stack of photocopied pages in it.
I admire the thought experiment, and I bet there're a whole bunch of nerds here who are going to get all in a tizzy about cameras and lighting and whatnot, but this, and I mean it kindly, is a waste of time.
And if you think photocopying is too time-consuming, suck it up and bring the book with you - there are worse fates than slinging a backpack over your shoulder.
http://www.creaceed.com/prizmo Prizmo straightens perspective, does lens distortion correction, and OCR. Book curvature straightening is a feature we'd like to have not too far away...
Google - and libraries - use copiers designed to allow you to photograph a book page without mangling or destroying the book, and with the ability to flatten a page so that the resulting copy is clean. These copiers aren't hugely expensive. If someone can build one on their own, as referred to in an earlier post, that's probably the best solution.
I haven't tried the software, but it's recommended by the guys at "make" zine who assembled a DIY book scanner:
http://diybookscanner.org/forum/viewtopic.php?f=3&t=34
Alejo
Are we talking books that are not available in eBook format? Wouldn't it be exceptionally easier to buy ebooks?
I imagine the time required to photograph/scan a book of any length, as well as searching for and tweaking a application to correct these images is significant. Say we're talking a PDA which doesn't support common ebook formats, in that case wouldn't it be easier and much better quality to convert ebooks into a compatible format (even if removing DRM was required). That, or drop $100 on a user eBook. Finally, if money is the concern, where time is not, there's always the download route. Again, unless we're talking books where someone hasn't already done all the work to scan to nice electronic format...
All that being said and done, best of luck. Sounds like an interesting project for it's own sake.
Photograph them with a rig from bkrpr.org. Plans are there to build one yourself on the cheap.
A bit expensive, but an accurate pdf (jpg, etc.) to text OCR. Urban legend has it, this is old Russian software developed by the KGB, and back in the Yeltsin era there was a free loss-leader demo making the rounds for Macintosh. I doubt that's still available, but even at $400 a pop, ABBYY FineReader is pretty good stuff, IMHO.
``Tension, apprehension & dissension have begun!'' - Duffy Wyg&, in Alfred Bester's _The Demolished Man_
Surprisingly enough, an '09 Chevy Malibu seems to be able to flatten things you wouldn't even dream they possibly could.
If any program or technology makes it too easy to copy parts or entire works of literature, and many people start to do this, the book industry will start organizations that work against this, and try to ban programs, issue campaigns to tell the public what is morally right and in the end, try to limit your (or copyright infringers) access to public libraries.
So you took pictures of a book instead of scanning and now you want software to flatten the images like a scanner would? /facepalm
My sig of choice is Marlboro
right click the network interface and choose "repair". It won't do all interfaces at once, but it combines release and renew into a single step.
Visit freecycle/freegle and arrange a free scanner instead of using the wrong software to fix problems caused by using the wrong hardware. Seriously. You'll be glad you did. In fact, I have a redundant scanner here, which I'll gladly mail to you at no more than shipping costs.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
Easiest way is just keep the page flat! For many cases, a thin piece of metal, plastic etc clipped behind the page being photoed would do it. Of course it's work, but if there are just a few pages, it's worth the effort. 'sides, it works and it's cheap! In case you aren't home where the metal is, just use another book's cover. thanks for lis'nin' seekertom
The best solution that I have found after long/painful research is to use ABBYY OCR software for OCR and conversions and then finish the file (if needed) with either Photoshop or your preferred Adobe application. ABBYY full version also includes the best screenshot reader I have ever used. The program can scan to ocr, an image file, a pdf, a word or exel document and a few others. If you captured the pages with a camera, you can flatten(optimize) the images in PS, Acrobat or InDesign if it is available for web which can take a huge file of images and compress them for web. This will create a very small file for your PDA. Keep in mind that if your objective is to create a tiny/flattened file for your PDA the pages will not be suitable for printing. For a PDA you can also reduce the files resolution quite a bit since a PDA usually only needs about 320x240 instead of 800x600 min. for a computer monitor. This is a huge tradeoff. A scan withh ABBYY can also correct the problem of page edges, dark spots etc because you choose the exact area to capture. I use a stoneage antique model Epson 2450 photo scanner to ABBYY or PS which works great for just about anything The best part about ABBYY software is that if you can see anything on your monitor, you can capture and convert it to anything you want. Program security cannot disable it.
The best solution that I have found after long/painful research is to use ABBYY OCR software for OCR and conversions and then finish the file (if needed) with either Photoshop or your preferred Adobe application. ABBYY full version also includes the best screenshot reader I have ever used. The program can scan to ocr, an image file, a pdf, a word or exel document and a few others. If you captured the pages with a camera, you can flatten(optimize) the images in PS, Acrobat or InDesign if it is available for web which can take a huge file of images and compress them for web. This will create a very small file for your PDA. Keep in mind that if your objective is to create a tiny/flattened file for your PDA the pages will not be suitable for printing. For a PDA you can also reduce the files resolution quite a bit since a PDA usually only needs about 320x240 instead of 800x600 min. for a computer monitor. This is a huge tradeoff. A scan withh ABBYY can also correct the problem of page edges, dark spots etc because you choose the exact area to capture. I use a stoneage antique model Epson 2450 photo scanner to ABBYY or PS which works great for just about anything The best part about ABBYY software is that if you can see anything on your monitor, you can capture and convert it to anything you want. Program security cannot disable