Slashdot Mirror


How Google Is Solving Its Book Problem

Pickens writes "Alexis Madrigal writes in the Atlantic that Google's famous PageRank algorithm can't be deployed to search through the 15 million books that Google has already scanned because books don't link to each other in the way that webpages do. Instead Google's new book search algorithm called 'Rich Results' looks at word frequency, how closely your query matches the title of a book, web search frequency, recent book sales, the number of libraries that hold the title, how often an older book has been reprinted, and 100 other signals. 'There is less data about books than web pages, but there is more structure to it, and there's less spam to contend with,' writes Madrigal. Yet the focus on optimizing an experience from vast amounts of data remains. 'You want it to have the standard Google quality as much as possible,' says Matthew Gray, lead software engineer for Google Books. '[You want it to be] a merger of relevance and utility based on all these things.'"

16 of 58 comments (clear)

  1. Rainbows End by Toe,+The · · Score: 2, Funny

    But do they really have to shred all the books just to scan them?

    1. Re:Rainbows End by Samantha+Wright · · Score: 4, Informative

      Wait! I'm undoing all my mod points because I just realised that no, you're quite wrong. The printing process wouldn't be the same for the older books, and some of them have survived hundreds of years before we came along and scanned them.

      However, the story about books being cut up for scanning was about microfilm. I think it was an institution in Texas whose library was cutting them up mentioned as an aside in a submission about how they were converting their library into a lounge and computer lab.

      --
      Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
    2. Re:Rainbows End by delinear · · Score: 5, Insightful

      I would guess (read, hope) that while the process means books which are commonly available might be handled in the quick yet destructive manner, books which are more rare or have historic significance beyond the data would be treated much more carefully (at the lower end of the scale, someone with a hand scanner maybe, at the upper end perhaps even people manually transcribing). Ultimately, though, while I think it's a crime for a book to be destroyed, if it's a choice between it mouldering away in a basement somewhere until it falls apart or Google destroying it early in the interest of preserving the data, surely it's better that the ideas rather than the physical object are preserved (I appreciate in reality it's not just a black and white either/or choice).

    3. Re:Rainbows End by Merpy · · Score: 5, Informative

      Google doesn't destroy the books, they've got a patent on "unbending" the pages. http://news.cnet.com/8301-11386_3-10232931-76.html

    4. Re:Rainbows End by ElizabethGreene · · Score: 2, Informative

      But do they really have to shred all the books just to scan them?

      No. A book scanning machine is capable of scanning a book non-destructively. My unsubstantiated guess is that they are less harmful to the book than your average reader.

      You can build one if you'd like. Instructable The automated page turners on the commercial models are awesome. Youtube video

  2. Re:How does one write ... by MrHanky · · Score: 2, Funny

    With a pen.

  3. Why can't the text of these books be clearer? by bogaboga · · Score: 2, Interesting

    I have always wondered why the text in these books is not clear. The blurry fonts make my eyes hurt and surely, Google can create a better interface for the main page. Just 1 million dollars can do so much if some expert were hired to revamp the site. Come on Google!

    1. Re:Why can't the text of these books be clearer? by AdmiralXyz · · Score: 4, Informative

      It's because the book-scanning process is completely automated. I can't find a look to it, but a remember a Slashdot or Wired article about Google's automatic book-scanning machine. Basically it's too difficult to adjust for perfect focus for every book.

      I wouldn't worry about it though: Google is doing OCR on all these books, and they'll presumably replace the images with plain-text equivalents at some point (more searchable, portable, etc.) That's my hope, anyway.

      --
      Dislike the Electoral College? Lobby your state to join the National Popular Vote Interstate Compact.
    2. Re:Why can't the text of these books be clearer? by grumbel · · Score: 2, Insightful

      It's because the book-scanning process is completely automated.

      I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.

    3. Re:Why can't the text of these books be clearer? by ortholattice · · Score: 4, Interesting
      As someone studying certain specialized math books from the 1800's and early 1900's, I had great expectations for Google books, since they offer downloadable PDFs for public domain works. However, the focus quality of many (most?) of them is so incredibly poor that things like tiny subscripts are illegible blobs, making them essentially useless.

      While plain text solves this problem for novels, it is useless for math books, because OCR renders the equations (which are the essence of the book) as garbage characters. And it's not clear how one would communicate them as plain text anyway, unless the OCR was extremely sophisticated and generated say LaTeX output.

      Thankfully, some of the ones I need are in the University of Michigan Historical Mathematics Collection, with a much higher quality. But for the ones that are not there, I've used the Google pdf as a last resort - at least I can get an overview, if somewhat unpleasant to read. But for books I actually want to study, I've ended up making my own scan from a library copy (which, if done with care, is better quality than even the U Mich. version) when Google's is the only one I can find on-line.

      However, scanning physically stresses these old books. I think it is sad that I have to repeat what Google has done, when they (presumably) could have scanned them with high quality with a little more effort or better equipment with automatic focusing. In some cases, the books have been in the rare book section of the university library, which can't be checked out, and making copies of the whole book locally is frowned upon because of possible damage and sometimes, depending on the book's condition, not allowed.

    4. Re:Why can't the text of these books be clearer? by icebraining · · Score: 2, Informative

      One way they do it is through reCaptcha. When you're typing them, you're also helping the OCR process.

  4. Re:Scientific books by jank1887 · · Score: 2, Informative

    they already do that via Google Scholar. Scientific paper searches often (maybe not often enough) bring up textbook references. I know searching through regular Google does quite frequently.

  5. Re:Books Contribute to Global Warming by WillAdams · · Score: 4, Informative

    You're not taking into consideration the energy required to make the book, or to transport it to the marketplace. The amount of carbon sequestered in the physical pages of a book is insignificant in comparison.

    The production of a book releases 8.85 lbs. of CO_2:
    http://latimesblogs.latimes.com/emeraldcity/2008/06/paper-vs-paperl.html

    Here's a page which indicates most CO_2 production is for energy:
    http://www.eia.doe.gov/oiaf/1605/ggrpt/carbon.html

    And here's a page which indicates that CO_2 production is a much larger problem for the manufacturing of electronics:
    http://www.energybulletin.net/node/49730
    w/ a ratio of 12 to 1 for energy usage to weight, so my PRS-505 weighs roughly 9 ozs., so presumably required 108 ounces of fuel to manufacture (on-going energy usage is trivial and not considered)
    http://www.epa.gov/oms/climate/420f05001.htm
    gives us a figure of 19.4 pounds of CO_2 per gallon of gasoline which equals roughly 16.36875 pounds of CO_2 to make the ebook reader.

    So getting two books for the Sony should make it roughly break even, and each printed book beyond that which is not purchased should result in a net reduction of CO_2 emissions, since the energybulletin.net page indicates that the embodied energy usage for electronics is much greater than the lifetime usage.

    --
    Sphinx of black quartz, judge my vow.
  6. Re:How does one write ... by JustOK · · Score: 4, Funny

    there's a whole branch of science that studies writing and drawing while in an ocean, it's called oceanographics

    --
    rewriting history since 2109
  7. Re:VSM by delinear · · Score: 3, Insightful

    I suspect this is as much to do with the uptake in ebook readers as any change to the search indexing. Previously, if you were searching for this book you probably had a very specific interest in it and often wanted to buy a copy, now the people searching are more likely looking for free reading material, so the ranks have adjusted to accommodate that (since "people looking for free stuff" is a much wider market than "people with interest in a particular book", so it's easy to swing the ranking in favour of the former).

  8. " 'There is less data about books than web pages.. by unitron · · Score: 2, Informative

    Shouldn't that be "are fewer data"?

    --

    I see even classic Slashdot is now pretty much unusable on dial up anymore.