Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Problem of Search Engines and "Sekrit" Data

Posted by Hemos on from the how-to-choose-data dept.

Nos. writes: "CNet is reporting that not only Google but other search engines are finding password and credit card numbers while doing its indexing. An interesting quote from the article by Google: 'We define public as anything placed on the public Internet and not blocked to search engines in any way. The primary burden falls to the people who are incorrectly exposing this information. But at the same time, we're certainly aware of the problem, and our development team is exploring different solutions behind the scenes.'" As the article outlines, this has been a problem for a long time -- and with no easy solution in sight.

5 of 411 comments (clear)

  1. Re:Oh Yeah? by Karma+50 · · Score: 5, Funny

    Just search for your credit card number.

    By the way, does google have that realtime display of what people are searching for?

    --
    http://www.thehungersite.com
  2. I've got a solution! by CraigoFL · · Score: 5, Funny
    Every web server should have a file in their root directory called "secret.xml" or somesuch. This file could list all the publicly-accessible URLs that have all the "secret" data such as credit card numbers, root passwords, and private keys. Search engines could parse this file and then NOT include those URLs in their search results!

    Brilliant, huh? ;-)

    On second thought, maybe I shouldn't post this... some PHB might actually think it's a good idea.

  3. Oh, for regular expression searching in Google by EnglishTim · · Score: 5, Funny

    I could be a rich man...

    (Not, of course that I'd ever do anything like that...)

    Searching with regular expressions would be cool, though...

  4. Business Model by Alomex · · Score: 5, Funny

    A while back there was a thread here about the weakness of the revenue model for search engines. Maybe we have found the answer, think about all the revenue that Google could generate with this data!

    Anybody knows when Google is going public?

  5. Re:Stopping Google won't stop the problem... by mobiGeek · · Score: 5, Funny
    but Google undoubtedly uses techniques beyond that of the casual browser

    Uhh...no.

    HTTP is an extremely basic protocol. Google's bots simply do a series of GET requests.

    It would be possible that Google's bots have a database of username/passwords for given sites, but the more likely scenario is that they have stumbled across another way to get the "protected" information:

    • a link which contains a username and/or password
      /protected/show_article.pl?username=foo&passwo rd=bar&num=1
    • a link to the pages which by-passes the protection scheme
      /no_one_can_find_this_cause_Im_3l33t/article1.html
    • someone else posted the information elsewhere, and this is what is actually crawled

    I ran robots for nearly 2 years and was harassed by many a Webmuster who could prove that my robots had hacked their site. They'd show me protected or secret data. It typically took 3 to 5 minutes to find the problem...usually the muster was the problem themself.

    HERE'S A NOTE OF WARNING TO WEBMASTERS:
    Black text links on black backgrounds in really small fonts are NOT secure.

    Maybe I should get this posted to BugTraq...or would MS come after me??

    --

    ...Beware the IDEs of Microsoft...