Slashdot Mirror

← Back to Stories (view on slashdot.org)

White House Website Limits Iraq-Related Crawling

Posted by simoniker on from the intriguingly-specific dept.

oscarcar writes "Dan Gillmor is reporting on the White House website's use of its robots.txt file to disable search engines from crawling certain material. Many excluded items in the robots.txt file involve mentions of Iraq, possibly to prevent people from finding changes to past statements and information when archived elsewhere."

7 of 837 comments (clear)

  1. Funny by sulli · · Score: 5, Funny

    whitehouse.com doesn't have that problem.

    --

    sulli
    RTFJ.
    1. Re:Funny by sulli · · Score: 5, Funny
      Also, TRUE God loving Americans would absoltely love to see that filthy, degrading website taken down because of the damage it causes to children who go there on accident.

      I feel the same way about whitehouse.gov. Couldn't have said it better myself.

      --

      sulli
      RTFJ.
  2. upside by 514x0r · · Score: 5, Funny

    it's good to see the whitehouse embracing technology so much.

    --

    !(^((ri)|(mp))aa$)
  3. Truly Frightening. by Dlugar · · Score: 5, Funny

    Obviously, they're keeping people from accessing the top-secret teeball Iraq files ! Besides:

    Disallow: /teeball/iraq/
    check out these other frightening examples of censorship:
    Disallow: /kids/spotty/iraq
    Disallow: /kids/eggroll/iraq
    Disallow: /kids/barney/iraq
    Disallow: /easter/iraq
    Disallow: /mrscheney/iraq
    Disallow: /national-anthem/iraq

    Truly frightening.
    --
    Computer Go: Writing Software to Play the Ancient Game of Go
  4. I, for one... by wardomon · · Score: 5, Funny

    welcome our White House Robot Overlords. It would be funnier if it weren't true.

    --

    - - - If the sun is a star, why can't I see it at night?
  5. Barney, agent provacateur of the CIA? You Decide by mykepredko · · Score: 5, Funny

    Downloading the "robot.txt" file and doing a quick ctrl-f on different words, I discovered that there are six instances of "Barney" coming up in the robot.txt:

    Disallow: /holiday/2002/barney/iraq
    Disallow: /holiday/2002/barney/text
    Disallow: /kids/barney/iraq
    Disallow: /kids/barney/text
    Disallow: /kids/photoessays/barney/iraq
    Disallow: /kids/photoessays/barney/text

    Which is the same number as "cheney", "powell" had 4, "saddam" didn't have any and "bush" only comes up with "bushpets".

    Clearly, there is something to do with Barney and Iraq that The White House doesn't want you to know about.

    myke

    --
    Mimetics Inc. Twitter
  6. Re:Other, arguably more reasonable explanations by greenhide · · Score: 5, Funny

    How can they be Iraq related if they didn't exsist to begin with?

    A question that GW gets asked all the time. :-)

    --
    Karma: Chevy Kavalierma.