Slashdot Mirror

← Back to Stories (view on slashdot.org)

White House Website Limits Iraq-Related Crawling

Posted by simoniker on from the intriguingly-specific dept.

oscarcar writes "Dan Gillmor is reporting on the White House website's use of its robots.txt file to disable search engines from crawling certain material. Many excluded items in the robots.txt file involve mentions of Iraq, possibly to prevent people from finding changes to past statements and information when archived elsewhere."

1 of 837 comments (clear)

  1. Re:Funny by CGP314 · · Score: 0, Redundant

    Don't forget Whitehouse.org, the real whitehouse webpage.