Slashdot Mirror


White House Website Limits Iraq-Related Crawling

oscarcar writes "Dan Gillmor is reporting on the White House website's use of its robots.txt file to disable search engines from crawling certain material. Many excluded items in the robots.txt file involve mentions of Iraq, possibly to prevent people from finding changes to past statements and information when archived elsewhere."

10 of 837 comments (clear)

  1. Oh please by shrikel · · Score: 0, Flamebait
    possibly to prevent people from finding changes to past statements and information when archived elsewhere

    Congratulations to simoniker, poster of the most inanely paranoid comment I have ever read here on slashdot. And that's saying something.

    --
    Any sufficiently simple magic can be passed off as mere advanced technology.
  2. shut up! by OwlofCreamCheese · · Score: 0, Flamebait

    oh dear lord, thats the most paranoid thing I have ever heard. lets turn off the internet now.

    --
    -You're wasting your time. Alfador only likes me.
  3. Are you surprised? by wrinkledshirt · · Score: 2, Flamebait

    If you're surprised by this, THAT's the news, not what the White House is doing with this information control. Click here for a list of the White House's policies with restricting FOI and other related requests since Sept 11th.

    This isn't partisan politics, either. The Republican party has been trying to keep Bush from violating the Presidential Records Act.

    Yes, yes, the country's at war. Makes you wonder why Bush doesn't want anybody to know about communications between Reagan and his advisors.

    --

    --------
    Bleah! Heh heh heh... BLEAH BLEAH!!! Ha ha ha ha...

  4. 1984 by spektr · · Score: 1, Flamebait

    The technology they introduced in 1968 allows them to rewrite the books and the newspapers. The next step is to make sure that the old copies get burned reliably. America was always in war with Iraq, you know...

  5. Shady? Nothing's private here by billstewart · · Score: 0, Flamebait
    shady spiders that ignored robots.txt?? This isn't somebody's private website that has data that non-privileged people shouldn't be looking at. This is the public record of the political statements made by OUR government, and they put things here for us to see. Unfortunately, I'd be surprised if they're not doing a lot of log analysis to find out who's reading what parts of the site, to look for political opponents of various sorts and other patterns that could be useful.

    It's appropriate to use robots.txt or similar mechanisms to discourage spiders from requesting data too fast and slashdotting the site, and while you'd hope that spammerbots would have the sense not to search the site, as long as they're rate-limited, that's ok too. There may be some honeypots to slow them down...

    (Stuff they don't want us to see goes on cia.gov or halliburton.com.)

    --

    Bill Stewart
    New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
  6. Re:Funny by Zeinfeld · · Score: 0, Flamebait
    Apparently some poor fool made similar remarks on k5 a while back, and did indeed receive a personal visit from the SS. No charges filed, but 'tis a rude awakening indeed when your online words come and knock on your door.

    Yes, but the comments in question were conditional on the day we become a dictatorship so the probability they will attract notice is much less.

    There are quite a few people being watched because they threatened the life of a current or past president. There is a federal law that requires the secret service to investigate every single one.

    Of course in the new Ashcroft USA where the bill of rights is now regarded as advice that the executive is free to ignore when inconvenient it probably isn't such a great idea to say such things.

    It is probably a better idea to work to prevent the election of Bush in 2004. Between Diebold, Fox News and Ashcroft it may be the last time we get the chance.

    --
    Looking for an Information Security student project suggestion?
    Try http://dotcrimeManifesto.com/
  7. Re:Funny by Zenjive · · Score: 0, Flamebait

    Yeah, sure, if you repeat it enough you might actually start to believe that. I pray that worms invade your intestines.

    --


    A vacuum is a hell of a lot better than some of the stuff that nature replaces it with. - Tennessee Williams
  8. Re:Furthermore... by Master+of+Transhuman · · Score: 0, Flamebait

    IANAP.

    Thank heaven.

    YANI. (You Are An Idiot)

    --
    Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
  9. Re: and your ... by placeclicker · · Score: 0, Flamebait

    Liberals, Republicans, Nazi's, and Libritarians agree, the media is out to get them.

    --

    Browse at -1, because trolls are often the most creative part of /.
  10. Re:country is not at war by Scudsucker · · Score: 0, Flamebait

    An honorable country would not pack 19 men onto airplanes to crash into civilian buildings, trapping the people inside to choose between a burning hell or a jump to certain death.

    The people in Guantanamo are from Afganistan (and probably now Iraq). The people who hijacked the planes were from Saudi Arabia. Iraq and Afganistan: invaded. Saudi Arabia: not invaded.

    Dumbass.

    Yes, I DO question your patriotism, because it seems like you sure as hell don't love what our country stands for.

    Uh huh. It also seems that you have no grip on reality, dumbass. But thats okay, this is a free country, and you are feel to free to wave a flag while sitting on your ass and call yourself a patriot, blindly believing whatever Karl Rove tells you.

    Dumbass.