Slashdot Mirror
White House Website Limits Iraq-Related Crawling
oscarcar writes "Dan Gillmor is reporting on the White House website's use of its robots.txt file to disable search engines from crawling certain material. Many excluded items in the robots.txt file involve mentions of Iraq, possibly to prevent people from finding changes to past statements and information when archived elsewhere."
Many excluded items in the robots.txt file involve mentions of Iraq, possibly to prevent people from finding changes to past statements and information when archived elsewhere."
Maybe, but I would think they might also be looking for "shady" spiders that ignored robots.txt. I wouldn't be surprised if there aren't a few honeypot pages in there too.
To ensure perfect aim, shoot first and call whatever you hit the target
Perhaps their goal is simply so that when people google or whatnot for information on the Bush Administration and Iraq, they will be likely to find the Bush Administration's current views on and actions in Iraq, rather than outdated material?
Completely ignoring for the moment the fact that these views and actions are really somewhat embarrasing for the Bush administration, this really makes sense from a practical viewpoint. Few things are as annoying as searching for something news-ish and finding primarily material from two years ago. And after all, if they ONLY were interested in people forgetting the old materials, they could have just removed those materials from the site totally. (Though perhaps they were aware removing the materials completely would cause mirrors, which would be fully searchable, to spring up.)
Irritable, left-wing and possibly humorous bumper stickers and t-shirts
While anything is possible in politics, is it possible that the web admin is trying to limit the amount of traffic on the site? Is it possible that his analysis of the weblogs show a lot of traffic from robots looking for Iraqi-related info?
If you persist in contemplating a world where whatever statements that the WH puts out, no matter how they might seem to contradict previous statements, are not totally true and correct, then a relocation expert from Guantanamo will be by in a few minutes. Just step away from the computer.
This gets modded up as Insightful? I mean, the White House is routinely editing their trascripts, and if bots like Google and Wayback can go and find that no, Bush said that we found weapons, not a weapons program, then there goes Bush's latest FUD... *thud*. Just because it's a tinfoil hat worthy theory doesn't mean it isn't true... most aren't, but therein lies the issue: most.
#define DRM chmod 000
Here's a minor example of something those two sites didn't catch: Remember Iraq's so-called "mobile biological weapons factories"? A month after the story broke that they were for weather balloons, the CIA moved their report's URL.
An intriguing fact about this whitehouse.gov/*/iraq thing is that they do in fact cover some of the important statements which are apparently not duplicated in the press release, conference, and briefing directories. Perhaps there was a "unique urgency" to cover up some poor choices of words?
true. true. Apparently some poor fool made similar remarks on k5 a while back, and did indeed receive a personal visit from the SS. No charges filed, but 'tis a rude awakening indeed when your online words come and knock on your door.
Where have you been living the past five years? Journalists don't criticize Bush.
They still have not published the fact that he deserted from the national guard during Vietnam and they practically ignored his DUI conviction.
The GOP has the media cowed with their constant 'liberal media' babble. There number of journalists who are prepared to hold Bush to account is tiny - Krugman, Conanston, Irvins, Alterman. After that its Al Franken, Jon Stewart and David Letterman.
it would create an incredible backlash as soon as detected. what purpose would this serve?
The chances that the mainstream media will pick this one up are very small. Just think how they would have reacted if it was Clinton!
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
The crux of the matter is that he refused to have his pilots medical just after the Pentagon added a check for illegal drug use.
You can try to spin this whichever way that Karl Rove tells you but the facts are against you. The fact is that your great leader is a coward who ducked the draft and then deserted to avoid a drug test.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
While I agree that by now there ought to have been more transparency by the US govt regarding the Guantanamo Bay detainees by now, if you had SHOT at U.S. soldiers in an engagement in Great Britain, you'd be an illegal combatant. This is pretty much why these people are being detained.
These people were probably by and large draftees, which unfortunately in Afghanistan, meant they weren't going to _get_ a uniform. They certainly have a right to public trial, but by and large they were probably arrested legitimately. I see this more as an indictment of the unfairness of the Geneva Conventions with regard to poor nations, or forces that aren't backed by recognized governments.
It would be a lot easier to classify this as "disgusting" if we knew just what was happening down there. Right now, we don't really know much of anything, which is disturbing on several levels. But isn't disgusting in the way I'd classify the very well documented types of supression that were commonplace under the government these combatants were fighting for.