Britain's Conservatives Scrub Speeches from the Internet

An anonymous reader writes news of an attempt to erase a bit of history. From the article: "The Conservative Party have attempted to delete all their speeches and press releases online from the past 10 years, including one in which David Cameron promises to use the Internet to make politicians 'more accountable'. The Tory party have deleted the backlog of speeches from the main website and the Internet Archive — which aims to make a permanent record of websites and their content — between 2000 and May 2010."

Comments

Archive.org should not respect robots.txt

[Anonymous Coward]

People have used robots.txt to buy up domains they want to censor.

For example, this happened with partyvan.

Re:Archive.org should not respect robots.txt

[morgauxo]

The problem is that people are buying up the domain names of old websites which no longer exist just to publish a robots.txt file. Then archive.org automatically deletes, or at least blocks access to the entire history of everything that ever happened at that domain including the past website which the new owner has nothing to do with.

I suppose they are just trying to honor site owner's wishes even when they may have initially forgotten about robots.txt and added it later. The robot doesn't know that the old content belonged to someone else who DID NOT wish to block it. Maybe a good solution is that when they notice a new robots.txt everything for the last 'X' months get deleted. (go ahead and debate values of X) Data from prior to that should be left alone. Even if it was posted by the same site owner who is posting the robots.txt today. Tough cookies! If you want to control how your data is used I don't see a problem with requiring you actually take the time to learn about things like robots.txt before you publish. It's really no different than releasing source code under the GPL and then later turning it into a closed source product. All your new work belongs to you but you don't get to force everyone to delete ever copy they might have of the old code and you can't stop them from forking it.

-- I would totally consider an 'X value' of zero as being on the table btw

History will be lost

[Anonymous Coward]

There's a theory out there that states that because most of what we do in the so-called Information Age is stored is somewhat fragile digital storage systems (as opposed to, for example, parchment) historians in the future will have very little to base their research on about our age, as most of the info will be permanently lost.
Well, hundreds of thousands of posts on BBS systems from the 80's and 90's are already gone, delete the Internet Archive and the Web is gone too, any thoughts?

Re:Doesn't that kinda defeat the point of the arch

[pixelpusher220]

couple that with the google cached copy of the site has a 'search for speeches' section which now is, interestingly enough, missing as well.