Facebook Crawler Speaks Back

Last week we ran a story about Facebook suing to get a crawled dataset offline. This week we have a bit of a response written by Pete Warden, the guy who actually did the crawling. He followed robots.txt, and then Facebook's lawyers went after him. It's actually a quite interesting little tale and worth your time.