Is Spidering Content from the Web Illegal?

Lysol asks: "I'm setting up a new site that I want to have spider some industry sites and grab their news. The problem is that no one has been able to tell me if this is illegal or not. I'd be fine with it, but I could see if a site was touchy about this and they were monitoring their logs and found a particular server on my site grabbing pages every few hours or so and then proceeding to sue me or something." As I understand it, it all depends on the fine print. Some sites have notices that allow you to freely copy the content of a web page as long as credit is given. Many commercial sites are much more restrictive.

Comments

Be very very careful

[copito]

#include "disclaimer.h"

www.freerepublic.com a conservative news discussion site is being sued by the LA Times and the Washington Post for copying news stories for discussion. Sort of like Slashdot without the links. The judge ruled that this was not free use. A final ruling in the case has not been reached.

Linking to content is likely to be much safer than copying it or framing it, although copying headlines might be safe.

Remember it is not whether someone can sue you that is the important question. Anyone can sue you. It is a question of whether you will piss them off enough they might sue you, and whether it is easy to get the lawsuit dismissed.


--