Is Spidering Content from the Web Illegal?

Lysol asks: "I'm setting up a new site that I want to have spider some industry sites and grab their news. The problem is that no one has been able to tell me if this is illegal or not. I'd be fine with it, but I could see if a site was touchy about this and they were monitoring their logs and found a particular server on my site grabbing pages every few hours or so and then proceeding to sue me or something." As I understand it, it all depends on the fine print. Some sites have notices that allow you to freely copy the content of a web page as long as credit is given. Many commercial sites are much more restrictive.