They were paying their subscription to Congress, but last month they forgot and now none of their congress-people will respond. Of course, had they bought a few individual senators, instead of leasing all of congress, they wouldn't be in this mess, as they would still have those. When you buy senators, you keep them for life. But, it takes a lot more money to fill an iPod, err Congress, when buying individually instead of just subscribing.
I learned my lobbying strategies from Napster.
Yeah, fuck Walmart. They are responsible for so much trouble in middle America. Now they are slowly approaching the big cities. Makes me sick, it does. I should defecate in random places in the store.
(Score: -1, Fucking Redundant)
Enough said, you just repeated your post, asshole. Dude, you just repeated the same thing you said in an earlier post. Thank you come again.
your friendly neighborhood troller general
Well, it seems if your website is getting excessive crawling, it would likely have more than two pages, else you wouldn't notice it that much. Therefore, crawl-delay could work. If you have say 100 pages (not excessive for an average website with dynamic content, like an image gallery), and it can only request every two minutes, that would take three and a third hours to go through the whole website. Unless they are spidering constantly, it seems like that should be about enough.
your friendly neighborhood troller general
You also might want to look into the Crawl-delay: tag for robots.txt. I decided to let MSN spider just my front page (no sub domains or picture galleries), but I put 'Crawl-delay: 120' in my robots.txt under msnbot to prevent them from coming to often. You might want to try that as well.
Slashdot Mirror
shit
They were paying their subscription to Congress, but last month they forgot and now none of their congress-people will respond. Of course, had they bought a few individual senators, instead of leasing all of congress, they wouldn't be in this mess, as they would still have those. When you buy senators, you keep them for life. But, it takes a lot more money to fill an iPod, err Congress, when buying individually instead of just subscribing. I learned my lobbying strategies from Napster.
Like a Mac user, don't drink the Cool-Aid. Oh Yeah!
I know you. I've seen you before. I've watched you get dressed and take a dog for a walk. I've seen you in your dorm room. Have fun in Canada.
Yeah, fuck Walmart. They are responsible for so much trouble in middle America. Now they are slowly approaching the big cities. Makes me sick, it does. I should defecate in random places in the store.
(Score: -1, Fucking Redundant) Enough said, you just repeated your post, asshole. Dude, you just repeated the same thing you said in an earlier post. Thank you come again. your friendly neighborhood troller general
Well, it seems if your website is getting excessive crawling, it would likely have more than two pages, else you wouldn't notice it that much. Therefore, crawl-delay could work. If you have say 100 pages (not excessive for an average website with dynamic content, like an image gallery), and it can only request every two minutes, that would take three and a third hours to go through the whole website. Unless they are spidering constantly, it seems like that should be about enough. your friendly neighborhood troller general
You also might want to look into the Crawl-delay: tag for robots.txt. I decided to let MSN spider just my front page (no sub domains or picture galleries), but I put 'Crawl-delay: 120' in my robots.txt under msnbot to prevent them from coming to often. You might want to try that as well.
You don't need folders, you douche.
apple is dying, netcraft confirms it