Slashdot Mirror
When Does Website Monitoring Go Too Far?
"Though I believe they are a reputable company, they are doing some things I do not think are good: checking for the domain names on the TLD servers once per second, downloading various files from the site once per second, and sending email to themselves once per second.
Our first response was to talk to them and explain what we needed them to do, including a list of IPs that we used for customers so they could adjust their monitoring to suit what we thought was reasonable. They chose to ignore the first discussion and continued to abuse the servers. After the email server required a half-day of cleanup, the CTO simply shut them off at the firewalls. Rather than using the contact information they had, they chose to complain to our mutual customers instead. (I should note we do significant monitoring of the servers ourselves, and typically know if something is wrong within minutes of the event.)
Is this typical behavior of monitoring service companies? I know some of them are not reputable at all (due to spamming) however these guys seem to know what they are doing, and yet managed to effectively attack our mail and web servers, as well as doing some things I would not do to the TLD servers. It is hard to feel justified to shutting off someone else's cash-flow, but at the same time we need to defend servers from over zealous monitoring."
A server should not choke if the log partition is full. Is the log in a separate partition, isn't it?
Nagios.
http://www.nagios.org/
Cheers
Stor
"Yeah well there's a lot of stuff that should be, but isn't"
we typically set our monitor software to check every 5 minutes, with one request PER SERVER not per site. if it is down it will send an email to our support address, if it is STILL down the second time around, it fires off an email to the cell phone of the on-duty admin, plus one email when it comes back up
i've had some services set up for monitoring as low as 30 seconds, but those are specific cases.
obviously a 1 seconds check is WAY too low, not only it's a waste of bandwidth, it's prone to false positives. what happen when you have a slight delay in one of the core routers that cause your packet to get dropped/delayed by 1000ms ?
From a business perspective, monitoring is a service *you* should offer to your customers. Since it is your network, you have the ability to provide a much more effective and accurate monitoring service, and can set the resolution of the service according to your customers needs. All the problems you describe are because they are operating from the outside. What that monitoring service is effectively doing is stealing your bandwidth, and selling to your customers. If you want to get your lawyers involved, send them a C&D since they are affecting your ability to conduct business. personally I would firewall then as the CTO has done, and offer the same service internally.
First things first. These are your servers. Your network. I am assuming you have the standard abuse clause in your TOS. You need a lawyer.
Unfortunately, you are in a bad situation. They apparently have more resources than you, because they can bring your setup to it's knees. Not saying it's right, not saying it's fair.
A lookup of your TLDs each second makes sense if you are Yahoo! or Google. Their web monitoring levels don't appear to be reasonable. You already know the technical answer.
Personally, I would be worried about them stealing your customers. I mean the argument is going to be simple from their side. They will simply say, "hey look, their stuff folded under 'normal' monitoring, we have a hosting company we can 'recommend'" or they will just have the hosting company call them up out of the blue and ask if they are "unhappy" with thier current service..."oh, it goes down a lot"..."they can't handle simple monitoring"..."gee, that's a shame"..."well, we've worked with that monitoring company before, and we have never had any problems, in fact we routinely get 5 9s"...etc
Honestly, talk to legal, explain the potential situation, and have them make contact with the monitoring company. A couple of tortious interference this, and cease and desist that, will put the monitoring company on it's toes and maybe get them to leave your customers alone, or possible play nice with your servers. Notify your customers yourself and explain that they are being investigated by your legal team, etc.
No.
IANAL, but if you'll allow me to shoot from the hip for a bit, I'll take a shot at it...
1) Tortious interference with business relationships. The solicited the customers. They directly interfered with the business relationship by bringing the servers down by overzealous monitoring.
2) The outage was caused by the monitoring company. If just one customer leaves to another hosting company because of outages or what not, or if that customer lost business due to downtime. The damages are realizable.
No.
If they're letting their logs get huge before rotating them it would cause a problem every time the server tries to append data at the end of the file.
And they shouldn't be keeping the logs on the server anyway. It's static data that only they could need access to. It should be moved off site to a standard IDE harddrive for processing.
Statistical data should be created as the data comes in and not from the log files if they intend to let the customers have statistics for whatever.
As for my own site, I have Apache doing the combined log format and wrote custom software to process and analyze the data. Every month I move the log off the server and every 10 megs or so I rotate the logs and move the data into a second cumulative file that Apache doesn't work off of.
Ben
Work Safe Porn
one of such monitoring tools is nagios. it allowes for multiple users, with access limited to view information only on specific hosts/host groups. it's a pain to set up initially, but in the end it works quite nicely. www.nagios.org
--- d'oh