Slashdot Mirror
Should You Break TOS Because Work Asks You?
An anonymous reader writes "My boss recently assigned me a project that was all his idea, with two basic flaws that would require me to break multiple web sites' Terms of Service (TOS). Part requires scraping most of the site, parsing the data and presenting it as our own without human intervention. While we're safe on copyright issues, clearly scraping like this is normally not allowed. At times it might also put a load on those sites. The other is, for lack of better words, a 'load balancing' part that requires using multiple free accounts instead of purchasing space and CPU time for less than $2,000 USD per month. The boss sees it as 'distributed' computing when in reality it's 'parasitic.'
My question is: am I wrong about the ethics? If I do need to walk, how best can I handle it without damaging my reputation and future employment opportunities?"
...you build a system that closely relies on this nonstandard (and unsupported) method of getting information, they change it and it breaks.
Either by accident, or because they spot a load of particular access patterns from your address, figure out what's going on and intentionally break it.
I work in health care, so maybe it's different in your industry, but every hospital I've worked for has had a compliance officer with an anonymous 800-number for compliance questions. This is DEFINITELY the kind of stuff they want to know about.
So the few times I've had someone ask me to do this sort of scraping, my response is usually that sure, fine, it works, but it's very easy to spot on the logs, and the information is very likely to become unavailable at unpredictable intervals.
Depends on how you do it. I tend to use tor and a random wait time between gets to bring down the data over a few hours (up to a few days) and in one instance, because the URLs were easily guessed, I randomized the list to make it seem as if the hits were going to pages all over the place. I was never banned for any scraping activity that I have done.
In the long run, it's usually pretty futile to scrape in the first place. When you're stealing content just to drive traffic, you tend to have a crappy site. The only time I ever did a professional scraping app that was "justified" and "legal", the victim was another business unit within the same corporation, and we had every right to the data that they "couldn't" compile for us.
It's not futile. Scraping provides a plethora of information in a useful format from places that aren't willing (or unable) to provide data in the necessary format. I used scraped data of course schedule information from MnSCU to develop a weekly report that showed data about how many courses were filled at other area institutions. It was to our competitive advantage to have this information and while it was publicly available, the system wouldn't provide it to us in the DW. I used that data for a variety of different reports than I originally intended and it would not have been possible otherwise.
While I wish that the data had been provided in a better format for my use, it wasn't and that's what made scraping necessary. Plus *I* was the one who got to determine what information I was allowed to glean from the data rather than whatever the system decided was appropriate for our needs.
This is a shortsighted view of the problem here.
"You're getting paid to do a job, and you're not going to be personally liable should anything go wrong anyway."
Incorrect. His boss isn't breaking the Terms of Service, he is. When the website in question terminates their access, guess who's gonna get the flak? The person who *implemented* the system, not the one who designed/thought of it, especially if they are non-technical and rely on lower-order technical beings to do things for them.
Take, for example, a situation that I regularly come across:
Boss: "It's okay, we'll just copy all these Microsoft CD's and save a fortune on licensing."
Boss's Boss: "Okay. You know best."
Boss (to underling): "Copy these CD's"
Underling in theory: "Okay". Underling in practice: "We *can't* do that."
When things go wrong, the underling in theory is going to get the blame here, because it's his area of expertise and he *wrote* the system that does it. I get people suggest to me all the time that we could just install another license of Office that we don't own, or we can just copy CD's that have blatant copyright notices on them, or breach a Data Protection Act directive by doing X, or a million and one other things that I *know* we can't do. The people in charge of me barely understand the terms, let alone whether what they are doing is illegal. I have to sit and explain to my boss and my boss's boss why we can't do them. Trust me, if something got noticed, Underling in Theory would get sacked/sued every time.
"Are you really going to walk out of your job over violating the terms of service of a few web sites?"
Why not? I get asked to do all sorts of crap and I point it out and say no. If I *chose* to do it instead, then it's a different matter. But when I *refuse* to do something on legal or ethical grounds (we're not just talking ethics here - it also sounds like they have a "subscription" of some kind to the data that they are scraping, or that it's a competitors website) then if you *make* me, I will walk (been there, done that - I've turned down a good career move and more money in order to sleep at night - not that I was being asked to break the law, not that I was being asked to sell my children, but that I was being asked to do things that I didn't agree with [wasting money within a school on useless IT cruft and consultants while the kids didn't have books or paper]). I'll also report you to the BSA or whatever organisation I need to if you really press me, or the local press like I did in the above case (they didn't do anything with it, but I breathed a sigh of relief once I'd sent off the information to them - my part was done and I'd done good by myself - if the press decide to sit on something, that's on *their* conscience, not mine). You don't do illegal stuff if you're honest and your mortgage depends on a wage.
"It's not your job to worry about the ethics of the situation, that's probably not even your boss's job -- it's somewhere in your corporate legal department, the Board, or an Ethics or HR department perhaps."
Wrong. Because they won't even *know* what the problem is until it comes up in court and they have it explained to them in excruciating detail. However, someone who decides to do something that's part of their job, within their area of expertise and breaks a law (or even does something a bit stupid) that *they* should know about will get fired/sued by their own company once the shit hits the fan. So your boss *and* you might get sacked - you're still no better off and your employment reference is now a million times worse.
"just do what you're being paid to do and ask fewer questions."
It's sad that people think this is a good way to live. He's *being paid* to do his job. Which does not entail questioning his ethics or breaking Terms of Service (even if legally unenforceable) or anything else. His *job* is to stand up and say "Whoa, hold on, we can't do that". If he doesn't do that, he's not doing his job an
I'd advise against discussing it with HR. I've encountered the following situation: I talked to a HR manager about something that obviously should've remained confidential. However that same HR manager was part of the management team and thus had two hats on. She proceeded to inform the management team, to my astonishment.
I've come to the conclusion that HR is just a staff department and owes allegiance to, you guessed it, the management team. Not you.
8 of 13 people found this answer helpful. Did you?
Unless it's something about you, personally, then an HR employee has no requirement to keep it "confidential".
In other words, if you are talking about your health insurance, your personal information, etc., that's not general "company business" and shouldn't be spread around. But, if you bring them some information about someone doing something that could be detrimental to the business, they really do have an ethical requirement to pass that along. A good HR team would know not to bring your name into it if you are "snitching" until it was absolutely necessary, but sometimes that happens sooner than you would like.
What you were thinking of is a company ombudsman. These people are somewhat like your "lawyer" within the company, and are there for the employees. What they would do is explain to you your options (go to management yourself, let them do it and respect your anonymity as far as possible, never divulge your name even if that means the complaint can't proceed, etc.), and then help you implement them.
It's happened. ESPN connived a way to get to another sites private database and reported the data as its own. The website injected some fake data which ESPN picked up and reported and were caught.
--You will rephrase your request for me to go to hell. Goto statements are not acceptable programming constructs