Slashdot Mirror
Why Should I Trust My Network Administrator?
Andrew writes "I'm a manager at a startup, and decided recently to outsource to an outside IT firm to set up a network domain and file server. Trouble is, they (and all other IT companies we could find) insist on administering it all remotely. They now obviously have full access to all our data and PCs, and I'm concerned they could steal all our intellectual property, source code and customers. Am I being overly paranoid and resistant to change? Should we just trust our administrator because they have a reputation to uphold? Or should we lock them out and make them administer the network in person so we can stand behind and watch them?"
If it was really a worry, why outsource it in the first place?
That's the service they are offering. If you want someone to be on property so you can look over shoulders, hire an IT staff.
Either that, or learn to do it your damn self.
Obviously you want to find someone reputable, and bonded, but you're never going to get to a point where you can have a network infrastructure that is secure from the people who do your network infrastructure.
I've had enough experience with paranoid managers who hysterically insist that I'm reading their email, or their online banking passwords and crap like that. You think that some schmuck who is working fixing problems remotely really gives a crap about the plans for your Facebook-killer? Think that they care about your boring ass emails? You think they care about your customers??!? Are you kidding? You obviously don't sell networking, so what would be in it for them? Selling a customer list is like selling a used phone book.
No outsourced company is going to send a person to your building every time there is an issue, and frankly, you don't want them to because they'll charge you out the ass for that sort of service. Even if you did decide to pay the price for in-person service, anyone who is out to screw you will be able to screw you while you're watching them over your shoulder, because you won't know what to look for.
If it's really that important to you, bring it in house. And, word of advice, if you do bring it in house, don't treat the guy like a criminal or he's going to start reading your email.
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
At some point, you're going to have to trust SOMEONE
Can you trust your Significant Other not to get all stabby when you are in bed sleeping?
Can you trust the drivers on your commute route not to suddenly get out their guns and start shooting at you?
It's all risk management. If you have super-important data, then don't farm out the management to someone you don't trust. If you have regular data, then farm it out to basically anyone.
SH*T happens... but if you are paralyzed with fear that bad things are going to happen because nobody is as trustworthy as yourself, you aren't going to be leaving your house.
For the same reason you trust your accountant.
Tell me, do you trust your sales people with your customer database? In my experience, they're the ones to watch.
As a guy whose worked in-house and as a contractor I'll say that you can give me full access to the system so I can charge you a reasonable fee or you can lock me out and breath down my neck while I'm trying to work. At which point I'll hand you a BIG honkin' bill for the hassle.
BTW, if you're standing right behind me watching, you still won't know when I'm stealing your data. Not that I would, cause I don't care a bit about your stuff.
I just want to do a good job for you. Make it easy for me to do that and I'll go easy on you. Be a paranoid, obstructive so-and-so and I'll still do a good job, but I'll stick it to you on the bill when I'm done.
I would guess that it costs less to outsource this sort of work than to try to keep your own full time IT staff employed. I might be wrong though.
Palm trees and 8
If you think watching over their shoulder of a person that you aren't sure you trust will make a difference...it probably won't. If they're bent on stealing stuff they just put in a back door in the 4 seconds you're not watching them like a hawk and probably wouldn't catch anyway. You should probably back and decide how much of a risk it is to outsource the admin gig to begin with. If your files are that valuable maybe your business model should afford somebody you can trust and see on the payroll with stock options. Perhaps you need two admins. One the outsource company that obviously would have technical abilities you don't have, but maybe another one that you do trust that at least has minimal abilities to at least monitor for anything unusual?
Do you trust your bank with your money? Even though they don't keep it at your business and you can't stand behind them and watch what they do with it? Your fortune is at stake. Why do you trust them?
Do you trust your grocer to give you clean, fresh meats? Even though you can't go in the back,
see how they're stored and watch them being cut? Your health is at stake. Why do you trust them?
Do you trust your pharmacy to give you the correct medication? Even though you dropped the prescription off, will pick it up later and don't know the look of one pill from another? Your life is at stake. Why do you trust them?
I trust I've answered your question.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
You seem to be conflicted. You don't want to have inhouse IT, but you want them there and available anytime you need them onsite. I think you first need to determine which is important: reduced costs of outsourcing (And all the issues that goes with it) or the improved service of inhouse (and all the issues that go with that)
Even if they're onsite, are you going to have someone paid to stand over their shoulder and watch? if so pay that person to do the damn work for ya.
To be honest your probably safer with an outsourcing company since no sane company would risk their reputation by stealing your "zomg important" secrets.
Nobody should trust their BOFH.
Sadly, it just happens to be the case that we can't live without them, but trustable as a group, they are not.
Trust people, not jobs.
NO SIG
There are no dumb questions.
He's here asking for advice, so give it to him. Even though most of the people who read/post this board are heavily involved with IT, and it might be a common sense answer, the fact is that to this person it isn't as simple a solution.
In many cases, people have sensitive information that they are handling on their servers, and whether or not to trust the IT staff is a valid question. (not all geeks are trustworthy). Also, in many cases, (especially with startups) they dont have the resources to hire on-site IT staff, so they have to outsource it. It introduces a dilemma that many will have to deal with.
-T
And you come to slashdot to ask that question?
Start by hiring someone with real business talent to run it for you because you sound like your own worst enemy.
IF YOU CAN'T TRUST THE PEOPLE YOU HIRED THEN WHY DID YOU HIRE THEM?
Seven puppies were harmed during the making of this post.
...just hiring a real network administrator? Honestly, it's an employers market right now. There's lot of people who have been recently laid off who would kill for a job right now...probably even for a below-average salary.
It is certainly harder to trust an offsite guy, for monkey reasons(can't see the look on their face, body language, that sort of thing) if nothing else; but I'd be curious to know if you have any reasonable grounds to believe that you could detect malfeasance in person.
An atttacker, even a modestly skilled one, given the level of access an admin would need, could do all sorts of terribly serious things in the blink of an eye, whether or not you are watching him. When I'm wearing the admin hat, I routinely run executables on numerous client PCs, manipulate server settings, write and run scripts that gather all sorts of data, make backups, and so forth. Are you really going to be able to see the difference between me tarring the contents of your OMG_Sourcecode directory for backup and me tarring for backup && sneaking a second copy somewhere? And, if you are that good, why are you hiring me to sit there while you watch me, when you could just do it yourself?
If you are paranoid enough, you can use some sort of intrusion detection/exfiltration detection setup, with shell logging, and firewalls, and disabling usb mass storage devices, and uniquely barcoded hard drives, and cavity searches, and so forth; but somebody you trust will have to build that as well.
Obviously, going to Shady Bob & Pradep's House 'o Discount Outsourcing is a bad plan; but so is hiring Shady Bob to work onsite. I'm less sure, though, that there is a significant security difference between offsite and onsite people of otherwise similar levels of cheapness and shadiness.
Yup, you're a "manager", that's for sure. The post was about data access trust, not whether they're doing the job. Do you think an audit report is going to say sniffed network, copied browser caches, installed key loggers?
You really need to ask yourself if you want a professional or a peon? You write your question as if you want someone you can piss on, that tells me you want a peon. Heck, you'll save money on the peon, you can get one from any local technical college, they might even know what they're doing.
If you want a professional and don't want to pay for one, your outsourcing some part time work. You get a portion of a professionals time, that makes you a part time customer, a small fry for the outsourcing company. They are essentially offering a courtesy to you at all to work on your network in the off chance your company grows as this will leave them in a good position.
The bottom line is that professionals that live in your country need to be trusted, they have to much to lose. Most professionals will undergo a background check one to every two years. No professional is going to destroy their livelihood by leaking something like your customer list. No professional is going to risk going to prison or getting sued for crossing the line as long as they live in the same country as you. They will lose their ability for references. Outsource to India and the like and all bets are off, there's no reputation to maintain.
Really, the question is why would your customers trust your company, and is a professional service really any different?
The biggest problem is that the vendors you are talking to are being honest and setting your expectations and you don't like what your hearing. Your about to discover how every extra service has an additional charge and you'll quickly bury yourself in extra fees in the event your company does grow. If you want to position yourself for growth and don't want to be sunk under a slew of fees you should hire a professional in house and then trust them to do their job.
You say "Or should we lock them out and make them administer the network in person so we can stand behind and watch them?"
Given that you aren't administering your own network, I'd guess that you don't have the skills to do so. Would you know trouble if you saw it?
Would you know enough to see them setting up a remote service that they could get back into? Would you know enough to catch them copying sensitive files from where-ever they live to some staging directory, then later copying that directory off to a flash drive, or to some external server? Would you be able to catch them downloading a root kit and installing it?
In short, given that you don't have the experience to admin your own gear, do you REALLY think "standing behind them and watching them" is going to do anything but waste your time?
And IF you have the skills to admin your own machine, but want to outsource that due to some idea of "I have better things to do than this" - you have the time to stand behind them and watch them do the work, does that not imply you have the time to do the work?
Like others have said: If you are concerned, make them put up a bond.
www.eFax.com are spammers
A few weeks ago I read an investigative report on repair shops in Britain. Aside from over charging and finding non-existant problems they looked at and copied information off the computers that were being serviced. Have reason to trust anyone that you give that kind of access to. Then trust, with as much verification as is economical and doesn't unduly make the service provider think that you don't trust them, since unwarranted distrust chips away at the relationship.
I'm unclear as to why you think having them work onsite is more secure. The statement "administer the network in person so we can stand behind and watch them" implies that you have network skills at least as great as they have. In which case the watchers can do the work themselves.
Would you really notice if I ran a batch file that planted a trojaned your computer and uploaded your SAM file(s)? I doubt it. Your IT guy knows everything; that is just a fact of life. Hire a professional and it won't matter. Or you can hire Geek Squad level. Just plan on those "private" pictures of your wife to be added to his personal collection.
I also suspect that you might be hobbling yourself in other ways. (Unless your are geographically isolated or have a non Mac/Windows environment) there is a large number of consultants who will do on-site work. I know; I'm one of them. You will pay more, but there are some situations that require hands-on support. It is very hard to replace a power supply over a VPN connection.
Good luck, and I'm glad you're not my client.
I worked in IT for about 15 years, and always held that if a company doesn't trust its network administrators for a justifiable reason, then those people shouldn't be the network admins.
Remote/local doesn't matter. If they are not trustworthy and you can document why, then don't make them your admins. If they are, then don't worry about it until they do something to violate that trust. And if they do violate that trust, then go after them guns a-blazing (figuratively, not literally, OBVIOUSLY).
Most network admins want to be trusted - and need to be. Being untrustworthy is the kiss of death in that entire career path.
As others have said, local or remote doesn't matter. In-house or outsourced doesn't really matter. You need to accurately assess their trustworthiness and then deal with it in an appropriate manner.
Insanity is a gradual process; don't rush it.
There seems to be an assumption that you can "keep an eye" on an on-site network administrator, and that's why you can trust them.
How would you tell if they were up to no good? Will you be looking over their shoulder constantly?
I have worked in medium size IT shops (appro 100 people), and have seen the system admin team all stand around a computer as they go through their manager's CV (they had left it on there home drive). This was practically outside the manager's office, but you can't be everywhere at once.
Maybe you assume that you will only hire trustworthy people, but how can you tell if you can trust someone just by working with them?
Personally, I think the bigger risk to your operation will be if you hire a bad sysadmin.
Owen.
Democracy isn't about no one telling you what to do. It's about everyone telling you what to do.
There is some data that a sysad, whether internal or external, should not be trusted with.
Basic system administration should be required for business and management degrees, enough to maintain the disconnected key server and the separated subnet that handles all the most sensitive data.
Small networks are not that hard.
Computer memory is just fancy paper, CPUs just fancy pens with fancy erasers; the 'net is just a fancy backyard fence.
I can tell you right now, and administrator is going to tell you right where you can tar it if you stand over his shoulder while he's trying to work. I've been an admin for a long time and I've dealt with people like you and it always comes down to the same thing:
Either you will trust me to do the job you hired me to do or you can find someone else to do it. Being administrator inherently means I will have access to all your base. The fact that I'm a professional doing a job I was hired to do means all your base are not belong to me. Irritate me by hovering over my shoulder all day and that will change.
It is a mistake to think you can solve any problem with just potatoes.
My response is one of many just like it, but bottom line is you HAVE to trust your network admin. Whether he's on site or off, he has access to your stuff. And frankly, I don't care if anyone walks in and sees what I'm doing randomly, but outside of a performance evaluation, the day anybody steps into my office and starts watching what I'm doing is the day I quit.
"People who think they know everything are very annoying to those of us who do."-Mark Twain
I am a remote administrator for dozens of companies. I have been doing this for many, many years. My business success is directly dependent upon your business success. I have a vested interest in every single one of my customers growing and flourishing in business. As such, I only recommend solutions that are justifiable in direct, easy to understand terms.
You have proprietary information? So what. So does every other company and government agency I do work for - all of which is done remotely. Only on rare occasion do I visit on site.
If you cannot place your trust in the people holding your admin password, then administer it yourself. Otherwise be prepared to pay 2-3 times more for simple administrative tasks.
I'm sure I have access to tons of proprietary information, sensitive information, etc. but so what - I'm an honest guy. If I see the stuff, my first reaction is do we have this properly protected? I know the first reaction in a criminal mind is "What can I do with this?". Criminals don't usually want to work for a living.
Good security is based upon reality and common sense. Common sense is a function of having common knowledge.
Here's a thought: If you hire an admin, you have ONE person who might potentially steal your data. If you outsource to a company that has 500 people who have the ability to remotely connect to your systems, you now have 500 people who might potentially steal your data. The chance of having one bad egg in 500 is much higher than having one bad egg out of 1.
Either you trust your sysadmins or you don't give them the access they need. Administrators require access to all of your files, your network traffic, your email, your financial data. Not all of the admin staff needs it, but at least one of them does need some access.
The problem with outsourcing is you are treating sysadmins like janitors, a necessary evil farmed out to the lowest bidder. Where the reality is the function is a critical professional appointment which requires vetting, just as you would your accountant and lawyer.
POKE 36879,8
You can't tell a story like that and just leave out the stupid questions.
Bear in mind that there's nothing to stop an angry local administrator stealing/selling data, and being more intimately involved with the company's business activities, he probably knows better where to look.
But, I'd suggest not outsourcing if posssible for a different reason. It normally doesn't work. The lack of local site knowledge is hugely detrimental to knowing wtf is going on. I was with a large aussie mining company that tried it - after 18 months they couldn't get away from the outsourcer fast enough. Main problems are that there is usually no continuity in who deals with a problem, no sense of personal responsibility, no problem ownership, and any admin who gets a clue at the outsourcer leaves and gets a real job as soon as they can.
You'll end up dealing with muppets who either don't care, have no clue, or both.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
This suggestion above is equivalent to proposing that managers have to learn electrician skills to wire the most important room in the building, for fear the paid electricians might sabotage it, or they have to learn locksmith skils to key the locks on the most sensitive file room, because they can't trust locksmiths not to share a copy of the key or sneak in one night.
The simple fact is the management of key systems should be entrusted to skilled IT professionals whose primary responsibility is maintaining consistent, operational, available systems.
That doesn't just mean setting up systems and forgetting it, it also means implementing secure backups, monitoring audit trails, managing the complex access controls, monitoring system logs, and correcting problems.
Make sure that you have a document to describe how to take back the network in case you decide to fire the IT staff. I used to work in this area, and I provided this to my clients even if they didn't ask for it. If I were looking to outsource, I'd certainly make sure that I had the ability to rip it back. Even if I trust the outsourcing company completely, which is requirement #0 in my book, I want to make sure that my company stays my company.
It also has me thinking about a boss I had who went nuts when he found out I could read his email. He wanted his own email server (and like who is going admin it?).
And see, had he asked that question (maybe not in front of slashdot, but at least someone who had a clue) that would have been better than what he did.
In any case I have to wonder about the future of this startup is the people involved are so inexperienced.
It sounds to me like he's trying to become at least a little less inexperienced. And we're calling him an idiot for it.
It's kinda funny, I joked about this very same idea, that the $2.00/hour outsourcers might be intentionally raping our servers for profit. Then the next day one of my support clients had that exact thing happen to him... one of his developers in India decided to create a bunch of email accounts and spam off of them. I have to admit, it makes perfect sense: he probably made more money selling spam runs for a few days, than a week of regular salary, plus he's not going to get into any immediate trouble... I'm not going to fly over there and beat the tan out of him, he just lost one smallish contract - big whoop.
It's not about "you get what you pay for", and certainly not a racially charged disconnect (at least not in my case), it's just the risk vs reward balance that's tipped against us. Globalization is a double-edged sword. White collar crime is just as big a problem in western societies, but we do it bigger and badder. As an American, if someone offered you $100 a day to sacrifice one of your clients, you'd probably tell him to blow you. In India, $100 might be equivalent to $1000 to us, maybe more. I don't know about you, but in my neighborhood if you want to make $1000 a day you either have to sell your ass, or sell gobs of crack and blow. The incentives vs risks aren't on the same scale at all.
I'm not saying we should treat all outsourcers as hostile crooks, we have plenty of those right here at home, on the payroll even. We just need to approach it sanely. If you underpay someone, they are more likely to fuck you over - that much should be common wisdom in the business world. It's the dirty side-effect of living in an entitlement culture.
-Billco, Fnarg.com
"Seems fair. Personally, i don't see why a company should refuse to do all service on-site."
Probably because the whole story went untold. While it can be true that small IT companies might not have the head count to offer on-site to their clients, I'd bet the untold part of the story goes more or less like "the company refused to service on-site for the peanuts I offered". Given that 8x5 on-site outsourced (I think that's the option he was looking for) will usually be overall more expensive than a direct hiring (since vacations, training, replacements, failed recruitements... all go to the provider's expenses) probably that's the point.
Data is valuable because management thinks it is valuable.
Bribing people to be ethical is probably more effective than attempting to force them to be ethical, but both approaches have limits, and the limits hit a lot earlier than managers want to believe.
Computer memory is just fancy paper, CPUs just fancy pens with fancy erasers; the 'net is just a fancy backyard fence.
Money can buy off the 'looking for other opportunities, including selling your data'.
Why do you think people that handle sensitive government information generally have their finances looked at? If you're hurting for money, you might try and pawn something you have access to.
True, some people will just take the money -AND- sell your crap.. some people will also take almost no money, but still not sell your crap.. what you're trying to buy is some insurance and CYA factor.
As for Managers -needing- to learn IT. .I think it's dumb.. IT mangers should know IT.. but does a Accounting Manager need to know IT? no.. they do need to be able to communicate their needs and concerns effectively to the IT manager, and the IT manager needs to know enough to relate those needs/concerns into their 'IT world equivalents', and make sure some relevant things are taken care of too (the Account mgr might not realize that some information should be encrypted in case of data-theft, that's the IT managers job to point out and bring to the table)
You cannot make someone an expert in everything; there simply isn't enough time or desire to do it... welcome to Specialization.. it's sort of why the human got as far as we have.
----- The internet has given everyone the ability to have their voice heard equally as loud.. even if they shouldn't be