Slashdot Mirror
Disempowering the Singular Sysadmin?
An anonymous reader writes "Practically every computer system appears to be at the mercy of at least one individual who holds root (or whatever other superuser identity can destroy or subvert that system). However, making a system require multiple individuals for any root operation (think of the classic two-key process to launch a nuke) has shortcomings: simple operations sometimes require root, and would be enormously cumbersome if they needed a consensus of administrators to execute. There is the idea of a Distributed Administration Network, which is like a cluster of independently administered servers, but this is a limited case for deployment of certain applications. And besides, DAN appears still to be vaporware. Are there more sweeping yet practical solutions out there for avoiding the weakness of a singular empowered superuser?"
Rule by a benevolent dictator has certain advantages, and rule by committee has certain opposite advantages. It was ever thus.
It is called: "Change Control" and usually goes along with "Revision Control" on configs.
If you change without recording the reason for change and without checking in the result so that the two versions can be compared and analysed you get a pink slip. Voila. Problem solved.
Baker's Law: Misery no longer loves company. Nowadays it insists on it
http://www.sigsegv.cx/
It isn't about respect, necessarily. I am a sysadmin that has the keys to a lot of things and I have wondered about this very problem. It isn't about how much respect I deserve but it would be nice to a have a distributed method in the event of some sort of catastrophe or something as simple as being sick.
That you have one person doing it. It's effective, and versatile.
If you have multiple people empowered to do exactly the same thing, you end up at the mercy of the one that decides to shut everyone else out.
If you then have a security admin that's the only one to be able to alter the login info, then you're at their mercy.
With the "dual key" type approach, what's to stop someone installing a back door along with a normal software upgrade? Does everyone have the same knowledge as your prime sysop? Can you afford to have one person that completely mirrors another, instead of distributing the skills across a time (with duplication covered across the team)?
What if both the key holders are in cahoots?
Interestingly, who is stopping your CEO from making those really bad decisions, or your FD from siphoning the cash, or a whole host of other areas where you trust one person to do a job?
Value the person, and make sure you treat them well enough to make it not worth their while to play you up.. Then you'll have no problem.
Screw them over at every opportunity, and you'll really have to trust their ethical views (you're still usually safe, but it's no guarantee then).
Mostly, except in very small organizations, there are several implicit safeguards to keep any one person from doing evil with the systems. They are subtle, but effective.
Peer review: Most sysadmins are hired by other sysadmins, or at the very least a technical manager. This means that you are hired based on your skills, reputation, track record, and demonstrated attitude. This means that ideally, you wouldn't even *think* about intentionally subverting a system, because that would mean breaking it or compromising it in some way, and most professional SA'a are simply too OCD to allow it.
Business continuity: Most organizations have several layers of continuity in place, such as disaster recovery scenarios, system snapshots, monitoring, and auditing. This means that unless you are VERY subtle, or work for an entirely incompetent team, you WILL get caught, and the damage will be minimized as you are being put into a police car, never to work in IT again.
There are no "indispensable people:" If you are a sysadmin, and you are the only one who knows your systems, you have not done your job. Every system and app should be documented, and there should be accountability for every change and decision.
No technical solution will ever replace good management and planning, and a design that eliminates the vulnerabilities of a system to rogue sysadmins, will also eliminate its flexibility. It's just a lot cheaper and easier to try and run a good shop.
-- lk t lv ll th vwls t f wrds. T svs lts f tm t wrt bt ts pn n th ss t rd nd mks m lk lk cmplt dpsht.
Everyone treats everyone else like adults and every one acts like an adult? Honestly, if you don't trust your admins, why are they your admins?
Also, simple change management alleviates most of these problems. Even if it's just a log for what happened so that the next shift or your colleague tomorrow knows what you did today. Then again, I guess that is really back to acting like adults.
"I use a Mac because I'm just better than you are."
Not really. It's fun to think I could do anything I wanted, but I don't want to. I like my job, I like the people I work with, I don't want to screw them over. It's nice to have an employer that trusts you too. If I wasn't trusted, I would probably just leave. If they want me to be able to administer and troubleshoot everything, I obviously need full access.
which is totally what she said
Peer Review, Change Control, Auditing, Maintenance Windows, Testing all changes in a lab before production, source and version control / maintenance. These are all best practices, work regardless of operating system and don't require any special software.
Why o why do you want to use software to take the place of established best practices? Best practices are there for good reasons, and those reasons usually have multi-million dollar lessons attached to them. You don't need special software, just a heavy that says yes you /must/ do it this way and raises hell when you try otherwise...
This.
if you can't trust the person at the top: then either they don't deserve to be there, or you need to find a new job.
when you're the person at the top: you better have earned the trust and respect of those under you. Subverting it does nobody any good in any long term.
You keep your passwords in a network share? Are you schizophrenic or just incompetent?
I hope that file is fucking well encrypted ... but even in that case, it's just a bad idea.
WTF am I doing replying to an AC at 5 A.M on a Friday night?
If you don't trust your sysadmin, they shouldn't be your sysadmin. Just like the accounting department probably has the ability to steal a certain sum of money before anyone will notice, your sysadmin is given responsibilities that could potentially cause grief if they are on the wrong team.
The solution here is to follow a reasonable security protocol in writing the sudoers file. Specifically, the default action is to prohibit. Permitted actions are then whitelisted. On a high-security system, no entry should allow a user to sudo su -. Problem solved.
Incidentally, I see no point in locking down users who have physical access to the DC.
www.wavefront-av.com
Trust, but verify. I believe the submitter is asking how to provide for verification without breaking operations.
Just as I'd be an idiot for handing my checkbook over to the sole control of an employee based solely on trust, I'd be an idiot for handing over the keys to my IT systems.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
Yep. And a single malicious incident could end my career. A career I've spent many decades and countless hours on. There's no way I'd risk it. And that's assuming that my morals would allow me to seriously consider jeopardizing it in the first place.
Obviously there are those with different goals and standards and it's not always easy to identify them. I'm not sure how to prevent that -- someone who over the years gradually gets more access and one day they decide to go rogue and do something harmful. Even minimizing the attack surface you usually have that single admin account that owns everything else. Maybe I should read the article.
The tricky part comes in at the point that, while most CEOs have at least a basic understanding of accounting and other departments under their watch, IT departments are *typically* a foreign land to the understanding of those in charge. Even if they wanted to audit proper usage of root it would be difficult or impossible. Small businesses have it hardest. At least in the larger ones there's a layering system so you can have higher-ups in IT auditing the lower guys.
I think you're missing the point. Auditing/logging systems are not meant to provide effective defense. They are meant to let PHB's mark appropriate check boxes on compliance forms and sleep better without worrying what those evil nasty sysadmins are doing. Don't confuse them.