Slashdot Mirror
Linux in a Business - Got Root?
greenBeard asks: "I work for a government contractor, and have recently convinced them to purchase a Beowulf cluster, and start moving their numeric modelers from Sun to Linux. Like most historically UNIX shops, they don't allow users even low-level SUDO access, to do silly things like change file permissions or ownerships, in a tracked environment. I am an ex-*NIX admin myself ,so I understand their perspective and wish to keep control over the environment, but as a user, I'm frustrated by having to frequently call the help-desk just to get a file ownership changed or a specific package installed. If you're an admin, do you allow your users basic SUDO rights like chmod, cp, mv, etc (assuming all SUDO commands are logged to a remote system)? If no, why don't you? If you allow root access to your knowledgeable users (ie developers with Linux experience), what do you do to keep them 'in line'?"
Much as I hate to break it to you - this is SoP.
End users Do. Not. Get. Root. Even allowing SUDO access to change file permissions, copy, or even move files is just asking for trouble.
Installing software or libraries? Hell no. Not on a live system.
If they have a development-type machine at there desk, that's one thing (just don't call for support if you break the damned thing). Even then, my preference is that they have limited access.
On large, shared systems, users get as much, and as little, access to do their jobs as necessary, and absolutely no more than that. I have to keep the system up for other users, I can't have power-user #1 screwing things up by changing permissions on something they really shouldn't be touching (let's take the compiler for example...)
A little knowledge makes one dangerous, and I'd just as soon noone other than those paid to admin the machines have access.
Ever heard of setuid root?
And even allowing chmod, mv, etc via sudo can be dangerous. Someone accidentally issuing a "sudo chmod 777 -R / ", having meant to type "./" for everything below their current directory, isn't going to be good for your system health and is going to be somewhat of a pain to recover from, even if you do know who screwed things up.
You are going to get a bunch of responses. most of them from people that will say something like.. "NO." "NOBODY GETS ROOT, PERIOD".
Well, in an ideal world, it would be that way. We would setup systems for people to use and they could just use them without root privledge. Unfortunatley we know that isn't possible if you want your users to actually be productive and get things done.
I work for a large software company. Trust me you'd know the name of it if I could tell you. We use linux on the desktop, as well as the servers. We also have some Microsoft servers that are either for legacy purposes (havent been updated yet) or for testing applications against MS environments. Anyway...
All my users have laptops with Linux on it. They all have the root password to their individual laptops. Many of the also have a server at their desk for their own testing purposes. They have root to that.
However, the "real" servers that are accessed by someone that isn't themselves, the users do not get the root password, ever.
I look at it this way. If you bomb your laptop or your test server, either you can fix it, or you can call me and I'll walk you through fixing it, fix it, or just give you a new clean configuration.
If you bomb my server, I'm going to make sure you never have access to anything, ever.
Don't Tread on Me
Developers with Linux experience are a LOT more dangerous than developers without linux experience. My experience has been (100% of the time) when I give "experienced developers" access to commands like 'chmod', I find all kinds of files mode 777 (among a list of about 10,000 random, stupid things developers do) because, well, I've heard pretty much every excuse you can imagine.
The problem is that as soon as people outside of the core sysadmin team have access to critical system commands (cp, chown, chmod) the integrity of the box is left to chance. There's always the possibility someone is going to do something outside the policy. Sysadmins make it their job to know and understand the impact of every change to a box. Developers tend to make changes in order to get their stuff to work, regardless of the consequence (hey, each group is just trying to do their job, which is "make it work!!" -- I'm not defending either side).
My rule of thumb:
- Developers get root in their dev environments.
- Sysadmins get root in the production environments (developers shouldn't even have user-level logins to these machines.) If your company is releasing software (even for internal use) properly, the IT group will be managing the code as a product, using developers as a help desk rather than letting them manage the applications directly.
Stick to this and everyone will be happy.
-- People who hate Windows use Linux. People who love UNIX use BSD.
I come from the other side of the fence. I am a developer of multiplayer servers. For my part, I couldn't do my job without root access. I need to do things like set the date and time on the machines, install to /bin, upgrade compilers, etc. If I had to ask the helpdesk every time I needed root, they'd just set up right outside my cube.
On my Windoze machine, OTOH, I have no need for system level permissions, and I don't ask for them. I can install software, but so can all the other developers (and, I think, anyone in the company). All I use that machine for is e-mail and testing client connectivity to my servers, when I'm not using my Linux test client.
Some people need root and some don't. Don't make blanket policies unless you're prepared to make exceptions. Oh, and, for everyone's sake, if you do restrict access, please, please make sure that at least one person who can change things is available 24/7. I can guarantee you that Peterson up in Accounting is going to have a system crash that requires help when trying to get the year-end reports out at 2:30 A.M. before the big board meeting at 9:00.
(Disclosure: I work for Sun and work with Linux since 1994).
Why would you move the modelers to Linux from Solaris? There is no real advantage....
Sure a Beowulf cluster is a nice piece of hardware, but hardware can only compensate a bit for programmer productivity... If their code is written using MPI or OpenMP or some other standard clustering environment then there shouldn't be a need to move the developers, should there? Just recompile and go.
It is really much more efficient to shove faster hardware under a programmer then to force the programmer to adapt to a different programming environment. Programming for a cluster is hard enough without having to take into account the details of the operating system, forcing them from Solaris to Linux might improve the execution part (on a side note, have you considered Sun's clustering tools?). But it *will* set them back in productivity while they move to different compilers and adapt the execution of the program to the Beowulf environment.
In my opinion you have forced your customer to make a move on questionable grounds.
Now to the matter of security. As you are aware, Solaris has the highest level rating for security. Secure Solaris is the defacto operating system at a number of government agencies. Linux cannot hold a candle to the multiple access levels of the Secure Solaris operating system. You state that you are frustrated at needing the helpdesk for file permission changes. What is your point? Are you using the fact that YOU don't like the limitations to change a customer from Solaris to Linux? Or are you complaining that the customer's environment did not deploy secure solaris with its multiple access layers? In Secure Solaris there is no need to muck with sudo. Each file can be managed properly from a security point of view (come to think of it, much of that can be done with Linux too).
Before I answer your question, let me state that I understand your point of view. When I joined the navy as a UNIX project manager, the admins gave me absolutely no rights whatsoever on the production systems. Their reasoning: '.. he can do things I don't understand, can't control or prevent.' There will always be a tension between the lockdown desired by the admins to keep their environment safe and secure and the users who want total freedom....
In my mind there is NO good reason to give ANY user root access in a secure environment. Period. If you have frustrated in the past by having to interface with the helpdesk, then the helpdesk needs to be improved. At the same time, I assume, any user has full access to their files.
You mention that you have convinced modelers to move to a Beowulf environment, then why the issue anyway. If they run cluster code then they run as user. All the need are basic user access rights, nothing more...
Maybe I don't understand your point....
Another sysadmin who is going to tell you that I don't give out root or sudo access to users. Most users who think they know enough, or even DO know enough, really know enough to make big problems. They invariably never check with me before making a change, or tell me that they made a change, or even admit to having made a change when they inevitably screw something up.
I make them come to me for everything. But not directly. That's what the ticketing system is for. The ticketing system justifies my existence, keeps any requests from slipping through the cracks, and helps to keep track of ad-hoc changes made to any given system.
Many times end users think they need root for something when they don't. For example, there might be some niche tool that they need installed on a system. Or do they? If the one user is the only one that is going to use it, I advise him to do something like "./configure --prefix=~" to build apps to install in his home directory. You don't need root to install apps anymore. Besides, if you want an app installed for everyone to have access to, sysadmin should be doing that anyway.
It might be a pain in the ass to make you go to the sysadmin for everything, but in the long run it will keep things running smoothly and perhaps force you to be a little more disciplined in your work.
I don't really get the original premise. Nobody needs to be root to run chmod, cp, mv, etc on their own files. The only command mentioned one might need root for is chown. Which would make me ask the question, why do you need to change file ownerships so often?
It would take a hard-core serious business case to convince me to grant someone root access, even sudo-limited root access to a production system. The fact that I might have a "log" of whatever broken thing they did to take a business critical machine down is fairly irrelevant to me. My job is to make sure that doesn't happen in the first place.
...and I do NOT WANT ROOT.
I have root on my workstation (cold dead hands and all), but not on a single server--not even a dev server.
sudo on things like mv and chmod gets you a root shell on the box fer chrissakes, why not just put the root password on a sticky on the rack?
When something goes wrong, I don't want to hear, "Maybe the dev did it." I didn't do it--no access. When we go to prod on something, I don't want to hear the admins complaining they don't know how to promote the app because some ass developer did it manually in dev instead of creating a proper install.
If you need root to chmod something, then your admin hasn't set up the box properly. Either he doesn't know what he's doing, or you haven't told him properly what sort of environment you need. Either get a better admin, or write up a clear description of all the functionality you require. Either way, you don't need root.
Of course, the smaller the business, the more likely an admin is a dev and vice versa. In that case, all bets are off.
There are some interesting 'privledge escalation' things that can happen on machines 'owned' by the user on a big network, though.
The one I experienced firsthand was a Windows NT machine that was my desktop that I ('naturally') had full admin access to. This was a machine on a large corporate network that was very diverse (there were Solaris, OS/2 Warp, Netware, and Windows NT servers on the network). I discovered, quite by accident actually, that if I ran the POSIX Interix (now SFU) shell on my NT workstation (something the company had bought for me, and I had installed myself,) that I could create any account I wanted on my local machine, and it would allow me, using that account name, to access shares on the network, doing whatever I wanted to files my username 'owned'. I am talking about the network that a company that makes implantable medical devices kept their work on. I suspect the 'defect' had something to do with NIS and 'travelling profiles' in Solaris, and the security system not being equipped to deal with other Unix-like hosts on the network that weren't secured. Incidentally, I didn't discover the problem by 'poking around where I wasn't supposed to be,' I simply noticed I was suddenly able to do things to files I normally had access to without entering my UNIX password as required in the past. Something clicked in my head, so I created a local account on the NT box that matched an important person's UID on the Unix system... yep, I had all his permissions.
Delete test account. Never touch again. Too scared to mention it to anybody. It's been enough years now that I can even mention it in public. I hope they've secured things a bit better now, because these days there are unsecured Unixy systems all over the place.
resigned
Hell, at time I think I shouldn't even give the users a keyboard and monitor. It's not a question of if the users will screw something up, but when. They are ALWAYS doing things they shouldn't. Thus the less they can do, the better.
The worst are the "I'm a sysadmin" types. For every one I meet that actually has the experience to make them a competent sysadmin, there are 50 that know just enough to be dangerous, but think they know it all.
For example some time ago I decided to roll Firefox out to the educational labs and make it the default browser. All other considerations aside, it's minority status in the browser market makes it far less of a target. Well a couple days later I get some guy in who's bitching about Firefox being installed in "his domain" and he wants it removed. Upon further questioning, it becomes clear he believes that programs are installed in user accounts. I cannot seem to convince him that the program is a local installation on every system and no, I'm not removing it.
Now for Windows systems, the damage someone can do is somewhat limited since all software installs are on the local system. However the UNIX systems all run off a central server. Like hell we are giving anyone anything but read access to that. All the time people want things installed or modified for their particular project. Quite often, they have no idea what they are asking, and what they want done would completely break the app, or worse.
I agree that access should be as limited as allows you to get the job done. Now, in some cases that needs to be total access. Fine, you get a system that's seperate and you assume responsibility for it. If you are doing something such that you need system access, you'd better have the knowledge to fix what you break. In other cases, come to us, that's what we are paid for.
We even operate that way internal to our group. I don't just go and change shit in DNS. Not because I don't know how to, not because I don't have the root password, but because it's not my area. Better I should ask the guy who is supposed to do it. That way, there's less chance somerthing that gets broken.
I think the problem is that some users have a real inflated sense of self importance and entitlement. They think that their project is real, real important, more important than everyone else's. Thus they don't have the time to wait to have the admins do things, they want to just be able to do them themselves. If it messes something up, well then the amdins can fix it. Of course people like that are also the most likely to do something that will break things for others.
The more shared the resource, the more you have to be strict with the access. Even on user desktops, limited access needs to be the rule. Support can't spend hours and hours fixing problems caused by users that don't know what they are doing. It's just not cost effective.
If you truly have the need and knowledge to run your own system, then fine, take it up with management. However part of that understanding has to be you can't bother the support team if you hose things. If you aren't good enough to admin the thing yourself, you probably ought not have admin permissions.
Take a lesson from this guy - he's smarter than he looks. When you find a security hole like this, do NOT report it unless you can do it anonymously. If you can't report it anonymously, then just sit on the knowlege until the end of time. This is your job, your life, and your paycheck. We've all read the stories about how the person who reports a security hole gets criminally prosecuted for "hacking". You might be a smart person, but everyone around you is a blathering moron. That is a FACT. That blathering moron isn't going to say "thanks for pointing out this embarassing security hole that my ass was hanging out of." The blathering moron is going to try to cover his ass by blaming somebody else, and the easiest somebody is YOU. That way he takes care of the problem and gets brownie points for uncovering a dangerous "hacker" within the company.
Next thing you know you're getting arrested by a nice FBI agent named Bob, and then getting cornholed for days in the local jail waiting for a judge to set bail. It's not worth it.
Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
On a production box, the admins have access to sudo, and root itself is locked down except for scheduled maintenance/upgrades or emergencies. No paperwork, no root.
As a developer with over 15 years *nix experience, I have never had root access to a box unless I was doing an install, except for my own desktop workstation. In the case of my desktop, the only reason developers had root was so we could kill rogue services during debug sessions gone bad.
Under no circumstances do I agree with any user installing additional software on a box. If it's needed, it gets approved and installed for everyone who needs the functionality, not by rogue users.
I do not fail; I succeed at finding out what does not work.