Slashdot Mirror
Linux in a Business - Got Root?
greenBeard asks: "I work for a government contractor, and have recently convinced them to purchase a Beowulf cluster, and start moving their numeric modelers from Sun to Linux. Like most historically UNIX shops, they don't allow users even low-level SUDO access, to do silly things like change file permissions or ownerships, in a tracked environment. I am an ex-*NIX admin myself ,so I understand their perspective and wish to keep control over the environment, but as a user, I'm frustrated by having to frequently call the help-desk just to get a file ownership changed or a specific package installed. If you're an admin, do you allow your users basic SUDO rights like chmod, cp, mv, etc (assuming all SUDO commands are logged to a remote system)? If no, why don't you? If you allow root access to your knowledgeable users (ie developers with Linux experience), what do you do to keep them 'in line'?"
You are going to get a bunch of responses. most of them from people that will say something like.. "NO." "NOBODY GETS ROOT, PERIOD".
Well, in an ideal world, it would be that way. We would setup systems for people to use and they could just use them without root privledge. Unfortunatley we know that isn't possible if you want your users to actually be productive and get things done.
I work for a large software company. Trust me you'd know the name of it if I could tell you. We use linux on the desktop, as well as the servers. We also have some Microsoft servers that are either for legacy purposes (havent been updated yet) or for testing applications against MS environments. Anyway...
All my users have laptops with Linux on it. They all have the root password to their individual laptops. Many of the also have a server at their desk for their own testing purposes. They have root to that.
However, the "real" servers that are accessed by someone that isn't themselves, the users do not get the root password, ever.
I look at it this way. If you bomb your laptop or your test server, either you can fix it, or you can call me and I'll walk you through fixing it, fix it, or just give you a new clean configuration.
If you bomb my server, I'm going to make sure you never have access to anything, ever.
Don't Tread on Me
"If they have a development-type machine at there desk, that's one thing (just don't call for support if you break the damned thing)."
The reason of existence for you, IT Lackey, is to make sure my machine is working. Then I, god-like Software Developer, can develop the company product which in turn gets sold by the rat-faced vermin in Sales and Marketing.
IT Lackey not giving support => IT Lackey should no longer be employed.
Have them share a group? They can always share files by allowing complete group permissions, can they not? If that is all they want to do.
Any guest worker system is indistinguishable from indentured servitude.
all well and good but I have worked in places where I definitly had more skills than the admin and had to wait long periods of time for the inept admin to fumble their way through somthing I could do in 5 seconds.
You're not a developer if you can't even maintain your own machine.
To be a decent developer, you have to understand not just how to write code (or, in too many cases, "move pretty icons on a screen") - you have to understand the environment it runs in, including file permissions.
Didn't your mother teach you to clean up after yourself?
So-called "god-like Software Developer" can't even maintain his own machine => waste of skin exposed.
I have always had this ability (in several of the largest companies in the US) but I have always started the conversation with an acknowledgment that the sysadmins are ultimately responsible for the network. Then, we focus on what functions I may need to do, how I avoid causing a problem beyond my own work, and how we can establish a regimen where I report what I have touched and where they are able to monitor to ensure that I have do only that.
This has never been a problem. Then again, they already know, prior to that, that they would be in bigger trouble if I were not trustworthy. I offer them more controls than they would have insisted on and this gets me more latitude than they normally would have offerred.
Yes it does.
Its usually the other group's reponsiblity to have the box and app running. That means changes are well communicated and planned, not ad-hoc.
Also, its to make sure the single app is the only thing running on the box.
Also, its to make sure that there is a known level of security on the box.
The surprise isn't how often we make bad choices; the surprise is how seldom they defeat us.
These days security is an issue. But with a compute cluster I'll bet I could come up with a setup where moderate errors wouldn't compromise the rest of the network. A compute cluster is easier because people won't be running browsers on it or reading their email there. That eliminates most of the user-caused problems. So at that point a tight firewall may be enough. On a small research machine used by one research project you're not really trying to protect users from each other. So many of the traditional concerns about protection don't apply.
One variation that I've seen work well is to allow people full access to their own workstations, but not on servers (or clusters in your case).
This limits the types of access people can have on the heavy server machines, but lets them install apps or do whatever they need locally, including installing a different OS or OS variation based on need or preference.
Of course, the more people 'adjust' their workstations the less support you can afford to give them. You also have to be very careful about how much servers trust any information from workstations (NFS can be very tricky to get right), and have to more draconian with your firewalling policies.
Security versus usability is always a balancing act. It just depends on which balance is right for your environment.
plus-good, double-plus-good
Speaking as a former user of such a university computer, I was *glad* not to have root access on my machine. It removed the burden of system administration, and left me to concentrate on the stuff I was supposed to be working on.
UNIX permissions are designed for just this situation, allowing users to work freely without giving them ability to break the system. Why is the orginal poster trying to circumvent this?
Suns also lack, as a default configuration, literally dozens of extremely useful tools that power users expect. That starts with a decent compiler built into the OS, (although I understand they're finally including gcc), standard X libraries and development libraries (because Sun's environment has always been way off the beaten path and need hundreds of man-hours to beat into shape with each new release at least up through Solaris 2.8), tend to be very short on RAM for the same amount of money, have a version of "tar" that inconsiderately makes it incredible painful to do a "ssh root@hostname tar cf - /etc/passwd | tar xf - -C /tmp" and yet have it overwrite your local /etc/passwd because it doesn't strip the leading slashes, have one of the strangest versions of LDAP I've ever seen, and their pkg package management system needs to be....
Well, I'm going to avoid using that kind of language on a bulletin board, but it's not good. But the amount of work necessary to weld together a cluster of 100 Suns into a large and flexible working unit with whatever software the users need to do their jobs is easily enough to pay for another 50 servers. The money saved by buying something other than Suns will buy the backup and cooling systems needed for the whole setup.
Man, I wish I hadn't posted in this thread, so I could moderate your comment.
You are the only poster so far who seems to have any understanding... Or at least the only one that doesn't let their understanding get clouded by their childish desire to "have root" even if they don't really need it.
With root access comes responsibility... and I don't mean that like the way they use it in a Spider Man comic book. It's not that you need to exercise caution, ethics, and good judgement lest you become evil; If you have root and something goes wrong, you are responsible. Even if you weren't the one that broke it. Root is a blame magnet. Period. End of story. Unless they're paying you the sysadmin's salary too, you should not want to have root access on any shared system.
Also, people who can't grasp the concept that sudo access to chmod is exactly the same thing as complete root access should have their *nix geek license revoked.
Unless you need to set the clock, signal a process you don't own, or listen on a well-known port numbered 1024 or lower (if it's not a well-known port, you don't need to use a low number. I don't care how much you insist. You don't have a good reason. I'm not listening anymore...), you do not need to be root. Yes, you can do every single other thing you need to do as a user without root. It's not even inconvienient. One must wonder how these people would have survived before PCs...
I've installed numerous versions of Ada and C++ compilers, window managers, gtk, qt, lyx, and more in ~/bin under Solaris without root. Almost all file permission problems are solvable by contacting the file owner directly without involving an admin, and that's usually the courteous thing to do anyway. Things that do actually require root are few and far between in my book. Even if you really need to do something regularly like restart a web server, it can be easily arranged with a one time change to a sudoers file or something. However, that is the exception rather than the rule.
This space intentionally left blank.
Beowulf can mean cheap hardware. Cheap hardware is not were most of the money is spent.
(You imply that Sun doesn't make cheap hardware, I state that you haven't brought yourself up to speed with Sun's current offering. The Opteron servers with Infiniband interconnects make quite a nice clustered environment.)
Programmer productivity is were the money is. Getting a program to work for a clustered environment requires a lot of skill. Often the code is hand-tuned to work over the particular configuration of a cluster. What we lack from the original poster is information on what environment the customer currently runs on and what their needs are.
My post is merely pointing out that it is an oft perpetuated myth that cheaper hardware equals more bang for the buck. It doesn't, the additional cost in developer time spent to make those applications suitable for a new cluster is usually equal or greater than the cost savings.
There are many issues related to programming for- and executing in- clusters. Only when you are using really large clusters on programs that are well understood does it make sense to make transitions (as in the ASCI red and blue) to benefit from new hardware. Usually program improvement is the best way to improve performance.
You might consider Sun a losing company, I don't have that feeling. Sun certainly has it warts and we haven't done the best job in market perception. However, do you really want a world without Sun? Sun is still a powerhouse of innovation, we are open source friendly and in our core we are still geeks that enjoy technology.
I have a somewhat balanced view of this, as I work for a University and have a variety of different interactions with Solaris and Linux. What follows is a few notes on Linux vs. Solaris and Access Rights across different categories of system
Firstly, our Production MIS Systems:
Nextly, our Development MIS Systems
Thirdly, Academic Development Systems
It's one thing for a developer to maintain his own system, but when all the developers work on the same multi-user system, you don't want them all dorkin' around with it. In our environment, sysadms manage the system, there is an area for one delegated developer (and ex-sysadm) to install and manage software for the group - no sudo privs needed. The one delegated package maintainer can coordinate upgrades with others, and because he's also one of the developers, can test and support those packages much more easily than the sysadms. Yes, all the developers would definitely benefit from having their own boxen to beat on, but only a third of them do.
Read a few "Ask Slashdot" questions, and you'll understand. Ask Slashdot can almost always be summed up: I'm old and cynical, and longer surprised by clueless fanboys with an exaggerated opinion of their own experience, intelligence, and skill.
I will never understand, however, how any of them manage to find jobs.
There are two issues you're arguing here, one having little to do with the article. First let's look into the administrative issue.
I agree that there's little reason to give any user root access. Note I didn't say in a secure environment, a cluster may very well not even be on the network. In general, a user should not have root access even in such an environment unless they are the person responsible if the system goes down. Limited privileges may be given through sudo, but any program with the ability to modify system files (editors, file modification commands etc) should be blacklisted entirely from sudo. Short scripts which the user may not modify might be exempt, for instance if the user is developing a kernel module and the module needs to be inserted/removed. In a production (non development!) environment, anything that modifies such low level parts of the system is out the window. Starting or stopping a service such as Tomcat may be negotiated through sudo, but this should again be done through scripts which have been examined by an administrator and is not modifiable by the user.
When it comes to clustering, I disagree wholeheartedly. I've worked with several variants of computational systems, including many large systems, and I have to say that Sun is really not the best choice for brute force computation. They make excellent workstations and servers, but for throwing a lot of CPUs at a problem Linux on x86 hardware and its variants is simply a better platform in my experience. The programming environment needn't be as different as you suspect, and there are a variety of excellent clustering packages such as Scyld and OpenMosix which make clustering quite painless if ease of use is more important than the results. Typically the speed of obtaining results is the most important issue of course, and this is the area where Linux/x86 shines. You simply get more CPU for dollar on commodity hardware these days.
GPL: Free as in will
On the rare occasions I am not overruled by marketting I tend to prefer the following development setup.
The developers personal machine. Full access, do what you wish, if you cannot trust your people with that then do not employ them. No developer worth his salt will ruin a desktop machine. Someone who either on purpose or by accident comprimeses his machine should be fired. Almost every developer has his own preferences and methods of working, I see no reason to restrict them on their own machine. Want to test a replacement for apache? Go right ahead.
The test server is the step where the locally developed code/setups etc are being tested. Access to this machine is limited by protocol. Basically any chance will have to be documented and be reasoned(?). So a chance would have to accompenied by who did it, why they did, on whose authority and exactly what was done. The test server is NOT a development enviroment, it is a proving ground for new developments. This is sometimes very hard to explain. So in caps, "YOU DO NOT DEVELOP ON THE TEST SERVER, YOU TESTIF the test server works as expected with the new chance then this will be ported to the live server. Friday afternoon is a bad time for this but somehow always seems to be the time desired by the guys who sign the paychecks.
But in principle I see no reason to deny any developer root access to any of these machines. What needs to be in place is proper protocol to make sure that people know how to deal with chances (no point in documenting all the chances if people don't read them) and that you have good people who do not mess with machines.
I have been the victim of bad restrictions to often to have any fate in people that create them. I had to personally subvert a production webserver to handle IM traffic because the office network blocked them and our sales support staff needed it. (Case of an outside department being absorbed in the larger organisation) It tooks over 3 months for them to finally get official permission to upgrade the firewall rules. I myself was denied SSH access to the outside webservers for a full week until I told them I would simply work from home permanantly until it was fixed.
If you have good people they can be trusted with root access. If you do not have such people then they cannot be trusted with being let into the building. My first IT job had a guy who installed a keylogger. He didn't have root access, he simply had a limited account on a windows machine and downloaded some exploit kit.
But in the same job I was being outsourced to a very large dutch company and had root access on their AIX production machines. I, a new then new newbie noob had to do my development on the production machines since my desktop was to restricted to install the software needed and in any case couldn't handle the filesizes involved (good luck opening a 2gig database dumb in either word or notepad on NT4.0). One morning I was early (so I could leave early and miss the endless meetings) and was asked by the director of the company to start a database. I was the only person who could do that, if I had not started that database the entire national compnay with a hundred offices could not have started the working day. (Was a temp agency).
A stable production enviroment does not come from limiting your employees, it comes from not letting your unix admin quit in disgust and having proper training in place so your critical servers do not depend on a hired developer who is still reading his Unix for dummy's manual.
If the above sounds fancifull then be glad I did not tell the complete story. It was the most insane enviroment I ever been in. It was so bad that when the company was bought by a rival and they learned about the true state of the accounts it even made the one national newspaper. While it focusses mostly on financial issues it also reported that they found the IT department to be a total mess. Not bad for your first assignment eh?
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
The system administrators in some environments may not have access to do some of those commands listed. That function may be reserved to the "security administrator" and no I'm not kidding. I'm a system administrator by trade and I know this is coming at some point in our environment. There are a great number of details that would need to be worked out before we run down this path. I work for a fortune 500 company and we do not work with the DOD.
In our environment, we have application accounts and we have different rules for different classifications of systems. On our development environments, the developers have access to the application account(s) associate with the projects to which they work. On production environments, because of good old SOX compliance, the developers can get access to the application account in emergency situations, but the security and the change management folks will have a full keystroke access log of what the developer had done and the projects AVP is notified of the unscheduled changes. At this time we are using PassGo's UPM (this is almost identical to Power-Broker) to manage and control access and it works pretty well for interactive commands.
Much of the documented need was for SOX compliance for our financial system, however we have some critical safety systems that the same rules were applied some time ago and rightly so more rules are coming.
Good luck.
You really don't want to have any more access then you really do need. If they gave you root access and something happened to the system they will point their fingers at you faster then you could type "df -k". And it will be up to you to prove that you didn't do anything to crash the system (assuming that you didn't do it). Governments employees are Union workers and Unions really hate contractors because we tend to be more efficient then Union workers without paying the union dues, so if they could find some proof of a mistake by a contractor they will be merciless. Hopefully you are being paid by the hour so you will get paid even while you are waiting for the sysadmin to install the package.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
There should be absolutely no reason that a user should need to sudo to change permissions, copy an move files, etc. That's something that should be done explicitly by the systes administrator.
We're currently setting up a Beowulf cluster and my job is to manage the queues, setup the resource management, and tune the scheduler to optimize the performace.
I've never seen a situation where anyone has needed to change ownership of a file except for where someone departs. De rigeur, you put in a request to the admin to chmod all files under that user id to g+r and directories g+rx. That's it. Anyone in the person's department can then copy out whatever they need.
Install software? We simply provide the software with instructions, and a log of installation on another machine -- or a binary RPM -- to the admin wit a request to install it. It's not like we install applications every day. This is doubly important in a Beowulf cluster since you need to sync the software amongst the compute nodes.
No, if you find yourself wanting root access for such things, then you are doing something seriously wrong.
I run into the same problem at my job, PHP programming and some system administration. My job would be much easier if they gave me the root password. Currently, whenever I need the permissions on a file changed, I have to ask the "sysadmin" (I do more system administration that he does) to login as root and change them. Not to mention that I write some scripts and run them on live sites, I have to ask my boss to login as root on my computer so I can write the scripts.
Their reasoning behind not giving me the password is that they are very secure about who has access and who doesn't. So, the workaround I've come up with, I just leave an SSH session running with a root account logged in. They know I do that and they called that a good idea when I told them. I don't get people sometimes.
Are you telling me that you don't see the connection between government and laughing at people? - Interviewer
Thanks to everyone that replied - I expected most of the "You will never ever ever ever ever need root, if you need it you suck!!!" posts. Glad to know that /. doesn't change over the years. :)
/.ers. :) I've never touched a Microsoft certification nor book, and at one time was working on getting my CCIE. Yes - I am just as clueless as the rest of us, but I at least used Netscape .99, learned Linux using Slackware with the 1.1.5 kernel on a 486 DX 100 (that was my news server), and got a overall 98 on the RHCE exam. If none of that means anything then cool - no big. :)
:)
Ok - to the clarifications.
First, our admins are for the most part clueless. One in the entire group has a clue, and he's the new guy.
Two - I am experienced but don't consider myself anywhere near an expert. I know I don't know everything and admit to much ignorance on Linux and Solaris. I don't like to mention them, but I'm an RHCE and a SCSA for Solaris 2.5. I can't give too many details because I know our IT guys are avid
Three - our environment is one of the most broken-in-the-name-of-security-through-obscurity networks ever. They don't even allow ICMP because of their security "mindset". I can't friggin ping my own servers to check to see if they're up. Yes - I know about opening a socket blah blah blah.
Four - The distro they chose is an offshoot of Redhat (remember, I am trying not to give TOO many details - I do value my job). Most of our Linux desktops are EM64T or x86-64. The problem is when we need to use 32-bit apps. The current distro is quite shy of most 32-bit libraries. Yes - I use LD_LIBRARY_PATH. Yes - I compile / use --prefix with configure to send it to a filesystem I actually have privileges for.
So - that's my background. The main need for sudo access is for changing file permissions on stuff I don't own, is completely wrong for our current standard (wrong groups, etc), and is usually in the wrong place. I do NOT want sudo / root on a production system (the cluster, for example). I am speaking of having it on my Linux workstation and the numerical modelers' workstations. Have I written a stupid set of C++ that looked for a specific library in a specific path? Sure - but I $#%^#$ed that up years ago. The main point I was looking for and admittedly screwed up in not specifying well was do you allow developers sudo on their personal workstation? If so, do you have problems. If not, why?
Yeech - I'll have to submit that later. Good discussion though.
I'd really like to see a business report on the global cost of security measures. I.e. all the forgotten passwords, all the time wasted figuring out that some security measure was silently put in place, all the scrutinizing of systems & code, all the new wonderous fix it products, all the infestations of viri and other crud of the net, all the heavy retro fixes (ever fixed up a TCP/IP stack?)...
Seriously, I suspect the cost out weighs the cost of a breach; if not locally, then globally. The fact is that the paranoid world of security eats up an insidious amount of resources.
Unfortunately, it's a cat & mouse game where one leads to the other; i.e. a bad apple leads to more security, and more security challenges a bad apple...
So do you also implicitly agree to support anything the users want to have installed?
Our Solaris sysadmins have a better (IMO) policy on our development servers:
(1) They will install software packages on request if a few users have a legitimate need or if it's otherwise considered to be really important for a product/project.
(2) If you, as a user, want to install something nonstandard, go ahead and figure out how to do it, but you have to support it yourself. I've done this with tools like mc, DDD, vim, and a number of other utilities I consider useful.
(3) Software installed by individual users will be removed if it causes any issues.
That's for DEV boxes, though. QA and Production servers are locked down tight, and only #1 applies, but they tend to be extremely fussy about what is allowed (almost nothing).
Mainframe/UNIX Bit Twiddler and long time Windows/Linux Hobbyist.
The Theorem Theorem: If If, Then Then.
I wonder, especially as Debian and Ubuntu become more popular: How much would users' desire for sudo access go away if apt / dpkg had the ability to install software for just the current user (and thus didn't require root)?
I know that in a home environment (such as if I'm setting up my parents' computer), I'd be a lot more comfortable having them use a version of Synaptic that installed software just for the current user. That would basically eliminate the need for them to have root access at all. Maybe a similar thing holds true even for most developers.
Granted people can usually install software for themselves by compiling the source code, but to require that is to basically ignore all of the benefits that apt / Synaptic offer.
(If you're a Gentoo, I think the same point can be made by using a find/replace on the terms apt/dpkg/synaptic.)