Slashdot Mirror
How Hard is it to Manage Different Unices?
vrmlguy asks: "Where I work has several Unix-based servers, all running the same vendor's OS. We are getting ready to buy another big server, and management wants to get bids from other vendors. However, our staff is only familar with our current vendor's OS. Yes, I know that any two flavors of Unix are more alike than not, and yes, I know about the Rosetta Stone for Unix that makes it easy to transfer skills. I want to know about the down-side: What's the difference in the cost of operations between a mono-culture and a shop running two or more vendors' OSs?"
You have a team of mechanics, and for the last 20 years all they have serviced, as well as driven themselves are Ford automobiles. Now, your boss tells them to jump right in and service Chevrolet autos too. How easy will this change be? Depends on the mechanics and how they've been trained I suppose.
It has always been my opinion that if you have people who understand the concepts and underpinnings of how *nix systems work, the flavor of the OS doesn't matter. People who have a good understanding from an abstract point of view will easily pick the differences in syntax, location, etc.
"I'm just here to regulate funkiness."
Your biggest expense is going to be training, but your company will probably choose to take it out of your clients' and employees' pockets by doing it "on-the-job". Next up would be licensing fees.
Unless Vendor B is offering a competitive upgrade from Vendor A's software, it would be much cheaper to negotiate an additional Server and Client license pack from your existing vendor than to enter into a new business relationship with some new vendor. Unless, of course, the new "vendor" is (sigh) Linux.
If you fall off a building, go real limp, because maybe you'll look like a dummy and people will be like hey, free dummy
For the users, all of the familiar commands shoud work fine. But maintaining the boxen will have a cost. For instance, I know how to create a disk partition under both Linux and AIX and can say that the process is totally different. Also, you'll have to keep two different platforms up to date with the latest patches. And don't forget your apps, which probably won't have binary compatibility. You'll have to make sure that all of the apps that you wish to run are ported to your new Unix flavor of choice.
"Avoid employing unlucky people - throw half of the pile of CVs in the bin without reading them." -- David Brent
The same principle applies to natural and computer languages - the more you know, the better you understand the fundamentals.
:)
Sure, you might know how to do x,y,and z on your Solaris box, but once you understand how to do it also on RedHat and AIX, you'll understand much better how it works conceptually. Then when you get an HP box, it'll be pretty easy.
Of course, don't run killall on HP.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
It is fairly easy to transfer sills from one version of UNIX to another.
Plus, it is greate for the resume. When you get tired of this job, get fired, laid off, or transfered you will find it much easier to find another job.
Some of the differences between different versions of UNIX include:
BSD or AT&T based
Disk tools
Adminstrative interfaces and GUIs (SAM, SMIT, etc)
Startup / Shutdown scripts (rc.d vs init.d)
User management
Included tools ("top" is a big one)
Backup and recovery (hp includes fbackup / frestore)
X-Windows (CDE, VUE, etc.)
Some if the similarities include:
user land tools (ps, ls, find, etc)
Directory structures are slowing becoming the same
Generally, this is not difficult to do, as long as your admins understand the bases of UNIX. (Vendor-centric admins sometimes don't, as they get dependent on their vendor's tools.)
The problems can arise with:
1. vendor-centric admins who aren't willing to learn
2. different service contracts creating differing expectations of uptime between systems
3. added costs from maintaining multiple service contracts and training on multiple platforms
4. finger-pointing, if the systems interact
5. rewriting in-house tools which are needed on the new platform, but were not written generically before
6. 3rd party licensing costs may differ (if you are licensing the same product on both OSs)
7. dilution of expertise, since your admins will have to be more generalists (this is often overbalanced by the expansion of perspective in problem-solving that comes from broader experience)
Other than that, I can't think of anything off the top of my head which would make this hard. Generally, it is not a problem to do.
-- Two men say they're Jesus. One of them must be wrong. - Dire Straits
Sure, an environment with only one vendor's OS deployed is easier for the admins to handle. However, if a problem develops, that problem will affect EVERY SINGLE MACHINE you have. Don't lose sight of that in your zeal to minimize the admins' workloads.
Learn to spell: nickel, missile, lose, solely, amendment, speech, kernel, probably, ridiculous, deity, hierarchy, versus
Statement of Bias: I "administer" several UNIX OS versions (Solaris, IRIX, Linux, occasional HP-UX), but in an isolated network with no outside connections (so very little emphasis on security).
Two factors come to mind:
No matter how close the systems are, you will still "loose" time to training (either formal or OJT) requirements for the new system. This may actually be a benefit for your staff (wider perspective, more to put on Resume).
Depending on how much focus is placed on security, you may end up doubling the time required to track vulnerabilities and install patches. Again, this may be an advantage as well since a single-os shop tends to have equal vulnerabilities on all systems. In a multi-os shop an attacker will have to work harder to get control of everything.
It is easist to manage servers from only one vender. Unix makes ti easy to transfer skills, but here is the contradiction: It is easier to manage servers from many different venders and versions, than to manage just one server that is different from the rest.
When you have all OSes the same it is easy because everything automaticly transfers. When all are different it is harder because you always have to remember the correct incarnation of each procedure, but because they are all different you get in the habbit of looking it up each time. When all are the same except for one machine you forget on that one machine that everything is different and you aply the wrong incarnation (ofte with disasterious results, see discussions of killall linux vs hpux on comp.risks) Because of this, the one different machine will get [invalid] complaints often due to these differences.
If you can't stick with one vender, then you should go with many so you are in the habbit of checking the differences. At the very least get some linux (debian, redhat, suse), and bsd (free, open, net) machines in house now, and use them for production. You need to make sure that your admins are used to subtile differences. The other alternative is to just stick with one vender, but not only do you pay more, but your admins become lower quality as they learn only one system. (think of it is a resume builder, you want different systems on your resume!)
It seems to me that the biggest cost is in sysadmin time. I figure it this way, at work I use a few UNIX systems. We have one machine running IRIX, a couple running BSD and one running Linux. Now, when I write a script one one of the BSD machines, it works on all of them, but it may or may not work on the Linux machine, and certainly won't work on the System V-esque IRIX machine.
Now, if your sysadmins employ a lot of scripts, figure you'll have to spend twice the time maintaining them if you have two different platforms that are not fully compatible. You can minimize this if you stick to POSIXly correct scripting, but you'll never completely eliminate it.
The same goes for custom programming. For instance, if you're running everything on BSD, and you want to take on a Sun machine running solaris, there may be some issues with the occasional socket call that Sun implments differently from the rest of the world.
So, the more custom scripting/custom apps you have, the more time your sysadmins will have to spend maintaining/porting/testing the stuff.
---
Play Six Pack Man. I
The short term costs to retrain staff for the new system will be higher but the long term benefits will definitely outweigh them. Once you build a multi-OS capable IT department the cost to add new hardware later on becomes significantly less. By not being locked into one OS (and one vendor) you have the freedom and flexibility to choose the best solutions for future problems (as well as hunt for the lowest cost). The smart thing to do is diversify your IT shop as much as possible so that you can insure you always have the right tool for the right job. No single vendor or OS can provide all the answers, regardless of what IBM/Sun/Microsoft may try to tell you.
Just keep text-file logbooks as you learn new things about the different UNIX implementation. Keep them in a hierarchical database on an NFS file system or web server somewhere, name the directories and files consistently by OS and topic (topics such as DNS, network booting, firmware, SCSI naming conventions, package management, etc.).
I do this at home to juggle Solaris, OpenBSD, and Linux and it works well. If I forget how to setup DNS under Solaris, I just go to <base_dir>/Solaris/8/DNS_Setup.txt, for example.
Also, install all of the on-line documentation you can and have it network-accessible. For example, when the man pages aren't detailed enough, on-line Solaris Answerbooks can save the day.
Also, keep well-organized bookmark lists for useful websites, such as http://docs.sun.com or http://sunsolve.sun.com, that cover your particular UNIX.
Having any number of UNIX implementations really isn't unmanagable (unless they have broken network protocol implementations). The key to success is documentation and more documentation (and unambiguously sharp sysadmins). On that note, if you don't have faith in your system and network administrators, you should just give up and stick with one OS, since no amount of documentation helps a truly stupid human.
Healthcare article at Kuro5hin
Money.
You obviously already know that managing support contracts from multiple vendors is going to suck. I would also recommend taking a long hard look at ongoing support charges.
For example, we have both HP/UX and Sun platforms where I work. We have both servers and workstations. For the workstation support contracts on similarly sized machines, there was a world of difference in cost.
The annual fee for an HP C240 workstation was somewhere between $2500 and $3000. The same annual charge for a Sun Ultra of equal speed, was between $1000 and $1500. Multiply that by the number of workstaions you have to maintain, and it can add up very quickly.
The up front cost typically isn't where they get you. It's on the back end. I would research the back end on all of the platforms you are considering very carefully before making any final decisions.
Hope that helps a little.
Most companies I have worked at or know people at go with a third party backup solution such as the ones from Tivoli or Veritas.
Makes your backup/recovery fairly consistent across different products, plus everything can then be managed from a central console.
all the good sysadmins are probably working and not monitoring these posts
Hire better admins. They are out there, and a lot of them are unemployed right now. Any problem in an organization that persists past a few days or at most a few weeks is a management problem.
-- Two men say they're Jesus. One of them must be wrong. - Dire Straits
But not as hard as you'd imagine. I use AIX, Solaris, and HP-UX on a weekly basis, as well as MacOS, MacOSX, and w2k (workstation and server).
/opt/apps/local
The biggest problem with a mixed environment is keeping it up to date with patches etc. Keeping track of that stuff is a complete PITA; I can't imagine how much more difficult it is in Linux, where the patches aren't on the vendor site (are they?).
Besides that, the big thing that you'll need to do is make sure everything is sort of in the same directory structure. For apps that you install, put them in the -exact- same directory. For example, all my unix boxes have the same layout:
/opt/apps
/opt/servers
/opt/data
/opt/src
/ usr/local ->
That way, it doesn't matter as much which box I'm on, and I don't have to remember exceptional cases. It also makes maintenance easier, because all the exciting stuff (non base operating system) is in a known structure. That means you can write scripts, etc to monitor everything and you don't have to change them on a per-host basis. It also means you can just copy the config.status from box to box (or directory to directory) and build without reconfiguring everything.
'Luck!
Not all of us are dead. Some, like me, are unemployed and looking for our next job. Given your description, I'm sure I could do better than two or three of your existing admins. Where are you located?
all the good sysadmins are probably working and not monitoring these posts
Dead wrong... All the good sysadmins have automated their jobs and have all day to surf.
Well, there's a few of us old dinosaurs out there... but the companies look to hire "certified" admins who work cheap.
Good old seat of the pants generalists often are overlooked in favor of the latest Whiz-Bang rookies straight out of the memorize for the certification test prep school.
I was a trainer doing sysadmin training for one of the big iron multiprocessor Unix boxes -- and in '93 you could see the beginning of the end as folks who were basically operators became sysadmins.
Bill Pechter
First thing I do when I start administering Unices is to install GNU utilities. At least this will lessen your headache.