Slashdot Mirror
Using Debian in Commercial Environments?
sydb asks: "I am currently persuading my employer to try out Linux. We are heavily dependent on IBM software technologies just now, and it's a very conservative operations organization. As a challenge, I am trying to persuade them to use my preferred distro but there are hurdles: IBM doesn't officially support Debian as a platform, though I have anecdotal evidence that most of it can be persuaded to work (with alien etc). Does Slashdot have experience shoe-horning Debian into this kind of scenario? Most importantly, how have things gone getting IBM support? My rationale for pushing Debian boils down to its vast array of packages available to apt-get, easy upgrades, apt-get itself, and the overall quality and consistency of the system."
In general, you're buying IBM software because you can call them up, tell them "it don't work, nosirree" and your contract says they get to send out some engineer(s) and make it work.
If they support your environment.
The gains you might think you'll get by using Debian are absolutely not worth losing your service contract, which you've likely already paid for. There's nothing horribly wrong with SuSE or Redhat, both generally supported IBM environments. If you succeed in getting your boss to install Debian, you're on the process of going up a river without the proverbial paddle.
MORTAR COMBAT!
You want to put Debian on the systems because of the vast array of software available for it.
They want to run IBM solutions because they can trust that the few apps that they actually want to run on the system will run with no trouble.
The trouble here is that you want Debian on the systems for your own selfish reasons. They want to run their systems as reliably as possible. Since this is a business and not a college dorm room, the business case will always win out.
Debian is a fine distribution. But no company in their right mind would go through a migration just so you can install the latest and greatest software via apt-get. You see, they've already got the software they need running on the system.
you focus on whats best for your company and ultimately your client by using the right tool for the right job instead of trying to hammer the proverbial square peg (Debian) into the round hole? Sorry to not really answer your question but hobbies and personal preference shouldnt take the place of a better solution (e.g. whatever distro of Linux IBM prefers for their hardware)
Ok, I love Linux. I use it at work. I work in a really big, international company.
..
Here's my take .
If it's not supported/approved by IBM and you are dealing with IBM then find out what they support and use that.
Why?
Because 1) it's easier, and 2) you want to succeed.
Your job is not to move the organization. Your job is to make your boss look good. IBM is very very talented at making their customers look good at very reasonable prices. You will make your boss look better with IBM's willing help than by trying to fly it yourself.
Apt-get is nice and all, but frankly, support is nicer. If you don't understand that, btw, then you are not experienced enough to be making the decission on what to move forward with. I'm not saying this to be an ass . . . but simply because it's true. Moving them to Linux is smart, but moving them to something the hardware vendor doesn't support is stupid
Everybody get your fire-retardant suits on for the ensuing flamewar...
The core differences between distros are package management, the version of the kernel, and the version of libc. Debian might work fine for what you want it to do, but a subtle problem might occur that you didn't catch during testing, due to a version difference. I've found that shoehorning, as you mentioned, is generally a bad idea. Shoehorn too much, and your feet will hurt.
Given your conservative environment, I think RedHat's Enterprise Linux product line is more appropriate. RedHat can sell you a commercial support contract, and they promise software updates for 5 years. Also, future Linux admins are more likely to be familiar with RedHat, which avoids needing to learn Debian's quirks. Also, IBM or other commercial software (like Oracle) is more likely to be supported on RedHat.
make a sandbox running Deb on your network to start showing them what it can do. this is what I did at my work, and it worked. Currently CVS and the Build machine are running my Linux distro of choice; Gentoo, for mainly the same reasons you mention.
RHCE's aren't going to do what we can do with *our* distro's, it's more than just LInux to us.
CB
free ipod and free gmail!
So, you need to ask these questions to yourself and your co-workers:
If you have a stable working enviroment, why change?
Is this move going to be cost effective?
Is the distro I use going to be the proper one?
Why am I really using this distro? If you say, because it is the one I use at home, then you need stop this project right in its tracks.
How easy is it to manage this distro in my enviroment. Running "apt-get upgrade" on 500 servers is not do-able.
Is there proper management software out there for my distro/platform of choice?
Does my software I need even run on my distro/platform of choice?
What about support for my software on my distro/platform of choice?
Can I keep my system software in sync across all servers?
Can I easily manage the distro install process?
Can I trim down the install time?
Can I make the install process automated?
These are just the basic questions you need ask. Don't get stuck on one distro. Be flexable and look around. Redhat or Gentoo or something might be better choices.
Linux O Muerte!
The fact that you're talking about "shoehorning" Debian in, using "anecdotal evidence that most of it can be persuaded to work" should answer your question.
This isn't a PHB issue, either. Anyone with a real production system should be scared off by language like that.
You could not be more wrong. Production environments require stability, period - not the latest glitz. Most Sun shops are using Solaris 8 - it's ancient. Most windows shops are using Windows 2000. Conservative and stable is debian's strength. The reason Red Hat is such a mess is because they keep changing shit, and the wrong shit into the bargain. Worse yet, they put "enhancements" and bug fixes into up2date, which is in my opinion a big no no. I've had up2date break systems more often than update them. You point was?
They have in the past been comfortable with their software environment but now money no longer flows like water. They are now realising that proprietary Unix comes at a cost, and it's becoming hard to justify.
Whatsmore, the overhead of a highly regimented IT operations organisation is more and more apparent. There is a balance to be struck between every technology meeting the corporate checklist, rubber stamped by all and sundry, sticking to the tried and test, and actually being able to implement change quickly enough to keep up with business realities.
Please don't answer my question so tritely. I think it is a reasonable one.
Yours Sincerely, Michael.
"sydb". What are you thinking? Seriously?
You have a working system. What is your rationale for wanting to change ANYTHING, much less your OS?
You've paid (if my own workplace is any indicator) at least tens of thousands of dollars just for the IBM support (which is superb, if you're running approved software).
You probably are using other software, all of which you've paid support contracts on.
All these contracts will become null and void if you should do something completely insane, like switching your DE to a distro that is not supported.
Well, go for it, it's your career. I'll say this, however. If you were employed at my workplace, and suggested such an insane course of action, you wouldn't be working here for long.
Boycott everything - they're all trying to fuck you one way or another
Interesting, you've gone from
IBM doesn't officially support Debian as a platform, though I have anecdotal evidence that most of it can be persuaded to work (with alien etc).
to
It's not about choosing the right technical solution, because I have ample justification for Debian being the right technical solution.
So, your anecdotal evidence is now ample justification? I would say Mike (great-grandparent post) is right on the mark with his comments.
I work in a Windows shop. Well, okay, we have a whole IBM AIX side of the company that runs the Peoplesoft stuff, but for all the rest of the company it's Windows. We tie peoplesoft and pretty much everything else you can think of into Active Directory. It works.
But there's places where I can see Linux boxes excelling where other software falls short. One of them is our Spam "solution." It was very expensive and it doesn't work for shit. 80% accuracy, maybe. Lots of false positives. In 2002, it was really cool shit. But that's the problem - things change fast when it comes to certain things like Spam and when you pay $50,000 for a license to filter spam you don't want to upgrade or change softwares every six months.
Enter OSS - My (*gasp*) spamassassin+dspam+amavisd-new is easily doing 99.99% of the spam with extremely low occurances of false positives. Is it supported? Nope. Wait, yes it is. I SUPPORT IT.
Some companies are all about support, support, support. They don't trust their IT staff, they consider them expendable. I don't work at a company like that. They put weight in our abilities. If you can make a good case for an OSS solution, one where you can support it yourself and train others, it will be seriously considered. Apparently there's other companies like this too, since a lot of places are running Linux now and not all of them use RedHat Enterprise.
- It's not the Macs I hate. It's Digg users. -
I have to comment on this...
:)
There are plenty of good business reasons to want to use Debian... the very same reasons you or I use it.
1) Security updates are done quicker than recompiling/manually installing (The competition is catching up).
2) Software installation to a new machine will take less time on a Debian system because it will update to the latest versions automatically instead of applying patches over the original install (competition is catching up).
3) More software packages prepackaged means that there are fewer custom compilations/installs, which means installing/upgrading client machines will take less time.
4) Setting up your own APT server means you can distribute your own upgrades internally, and even package non-standard software yourself. This means you can write one install/setup/upgrade script for oracle, and have it automatically propogate through the network instead of installing it on a per machine basis.
Every one of these points saves time. If a company is under pressure right now to save money, applying some of that presure on IBM might be a good way to get the ball rolling toward getting support for Debian. IBM only supports SuSE and RedHat because that's what everyone else uses. There is enough room in the market for another supported distro, especially one as easy to support as Debian.
I wouldn't sacrifice support, because that would put your job on the line, but I would lobby them to ask IBM to support Debian. If enough people in your position do, they'll add it to the supported list. You might want to have them run a test on the next server upgrade/install by installing Debian on it. If that means that IBM doesn't get service fees for that server, and you tell them so, then they'll start paying attention. You're company can always switch a single, not-so-critical system to a supported platform at any time without a significant loss. You just have to convince them that the potential economical gains are significant enough. If that server sits in the corner doing it's job without anyone touching, they'll start to see the wisdom. If you suggest something like a single server as a test bed, they'll see it as more of an experiment to try to save money, and if it fails, it probably won't be your job, but if it succeeds, and you implement it company wide and save a lot of money, then you will probably have eliminated a need for your job, and your boss will get a raise from the portion of your no longer needed salary.
Karma Clown
You might be able to convince them based on the licensing and service costs. Try making it a business case, exposing how much would it cost to have inhouse support for Debian vs Novell support for Suse. Be realistic, don't be quick at dismissing the costs of inhouse support for Debian. If you can, get some of the folks at IBM to back the feasability of the case, telling that, though unsupported, they dont forsee any trouble.
Depending on how critical the production end of your environment, you might be able to pull it off. Always bear in mind if for any reason the tested scripts will not run on the production end, the excrement will be flying your way. This decision might come to haunt you later if you keep your current employer.
My other OS is the MCP!
Um, no, Not even a little bit. It doesn't matter if you think Debian is the greatest thing in the world, or something you found at the bottom of your garbage can, there's one key difference.
Imagine some updated package broke all your applications. And your quarterly statements are due tomorrow. And the CEO is touring your server farm. And the planets are aligned infavorably. And it's Friday the 13th. Let me show two different scenarios:
And the alternative:
There is no sig, there is only Zuul.
apt-get install kernel-image-2.4.18-1-586tsc 2.2 is the default, but specifying bf24 on the boot prompt will install 2.4. And yes, still using the stable tree. Debian isn't *THAT* out of date, I use stable on my servers, and testing on my workstations.
Bored? Why not join a decent mess
A few things I've learned from deploying Linux in the enterprise (some of these may not apply to everyone, depending on how large the enterprise is)...
1) An installation needs to be supportable. This does not mean that you can get tech questions answered quickly via IRC or mailing lists. This may or may not mean the availablity of a hotline to call when everything hits the fan, and you are loosing big bucks every hour. It most definetly means that you can get install third-party software, and when that software hickups, you can call the vendor and have them help you, and not tell you they don't support your installation choice. Support also includes an assurance that someone has a _financial incentive_ to provide timely security updates and bug fixes for the product.
2) An installation needs to be repeatable. Which means that installing a distro that doesn't baseline their releases won't cut it. What I mean is, some distributions come out with a version, say 11.2, and will put out a series of fixes in the form of a couple updated package files every week or so. Thus, if you set up a server today with versin 11.2 and all current fixes, then next week if you do the same thing you will get a slightly different install. So what is needed is for the distro to have the concept of maintance levels, or patch levels, which defines a line in the sand so that you can at any time install 11.2 patch-level 13 and it will always be the same. (This also makes it easier for patches to be reviewed and signed off on by your patch-review board).
3) An installation needs to have a good chance of being maintanable by someone off the street. There are more enterprise-class unix admins out there than enterprise linux admins (that is, at least 5 years experience supporting a minimum of 50 systems that are in use 24x7 with stirct uptime requirements). And since most enterprises and their vendors are going with one or two linux flavors, a shop has a better chance of getting an admin in a pinch if they go with one of those two major linux players. And just knowing how to troubleshoot and upkeep linux in general isn't enough for a production system. Any linux distro has it's particulars that you don't want someine learning about during a crisis.
Unfortunately, most distributions fail one or more of these tests (or other tests that I didn't mention). For example, with Redhat Enterprise, their only supported methods of updating are to use up2date, which grabs the latest patches for all installed packages (which means you can't baseline), or you have to grab the patches one-by-one. If you download their update CD's, they don't provide an easy way to apply all the fixes (rpm --freshen doesn't cut it, cause sometimes you run across a patch that has prerequisites that the previous version didn't have, and rpm doesn't automatically resolve dependancies. Of course, there is always autorpm, autoupdate, apt, and yum, but these aren't part of the base distro, so you aren't guaranteed of it always working with that distro).
Conversley, if there's a bug in the default xfs setup in the default redhat kernel, IBM calls up redhat and says "fix it" and redhat says "sir yes sir I love you sir would you like coffe with that".
It doesnt get thrown onto some mailing list, argued about for a few days, crammed into somebodys bugzilla or wiki, opened and closed three times, moved catagories, sit through a developer moving appartments, ignored by an irc channel with 60 idling people, dissapear into usenet, etc.
99% of someone saying they "offer support" is just the fact they they have the balls to say "we're so sure this works we're prepared to accept the dent supporting it will make in our budget". For instance with redhat, the very fact that nearly all their customers can file a support request with them now, means that if they didn't have a damn good product, they would lose all their money to support costs. Plus, when there are genuine fixes to be made, they can use their margins to hire full time programs to fix exactly what their customers need fixed pronto... not when some package maintainer gets around to it. You'll notice this is why a metric fuckton of open source projects have @redhat.com email accounts on their credits page. You'll also notice that redhat's commitment to the GPL is near debian like, they even buy other software products and gpl them. When you're paying redhat to support your linux, you're actually in a large part paying them to improve linux to a point where it needs less support.
I didn't mean to turn this into redhat praising, but merely to counter the insane, annoying, and far far to prevalant attitude around here that redhat is "screwing" anybody with their pay model or "turning their backs on the community". If anything paying for redhat is the easiest way I can think of to support linux development (especially the kernel).
As a linux 'advocate' working in a large IBM customer (top 20), I feel your pain. However, give up on debian.
Seriously. If you try to run this stuff on anything other than an IBM-supported distro will start to refuse your support calls, charge extra for incidents and basically make pricks of themselves.
Your best bet is either:If you're already paying for DB2, Websphere *and* tivoli, you're looking at a few million a year. What does redhat cost, ~1k, just pay it. From there you can advocate JBoss/Tomcat instead of websphere, Postgresql instead of DB2 etc. etc.
If you run IBM stuff on another distribution, who do you think will be up against the wall when your fixed price call out suddenly becomes a ~$1k/hr (lab rates) fix?
Cheers Koz
I also worked at an ISP that ran its mail system on IRIX and migrated it to Debian, and our experience was nothing at all like yours. While I no longer work there, they are still running Debian and to the best of my knowledge (which is pretty good; I still keep in touch), they have delivered 100% mail system uptime since migrating to Debian, something we were not able to do with our SGI boxen.
Partly on the basis of that experience, I moved from running RH on my workstation to running Debian, and I've never been sorry about that, either.
Our migration from IRIX to Debian was a complete success because of two things:
1) We had, collectively, a lot of talent on Linux;
2) The sysadmin put in charge of the project had a lot of talent and experience on Debian; the rest of us had most of our experience in Solaris, BSD, and Red Hat. The IRIX guy had moved to another department by then.
What was the difference? Not lack of talent, I think. It sounds like you know what you are doing. Perhaps a matter of choosing appropriate hardware, though. We didn't screw around with ATA RAID (this was in the pre-SATA days, but that wouldn't have mattered) or anything that was less than server grade. This was a mission-critical system, and we used only server-grade hardware that was known to be very well supported.
The hosts we used were six dual-CPU rackmount cases running SCSI disks (RAID 1) for the OS install, and all the important stuff was on SAN (RAID 5 there).
Everything was absolutely bulletproof. How bulletproof? We installed Woody, with the 2.2 kernel (this was the late 1990s, and 2.4 was still experiencing some growing pains) and it worked perfectly right out of the box.
As I noted at the top, they are still at 100% mail system uptime to this day, to the best of my (fairly good) knowledge. They are still running Debian Stable.
Many other people can tell you stories just like this. Debian most certainly has a place in a shop that needs to get things done, a place that can perhaps only be taken by FreeBSD (with the possible exception of Slackware, Debian Stable is the only Linux distro I've ever used that can match FreeBSD for stability, or at least come very close).
I'm not saying you don't know what you're doing, I'm sure you do. You're probably a better sysadmin than I am. However, I do see one thing that you did wrong. You chose (or perhaps the customer's budget chose for you) what some people would call "toy hardware." Debian Stable often isn't the best fit on the block with that stuff. But if you had been using a proper server box with SCSI (or at the least parallel ATA; I *still* don't like SATA support under Linux much), I think it would have been all right.
One other thing I would have done differently is this: as soon as I found that I had problems with the hardware and the distro I had chosen, one or the other would have been jettisoned. For a server application, it would have been the hardware if I had the latitude to make that decision. Even today, a server you need to depend on should use SCSI disks (I'm still partial to Adaptec adapters) and known top-quality parts.
With all due respect, while building an identical machine in your lab was the smart way to do it, investing hundreds of hours into making Debian work with that hardware was not. It would have been cheaper to *buy* a proper box and just *give* it to the customer. Alternatively, if that hardware was cast in concrete, early on you should have chosen a different distro, one that is focused on a single hardware platform and that places more emphasis on supporting the bleeding edge than on rock-solid stability for tried and true equipment. Debian is not that distro (not to say it doesn't work fine on most stuff; I install Debian Sid on Frys' sale-quality hardware regularly without incident).
I would advance the idea that Frys sale-quality hardware (such as SATA-RAID) has no place in a shop that needs to get things done. You probably won't ex
" Yeah, apt-get, but then what happens when apt-get goes wrong and there's no support contract?"
First of all if you are using stable (and a corporation should) the chances of apt get going wrong are just about null.
Secondly you could buy a support contract. Just like you could buy a support contract from MS.
Finally this is OSS. You can get support even though you didn't buy it. The debian community is especially clueful and helpful. Chances are you'll get better support for free then the first or second level droid at your other company. In most cases you should solve your problem in less time then it would take to escalate it with MS.
"new one has no support contract, and the new one goes wrong, it's all YOUR FAULT. If you use the existing system, with a support contract, and it goes wrong, it's the fault of the contractors, or whoever installed it, not you."
Maybe where you work you can simply say "it's Microsoft's fault" and go home. Not where I work. Your ass is on the line when the server goes down. No ifs, ands or buts about it.
evil is as evil does
Either you live in some alternate universe in which vendors work on bugs for individual users, or you've been smoking some exceptionally strong weed. Or, possibly, you don't have a clue.
I don't believe in alternate universes.