Slashdot Mirror
Ask Slashdot: Paying For Linux Support vs. Rolling Your Own?
schmaustech writes:
A lot of businesses pay for Linux support. But at what point does that stop being worth the money? When would a company be better served by setting up their own internal support? When does it make sense for them to write their own patches, which could be submitted back to the community? The inherit risk is that the organization is accountable and accepts the risks if a major bug is encountered within any of the open source applications they are using. What's your perspective on this, and how many major corporations are taking this approach?
I work with clients ranging from small business to Fortune 10 companies. On the SMB side most do support their own, though they rarely write patches. I don't know a single large enterprise using Linux that doesn't pay RedHat or whoever for support though. There are many reasons for that. SLAs are easier to hold a third party to than an internal organization. It makes the C level people feel better to have a company they are paying accountable for support. They do not have to carry the burden of the extra staff needed (that's a big one). The list goes on.
I work in pretty homogenous networking environment: Linux, BSD, Windows, Mac. I NEVER, EVER call for support, even if I work for a place that has paid for it. Why you ask? Pillars of intransigence is why. All vendors, to a man, do their best to give you the least. I've found that by understanding the problem, using Google, and pinging others in the shop, we can usually figure out the most complex stuff within a day or two. Linux support is for shops with people who don't really understand Linux at all. Linux networks, even machines running mostly Linux, is usually pretty easy to sort out with a little understanding of the issues. CentOS is popular largely because people can sort it out without paying the exorbitant costs associated with Red Hat support. I've used Red Hat support in the past. I don't plan on ever calling them again.
It really doesn't make sense for large organizations who are supporting mission critical apps. There probably aren't any managers on the planet who will willingly make the decision to support it themselves because one critical issue and it's their job. Instead, they'd much rather have a 3rd party to strangle if and when they have a critical issue
"The inherit risk is that the organization is accountable and accepts the risks if a major bug is encountered within any of the open source applications they are using. "
There's no liability for patches (except for intentionally malicious ones) to an open source application. If there were, nobody would submit one.
"Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
Run the numbers for your particular business, dumb dumb, and stop asking for vague, qualitative, subjective responses to validate your biases.
Take how much it costs for external support and compare it with hiring someone or more depending on much support you need. IOW, one guy cannot work 24/7; so if you need that kind of support, you'll obviously need to hire more people.
Also, consider where your business is. If you're in a large metro area, Linux people will be easy to get. Some little town in a fly-over state, well you'd be better off outsourcing support.
Seriously? Who TF is editing this?
Mod points: Guaranteed to remove your sense of humor.
Side effects may include gullibility and temporary retardation
To maintain and support an entire OS takes a lot of work. We aren't talking about just development here, but checking to make sure things run properly and making the changes needed to ensure stuff is supporter. The point I would start looking at rolling your own distribution and supporting it is the day you decide to start selling your distribution.
For internal use, sure you might have to have a team to do internal work to modify certain sections in order to make the OS work for you, but they are relatively minor compared to ensuring an entire distribution works as needed. Let another company do the heavy lifting and just have your company modify it and submit changes back through the system as desired. Feedback works as well.
To run an entire distribution and all the subsystems takes billions, look at IBM donating to Linux as a whole they give value back to the community rather than trying to extend and embrace for their own purposes. Redhat does the same and they do distribution and sales. Other companies are the same. I guess you can make the decision on your own but personally I suppose the time to switch is when you have support fees in excess of what it would cost to maintain an entire distribution. I'd assume someone around a thousand people focused on the project would be about right. A thousand people's salaries would buy a lot of support. A better idea might be to hire developers for the subsections of the OS that you need and have them work with the community.
/* TODO: Spawn child process, interest child in technology, have child write a new sig */
The college I work for (not in IT but in academic technology) has a support contract with IBM for Linux on the p-series boxes that have replaced the mainframe and zVM. Needed too due to some network issues.... Of course, since there was a support contract for the mainframe, not much has changed as far as the bean counters are concerned.... Note that while they use RedHat on some x86/amd64 boxes, they don't have a special support contract for those - just for the "new mainframe replacement systems".
Don't blame me, I voted for Kodos
For IBM, yes. They do, but only for the parts they support. They buy support for the rest.
I work for a software manufacturer. We internally support the libs that we ship with our products. Usually this just means getting the latest source drops and building them. When there's been a problem we could solve it in our code, but being able to debug into the lib code was the only way to find it.
For major system components, and packages that aren't part of our product, we buy support.
For the average company, there's no way it's cost effective.
Design and manufacture your own servers too!
The only issue is that certain employees have the knowledge and should something happen to them, the business may be in trouble. By paying for support, you place the burden on an entity, which is responsible. It all comes down to question for CIO/etc if they should lose an employee, how will that impact the business.
A lot of businesses pay for Linux support. But at what point does that stop being worth the money? When would a company be better served by setting up their own internal support? When does it make sense for them to write their own patches, which could be submitted back to the community?
The core competence of most businesses does not lie in the internals of an operating system.
It can make perfect sense as well to "outsource" clerical work to Microsoft and Office 365, accounting to specialists in corporate accounting, and so on.
Contributions to open source that build on a deep investment in what you are really, really, good at, and perhaps better at than anyone else in the world, are far more likely to be enduring and influential.
Open Source
skilled staff in this area are expensive, it is almost never cost effective to do your own support. The exceptions maybe if you have some very niche requirements that make paid support difficult, but for the average business small or even large enterprise it doesn't make sense, a support contract is far cheaper than the staff needed to do it yourself.
Have you lost your vulcan mind?
The best thing you can do for your company is build off and rely on others who have a broad range of support and can focus on the given area. Otherwise your just going to be wasting resources and getting poorer results.
I think it also highly depends on the companies for which your working with and the skills / connections / etc of those companies. There are certainly things that even little companies can bring to the table. I work for a large company who could go out and test every piece of hardware that exists, but we found that we're just not that adept at it compared to another smaller company which specializes in it. We understand the general issue and why that company is successful, but we're not going to hire a few people to work full time on making those connections, certifying hardware, and stocking hardware we may or may not utilize now or 10 years from now. When I need a parallel port card for a 1990's computer that runs mission critical software I need it now. How often does that happen though? It's best to go to the company which is experienced in this department, stocking those components, etc. Right now that's ThinkPenguin in this area. They're the ones with the connections to the developers in the hardware arena and upstream companies designing chipsets. They may be small, but they know who to talk to, they've done the talking, and they'll tell us exactly what works and what doesn't.
I'd say the same is true in other arenas. For example in the desktop arena we go to Canonical for desktop support. We go to Redhat for server support. We don't mess about rolling our own distributions when there are entire organizations focused on particular areas that we know we'll never be able to compete with (and wouldn't want to, because thats NOT our area of expertise- it's not our business). We DO have our own internal Linux support, but at the end of the day we go to the companies and entities that are specialized in a given area and/or build off those companies products when we don't have the answers. The sooner you recognize you can't be an expert in everything the sooner you'll be equipped to run things.
Money has little to do with it as in a large corporation the resources spent will be insignificant short of doing everything yourself. If your doing everything yourself, your doing it wrong. You don't understand the benefits of free software. Free software enables you to work with others and/or fix problems that aren't important to other entities. Maybe you need a bug fixed for a 1990's error device. If it's free software you can fix it yourself. However if you don't have to because you've been using another company which focuses on providing hardware support then why would you?
The kernel is a pretty complex beast. You think in a staff of five or six people you're going to have someone with the skills to fix anything from the VM system, the VFS layer, networking, a dozen different file systems, dozens of device drivers, the scheduler, etc., etc.
Now consider user land –apache, ssh, virtualization, your storage and backup solutions, etc., etc.
And these five or six people are going to have the connections to the developers? Either to get help from or to help get the fix into upstream? (Or do you really want to carry your own fixes for the bugs you've fixed in perpetuity?)
> ... patches, which _could_ be submitted back to the community
Depending on the license and how you deploy the software that could be _must_ be. YMMV.
If the burdened labor rate in your area is like mine, five people cost over $1M per year. Even in the less expensive places in the world, e.g. eastern Europe, India, and South America, experienced software developers aren't cheap. Support licenses could be bargain.
Companies like SuSE, Canonical, and Red Hat employ many of the people who are the primary developers of the software you're using; paying for a support license helps pay their wages. There are a lot of reasons why that's a good thing, I won't belabor the point other than to say that nobody really approves of the leeches who are making money using someone else's software without, you know, actually contributing something back in some form. It's called quid pro quo. The easiest and most ethical thing you can do probably is to pay for support. It's just the right thing to do.
The inherit risk
The inherent risk
Inheritance is the sincerest form of nepotism.
If you're capable of rolling and maintaining your own OS and the apps that you use, why the hell aren't you selling them?
In other words, no you don't write your own fucking patches.
Scientific Linux is partially the "physicist employment act". From most of the public information, most of what is done appears to be just take the RedHat sources, recompile, and add in a few packages. One could reduce the cost (and the employment of physicists?) by just adding the packages to a CentOS base.
Linux support means different things to different people. Some (significant) part of paying support to a large enterprise linux vendor goes not only into addressing operational issues, but goes into the support of Linux itself, including support for new hardware and features. You are, in essence, paying it forward. Some will look at that as a valuable investment. Others, not so much.
1) Many hardware vendors, especially storage, will give you trouble or refuse to support their gear if you're running an OS they don't bless
2) It can make business continuity easier; you'll find more people who claim RHEL/CentOS support than Debian (though Ubuntu is pretty wide-spread these days).
The rest are pretty minor. Some C level tools want to feel like they're magically protected by vendor support.
The reality is I've pretty much always had to do my own diagnostics, RCA and come up with a resolution. If there's no patch in a RHEL blessed kernel but there is in mainline, I'd just as soon pull it in and patch it myself, which is way easier to do IMO using something like Debian, or write a patch myself if necessary.
So I'd say if you're having to deal with a lot of hardware support it might be worth it just to get less pushback but if you're mostly cloud based or just have a few machines go with whatever you think rocks (unless you suck, but you would never realize that :P)
At one shop we all wanted to use Debian but the vendor only "supported" suse/rhel etc.. so we went so far as to modify uname and whatnot to pretend we were a redhat shop ;)
I don't think your perspective on this is right. There is no hard cut-off between fully depending on support an doing everything yourself. At the very least you will need someone to talk with support. You will always have someone who is acting as an administrator and who will solve problems. Sooner or later you are going to run into problems that can be fixed without support. If you don't want to keep fixing the same problem over and over you are better of sending the fix upstream. With or without the help of payed support.
You're asking the wrong question. The question is "what is our business?".
If it's not your business, you hire experts to take care of it.
My guess is "producing a Linux distribution" isn't in your business model.
I do not fail; I succeed at finding out what does not work.
It's no different. If being a sysadmin, etc. is part of your business's core competency, then by all means hire some folks and do it in house. If not, then contract it out.
plumbing and electrical work is the epitome of open-source.. there's no secret knowledge, it takes some training to be good at it, although most folks are capable of it, just like software development.
Some small firms I've worked at have done their own plumbing and electrical work, some have not, depending mostly on the scope of the work and the competency of the people at the business. Virtually all of the large firms I've worked out contract it out.
with in house Sysadmin/support work, you have to ask "are the people you hired to do software development on your product best utilized doing that, or utilized doing sysadmin/support work".
I have managed Linux servers in a professional environment for 20 years and for "production" environments I highly recommend paying for support.
I have had the good fortune of getting a tech on the phone to handle issues that were solved by the people that know the software the best. Red Hat and SUSE both employ top notch support folks. I have not experienced any others, I would expect they are also quite good. Talking to another tech on the phone that you can trust is totally worth it to you and the business that you are working for.
Contrast that with hoping that someone has had the problem you are trying to solve and has solved it and has posted to a forum you search is just not the route to go. In my opinion, you should not allow a business to make this choice. It is insurance and not a guarantee of success but that is why the business pays for it and other types of insurance.
Also these paid support contracts help fund the open source world and are, again in my opinion, the responsibility of a business using open source software.
How else are junior sysadmins going to learn how to do this stuff? A trade school? Please.
"The inherit (sic, seriously editors?) risk is that the organization is accountable and accepts the risks if a major bug is encountered within any of the open source applications they are using."
I'm always surprised by this being raised as a contrast to proprietary software. As pretty much anyone who has ever relied on proprietary software for their business and had that threatened by a major bug will tell you, there's no magical protection brought about by the license agreement being closed source. I've created massive complex workarounds for bugs in software we were paying tens of thousands of dollars while waiting literally years for the vendor to acknowledge the issue, let alone fix it. I won't call out my specific employers or vendors, but I can't help but assume a lack of experience on the part of the writer when I read something like that.
In my experienced opinion the best scenario to stake your business on is open software with strong commercial backing. That way when something goes wrong, you've got a third party with incentive to help you, a community of eyes, and access to fix it yourself.
Open source or not, at what point is it more profitable for a company to have an internal IT department instead of having contractors or service companies? Open source is no different, except that when the source is open, you can get down to the fundamentals of the software and fix fundamental problems yourself. You can't do that with non-open software. Closed source software is like buying a car with the hood welded shut. You might not know how it works or how to fix it yourself, but you can hire someone who does. But not with closed software. Decompiling and reverse engineering is much harder than just cutting a weld on a hood.
Read the source, Luke. I have years of trouble free CentOS use on many servers in an internet facing environment. We Graybeards know that stable is good.
As for patches, I have found very few bugs in mainstream packages. Many eyes kept the number of defects low. I have fed back desired features to devs as needed.
My org is less than 50 humans. We have more servers than humans. IT staff of 3.
If you can't find the answer on the internet, consider another career.
In most companies the question is not whether they do or don't pay for support but rather what systems are supported to what level. No company has the technical depth to deal with the entire Linux stack, with the possible exception of IBM. Low criticality systems can have problems but because they are low criticality the company has time to find resolution so they tolerate greater levels of mistakes in exchange for lower costs. As you get closer to the center support becomes more important because they want a depth of experience that only a contract can buy. With bigger companies that support contract is held by a vendor who has multiple support contracts.
I'd say for most small companies the best approach is a good IaaS/PaaS provider with very little internal. For larger companies multiple overlapping systems of support including a wide range of internal knowledge.
how many major corporations are taking this approach?
I can think of a few companies that support Linux in-house. In no particular order: Red Hat, Novell, Canonical, Oracle, Google. I'm sure there are others.
-- Jeff Woods
Why is it either roll your own distro or pay for support? You know if you can run your own servers you can run CentOS which gets all of redhats packages. Your question is equivalent to "Should I get a warranty on my new car or should I build a car myself?".
truth is, when you hire someone to provide 'Linux' support, you get someone that usually does so much more. the generally understand network design and implementation, process scheduling/automation, basic programming/scripting skills, client server concepts, mail servers, dns, chat server, the list can be extensive.
Tis question is not exactly but largely analogous to the question : 'Do we self insure or purchase insurance'? The risk evaluation is complex and the answer is dependant on the a number of weighty factors not least of which is the amount of liquid assets available to address a catastrophic failure. Hire a risk analyst.
"The inherit risk is that the organization is accountable and accepts the risks if a major bug is encountered within any of the open source applications they are using".
In order to use the software, the end user must accept the license, which in no-way-shape-or-form makes the organization accountable for any such defects.
Limitation of Liability.
"In no event unless required by applicable law or agreed to in writing will any copyright holder, or any other party who modifies and/or conveys the program as permitted above, be liable to you for damages, including any general, special, incidental or consequential damages arising out of the use or inability to use the program (including but not limited to loss of data or data being rendered inaccurate or losses sustained by you or third parties or a failure of the program to operate with any other programs), even if such holder or other party has been advised of the possibility of such damages."
When I first started out my business I had a shoestring budget, if that. I could not afford conventional Microsoft software like the OS and office suite, anti virus, graphics design software, accounting software and so on. GNU/Linux seemed the best route to go. However, I knew very little about GNU/Linux either. For a small fee, I paid for GNU/Linux support and had someone there to hand hold me through crucial moments when I did not have time to research problems on my own or when I got stumped by a bug. It was great to know someone was there, on retainer, whom I could call when things got rough. Now I do not need paid support, at this time, because I can sort out these issues on my own. However, if my business was to grow, I know not all employees can handle the GNU/Linux OS, so paid support is a good idea if I am too busy to explain everything. Paid GNU/Linux suport is one of the best things I ever invested in. Sure, there are some weird support staff at there, but that made for fun conversations and a lot of learning.
"SO we bide our time, waiting for a purer kick to bloom and the future is still bleak, uncertain and beautiful" -GSYBE
Linux Support? Isn't that an oxymoron? Unless of course you consider some suspender wearing douche-bag with Cheetos in his beard typing, "RTFM Noob!" into a forum to be support. Linux support. Snort! Oh the man pages... apt get a-life.
Shouldn't this be a problem for a fifth grade economics class?
When the cost of external support for n machines exceeds the cost of an in-house IT contractor, then it's time to shop around for a contractor. It's really that fucking simple!
Political debates have me rolling my eyes so much I think I got optical whiplash. I should sue. - Foamy The Squirrel
I've never read a EULA that accepted liability for bugs that went beyond the cost of the software license.
This only makes sense if you have contributor(s) in house. And i mean the plural, because just having a kernel contributor in house is not enough.
If this is not the case, don't even think about it.
On a long enough timeline, the survival rate for everyone drops to zero.
They all would have been harder to fix if we didn't have Red Hat support. And because we had Red Hat support, it made explaining to our customers and vendors that we had patched the security holes so much simpler.