Slashdot Mirror
In Favor of Homegrown IT Solutions
snydeq writes "Today's IT organizations turn too readily to vendors, eschewing homegrown solutions to their detriment, writes Deep End's Paul Venezia. 'Back when IT was "simple," several good programmers and support staff could run the whole show. Nowadays, [companies] buy hefty support contracts and shift the burden of maintaining and troubleshooting large parts of their IT infrastructure on to the vendors who may know their own product well, but have a hard time dealing with issues that may crop up during integration with other vendors' gear. ... Relying solely on support contracts and generic solutions is a good way to self-limit the agility and performance of any business. In short, more gurus equals less hand-wringing and stress all around.'"
When you treat people like second-class citizens by making them contracted labor, especially in IT, this shouldn't be a surprise.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
The general in-house versus outsource vs commodity question here is a bit inextricably tied up in the more specific "enterprise software sucks" problem. I've seen moving from in-house solutions to third-party stuff work well, when it's good third-party stuff. For example, near the end of my time there, my university switched from an aging home-rolled email setup to a Zimbra installation, which, while not perfect, was generally better and more reliable. On the other hand, there is certainly plenty of crap that they pay Oracle and Microsoft $$$ to run that doesn't serve anyone's needs very well, or integrate with anything else.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
I couldn't agree more with this. We run an in-house development shop that continually out-performs areas of the organization that purchase COTS stuff (and then spend millions trying to customize it). In the beginning we got a lot of crap for having a "not-invented-here" approach and coming up with custom solutions. The first time we replaced one of these multi-million dollar solutions with something much cheaper (and easier to maintain) the comments stopped. This isn't to say we don't use commercial frameworks, appliances, etc. But these are tools (sometimes power tools), not pre-fab homes.
Loaded cost for an employee is typically 18% of salary + $320/month for real estate overhead. So a $90 K employee ends up costing about $120,000 with benefits.
-- $G
A high turnover of employees creates problems with in-house development and maintenance of software. The "organizational memory" -- how did we get here, what were the problems, how were they solved -- is lost.
In the U.S. military, cognizant personnel are often rotated to new assignments every 2-3 years. This has the same negative effect on long-term maintenance and evolution of software for military uses. For this reason, military software projects are (or at least were) out-sourced.
For 24 years, I worked for the System Development Corporation (SDC), which eventually became part of Burroughs which then merged with Sperry Univac to form Unisys. We worked with the Aerospace Corporation and with Lockheed. Together, these three companies held the organizational memory needed to maintain computer systems for operating an ever-changing array of earth-orbiting space satellites. Our role at SDC-Burroughs-Unisys was to receive software packages from 10 or more independent software development companies (sometimes the same companies that built the satellites) and integrate them into a single system. We audited the developers' specifications and tests, tested the merged packages, performed configuration management, prepared user documents, conducted training for the end-users, and diagnosed suspected errors. On occasion, we even rejected software and sent it back to the developer company to rework. Contrary to current practices, the most senior professionals also provided "help desk" support. In all the time I worked on this project, not one space satellite was lost due to a software error. Considering the cost of a space satellite, the fact that our task doubled the overall cost of software development was money wisely spent.
While the project on which I worked was technically out-sourced from the U.S. Air Force, the repeated renewal of our contract and the contracts of Aerospace and Lockheed created an in-house professionally-skilled environment for acquiring and evaluating software. As a result, a very large software system with an expected life-span of 15 years evolved and was used for over 20 years.
This brings up a question. My organization replaced our old ERP and CRM-like system which was bought 20 years ago with the source code and heavily customized. The administration (through thir consultants-ugh) declined to buy the source code licenses for the new applications because "modern organizations don't buy source code licenses anymore." Now, predictably, people are upset because we cannot tailor the apps to our business rules. My question is whether the statement of the consultant is crap or not: do companies nor buy the source code license and solely rely on vendors to make changes via upgrades or custom programming?
While I would love to wave this article at my management and say "hire more gurus", I find it somewhat disconnected from reality. This concept would only work if you had a department dedicated to in-house development, with unlimited permanent headcounts all of whom would be flawless in developing, documenting and supporting their respective applications in a uniform, regulatory-compliance friendly manner and who would never, ever move on to the greener pastures. In reality, you have self-proclaimed "developers" from various departments, writing spaghetti code designed to address their specific problems, then eventually quitting and leaving IT to struggle with supporting the uncommented, undocumented application that now cannot be replaced because it contains "all customer data". And when your friendly neighborhood ISO 27001 auditor comes along, you end up hiring 3 more people to fix every missing data validation, credential management and change control problem in this irreplaceable creation, and then, maybe, it becomes that wonderful application the author is hoping to push for.
On the other hand, if you get a third party vendor to provide you with a solution - your upfront costs will seem higher, but chances are - unlike your departed headcount, that vendor exists for the sole purpose of supporting their solution. Their features, functionality, security and regulatory requirements have either already been hashed out by other customers, or will be hashed out for your if you ask for them. And unless they're a small enough vendor to go out of business without selling their assets to someone else who will take over, they will be there to support that application and provide feature updates while your in-house developers come and go...
Bow before me, for I am root.
There are IT gurus out there with free time. Some of them are working in environments that have completely outsourced to vendors, and the gurus end up educating the vendor's minions, sometimes on the most basic operations. Personally I find it easier when I open a ticket with the vendor to copy/paste the exact commands for them to run on servers on which I no longer have root. It saves time.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
While having in-house solutions is great, but what happens when people move on? I work in the EDU part of the IT industry, and we have a particular system that was designed by a former employee, picked up by a second, redesigned by the same person, who denies that anything could be wrong with it. Support calls to them go unanswered, and they're rarely in the office. And they are one of the three Directors in IT. Personally, I work on our Windows 7 deployment, and all the underlying AutoIt scripts, plus the virtualization of our applications. I have trained all of my colleagues in what they may need in the event of my demise (or less worrisome event). And I get paid about $34k a year.
Those must be some bad MBA schools in your area. I got an MBA and I was never taught we should outsource everything.
We were taught to get venders when the requirements word distract the existing staff from their mission focus. I had to read case studies where outsourcing worked well and when it failed miserable and should have kept the inside staff. We were taught the complexities of global business and that American staff tend to be more productive and creative even though they cost more. How bean counting causes you to miss the good envestments. And a good HR policy means treating your workers right and at a good pay.
I am willing to bet there are less MBA but BBA who are out of a 4 year business with no experience, trying to save money by stepping on the backs of anyone who gets in their way. The MBA program is far more responsible.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Wrong. If you don't have the developer, then you don't need as much floor space for your office, you don't have to buy cubicle furniture, you don't need as large an IT staff, etc. Obviously if you get rid of one developer, you're not going to immediately save this money, but when you're looking at hiring a whole team of people versus contracting something out, you probably will (as for a whole team, you'll need to rent more floor space somewhere, unless your company happens to have a bunch of unused floor space, but most don't as that's wasteful).
Also, 401k, SS, and health insurance absolutely do immediately increase each employee's cost.
Hell sometimes you have to educate the vendor's minion's on what their product is supposed to do!
I didn't intend it as anything more than a maybe funny, snide remark. The article was about contracting versus in house gurus, and every month it seems there is always an article about the lack of gurus, hence the comment. The "we contract everything at our detriment" crowd, who complains about the lack of gurus, would contract to get an in-house guru, get it ?
;)
Of course, I'm a guru, but I don't want to work for the "we contract everything" crowd, so maybe thats the problem.
Some of us have worked for employers that made the extra hours worth it; that doesn't mean you'll have to exclude large businesses either as well.
How about fixing the overtime law to remove the IT exemption, along with something that makes requirements more reasonable(e.g. if you can't find someone, you're going to be on the hook for directly hiring someone - not as any form of a contractor - and training them as an FTE at full wage)?
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
The company I work for has the best of both worlds. They go out and buy a $500,000 piece of Enterprise Software*, forgo the expensive contractors and dump the setup and configuration on 2 or 3 in-house developers, a project manager (who is usually an outside contractor who happens to be friends with an executive -- a budget locust, if you will) and an IT manager. After about a year the esteemed project manager moves on to the next project, the manager in charge gets promoted, the software is blamed for the lack of results and a new $500,000 purchase is made.
*For those that haven't used the stuff, Enterprise Software doesn't actually work out of the box. It's much like a do-it-yourself plane kit with lots of manuals on FAA regulations, a glossy guide full of pictures of planes "other customers" have built and a box full of parts (with a few random parts missing) but no actual assembly instructions.
Imagine - you are trusting a PRIVATE party with your sensitive stuff. they can do something stupid and go bankrupt, get sold, this that. you have no power over hirings there, so you wont know whether they are hiring reliable individuals or people who could leak your stuff at any given point. what are their goals their policy changes this that.
basically you are giving your balls to them. and they grip tightly.
i.t. became too complicated now indeed. but, is that much complication really necessary ? KISS rule (keep it simple, stupid) is applied in software development, but, ironically it is not applied in setting up i.t. infrastructure of an organization - nowadays people try to incorporate every 'next big thing' into the setup. and you get a mess.
KISS outside, KISS inside the infrastructure. And then keep your own infrastructure. That's the key.
Read radical news here
When I worked at an F500 high-tech company, they accounted the total cost of each software and hardware engineer as 2.5 times salary. This included the buildings, computers, training, and all the other stuff necessary to keep the engineer productive. For big companies that's probably still pretty reasonable.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
Doing that now with a product being developed by a major company which shall remain nameless. I'm having doubts the corporate culture is actually capable of engineering such a complex product. We'll see.
The biggest drag about contracted services is that, even if you are lucky and they actually save time rather than waste it, they have external costs in that some of your projects get hamstrung waiting for vendor fixes. The flip side of that is at least you don't ever drown in a sea of options.
Someone had to do it.
This is true, but you have to be selective. Sometimes pre-built solutions makes since.
I've seem way to many people try to build solutions when an off the shelf solution would have been easier and cheaper in the long run. (say after a failure, and sometimes before!)
If you need a mail server with lots of accounts, but no bells and whistles. Build it yourself. You need an mail server with all the bells and whistles. (calendaring, etc) Buy one off the shelf. You will save yourself a lot of head aches. (providing you're not stupid in implementing your off the shelf product)
I have a couple of name brand HA NAS devices. I also have a couple of Linux NFS servers running DRBD and heartbeat. I knew where to buy off the shelf and where I could do it myself.
A lot of the time it's because when shit hits the fan then management can shift blame to the vendor and/or support contract.
this is my sig
I'd like to see software "kits" for families of application domains. One purchases the software kit's source code and then customizes it, perhaps with an optional subscription to get future doo-dads.
Doing everything from scratch takes too long and buying pre-built solutions shoehorn you into something both missing features you need and that carries the baggage of features you don't.
Write them in common languages such as Dot.Net, Php, Java, etc.
However, enforcing licenses may be tricky.
Table-ized A.I.
In my MBA program, we discussed that outsourcing almost always costs more than insourcing for the simple matter that if it costs them $10 to do it, or costs you $$10 to do it, they'll mark it up 50% and sell you the service for $15, but if you'd had the capability to do it at cost, you'd be out $10. Oh, and that the single greatest "cost" of outsourcing is almost never counted. Risk. What's the risk the outsourcing company will close down? don't know? Then you shouldn't be relying on them for a business critical function. What's the risk that your competitor could pay off your outsource company to get access to your systems? Don't know? Then why are you outsourcing?
Learn to love Alaska
My response is:
As with many things in IT, it comes down to the fact that the developers are not the ones with the authority to do these things. The authority and the responsibility rests with the managers and executives who make the decisions and set policy. As an "IT nerd" (read "techie, guy who's paid to make the little boxes with the blinkenlights do their thing") I'm often caught between the desire for good project discipline and the reality that management doesn't want good project discipline because it interferes with delivering the most features in the least amount of time (notice that I said "features", not "bug-free working software"). And I can't tell the CIO "No, you're not shaving 4 weeks off the project schedule. No, you're not assigning Joe to another project. No, you're not adding those 5 new requirements to the list without adding additional time to the schedule.". I'd love to, but I'm not his boss so I'm not the one giving him orders. And if he ignores what all the people under him are telling him, there's only one person responsible for the resulting trains-wreck. But his bosses won't hold him accountable for it, so it's no skin off his nose.
I understand that some environments can be more flexible or more dialed-in to company/user needs with a full time, active development staff doing everything homegrown.
But the talent pool for this sort of thing is woefully limited. I've seen "in-house" development groups come up with some of the nastiest, most byzantine pieces of crap-hackery you could possibly imagine. And there's ZERO planning for what to do when the system reaches obsolescence. And don't give me any crap about how it won't ever happen. It WILL. Then, what's the upgrade path? How do you get the data out? And a million other niggling little things.
There's also the problem of relying on a group of individuals like this. It's essentially a thinly disguised form of vendor lock-in. Save the vendor is a group of your own employees. And what happens if they all up and move on to greener pastures? How do you maintain the system? CAN it be maintained? Can it be extended? Can ANYTHING be done with the system without bringing it crashing down?
How do you know Joe WannaSecureMyJob didn't back-door the system?
Yes, a lot of these are concerns you face with vendors too. But with vendor approaches, if you dislike the direction the project is heading, you can kill it, cut out the vendor, and move on to something you find more acceptable.
Not saying it CAN'T work. Just that the level of care you have to take when risk managing is different from what you need with outside vendors.
Chas - The one, the only.
THANK GOD!!!
In my experience, the biggest difference in outsourcing operation of key software is that it forces internal customers to rethink their expectations. If software is maintained in-house, they expect it to fulfill their every whim. When the IT dept says "It will take 3 of our developers 6 months to do what you want", then say "Ok, we need it! Do it now!". But when they are dealing with a software vendor, and they say "It will cost you $175K to do what you want", they say "Hmm...well, that's kind of expensive, I'm not sure we need it".
When you have your own developers, they can tailor the technology to meet the needs of your business. When you purchase pre-packaged software, the business tailors its needs around the software.
The article was about contracting versus in house gurus, and every month it seems there is always an article about the lack of gurus, hence the comment.
I suspect the problem is corporate executives who know the cost of everything and the value of nothing. There is a simple way to increase the supply of something: Pay more. If companies would pay competent IT people more money, then more people who would otherwise go on to be tax lawyers or securities traders will go into IT instead.
By contrast, what you hear in the media is the executives thinking with their MBA brains: If you want to increase supply, you can pay more... or you can go to the government and create artificial incentives to increase supply. More H1B visas. Government education subsidies for tech majors, to divert labor supply from occupations that pay the same or less than IT into IT. More supply at the same price.
The problem is that the latter doesn't create "gurus" -- it creates paper MCSEs. It makes the problem companies have in hiring competent staff that much harder, because you create a population of applicants who have degrees and certifications and even experience, yet have no earthly idea what they're doing. It attracts exactly those people who are too stupid to understand that a $1000 scholarship is a completely asinine way to make a career choice, instead of those who are smart enough to do just about anything and who make decisions based on forward thinking criteria like which career will allow them to afford a house in a neighborhood with better schools and a comfortable retirement.
It's the same disease that allows them to make the IT department a cost center: They count all of the salaries and equipment and ignore the productivity improvements that accrue to other departments as a result of their existence. Which makes it look like cutting staff or replacing them with less qualified but lower paid employees will save them money: The cost savings goes straight onto the spreadsheet, without accounting for the lost profits that will occur when a major system falls over and there is no longer anyone competent working there who can get it running before you lose a big client.
I'd +1 you ... if I had any mod points. One of the more lucid assessments I've had the pleasure to read.
You need to decide what business you want your company to be in - if you want to be a SW development company, fine - be a SW house. However, if you want to be anything else, then don't write your own SW. Keep your business focused on what you really do. You don't want to spend resources tracking, designing and coding the annual changes to the tax code, or all the deprecated functions in your chosen framework or the latest trends in user interface design. There is no way you can do all the industry research, application design and code maintenance for the price of the annual SW maintenance, let alone match the amount of resources a large commercial SW vendor can devote to the same problem. His R&D costs are spread across all of his customers - are yours?
Oh - and lets not forget a little thing called Sarbanes-Oxley. Do you really want to prove to your auditors that you have built the same level of controls into your homegrown ERP system that are put into tier 1 or 2 commercial systems?
In the vast majority of companies, the "unique business processes" are a very small percentage of the application - and many of those are simply stubborn and egotistical business users who refuse to believe that the vanilla solution would also work just fine for them if they were just willing to try to understand it.
apprenticeship system. Take today's tech schools and add apprenticeships to them.
CS degrees build theory and a lot of that is high level stuff with out the skills of working on systems / working with stuff at the hands on levels.
Now with a apprenticeship people can build real world skills and companies get people who are not people who can cram for a test and be come a paper MCSE
And what does CS have to do with IT?
And what does CS have to do with IT?
Exactly. This. This is part of the problem.
There's a disjunct between how academia sees Computer Science as nothing to do with IT and how business sees a CS degree as the basic starting point for a career in IT.
Can we please either have a Computers in Business degree that teaches useful skills, or a business culture that doesn't expect academic degrees to be vocational qualifications? I don't mind which, either is good.
Also, the reason your company doesn't have any gurus is that no-one is prepared to spend any time or money training their staff, or even giving them self-development time to train themselves. Companies that do decent training have gurus. It's pretty simple.
Business/App ideas are like arseholes: everyone's got one, they're mostly shit, but very rarely they contain a diamond
In terms of teaching useful skills. Tech schools are better but HR / business culture does not see them as good qualifications.
IT is at the point of plumbers, HVAC, car repair and so on. In where only so much can be learned in class room and only so much theory can get the skills needed to the most common work and 4 years is to long for a starting point even 2 years pure class room is pushing it.
Now say 1 year for basic IT and then maybe some kind of a apprenticeship with on going class and then maybe after that have higher level stuff NOT CS stuff but things like advanced networking, advanced security and so on. CS is way to much on the theory side and the tech schools are lacking the real work place experience.
Right now some can say do a 4 year advanced security and miss out on the part doing the basic work and end up pushing advanced security stuff with out haveing worked with doing stuff at basic level where you find out how at times that advanced security does not work as planed or that you can get by with lot's time wasting work around / paper work coming from a poor security plan.
I keep seeing this posted often on Slashdot. Of all the industries, IT is the absolute worst example you could name being a candidate of some sort of apprenticeship program. That's because Information Technology is fast moving target that defines progress and changes in paradigms. It's also why even IT college degrees are almost worthless too. I'll leave CS out of this because they actually rely on math and other proven techniques that have wide reaching applicability. But certifications such as an MCSE and CCNA only prove familiarity. They do not however prove experience. In fact, I would state that these certifications are best suited to compliment your resume of existing experience.
Life is not for the lazy.
Although a union to say "You don't have to be forced to give up having a life, just so someone can get their spreadsheets at all times of day" would be nice.
Everyone wants a 24x7 IT system. There's a way to do that; lots of money on the hardware, and three complete teams of core staff who work shifts (with the commensurate shift salary augmentation).
But no, what business wants is a group of IT staff who work the same hours as everyone else, for the same kind of salary as the average pen pusher, who will then, at no notice, respond to a phone call at any time of day or night and get to site (or at least connect up remotely) and spend hours diagnosing network/server/PC/application problems (possibly calling up other IT staff), and then being in for work the next day as if nothing happened.
Why do people think that contractor = second rate citizen? I don't know any contractors (including myself,) that want to go full time. I don't understand the mentality that choosing to be paid a rate per hour and have no other connection to the employer is somehow a bad thing.
You can't handle the truth.
Perhaps they should have taught you reading comprehension. The OP did not say that the MBA program taught outsouce everything (I'm not even sure that he was referring to an MBA program, it appears to have been an IT management training program--at what level appears ambiguous). The OP said that the IT management training program he was part of taught a whole bunch of stuff such as outsourcing, best of breed, vendor support as primary skills and that actual IT skills as not terribly important (nice to have for an IT manager, but dispensable).
I have seen this in many areas, not just IT. The idea that management does not have to have any of the skills needed to do the jobs subordinate to them is very prevalent. While a manager does not need the skills to sub in for all of their subordinates (although it does help), they should be able to do the job of a significant number of their subordinates. Otherwise, they will have trouble recognizing the relative value of different staff members contributions.
The truth is that all men having power ought to be mistrusted. James Madison
Unfortunately, this is all too true. Just recently, I had to contact support for one of the largest IT companies in the world. They will remain nameless, but the company has a 4 letter name and they were at one time the largest PC manufacturer. I asked to speak to someone in the support dept for the product I was using, and the people on the other end of the phone had no idea that product even existed.