Slashdot Mirror
Overwhelming Bureaucracy in the IT Department?
Nedry57 asks: "I am in the somewhat unique position of being a technology worker, who lives outside of the IT department in my company (a very large organization in the US). By far, the biggest challenge I face is getting anything done due to the bureaucracy that exists, within IT. There are certain tasks (i.e. anything that happens in the data centers) that I don't have the access to do. Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration), thus costing many times more than it should. The lack of agility is maddening, because I know we are missing significant business opportunities. My management is extremely supportive and despite our excellent track record of success in creating robust/secure applications--our work has passed audit numerous times with flying colors--we get no support from IT. Even senior management can't break through the barrier. I am very interested in hearing the experiences Slashdot readers have had in similar situations." How do you get your technology work done, when your IT department is more hindrance than help?
You don't. You fire them and outsource their jobs to India.
This guy is the head of my company's IT dept. BOFH
and leave!
Seriously, it seems that you have fought the good fight. Your managers ahve supported you, you have been at this for a long time, without effect. You now have a choice: accept that it probably won't change and that you can live with it, or leave.
The real "Libtards" are the Libertarians!
It's simple. Either you get the buy-in of upper management, CIO, CFO, CEO and effect a change in the present system or you bail out and get a job in another company. You and your immediate supervisor, obviously an inconsequential middle manager, will hold no sway and make no changes. All that you and he will do is rock the boat and develop a bad reputation in the company. Get upper management buy-in or bail out!
P.S. It sounds like you need to acquire funding for a development and testing lab that is not under IT however, do not expect to connect such a lab to IT's network.
its easier to ask for forgiveness than permission
This is not an IT-specific problem: all functional areas in large organizations are vulnerable to this sort of bureaucratic barbed wire.
Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration), thus costing many times more than it should. The lack of agility is maddening, because I know we are missing significant business opportunities.
If you know that there are real costs associated with the lack of agility, you should a) document in detail the actual losses, b) present these figures calmly and respectfully, and c) gauge the reaction from senior management.
I want to drag this out as long as possible. Bring me my protractor.
Get a new job working for yourself or a start-up. Large companies (like the one you are working for) tend to have a lot of bureaucracy. Smaller companies tend to have less bureaucracy. Not to say this has to always be the case, there are certainly exceptions. Good luck changing the IT culture. Once a corporation or a department develops a certain culture or way of operating it is usually very difficult to change. Sorry, this is probably not what you wanted to hear.
Bradley Holt
Somewhere in the senior echelons of your organization exists a guy. This guy (likely at the CIO level or higher) is either willfully ignorant of the nature of the IS organization which reports up to him, or he's actively encouraging the situation.
If it's the former, you need to find out who it is that's allowing the inefficient environment to foster and take steps (and obviously "you" aren't the answer, but one of his peers or superiors is) to educate him on how things could improve.
If it's the latter, and he's actively promoting that method of interaction because it keeps their costs down, or reduces headcount, or whatever AND if he has the buy-in of his peers and immediate superior, you're screwed. I suggest looking to outsource your department's IT requirements to a 3rd party if you can't bring them into your own group.
Reason why there is hope for the future generation #364:
"I wish my grass was emo so it could cut itself."
Progress and stability are often conflicting goals. IT departments generally prefer stability, and that's why your deployments have probably been so stable and passed so many audits. Developers, of course, are charged with driving progress.
The real answer if you need flexibility with regards to "non-production stuff" is to not let IT have anything to do with it at all. Create a separate sub-net if you have to to keep the non-production machines off the IT network, and a firewall between your network and theirs to prevent any viruses, or other effects, from leaking from your net to theirs (this may require having to VPN through it just to work with these machines, c'est la vie). Keep the machines in a different room than the official server room. Maintain them all 100% yourself. Then do what you need to. Anything less and you're asking IT to aid in your development, a task they're probably not equipped to do while maintaining stability.
It's not uncommon for companies to have a "developer", "staging", and "live" system setup that are all completely independent, with some established mechanism and metrics to push products from one level to the next.
E pluribus unum
Moving to another company besides Microsoft?
Management can keep holding long meetings to find out why work is not getting done.
... and with a subject like that.. you aren't going to here what you expect:
The military (USAF) had a very good IT setup (overall) that was basically setup the way you'd set up a good memory architecture.. you have a hierarchy of IT with the most used/essentialy tasks able to be done close to where they requests come from, and build upwards and outwards. Those local people were "fired" (in the government people don't actually get fired, but moved) if they didn't perform, and they were basically giving the keys to their kingdom. In the grand-scheme of things, it actually ran pretty effeciently, and we were never waiting on IT for more than a day, except in the most extreme cases.
That being said, the military has a pretty large vested interest in people being able to work and use their computers (ie, the cost of failure can be scrubbed missions which equates to huge amounts of money down the drain) so things tended to Darwin into a workable system. It sounds like your company's IT organization is just immature or flat-out poor (I don't mean in money... althought that could explain poor quality, also), and the powers that be don't seem interested in fixing it.
Where I work, it takes 6 months, minimum to get a server in a datacenter rack. Then the department pays in excess of $30,000 per server to "maintain" it.
IT departments run amok waste outrageous amounts of money. Those million dollar Oracle licenses and SANs have to be paid for somehow; and bueracracy helps cover up where the money is going.
Conformity is the jailer of freedom and enemy of growth. -JFK
It sounds like the bureaucracy is going to be tough to change. However, is it possible to get your group moved *inside* of IT so you can get the job done? It might require less work to do this and still let you get your job done.
It sounds silly, but if you can't beat 'em, join 'em.
Look, the people you are after are the people you depend on. We install your memory, we code your apps. We run the internets, we guard you while you sleep. Do not... fuck with us.
Dear IT Professional:
Please don't change anything about the way your IT organization does business. We love the way you and your team fail to communicate; the way mindless mandates from on-high drive pointless initatives; the way the latest technology trend shifts focus from project to project like the attention span of a two-year-old.
Especially don't pay any attention to streamlining the use of hardware and software investments that you've already made. You and your team need MORE MORE MORE to get this project wrapped up on time. Have you upgraded to the newest rev of our software? Can't you just taste the new-and-improved speed of our lastest hardware?
In summary, we love the way your IT organization is today, and wouldn't change a single thing.
Yours Truly, Your software & hardware vendors
The surest sign of intelligent life in the universe is that none of it has tried to contact us. -- Calvin & Hobbes
"Our SAMBA connection is broken. Something changed over the weekend."
"Nothing changed over the weekend."
"You sure about it? Why does the AD server report it's running Server 2003 now?"
"Oh that? We tried to implement Windows Server 2003 to replace our AD server, but we backed it out."
*boggle*
That conversation was with our IT dept. In any controlled environment, things should be thought out, documented and multiple sanity checks performed. Even a dev system can impact a production system if they run on the same segment.
Now, having said that, our IT dept tends to mindlessly enforce rules without thinking about them and getting them to wake up to new technologies (e.g., SOAP, web apps) is like trying to bring around a corpse with smelling salts.
A good IT department should make sure things happen in a controlled and documented way, but should also make it as painless as possible to follow the rules. They should be proactive so if you come to them with something new you want to implement. Not only will they know what you're talking about, but have already prepared a white paper of preferred architecture for performance & security.
A really good IT department brings something to the table.
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
At a very well-known, well-funded, academic institute, I had to write a formal business case to submit to not one but TWO directors to justify why I needed an extra 512MB in my laptop...despite the fact that it would at worst be about fifty bucks and, regardless, it was a FREE upgrade. A "business case." Honestly. I didn't have to write a !#%ing "business case" for the laptop itself! The amount of time spent biatching over that $0.00 basically could have paid for the whole g.d. machine, gig included.
You do the best you can. If you care enough about the job to stay, I would make sure that senior management knows where the bottle-neck is. When giving status reports diplomatically remind people that "Item X,Y and Z" are not released due to delays in IT. I feel for you, my company was infected by middle management about 8 months ago, now releases that took me 30 seconds (literally I timed it) now take 2 weeks at the minimum.
First off, 9 months seems excessive. Very little should take longer than a business quarter.
..don't assume the issue is on the IT side I guess is the gist of it.
However, in my experience every person outside of IT and security groups has this mindset that IT groups hinder them for no real reason.
I do not doubt there is bureacracy that slows every company's process. However, the fact that you want a change made to one system now doesn't change that these IT people are responsible for the effects any change might have on an entire organization. I don't know how many times I hear "But all I want is X". And that person requesting 'X' doesn't realize that 'X' has these 3 possible security issues associated with it. Maybe it won't effect his server even if it is exploited, but that risk has to be evaluated, approved and lord knows what else.
The fact is, every change *must* go through a certain amount of bureacracy to make sure all that it could effect have taken the appropriate level of responsibility.
My best advice is work through your own internal processes to see if turnaround time can be expedited. Maybe all they need is a motivated developer type with your skills to assist in making their change control system better. Or maybe there are things you don't see. Don't assume IT folk are just pushing your stuff back because they don't like you (though that could be a factor). If you can get a 'champion' type in your IT group that can help you get your stuff moved through the most efficiently.
But in the end, it is not up to you to decide what priority your request is given over someone else's. Even a simple request should be evaluated properly and must be given priority that is likely outside the IT drone's choice... Maybe your manager/director type needs to champion your projects to get them pushed through with greater priority .
Oh, and Bill said he didn't wanna give you your ram because you ate his pudding cup.
He and others in the IT department tried doggedly to get security noticed, only to be shot down by executive management. To paraphrase the CFO and strip out the gratutious profanity, "We're a meat company. We turn happy cows into happy steaks and happy pigs into happy bacon. We're not freaking NASA. We don't need to worry about our computers like Lockheed Martin does."
Several months later a virus hits the company and the phone system, which includes all sales offices, dies. I rush and get the tools to remove the virus in every hand possible.
Ultimately, as I was leaving the company, they finally hired a security manager. This was only because of Sarbanes-Oxley, and that person was given the role of a paper tiger--no authority to change things to be more secure, but a perfect picture for blame should something go awry.
When I left, I entered another office with other politics, but it is nowhere as bad as it was there.
--Chag
I get on /. to try and escape this crap for a few minutes! Thanks a bunch!
In Soviet Russia, Chuck Norris will still kick your ass.
what was your username again? *clickety click*
Sure -- if I can read between the lines of what you seem to be saying -- the chicks might not care if you're good at your work or not. But some of those mere "other men" you mention might also happen to sign your paychecks.
The guy was complaining that his company is missing significant business opportunities. Translation: The company is missing significant business opportunities that he could have been instrumental in acting upon. But he can't, because of IT bureaucracy.
OK, so it's not his fault -- but do you think that's going to matter next time he goes in for a raise or a promotion? They'll want to see all the forward-thinking plans he's executed on, and he's going to have nothing, because trying to do anything is like wading through mud.
Even worse, what happens when it's time to a round of layoffs? What justification will he have to keep his job then?
Maybe it's easy for you to just sit there and be grateful you have a job. If it is, it's probably because you've only had one or two entry-level jobs. For people who have had a job for a number of years, however, just having a job no longer seems like Goal #1. Those people start to have other ambitions -- like buying a house, for instance, or a new car, or providing for their families. Maybe you've put yourself through college. Have you put anybody else through college lately? Dads sometimes like to do those kinds of things. They're hard to do when you've spent the last five or ten years sitting at the same desk in the basement, just spinning your wheels.
Breakfast served all day!
I think the most important thing to remeber about Large Companies is that most large companies are old companies.
Most Old Companies are very slow. They are slow becuase they have learned a lot of very painful lessons over the many years. They purposfully slow things down to insure that all the old lessons and painful experiences are taken into account.
The way this is done is through paperwork, meetings, agreements, etc... Think of it as the company is protecting itself from the stupid decisions of the past.
Fantasy remains a human right; we make in our measure and in our derivative mode... -- JRR Tolkien
Make friends with somebody in IT, grease the proverbial wheels. A case of beer can do wonders for motivation.
SBC corporate (now AT&T) is exactly like this. I was a contractor building an application in 3 months. IT said that it would take up to 12 months AFTER applying for a server in NEXT YEAR'S budget. That's right, it was going to take 16 months and several layers of approval. The VP of the entire division (only 1 person down from the CEO) couldn't bust through that red tape.
Now THAT was funny... 3 months later I had a working application sitting on a shared server, and I had to go. We had about 1 week's worth of data in there, but that was almost 100,000 rows in most tables.
Behold the glorious bragging rights
Some number of years ago, I found myself in charge of a private infrastructure. We had maybe 50 servers and 400 users exchanging sensitive information completely seperate from the main, public network.
Because of the percived importance of uptime on this network, everything required mountians of paperwork. Installing and removing nodes from the domain required three administrators, setting up a new machine required a month on a private VLAN being monitored by a sniffer, memory and hard drives were obselete before they got to the customer.
Anyone who ever worked around an UPS knows how they die. They give plenty of warning. Having an UPS fail is a rediculous way to lose your backbone infrastructure.
My predicessor had done a wonderful job of installing an UPS for every router and switch in the datacenter. Problem is, both power supplies in the routers and switches were connected to the same UPS. In cases where an UPS was about to fail, he unplugged the UPS from the wall and plugged it into, you guessed it, another UPS.
He didn't do it out of ineptitude; it was done because the only option was to clash heads with the IT overlords. They would require studies about how many UPSs failed and if it failed before the MTBF, they'd want us to try and recover money from the manufacturer. They'd want contractors to come in and examine the UPS to bid on a UPS monitor and replacement contract.
In short, asking the overlords was like asking to be turked by a syphalitic bear.
So, some BOFH, overwhelmed by the prospect of repairing the power system, chose another path. He walked over to a failing UPS and simply turned it off. He was the only one with the access to turn it back on, so he had no reason to worry.
Within two hours, all in-progress meetings were cancled. The Supreme Overlords demanded from on high that this lowly tech was to get a blank check and a blank trouble ticket (approved by the Supreme Overlords) to do whatever he needed to do to prevent that from ever happening agian.
Electricians installed two seperate power feeds into every rack.
Each power supply got a seperate UPS.
Old equipment was updated.
Everything was strawberry fields and unicorn giggles after that for the infrastructure department.
Now, to answer your question: You have something that someone wants. Hold it hostage till you get what you need.
I'd rather you do it wrong, than for me to have to do it at all.
Most of the Pharma data centers i've dealt with in the last 5 years are locked down VERY tight. They have to deal with 21 cfr part 11, Hippa, SOX and a list of others issues and as such sound EXACTLY like the situation you describe. Example, in order to upgrade from MSSQL 7.0 to MSSQL2000 on a Pharma house manufacturing server requires changes to the following: Changes to the original Validation Plan Detailed Design Specs Functional Requirements Specs IP's - Installation protocols OP's - Operational protocols QP's - Qualification protocols ALL of the above operations require testing-testing-testing, a multitude of meetings and of course approvals from God all along the way. Then QA-IT-Engineering has to oversee everything. Its a very cumbersome, expensive process. This is for your-our own good. I have seen manufacturing data corrupted if this process is not followed exactly. Remember this the next time you think about the consistancy of those pills you take.
*--- Sometimes a majority only means that all the fools are on the same side. ---*
Now I run my own company with lots of production severs.. No paperwork required, and I've automated most stuff.
If you are stiffled, go out!
-- these are only opinions and they might not be mine.
Meticulously document how much of a barrier the IT department is to productivity, and why you don't get things done. Keep a record of every e-mail, and make sure all communication is at least repeated in summary by e-mail, so you have proof. Present the evidence to senior management when they ask why things haven't happened.
Ultimately if the management chain doesn't see it as a problem, then it's not. Or rather, it's not a problem you will ever be able to do anything about. So once you have that documentary proof, by all means sit and read Slashdot or twiddle your thumbs while you wait for IT to do their jobs. Or even better, use the time to experiment, learn, and gain skills.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
Interesting comments...
I've been in IT for close to twenty years in a couple small startups to some multi-nationals and in my own consulting business. One thing that lots of IT folks lose sight of is that IT is first a support organization within the larger organization. If the larger organization is sufficiently forward thinking, then they can (arrg, PHB-speak) *leverage* IT to be more competitive. But IT folks still have to make sure the website is up, the file server is accessible, users can login, etc., *before* you start thinking about the add-ons.
If the business doesn't want to spend money on the servers, then document what the consequences and benefits are for their decision. Don't just write that they'll have slower machines, but play Devil's Advocate and write up the business case for not adding memory.
Or, figure some way to optimize your resources so that less memory is required. This can be as simple as turning off services, or as complex as setting memory and processor caps within the virtual partition. And if you've tried all these and you're just short of memory, let them know.
In my consulting business my first goal is to keep my customers' infrastructure running. Next is to save them money versus some other consultant. Sometimes they need to spend money up front to save more down the road. Let them know if this is the case.
Here's what you do: take a simple task like adding RAM to a non-production server, and go through the entire, exhausting process in letter-perfect fashion, meeting every paperwork, audit, and permission requirement. Along the way, document every minute you spend on the process, showing exactly what you're doing, how long it took in minutes, and what requirement you were meeting. At the end, create a spreadsheet showing in careful detail that adding a $500 SIMM actually cost the company $5,000 in processes.
That spreadsheet becomes the club with which your managers and directors can beat the IT department because they're effectively offloading cost onto you at a rate of 1,000%.
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
I usually try to figure out ways of either circumventing them, mocking them, or getting them to want to do it because not doing it hurts them personally. Three examples:
1) Circumvention: Recently, I needed a DNS change to point an existing subdomain to a different IP address. Our not-very-useful IT project manager told me they needed to come up with an LOE for changing the DNS entry. Three days later, they told me they hadn't had time to calculate the LOE and would not be able to complete the change by the following week's deadline.
I went to the head of our corporate marketing and branding group, asked for her help. Even though this is a very large corporation with more than 30k employees and a very significant IT organization, within 10 minutes, one of the staff members on the marketing and branding team physically made the DNS change herself.
2) Mockery: I needed our web team to add a link to the bottom of our company's homepage linking to my program's home page. Three weeks later, the guy who was going to do the change saw me in the hallway and asked me if I had lined up testing resources from our testing outsource company to make sure that the link worked.
I responded, very loudly and within earshot of the web developers: "open bracket a href equals quote h tee tee pee colon slash slash my dot domain dot com close quote close bracket My Site Name open bracket slash a close bracket and then click on the damn thing"
2) Fool them into wanting to do it for their own purposes: We decomissioned a website a few months ago, and it is no longer publicly available. However, we've kept it around while we make sure we got all the old documents. However, we are still getting monthly reports extracted from the back-end database. I contacted our IT reporting team and asked them to stop delivery of these reports since they're no longer needed. They sent me a form I needed to fill out justifying why I needed these to stop and aking for VP signatures and notarized copies of the marriage certificates for every gerbil I've ever owned.
I told them they had to be kidding, then I set up a rule in Outlook that automatically bounces back the reporting emails to them and deletes them from my inbox. I don't have to worry about it and once they start getting these every month they'll try to figure out what's wrong and fix it. Once we fully bring down the system, I imagine that the report engine will start throwing all sorts of error messages and they'll see fit to do it on their own if the auto emails don't do the trick.
It's sad that IT, something that shoudl enhance productivity, has become a huge obstacle for us to do business.
***Foucault is watching you..***
I think I'll be remembering how I'm paying ridiculously higher prices than even inhabitants of other similarly rich countries for those drugs, for huge amounts per pill or (I kid you not) $130 (before insurance kicks in) a 59-mL bottle of rosacea face cream. I think I'll be thinking of the ridiculous patent crap pharma companies pull to keep generics, which are sorely needed to rein in abusive ripoff pricing, off the market. I think I'll be thinking of the pharma industry's focus on "blockbuster drugs" instead of actually trying to help people by focusing on the basics and on pulling drug prices out of the stratosphere.
Nice to know a lot of my rising-ridiculously-every-year insurance premiums are going toward horridly inefficient bureaucracies.
Shape up.
i am a soviet space shuttle
To make IT believe they spotted and fixed the problem. It's an ego thing; if you tell them the problem and the fix, they have nothing to do anymore.
Tell them your machine is really slow lately and the harddrive runs like mad. Sometimes you get a 'Not enough RAM' error, but you have no idea what that could possibly mean.
Chances are you'll have your stick within a day.
Alternatively, ask your cute receptionist to go over in miniskirt and take a few sticks of RAM; they'll never know what hit them.
'For we walk by faith, not by sight.' II Corinthians 5:7
You forgot marketing budgets. ;-)
Is it just my observation, or are there way too many stupid people in the world?
There are certain tasks (i.e. anything that happens in the data centers) that I don't have the access to do. Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration)
So, let me get this straight...
User is frustrated because request to make standard servers non-standard with a custom request in a datacenter requires paperwork and time. User is upset because IT has formal procedures for change control, service level agreements, and standard hardware configs. User doesn't get ram upgrade and posts rant to Slashdot.
User is technical, probably dual boots their desktop to non-supported OS, probably hacks computer stuff at home, probably very smart and capable of supporting five or six computers by him/herself.
IT department probably supports 1000+ machines, and that number has doubled in the last year or so while staffing has been cut.
IT department probably has 200 servers per admin and only maintans this ratio by with consistent server deployments that maintain standard configurations.
A good IT organization understands the company's business.
A good technology worker needs to learn to work with IT to get what they need. You would probably be able to request and justify 10 servers and get them in the time it takes to get a one-off upgrade
The lack of agility is maddening, because I know we are missing significant business opportunities.
Lack of planning on your part does not create an emergency on my part.
Learn how your IT organization works. Work with it.
we get no support from IT. Even senior management can't break through the barrier
Perhaps you don't see the big picture. Perhaps you don't see the corporate IT budget and where you/your team/your project is on the priority list for that budget.
I'm sure there are all sorts of IT departments, but the *good* ones understand the core business and know what's important to the company's bottom line. If your IT department doesn't understand that, then I'm afraid you're going to have to become the IT liason and teach them. Provide them with your requirements well in advance so that they can plan proper deployments. Work together so that you can understand the pain points of IT, and they can understand your hardware/support requirements and the *value* that this will provide to the company.
It seems to me this is an increasingly common and distressing phenomenon, and I'm convinced that it's driven largely by the dramatic increase in regulatory compliance concerns. Case in point: in 2002, I collaborated with one person for two weeks and redesigned our company's website. I work in the marketing department, but I had a little bit of web design experience; my coworker is more of a hacker type who doesn't really have a title, but he's in I.T. We got the job done, great. Now, almost four years later, I'm still getting requests to make website updates, a procedure which has turned into an eight-headed monster due to the slew of 'systems' which have been put in place to regulate our public website content. This means that making a single, minor textual edit means filling out a Request for Testing form, logging into a semi-automated online QA application and submitting a second request for testing, making the change and updating the CVS repository in staging (which, incidentally, took me a year and a half to be granted access to, since I'm not 'officially' an IT guy), then sending a follow-up e-mail detailing exactly what was changed and listing all of the files affected. To top it off, we'd used a popular WYSIWYG HTML editor when we originally designed the site, and we opted to utilize all the cool automated site design features contained in the software. This resulted in a forest of static .htm pages (see, I knew little or nothing about server-side scripting way back then) with a ton of duplicated code. Needless to say, all of those automated features got tossed out the window a long time ago. Adding server side scripting is not an option, since all the pages would have to be renamed, links rewritten... which is straying from the point- which is, I.T. departments are running scared. Litigation is everywhere. Keeping things stable has become the sole priority in any company under any sort of scrutiny, since the stakes are higher than ever before. Nobody wants to pay fines or do jail time. Suddenly, the free for all has turned into a mine field. This is all the more reason that startups are likelier to gain traction in the marketplace. Agility is the key word- and it has been destroyed in all but a few larger organizations.
If it's any consolation, you're not alone.
The IT department exists to make sure they have regular, gainful employment. They do NOT exist to make your job easier, or anyone's for that matter, who does not have direct or closelly indirect firing power over them. There are mouths to feed, mortgages to pay, colleges funds to fund, retirement to dream about.
Cynical? Yes, but also very true. The above is the root of the issue. I'll put it in the terms that IT would:
ITs job is to keep the servers running, smoothly, with as little interruption to daily work as possible. As with any complex undertaking, different users have different priorities. CxOs come first. Period. Internal needs come next (see: "servers running, smoothly," above). High profile departments are next - marketing, sales, accounting. The last one is mostly because it comes under a CxO (F - you can choose what it stands for) who is intimitely involved with the month-to-month operation, and through which everyone gets their pay checks (including previously mentioned CxOs). Development is pretty far down, as you can see. You must understand - you don't bring cash into the organization (sales), nor do your efforts directly affect the price of company stock (marketing), both of which are of top importance to the CxOs.
That does not mean that you are not essential. But you are essential in a way that is ongoing - like the janitorial staff. If they lose development, things will slowly start to degrade, but it will be a while before there is a crisis. Either way, its an expensive mess to clean up, but if you throw some cash at it, you can bring things back to livable.
Now, lets look at the flip side. If IT goes down for a day, there will be hell to pay, and heads may roll. Every IT person knows this. Anyone who has dealt with complex modern systems knows that it's a house of cards. There are so many things that can go wrong. One failure, if not just costing your job, is certainly going to make for a long night getting things back in order. That would be uncompensated overtime, remember. Also, ten years without a single failure will not make you a hero, like landing a new sales client, or scoring a great marketing campaign which lifts the stock price or sales. It will make the company think you're reliable, but boring. Bonus aren't given out for boring. One failure, on the other hand, makes you a villain.
Now, if you've made it this far, how much value is there - for the IT professional - in helping you get your job done faster. In case you've skimmed, I'll tell you: none. It's like playing russian roulette for fun. Unless you just happen to like the life-or-death thrill, or have nothing to live for, it's a fools game.
I wish I had better news for you, but if you have a large corporation, than you have an ingrained corporate culture, and IT subculture. And they don't drift your way.
Oh, I've never been in IT. They piss me off 'cause I'm an engineer and just want to get shit done, and they want to worry about making sure the CEO's internet never goes down. I've learned over the years that, in effect, that is their job. I've stopped fighting them and learned to either (a) work with them or (b) work around them. The latter is done carefully to avoid stepping on toes. Just as they are under the thumb of uper management, they like to exert their power where they can. That would be against you and me. You don't tunnel under a mountain if there's a reasonable way to pass around it.
Is it just my observation, or are there way too many stupid people in the world?
Great points. It's difficult sometimes to just "not care" - not only because it means you miss opportunities but because it's difficult to take pride in your work. I think I'd rather continue to struggle against the situation, as you say, rather than give up and play solitaire all day.
GO to the CEO, tell him about missed opportunities, and tell him the IT department is hostile.
Tell him what needs to be done, and how you owuld do it. Get his buy in.
Then do it.
3 things can happen here:
1) S/He fires you. You were going to quit anyways.
2) S/He gets IT to start becoming more agile
3) S/He say "Do it". At this point you have to do it. Point all bottlenecks you can't deal with to him. In fact, when you have a people issue you can't solve, go to him and ask for help. You must be successful. When you are it could mean a good promotion, perferably over all those enemies you just made. Enemies are all right, you just have to deal with them calmly, and with authority(or assumed authority if you don't have any real authority.)
Hen you consideringf quiting anyways, you might as well take the risk and shoot for the stars.
The Kruger Dunning explains most post on
I work at a top 25 law firm. The CFO and a few of his traveling crew from one of the largest cell phone companies in the US was using a few of our vacant offices. We recieved a request that they needed network connectivity and a network printer if possible. We had him up and running on our public vlan with internet access and a laserjet printer in about 10 minutes. He commented that at his facility, it would take about 6 months for something like that to happen.
Beside which, what better way of making things better for those who stay? If management loses good people because of some problem, they just might address that problem.
I am in the same situation at the moment working for, err... a major british telco. I've done it before as well, I spent 10 years working in the civil service. That was pretty tough.
I have a consistent approach to the problems of working this way: I do everything myself. If I need a server? I buy one, charge it, plug it in, install it, support it. If I need hosting? There are plenty of hosting companies out there selling services; I buy it and set it up.
I do try and keep the things I do clean and secure and "away" from the IT department.
I do try to point out, whenever I can, what a clueless bunch of losers the IT department are.
I do try to get people on my side by doing favours for them with "my" resources as quickly as I can. If you can save an important managers pet project by judicous installation then all well and good.
I tell as many people as possible about what I am doing; taking care to point out that if I relied on the IT department I would never have been able to achieve success.
Mostly this approach is getting easier. It's easier to buy powerfull servers that can host masses of virtual machines; it's easier to get the hosting you need.
Lastly, remember that IT departments are so swamped by their own dumb rules and ineptitude that they have very little time to concentrate on trying to stop someone who knows what they are doing.
This is the sort of complaint I hear constantly. So, speaking from the IT side of the house.... My job is to keep existing systems that generate revenue and enhance productivity up and running and secure. Downtime costs serious bucks in lost revenue. On top of that, I do indeed have an overwhelming bureaucracy to deal with, doubled in difficulty and complexity by Sarbannes-Oxley. The S-Ox auditors are not techies, they are accountants, which means a great deal of irrelevant detail has to be audited. Exceptions to existing application environments and frameworks are extremely expensive in terms of allocating dedicated hardware and dedicated people that could potentially be servicing ten times the resources, but those economies of scale are lost when we have to do special things for someone's one-of project. Handling exceptions is very expensive in a large scale environment. If we need something new, lots of planning goes into it, to make sure we can keep it up and running, and scale to much greater than anticipated load. If you want agility, you either have to find a way to achieve it within existing channels (in our environment, the turn-around for J2EE or Oracle apps is quite short), or you need to convince upper management of the value of a skunk-works type mini-DC for such "agility", with the understanding that anything successful will need to be reengineered to be robust enough for the main DC. Most of all, you have to have a value case. It's not enough to talk about lost business opportunities. You have to be able to quantify the projected value of the opportunity, and balance that against the cost of handling an exception in the datacenter. If IT cannot quantify the cost of an exception, bad on them.
From the IT side, you can't always respond to every change request ASAP. Simple things like adding more ram and stuff are all easy to actually do, but sometimes there are roadblocks, for instance who owns the system? If it's IT, they may not have budget to add more ram whenever someone asks for it. If it's you or your group, can you get a PO approved?
Does the system really need more ram? I used to get requests for more internet speed all of the time. It happens a lot less often since I started parading out the metrics to show people here that a) we are not using all of our bandwidth to the internet when their issue occurred, and b) I can prove that we can and do use up all of our bandwidth at times.
Policies can slow things down too, but to operate without them is a very slippery slope. I used to hate policies but as I moved up the chain in IT and we began to get requests for things that would create a great deal of work for very little return, or even more important, to deal with difficult HR situations, it became much simpler for everyone to be able to point a the policy that says "As far as the company is concerned, there is no personal data on that company supplied laptop, and you need to hand it over now"
From the non IT side it can be very frustrating dealing with IT some times. If they are really competent,and your requests are reasonable, they will get to your request in a reasonable amount of time. If not, well....
Here is all I can recommend if you aren't getting the service you need. Make the business case to your manager. Show him or her what these delays are costing the company, and allow them to take it up the chain of command with the data you have provided. In any well run company, showing how you can improve the bottom line should be enough to get things moving. Keep in mind that you will win some, and loose some. There may be issues you are not aware of behind the scenes (partnerships, politics between groups, etc.)
Leaving is an option, but save that one for when you are certain that you are dealing with real incompetence and you are sure that there is no way to fix things. If you think you have a good company, do what you can to make things right.
Eschew Obfuscation
I used to work in a *very* bureaucrat infested research lab. This is the place that firewalled "new" ssh but let the "known protocol" telnet, out... Friend of mine was running very numerically intensive spectral analysis/matching on samples. Bung in sample. Get data. Process for about 8hr. So, do last thing of the day and you've got the results next morning. Until, in the interests of a uniform computing experience *all* boxes were required to have the same basic setup and were bolted down tight. This included everything. Including the screensaver that seamlessly blended from slide to slide of the company's publicity shots. Bingo! 100% CPU when the screensaver kicks in and the analysis runs can no longer work unattended. Bummer! OK so my friend takes an old mouse, a clamp stand, a magnetic stirrer and flea, and some epoxy. Glue magnetic flea to mouse ball. Clamp mouse over stirrer. Stirrer agitates mouse. Screensaver never gets to run. Once again work can happen!
Red to red, black to black. Switch it on, but stand well back.
My situation is very similar to yours. I am a tech person for a department outside of the formal IT group which sometimes seems to be inefficient and/or ineffective. However, I don't believe that IT is the enemy and tend to sympathize with them.
In my current position I've seen some of the worst behaviors (in system administration, application development, etc.) practiced by tech people outside of IT who then expect IT to automagically make everything work and clean up any messes the non-IT folk created. On the other hand, there are also times when our IT department really does drop the ball.
This has created deep-seeded animosity between some non-IT departments and IT, I think. The non-IT folk believe IT are bureaucratic obstructionists who don't know what they're doing; while IT believes the non-IT folk are disorganized, loose cannons who don't know what they're doing. Unfortunately, to some extent, I think they're both right.
That said, my best advice to you is to help IT help you. Try come to some agreement or understanding with IT and define what it is they they need in order for them to be more responsive to your needs. Respect their needs as much as you want them to respect yours.
Also, don't undermine or bypass policies and procedures defined by IT. It might seem like you can get around IT's requirements and do something your own way, but that just perpetuates the problem. If you think IT is being unreasonable with their policies, find out why their policies are the way they are. You might discover that there is a good reason for it.
Think of IT as a finite resource -- don't squander it. I've never met (or worked in) an IT department that wasn't overwhelmed with things to do. Keep in mind that any system you implement may require some amount of time and effort for IT to support and/or maintain it. And keep in mind that there is always a Y2K or Service Pack N+1 or something like that around the corner keeping IT busy. So, as much as possible, budget your IT-time wisely. And, of course, plan ahead.
You may already be doing all of this, which makes your situation a more bitter pill to swallow, in which case I'd suggest helping other non-IT departments do the same, if they're not already.
And, of course, doing all of this doesn't guarantee that your IT experience will improve. But, I think it's a case of "you catch more flies with honey than you do with vinegar" and "you who are without sin may cast the first stone."
may require much more than you realize. Case in point. A developer needed a single column added to a table, and we had done test and acceptance testing. He wanted the column added during the day, so we put it in with an alter - no big deal right? After 50 seconds or so, the alter timed out, and took down users all over the country with it!!!! And the alter did nothing wrong, but it needed exclusive access to the table - and could not get it.
We had to step back and put the alter in in the middle of the night on a Sunday. And with our usage, we can't even get that time every week.
Bottom line? Get over yourself. You would do better to go talk to IT and find out WHY things are the way they are, and work with them, rather than against them.
First, the reason that IT organizations typically don't like technical folks outside of IT developing their own internal business apps, building their own infrastructure, or buying their own gear is pretty simple: we're the ones generally charged with ensuring predictability and security in the corporate infrastructure and we lose the ability to mitigate risk and provide reasonable levels of support with each bit of control that we give up. It's the same reason that the Legal department doesn't let you write your own contracts and why the Finance/Accounting department doesn't let you make journal entries.
I can't tell you how many applications, systems, and servers that my respective IT departments have had to inherit because the well-meaning business employee who developed or setup the system had either lost interest or moved on. When this happens we find that nobody left in the department knows anything about what is inevitably deemed a "critical app" by the department head (and is usually running on a server under a desk in a vacant cubicle). This scenario also applies to self-setup infrastructure of all kinds -- We regularly find rouge wireless access points, PCs and laptops bought and 'expensed,' application-ready mobile phones attempting to attach to our network and on and on.
The only way to deal with an increasingly technology saavy workforce wanting to do their own thing, in my opinion, is for IT to set clear policies and processes that allow for a certain amount of 'self help' but only within the guidelines of an IT ecosystem-friendly arrangement. We need to know about hardware and software you buy or make and we need to know where these systems and sub-systems reside, what data is on them, how that data is protected, who has access, and who is responsinble for maintaining them. In this day of increasing scrutiny (SarBox, etc), its more important than ever that we maintain some level of control.
Aside from all this ranting, I'll say that IT Leaders who do not realize that they are service providers at the end of the day are doomed to be loathed by business users. CIOs who stand fast with their arms crossed saying "no" to everything are obviously not familiar with the way a service organization is run. Unless a service-oriented culture is fosted from the top of IT, things will never change in your organization. The most successful CIOs that I know spend a lot of time with business department heads ensuring synchronization of priorities while also instilling in their IT employees a sense that proper, measurable internal customer sat is a standard part of doing business. Take to your CIO to lunch if possible and talk about this. I think you'll very quickly be able to tell if you have any hope of a culture shift.
There are some exceptions, but they'll charge you more money for the flexibility. That's the other way outsourcers make you money - precisely defining the scope of work and charging higher prices for anything outside of it. Sometimes that's a Great Thing - outsiders who want to charge money are often much more willing to do what you want than insiders whose reward structure is that they're a Cost Center incentivized to cut costs. But the kinds of bean-counters who outsourced your IT department on you are usually going to prevent you from getting the extra-value services if they can.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Simply break the machine in question. Be sure to inform the boss of the unfortunate situation. Not only will you get a day of light duty, but the problem in question will be IT's priority number one.
1. Their computer problem is much more important than any other computer problem that might be on your plate on any given moment. Oh, and they are certainly more important than you going home to the wife and kids or to catch the latest episode of Veronica Mars or whatever you IT people do in your off hours.
2. Even though computers are mysterious things to them, they know that it'll only take you a couple of minutes to fix any given problems they have with them. So, you can get whatever you were currently working on done, if you IT people even really work rather than surf the net and play video games all day.
3. The words from the Veruca Salt song in Willie Wonka and the Chocolate Factory, "Don't care how, I want it now!!"
You may know the following things:
1. It will be very tempting to work on the most obnoxious person's problem first just to get rid of them. Even though that person's problem may be irrelevant in terms of the organizations productivity or profits, since they won't let you alone you may take your valuable time and use it to work on it just to get some peace and quiet.
2. There is nothing more fun than to be pressured into working late to solve some irrelevant problem because you are being pressured into it by some obnoxious co-worker who may be important in the corporation.
Face it, most of us need some sort of layer or wall between us and them so that we can work on our manager's priorities rather than J. Random Employee's priorities. When you waste hours on someone's project and your manager comes and yells at you for missing your deadline on your real project, you're not going to be happy about how little "red tape" is in the corporation.
"MIT betrayed all of its basic principles."
I live in a regulated environment, and understand your issues with getting stuff done. However, there are generally two sides to every story. I will counter your discussion a bit.
You want RAM in a server. That company currently has over 2,000 server and they have a service level agreement that is currently not being met with the business. They also have people that use to take servers down to just do "one thing" and not document why they were doing it, then when someone else went to update the server later it was not in the state they believed it would be and it created more problems and thus the server was unstable after their upgrade or the downtime was far greater than expected.
So the I.T. department gets judged by the business on uptime and other service level agreements. They do NOT get charged on helping the business out. So they are very cautious on any change to the environment. They are so cautious that it has gotten ridiculous for any change to occur.
So what can be done? Well I would need a ton more information than you provided to make more suggestions. I will NOT believe that everyone in your I.T. department is a bunch of idiots and lazy. I bet that around 80% are average to good, 10% suck and the last 10% rock (Like every large company).
Now a few questions.
1. Do you have a CIO?
2. Where is the majority of your I.T. department located?
3. How does your I.T. department prioritize its' project?
Those are just the first three that come to mind. In short I need to understand the constraints on the department before any real suggestions can be made. It is far too easy to say "fire them all", and in most companies that would be a huge mistake.
Lastly, I can say that I have seen a company that making any changes to ANY router took forever. It flat out sucked, however the reason is that this company was part of a bunch of other sister companies and one parent company, and those same router guys use to make changes on the fly (quick), but then it would take down a sister organization for a day or so, until they realized the mistake they made. So because of the major impact to the other businesses those same router guys were not allowed to make a change without a ton of paperwork under the penalty of being fired.
The more I learn about science, the more my faith in God increases.
We outsource to IBM. The price they charge for ANYTHING is rediculous. Want to save money? Don't outsource. Don't put contracts on your server that give free reign to anyone to charge you the earth to change anything.
Right now we have several web servers on our Intranet. NONE have the options enabled for dynamic languages to access small databases (ODBC). Guess why? You got it. We outsource to IBM. They support our DB2. Hence everything is either plain HTML or a Programm that needs what DB2 offers.
I can not get a single flag in IIS changed so that we can use ASP to connect to a simple database. I can not put DATA up to a production server (static HTML content!!!!).
Yes, this is due to contractual obligations. However.. the people who sign those contracts don't listen to IT. Our ZOS mainframe does 97% of our processing and costs the same (ball parking and rounding here) than the 3% of processing done on our midrange. Why? We outsource midrange and mainframe.. but each midrange box is worth $40,000 per MONTH. *sigh*. Just give it to me and I'll go buy a new box every month and live the good life ever after.
Don't even get me started on IT Helpdesk support (also outsourced to IBM).
We're also not going to discuss upgrading single workstations.
Nor are we going to talk about how much they charge to load a program / PTF onto the mainframe.
Outsourcing only saves money if you are interested in not paying redundancies / wages / etc.
Excuse me.. I've got to go, I need a coffee.
Dammit, I pasted in a fury. Let me explain this one:
... just so people can publish static HTML documents!!! Pure instanity. At one meeting I told the web team that I could have their Intranet running on linux, apacahe AND have ODBC/PHP/Whateveryoudamnwelllike running with their current content. They said it couldn't be so (lots of reasons. Main one is: no servers on the network if they are not managed by IBM). I gave them to the URL of the box where we were 'testing' this idea.
"I can not get a single flag in IIS changed so that we can use ASP to connect to a simple database. I can not put DATA up to a production server (static HTML content!!!!)."
What I meant here is:
"We pay IBM to upkeep a web server who's job it is to serve HTML content."
It's a file server that people access by typing in a URL. Basically. Yet, no one on our side has the authority to copy files from a LAN drive to a folder on that server.
So, what do we do? That's right. We zip the files up and email it to them. They then unzip it to the wrong place (usually), restore the main HTML dir from backup, unzip the changes to the right place and then people see the new content.
To combat this we are paying for a brand new server, a WCMS and a whole team of people
Anyway. Outsourcing really can suck. I guess it would be OK if the terms, conditions and relationship with the outsourcer were okay.. but usually you run into the program the outsourcers are in it to make money. They don't want to hand control of the program back to you, or to complete a project when there is little hope of them maintaining it.
Anderson Consulting / Accidenture (accenture) ripped us off of more than 5 million, in one hit, then went for me. *sigh* We fired more than 600 people, and are still firing people to make up for the loss of 12 million $$$. It's like a ghost town here now.
Wait, dude, this is slashdot, I don't think you get it. You're being all rational and balanced. I admit, I was trolling a bit. You were supposed to respond with irrational vehemence. :)
The right answer is somewhere in the middle, not a bureacracy expanding to meet the needs of the expanding bureacracy, and not departmental IT Lords all deploying their own solutions, Linux here, Windows there. But I so rarely hear about anyone finding that middle ground. I've seen a balance at big tech companies, but a balance of centralized and departmental IT expending 75% of their energy in a tug-of-war. The departments and divisions of a tech company can sometimes effectively fight the bureacracy because there's geeks in all corners who know what they're talking about. At a software company especially, the product teams rule, they know it, and they can fight about IT issues on even footing with the IT bureaucracy. In most other industries, the key departments don't have that advantage, so at the end of the day the IT folks make the IT choices, always making noises about collecting and meeting business requirements, but free to say "no" without much effective pushback.
My basic point was about human nature. Even if you create that balance, with a central IT plus dedicated IT staff across the organization, eventually the centralized guys win because their chief sits at the table with the other C*O's and exerts more pull, making effective noises about standardization lowering costs. It's simple corporate politics. If that CIO sees the big picture and has some humility, s/he might end up leading an organization that does the right things. More likely, even with that CIO, the IT middle management underneath will still play politics and make arbitrary rules and decisions that benefit themselves and disempower everyone else.
On another note, I never meant to suggest that a NAS from Best Buy was a good choice for any office needs. It's just that 6+ month turnaround on upgrades or new solutions is what drives people to route around that crap and starting using things like that NAS, or worse, Microsoft Access. I guess it's kind of a similar phenomenon to the adoption of the PC and M$ software in big businesses in the first place, to route around the mainframe cult.
My general contentment with my job has always been inversely proportional to the size of the employer. This is regardless of whether it was a retailer, school, or non-profit. The college where I work now (I'm half of the IT staff) merged with a larger university recently, and it's gradually getting bogged down. The functions the university has taken over have become slower to respond, and now they're trying to integrate our activities into their change-management system, and it's going to slow us down as well. There are some legitimate reasons for why this has to happen (more complex systems are more prone to failure) but it's annoying as heck. (Fortunately our college has a large Mac population, and the univerity staff know next to nothing about them, so they don't interfere too much in that area.)
http://alternatives.rzero.com/
...someone gets to be an IT worker outside the IT department. In my experience, usually this happens because some dept. head is unhappy with IT. They think if they have someone they can control directly, they'll get things done 'their way' instead of the 'IT people's way'. So they go and hire someone.
This poor soul then comes into the company without any knowledge of the wars that ended up by spawning his or her job and gets all surprised because IT is less than helpful to him or her. If you think their job's simple existence means IT lost that war it becomes clear why IT reacts the way it does.
But feelings and corporate politics aside, usually and especially in complex environments, there's reason for what outsiders perceive as bureaucracy in the IT dept. This is not to say that sometimes structures ossify and start abusing their powers, by no means. That does happen, but I believe most of the time that 'bureaucracy' is just IT trying to cope with absurd workloads.
Remember that IT depts have been hit hard by cost-cutting measures. There's never enough warm bodies to tackle all the projects and the backlog is usually huge. Remember that, even if the 'IT person outside the IT dept.' is absolutely flawless in their skills, mistakes and security vulnerabilities, especially done to central resources, will ultimately be blamed on IT and IT will be the dept. expected to correct the problem. Combine these two issues and you begin to understand why IT depts. everywhere are pushing for centralized controls. There's no other way to make sure of a lot of vital things such as: changes are logged somewhere so people know who did them, why and more important, how to undo them if they have to; proper testing has been done before changes are implemented; backups are being done and spot checks are happening so those tapes are actually useful if they are ever needed; all (sometimes thousands) Windows workstations are having security patches applied regularly and anti-virus definition files updated at least daily; etc, etc etc.
We have what I think is a good plan where I work (state university) - and yes, I work in the IT dept.: you administer, you support; you want our support, we administer. In other words, if you have the root/ administrator password, you are self-supported. Why is that? Because our team of 8 people wouldn't have time to fix everybody's computer if all our 8000 users had the freedom to download and install whatever they want.
Although I believe 9 monthes is way too long for adding memory to a server, if someone is trying to do it right it's also not a 10 minute job. In our environment we do have to cope with state purchasing laws and regulations, for instance. Yes, getting a memory stick from buy.com and sticking it into the server is appealing, but it's illegal and that's not the IT dept's. rules. Beyond that, we want to make sure we're buying a trusted brand, the vendor has proper warranty, the server actually supports the part, the server downtime will not create other problems down the line. Not to mention, in times of tight budgets, checking if the additional memory is even needed. Maybe trying to be a bit more efficient in your code or database design would save the company a lot of trouble.
I've been an IT person for several places, some of which had no rules whatsoever, and some that had the nine-month memory upgrade syndrome.
I've determined that once an organization grows beyond a "small business", there cannot be a "no rules" approach. If there is, lots of money gets wasted on hardware for people that self-approve their purchases, and critical apps go down in the middle of the day. The apps aren't fixable until the only person in the company who knows the system gets back from lunch, because he has all the info in his head.
The other side can be worse. My last job was for a company that got the whole ITIL religion. Absolutely everything had pages and pages of documentation attached to it. Service requests got routed through several levels of helpdesk before they got to us. We had a full-fledged project management office that made us spend more time in status meetings than working on actual projects.
There must be a happy medium. Period.