Slashdot Mirror
Overwhelming Bureaucracy in the IT Department?
Nedry57 asks: "I am in the somewhat unique position of being a technology worker, who lives outside of the IT department in my company (a very large organization in the US). By far, the biggest challenge I face is getting anything done due to the bureaucracy that exists, within IT. There are certain tasks (i.e. anything that happens in the data centers) that I don't have the access to do. Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration), thus costing many times more than it should. The lack of agility is maddening, because I know we are missing significant business opportunities. My management is extremely supportive and despite our excellent track record of success in creating robust/secure applications--our work has passed audit numerous times with flying colors--we get no support from IT. Even senior management can't break through the barrier. I am very interested in hearing the experiences Slashdot readers have had in similar situations." How do you get your technology work done, when your IT department is more hindrance than help?
You don't. You fire them and outsource their jobs to India.
This guy is the head of my company's IT dept. BOFH
Oh, now come on. I don't want to seem the pedant, but I believe that describing paperwork as "massive mountains" qualifies as exaggeration. Nine months, perhaps, may not be an exaggeration - but I'm seriously doubting that there is anything remotely resembling even a small hill.
OK, so I'm pedantic. Sue me.
Ce n'est pas un vrai mouvement de robot!
and leave!
Seriously, it seems that you have fought the good fight. Your managers ahve supported you, you have been at this for a long time, without effect. You now have a choice: accept that it probably won't change and that you can live with it, or leave.
The real "Libtards" are the Libertarians!
It's simple. Either you get the buy-in of upper management, CIO, CFO, CEO and effect a change in the present system or you bail out and get a job in another company. You and your immediate supervisor, obviously an inconsequential middle manager, will hold no sway and make no changes. All that you and he will do is rock the boat and develop a bad reputation in the company. Get upper management buy-in or bail out!
P.S. It sounds like you need to acquire funding for a development and testing lab that is not under IT however, do not expect to connect such a lab to IT's network.
be grateful you have a job. it's obvious there is nothing you can do about it, so why are you sweating it? go with the flow and live a less-stressed existence. it's not worth creating ripples. the only people who judge you for your work aptitude are you and other men; no one else cares.
I guess it depends if you like your job or not. If you do, complain until it gets done, or until IT tells you it's impossible. If you don't, just sit back and wait for the IT department to do what you could have done yourself in 5 minutes.
Depends where you work and what your company is doing. Are you working for Oracle or IBM or any huge mega-IT company? Then, of course, the Powers That Be need significant background work before something seemingly as mundane as adding memory to a machine can be green-lit. Do you work for a hospital's IT department? Or a finanacial institution? Then certainly significant safe-guards need to be in place before changes are made. But if you're working for a relatively small, non-IT company and you have to jump through rings of fire for changes to non-prod system I'd have to wonder what in the world is going on.
$6.21 is the number of the beast before sales tax. Meh.
its easier to ask for forgiveness than permission
This is not an IT-specific problem: all functional areas in large organizations are vulnerable to this sort of bureaucratic barbed wire.
Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration), thus costing many times more than it should. The lack of agility is maddening, because I know we are missing significant business opportunities.
If you know that there are real costs associated with the lack of agility, you should a) document in detail the actual losses, b) present these figures calmly and respectfully, and c) gauge the reaction from senior management.
I want to drag this out as long as possible. Bring me my protractor.
Get a new job working for yourself or a start-up. Large companies (like the one you are working for) tend to have a lot of bureaucracy. Smaller companies tend to have less bureaucracy. Not to say this has to always be the case, there are certainly exceptions. Good luck changing the IT culture. Once a corporation or a department develops a certain culture or way of operating it is usually very difficult to change. Sorry, this is probably not what you wanted to hear.
Bradley Holt
Somewhere in the senior echelons of your organization exists a guy. This guy (likely at the CIO level or higher) is either willfully ignorant of the nature of the IS organization which reports up to him, or he's actively encouraging the situation.
If it's the former, you need to find out who it is that's allowing the inefficient environment to foster and take steps (and obviously "you" aren't the answer, but one of his peers or superiors is) to educate him on how things could improve.
If it's the latter, and he's actively promoting that method of interaction because it keeps their costs down, or reduces headcount, or whatever AND if he has the buy-in of his peers and immediate superior, you're screwed. I suggest looking to outsource your department's IT requirements to a 3rd party if you can't bring them into your own group.
Reason why there is hope for the future generation #364:
"I wish my grass was emo so it could cut itself."
Progress and stability are often conflicting goals. IT departments generally prefer stability, and that's why your deployments have probably been so stable and passed so many audits. Developers, of course, are charged with driving progress.
The real answer if you need flexibility with regards to "non-production stuff" is to not let IT have anything to do with it at all. Create a separate sub-net if you have to to keep the non-production machines off the IT network, and a firewall between your network and theirs to prevent any viruses, or other effects, from leaking from your net to theirs (this may require having to VPN through it just to work with these machines, c'est la vie). Keep the machines in a different room than the official server room. Maintain them all 100% yourself. Then do what you need to. Anything less and you're asking IT to aid in your development, a task they're probably not equipped to do while maintaining stability.
It's not uncommon for companies to have a "developer", "staging", and "live" system setup that are all completely independent, with some established mechanism and metrics to push products from one level to the next.
E pluribus unum
Moving to another company besides Microsoft?
Management can keep holding long meetings to find out why work is not getting done.
Once it gets to that point it most likely won't change, barring a CTO/CIO change.
It's infortunate, but that's how most companies operate. Top, down.
They stole your stapler too?!?!
... and with a subject like that.. you aren't going to here what you expect:
The military (USAF) had a very good IT setup (overall) that was basically setup the way you'd set up a good memory architecture.. you have a hierarchy of IT with the most used/essentialy tasks able to be done close to where they requests come from, and build upwards and outwards. Those local people were "fired" (in the government people don't actually get fired, but moved) if they didn't perform, and they were basically giving the keys to their kingdom. In the grand-scheme of things, it actually ran pretty effeciently, and we were never waiting on IT for more than a day, except in the most extreme cases.
That being said, the military has a pretty large vested interest in people being able to work and use their computers (ie, the cost of failure can be scrubbed missions which equates to huge amounts of money down the drain) so things tended to Darwin into a workable system. It sounds like your company's IT organization is just immature or flat-out poor (I don't mean in money... althought that could explain poor quality, also), and the powers that be don't seem interested in fixing it.
I've worked in similiar situations, except from the opposite side (the bureaucratic IT department). I recall having many employees outside the IT department being technically competent, who should have been allowed to work as they see fit, as though they were actually part of the IT dept. (assuming they just communicate with them) The issue was that we had to cover ourselves - ie, if we were responsible for something, then we sure as heck didn't want anyone, technically competent or otherwise, touching it. Efficient? no. It was just the way it had to be for us to feel like we were doing our jobs.
Register the editry.
It sounds like the bureaucracy is going to be tough to change. However, is it possible to get your group moved *inside* of IT so you can get the job done? It might require less work to do this and still let you get your job done.
It sounds silly, but if you can't beat 'em, join 'em.
Look, the people you are after are the people you depend on. We install your memory, we code your apps. We run the internets, we guard you while you sleep. Do not... fuck with us.
Dear IT Professional:
Please don't change anything about the way your IT organization does business. We love the way you and your team fail to communicate; the way mindless mandates from on-high drive pointless initatives; the way the latest technology trend shifts focus from project to project like the attention span of a two-year-old.
Especially don't pay any attention to streamlining the use of hardware and software investments that you've already made. You and your team need MORE MORE MORE to get this project wrapped up on time. Have you upgraded to the newest rev of our software? Can't you just taste the new-and-improved speed of our lastest hardware?
In summary, we love the way your IT organization is today, and wouldn't change a single thing.
Yours Truly, Your software & hardware vendors
The surest sign of intelligent life in the universe is that none of it has tried to contact us. -- Calvin & Hobbes
"Our SAMBA connection is broken. Something changed over the weekend."
"Nothing changed over the weekend."
"You sure about it? Why does the AD server report it's running Server 2003 now?"
"Oh that? We tried to implement Windows Server 2003 to replace our AD server, but we backed it out."
*boggle*
That conversation was with our IT dept. In any controlled environment, things should be thought out, documented and multiple sanity checks performed. Even a dev system can impact a production system if they run on the same segment.
Now, having said that, our IT dept tends to mindlessly enforce rules without thinking about them and getting them to wake up to new technologies (e.g., SOAP, web apps) is like trying to bring around a corpse with smelling salts.
A good IT department should make sure things happen in a controlled and documented way, but should also make it as painless as possible to follow the rules. They should be proactive so if you come to them with something new you want to implement. Not only will they know what you're talking about, but have already prepared a white paper of preferred architecture for performance & security.
A really good IT department brings something to the table.
The bitter lessons of a veteran coder: http://bitterprogrammer.blogspot.com
At a very well-known, well-funded, academic institute, I had to write a formal business case to submit to not one but TWO directors to justify why I needed an extra 512MB in my laptop...despite the fact that it would at worst be about fifty bucks and, regardless, it was a FREE upgrade. A "business case." Honestly. I didn't have to write a !#%ing "business case" for the laptop itself! The amount of time spent biatching over that $0.00 basically could have paid for the whole g.d. machine, gig included.
You do the best you can. If you care enough about the job to stay, I would make sure that senior management knows where the bottle-neck is. When giving status reports diplomatically remind people that "Item X,Y and Z" are not released due to delays in IT. I feel for you, my company was infected by middle management about 8 months ago, now releases that took me 30 seconds (literally I timed it) now take 2 weeks at the minimum.
Ignore them setup a lab in your work area for business development, then tell them hands off. Maybe they will get jealous.
But seriously it sound like your IT management needs a shakeup. Firing a few people now and then can also lead to improved performance.
lol
First off, 9 months seems excessive. Very little should take longer than a business quarter.
..don't assume the issue is on the IT side I guess is the gist of it.
However, in my experience every person outside of IT and security groups has this mindset that IT groups hinder them for no real reason.
I do not doubt there is bureacracy that slows every company's process. However, the fact that you want a change made to one system now doesn't change that these IT people are responsible for the effects any change might have on an entire organization. I don't know how many times I hear "But all I want is X". And that person requesting 'X' doesn't realize that 'X' has these 3 possible security issues associated with it. Maybe it won't effect his server even if it is exploited, but that risk has to be evaluated, approved and lord knows what else.
The fact is, every change *must* go through a certain amount of bureacracy to make sure all that it could effect have taken the appropriate level of responsibility.
My best advice is work through your own internal processes to see if turnaround time can be expedited. Maybe all they need is a motivated developer type with your skills to assist in making their change control system better. Or maybe there are things you don't see. Don't assume IT folk are just pushing your stuff back because they don't like you (though that could be a factor). If you can get a 'champion' type in your IT group that can help you get your stuff moved through the most efficiently.
But in the end, it is not up to you to decide what priority your request is given over someone else's. Even a simple request should be evaluated properly and must be given priority that is likely outside the IT drone's choice... Maybe your manager/director type needs to champion your projects to get them pushed through with greater priority .
Oh, and Bill said he didn't wanna give you your ram because you ate his pudding cup.
Seems like most IT Depts have one problem or another. Sometimes there's no structure to be able to get things done. Sometimes the management doesn't care, and hence can't get approval to get anything done. Or management cares too much, and you're spinning your wheels in meetings for most of the shift and can't get anything done.
Personally I'm considering getting out of the field. I love technology, I love playing with multi-million dollar servers, I enjoy helping users out of a problem (as long as they're reasonable about their problem. If they're not reasonable, they find the BOFH in me.) But the endless rat race in the IT field definatley wears on ya.
"If you insist on using Windoze you're on your own."
and your right.
Been there. Done that. I went to the CEO of the company the day I left and had a heart to heart for an hour. I was able to do this 'cause I'd known him since I was in my early teens. His wife happened to be one of my mother's best friends. Six months later he fired the head of IT. So it's really a matter of who you know.
I can't begin to tell you how many times I've fought this battle in both the public and private sector. Your best resource is your audit track record. You have to find a diplomatic way of putting a squeeze on them.
If you've got an internal billing situation for IT support, you need to find out how to cut the bill from them. This will pressure them to provide services. If the amount is to small to jump there radar you're pretty much SOL.
Alternatively, ignore them. Get your own budget from your own operation and freeze them out, pretty much the same as outsourcing to India. This will also get the attention of your mamouth companies budget office. If you can prove savings, it'll give you some more leverage against the IT rascals.
Go dumpster diving around their office after hours and see if you can find some malfeasance. Report it to the company auditors. Take over the IT dept. Man the barracades and raise the flags. It's a shooting war!!!
He and others in the IT department tried doggedly to get security noticed, only to be shot down by executive management. To paraphrase the CFO and strip out the gratutious profanity, "We're a meat company. We turn happy cows into happy steaks and happy pigs into happy bacon. We're not freaking NASA. We don't need to worry about our computers like Lockheed Martin does."
Several months later a virus hits the company and the phone system, which includes all sales offices, dies. I rush and get the tools to remove the virus in every hand possible.
Ultimately, as I was leaving the company, they finally hired a security manager. This was only because of Sarbanes-Oxley, and that person was given the role of a paper tiger--no authority to change things to be more secure, but a perfect picture for blame should something go awry.
When I left, I entered another office with other politics, but it is nowhere as bad as it was there.
--Chag
...but in some places it actually works.
:\
It is easier to beg forgiveness than it is to ask permission. Do what needs to be done, apologize later.
Just don't screw something else up in the process.
Karma: Chameleon (mostly due to the fact that you come and go).
I get on /. to try and escape this crap for a few minutes! Thanks a bunch!
In Soviet Russia, Chuck Norris will still kick your ass.
- Look for an external company that is dedicated IT support outsourcing (local, so they can be on site if needed).
- Invite them and ask them for a proposal to replace your current IT dept in some functions (make sure to get respose times and costs).
- Show them to senior management, an ask internal IT to make a counter offer.
IT needs to start treating you like a customer, not like a problem that needs to be dealt with.
what was your username again? *clickety click*
Once you have the contact, you need to work them. Make them realize you aren't a moron (i.e. a typical customer of theirs), and that you are willing to help them out where you can. I often offer to help out with simple s/w upgrades or to go answer some end-user's question about MS-Office or whatever.
You likely have to put up with their blabbering about their views of the OS/editor/language wars, or other geekout topics, but the idea is to keep them as a friend. Ping them from time to time asking about their kids, or their whacked out computer, or their modded coffee machine or whatever it is they are in to.
Also make them realize that you realize there's a bureaucracy that needs to be worked within. Offer to open support tickets for any work they are doing for you "outside of the loop" to make sure they get credit for the work. Also put in their name for "star performer" awards or whatever type of reward system your company may have.
pp.
It's a simple matter of complex programming.
About this time last year I was working for a large US govt and State subsidised hospital in the deep south. I won't mention the name of this place...http://www.umc.edu/ But said location was the biggest single waste of taxpayer money I've ever seen in my life. Ever.
I think you work there too.
I spent 8 months building a J2EE application stack, framework, data aggregation, system integration, etc. The darn thing can easily change the way that hospital runs, as it aggregates data from all their major systems into a coheasive, normalized, well formed data base with a clear, orthogonal object model on top of it. We used it to write a "system critical" application. It was beautiful.
Then it came time to pony up cash for the hardware and software to run all this. This place will drop -millions- on an IBM mainframe and _really_shitty_ HR software and consultants to make it work, but they would barely budge in making a major infrastructure and integration investment, when the entire campus was clamouring for it. The place is now able to deliver highly integrated, damn slick applications (thanks to me and my two cohorts who worked on it) but they dont -- because of the bureaucracy.
They also treat their employees like shit.
Never the less, after negotiating prices with vendors, we got the total cost for hardware / software to under $60k, (nearly 1/3rd the list prices for the software/services!!) and I was still walking around the office literally saying, "How many dicks do I have to suck to get $60k of funding around here?"
No kidding, I said it, out loud, repeatedly. At that point, I was actually -trying- to get fired. I knew the place was a hell hole, even if I did learn a ton, and really enjoy most of the work.
Eventually, we managed to get our funding without having to "take one for the team." And yes, I did in fact tell my coworkers that if it would let me get some goddamn work done, "I'll happily take a shot or two in the mouth if it means we won't have to put up with this bullshit for another three months."
Oh, and once we did get hardware, it took EIGHT MONTHS for the admins to get a STABLE INSTALL OF REDHAT AS on it.
NO KIDDING. EIGHT FUCKING MONTHS. I had to reinstall our software stack FOUR times because they were SO INCOMPETANT at INSTALLING A FSCKING OS. GAH!!!
(Ed Anderson, you're not incompetant, but you've become totally complacent. Thanks for at least trying for a while.)
However, David Massey, Mike Smith, Jerra Anderson, you should all be fired for being stupid fscks. Oh, and Steve Roberts, don't forget to say a prayer of that coffee that you charge your subordinates to drink, even though according to the CIO, "there's money in the personell budget I can't spend because I have no where to put it." Meanwhile, they want to pay Sr. Application developers 36k/yr to live in that shithole of town. Guh.
So glad I'm out of there.
In cases like this, where a problem is more groupwise than individual, it can be hard to solve. The situation is that the I.T. dept do not feel accountable, and prefer to pursue their own goals and practises, whatever they may be. The best solution is new blood in the I.T. dept, preferably lots of it. If that can't be achieved, a hard line from management can help. Sometimes, even that's not enough. If the dept are still (somewhat) fulfilling their function, then it's hard to edge them out based on simple frustration. They just fall back on "we do what we can/should", which is exactly what they want. I often deal with this kind of problem, and it's incredibly frustrating. As I have no real power in my org, I mostly have to just live with it, and do the best for myself. But if you have the backing, shining a light on the problem (preferably a light which reflects things in dollar terms, an unusual piece of the EM spectrum), may help. Alternatively, there's the "just do it" approach. Buy a development server, do whatever the hell you want without asking, and wait for them to start screaming when the most important systems start to move outside their sphere of control. Wag the dog a bit. Cheers, -MP
http://melbournephilosophy.com/
Problem solved!
Just realize that you never leave high school. There are still clics, there are still popular people, jocks, nerds etc... The sooner you realize this the sooner you realize it is hopeless. :)
I can only please one person a day. Today is not your day, and tomorrow does not look good either.
I think the most important thing to remeber about Large Companies is that most large companies are old companies.
Most Old Companies are very slow. They are slow becuase they have learned a lot of very painful lessons over the many years. They purposfully slow things down to insure that all the old lessons and painful experiences are taken into account.
The way this is done is through paperwork, meetings, agreements, etc... Think of it as the company is protecting itself from the stupid decisions of the past.
Fantasy remains a human right; we make in our measure and in our derivative mode... -- JRR Tolkien
Make friends with somebody in IT, grease the proverbial wheels. A case of beer can do wonders for motivation.
Don't! When management complains about productivity, calmly give them detailed chapter and verse written during your "on call" downtime. Then management can make informed decisions. Which they cannot if you keep covering for IT.
I'm with you. I'm a UNIX admin in a really big company now. The smaller comany I was in was recently purchased by said big company. It's been like swimming though syrup trying to get things done anymore.
One way to make things happen fast is to say that NOT doing X will incure a risk to the company. You say, we need RAM this week or production app X may break. Every day send out an email to the effect of "6 days until system failure..." The countdown of doom.
Another is to use process management judo. Attach your need to the success of a project, make installation of that RAM a line item in MS Project. Then just sit back and let the project manager drive. When he asks about that item you say, "I have been unable to get a response from IT on that. Oh my, I hope it doesn't hold up our project." The project manager will go nuts.
Learn to use your bosses to your advantage. You boss calls there boss and things will happen.
Remember that in IT you're dealing with a bunch of geeks. Aspergers syndrome, mild Autism in individuals is par for the course. Learn to recognize those people (hint, they're the ones with all the talent and strange personality quirks). The communication skills of those folks can be terrible. Get over that and make a friend that can be a resource for you in the future.
Everyone is accountable for their actions. It is OK to call the boss of group or person in question and politely say, "Hi, I put in this request some time ago and haven't heard anything back, can you help me figure out where this got dropped?" No manager wants his team to be percieved as dropping important tasks.
Finally, be sure you're not asking for things you have no authority to ask for. If you're asking that company money be spent then you should have written approval. If you're one of those people who constantly insists they need things that cost money or require major time commitments from others then they are right to ignore you without an approved project and management buyoff.
--Chris
Sometimes it's self-preservation on the part of the bureaucrats.
I've started insisting on emailed instructions and approvals from my boss and other internal clients, for protection when controversy erupts. Managers tell me we have a small, friendly company, but it doesn't feel that way when the CEO calls a meeting on five minutes notice, requiring the presence of everyone between him and me, to discuss something I did based only on someone's verbal approval.
org.slashdot.post.SignatureNotFoundException: ewg
Some number of years ago, I found myself in charge of a private infrastructure. We had maybe 50 servers and 400 users exchanging sensitive information completely seperate from the main, public network.
Because of the percived importance of uptime on this network, everything required mountians of paperwork. Installing and removing nodes from the domain required three administrators, setting up a new machine required a month on a private VLAN being monitored by a sniffer, memory and hard drives were obselete before they got to the customer.
Anyone who ever worked around an UPS knows how they die. They give plenty of warning. Having an UPS fail is a rediculous way to lose your backbone infrastructure.
My predicessor had done a wonderful job of installing an UPS for every router and switch in the datacenter. Problem is, both power supplies in the routers and switches were connected to the same UPS. In cases where an UPS was about to fail, he unplugged the UPS from the wall and plugged it into, you guessed it, another UPS.
He didn't do it out of ineptitude; it was done because the only option was to clash heads with the IT overlords. They would require studies about how many UPSs failed and if it failed before the MTBF, they'd want us to try and recover money from the manufacturer. They'd want contractors to come in and examine the UPS to bid on a UPS monitor and replacement contract.
In short, asking the overlords was like asking to be turked by a syphalitic bear.
So, some BOFH, overwhelmed by the prospect of repairing the power system, chose another path. He walked over to a failing UPS and simply turned it off. He was the only one with the access to turn it back on, so he had no reason to worry.
Within two hours, all in-progress meetings were cancled. The Supreme Overlords demanded from on high that this lowly tech was to get a blank check and a blank trouble ticket (approved by the Supreme Overlords) to do whatever he needed to do to prevent that from ever happening agian.
Electricians installed two seperate power feeds into every rack.
Each power supply got a seperate UPS.
Old equipment was updated.
Everything was strawberry fields and unicorn giggles after that for the infrastructure department.
Now, to answer your question: You have something that someone wants. Hold it hostage till you get what you need.
I'd rather you do it wrong, than for me to have to do it at all.
First off, your position is not unique at all. Perhaps it's unusual in your *exact* location/department, but rest assured you're not the first tech worker outside of IT.
Secondly, this is really your boss' problem. If they're supportive - great! Have them find a way to cut you a check, pass you some cash out of petty cash, bill their company credit card, whatever it takes for you to have a couple of piddly sticks of RAM shipped to your mail drop so you can pop it into your server. Then, have them duke it out with any BOFH ( and their bastard boss ) when they get pissed off about you doing their job for them.
If they don't or "can't" do that, they're not as supportive as you think, nor do they have the clout that you think.
Honestly, I've seen this type of problem many times before at many different companies, and it boils down to one thing: your poor overworked, underpaid staff down in IT already have their hands more than full trying to keep things going, and they've been burned more times and in more ways than you could ever imagine. They're busy trying to keep production servers running, and you're bothering them with a request for a non-production machine that seems to be doing just fine ? Your request, if valid, is so far down the food chain, might as well just say no to it. You might stand a better chance of getting a whole new machine ( configured to your specs as much as possible ) rather than having a part of it replaced or upgraded.
In fact, ordering a new machine is what I would indeed recommend, if you've tried just asking for more RAM _and_ asking your boss to authorize the purchase and do-it-yourself.
I would also recommend looking into the possiblity that your division/group/you/whatever could have a small set of machines and possibly even test network entirely outside of the realm of responsibility of the IT department. I once worked on a development team where making that move was the best thing we ever did for our own productivity and sanity. We couldn't ask the IT department for help, but... we didn't need it, having their involvement just got in the way. To our surprise, the IT folks loved the idea, too, as long as we kept the networks separate... which wasn't as much of a pain as you'd think. We still had our "main" machines on the corporate network, where we'd get mail and interface with marketing's Exchange-server-based crap, er I mean meeting schedules and such, but a large percentage of development work and all testing was done on the test network. Our group picked up a bit of a budget for equipment, and that was it.
I'm sure some corporate policy might prevent such a move, and I'm afraid that you could end up in a situation where, without some real leadership from someone above you in the company, nothing is going to happen. In that case, I'd have to recommend just doing your best, documenting the refusal of others to help out, and looking for a smarter company to work for if possible.
Consider the low memory condition of the server a "stress test" for your software... and keep asking, making sure that everyone sees your request as *necessary*. If the request sounds like "this works, but it's work better if...", then the extra memory isn't really *necessary*, is it ?
Why you already have?
Mice buzz word usage! You are the man!
Improved POI, data integration, seamless user interface, data aggregation.
Jackoff.
do it yourself.
Yes, if the IT dept is so difficult, start your own with your own machines and staff. Keep it discreet, obviously. Get your work done, and don't go shooting your mouth off about it.
#6495ED - cornflower blue
Now I run my own company with lots of production severs.. No paperwork required, and I've automated most stuff.
If you are stiffled, go out!
-- these are only opinions and they might not be mine.
Meticulously document how much of a barrier the IT department is to productivity, and why you don't get things done. Keep a record of every e-mail, and make sure all communication is at least repeated in summary by e-mail, so you have proof. Present the evidence to senior management when they ask why things haven't happened.
Ultimately if the management chain doesn't see it as a problem, then it's not. Or rather, it's not a problem you will ever be able to do anything about. So once you have that documentary proof, by all means sit and read Slashdot or twiddle your thumbs while you wait for IT to do their jobs. Or even better, use the time to experiment, learn, and gain skills.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
It often comes about that some business unit thinks that they can perform IT functions better/faster/cheaper by hiring a few people and make an end-run around the IT department. And they can...for awhile. And then something comes up that their junior admin can't handle, and they call IT. Or it turned out that they don't do backups, or RAID, or whatever, and they lose all their data.
IT is bureaucratic. Its because we know what we're doing, and you don't. We've made all the mistakes already. When you come to us thinking you know what to do, we create roadblocks to make you give up and go away. Then we just do it the right way. And that takes time and money.
To get the type of reliability businesses have come to expect is hard, and expensive. You can't see the difference between a white-box server and a brand name, but we can. You can't understand why we can't just use a perl script to run backups.
If we let you have your way, you'd buy a couple of memory modules at compUSA, walk up to the server, and drop them in. Why is that a problem?
1. How reliable is the memory? Is it warrantied?
2. Do the timings match the memory already there?
3. Does is have the proper error checking?
4. Did you use an anti-static wrist strap?
5. Did you tell the other people who use the server that it was going to be down (you did turn it off first, didn't you?)
6. What if it fails to reboot. Are you going to fix it?
7. Does the OS support that much memory?
8. Did you introduce any bugs by the change you made?
I could go on and on. The bottom line is, a multi-user server requires complex management, and skilled administration. That is what we train to do. So let us do our jobs.
At EA we were working with internal websites and opera.
A website developer came down to test it on one of our pc's. With 256mb of ram it took Opera about ~90 Minutes to properly load.
He turned off the computer went upstairs and within 5 minutes had 1gb of ram in that PC. Booted it back up opera took about 5-8 seconds. And that was just for a 10 minute task.
Efficient, and the way it should be done. An employees salary - Time wasted ratio is way more expensive than the cost of memory or "The right tool for the job"
Interesting comments...
I've been in IT for close to twenty years in a couple small startups to some multi-nationals and in my own consulting business. One thing that lots of IT folks lose sight of is that IT is first a support organization within the larger organization. If the larger organization is sufficiently forward thinking, then they can (arrg, PHB-speak) *leverage* IT to be more competitive. But IT folks still have to make sure the website is up, the file server is accessible, users can login, etc., *before* you start thinking about the add-ons.
If the business doesn't want to spend money on the servers, then document what the consequences and benefits are for their decision. Don't just write that they'll have slower machines, but play Devil's Advocate and write up the business case for not adding memory.
Or, figure some way to optimize your resources so that less memory is required. This can be as simple as turning off services, or as complex as setting memory and processor caps within the virtual partition. And if you've tried all these and you're just short of memory, let them know.
In my consulting business my first goal is to keep my customers' infrastructure running. Next is to save them money versus some other consultant. Sometimes they need to spend money up front to save more down the road. Let them know if this is the case.
Here's what you do: take a simple task like adding RAM to a non-production server, and go through the entire, exhausting process in letter-perfect fashion, meeting every paperwork, audit, and permission requirement. Along the way, document every minute you spend on the process, showing exactly what you're doing, how long it took in minutes, and what requirement you were meeting. At the end, create a spreadsheet showing in careful detail that adding a $500 SIMM actually cost the company $5,000 in processes.
That spreadsheet becomes the club with which your managers and directors can beat the IT department because they're effectively offloading cost onto you at a rate of 1,000%.
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
I usually try to figure out ways of either circumventing them, mocking them, or getting them to want to do it because not doing it hurts them personally. Three examples:
1) Circumvention: Recently, I needed a DNS change to point an existing subdomain to a different IP address. Our not-very-useful IT project manager told me they needed to come up with an LOE for changing the DNS entry. Three days later, they told me they hadn't had time to calculate the LOE and would not be able to complete the change by the following week's deadline.
I went to the head of our corporate marketing and branding group, asked for her help. Even though this is a very large corporation with more than 30k employees and a very significant IT organization, within 10 minutes, one of the staff members on the marketing and branding team physically made the DNS change herself.
2) Mockery: I needed our web team to add a link to the bottom of our company's homepage linking to my program's home page. Three weeks later, the guy who was going to do the change saw me in the hallway and asked me if I had lined up testing resources from our testing outsource company to make sure that the link worked.
I responded, very loudly and within earshot of the web developers: "open bracket a href equals quote h tee tee pee colon slash slash my dot domain dot com close quote close bracket My Site Name open bracket slash a close bracket and then click on the damn thing"
2) Fool them into wanting to do it for their own purposes: We decomissioned a website a few months ago, and it is no longer publicly available. However, we've kept it around while we make sure we got all the old documents. However, we are still getting monthly reports extracted from the back-end database. I contacted our IT reporting team and asked them to stop delivery of these reports since they're no longer needed. They sent me a form I needed to fill out justifying why I needed these to stop and aking for VP signatures and notarized copies of the marriage certificates for every gerbil I've ever owned.
I told them they had to be kidding, then I set up a rule in Outlook that automatically bounces back the reporting emails to them and deletes them from my inbox. I don't have to worry about it and once they start getting these every month they'll try to figure out what's wrong and fix it. Once we fully bring down the system, I imagine that the report engine will start throwing all sorts of error messages and they'll see fit to do it on their own if the auto emails don't do the trick.
It's sad that IT, something that shoudl enhance productivity, has become a huge obstacle for us to do business.
***Foucault is watching you..***
The place I recently left - had reached around 20% efficiency.
... from the outside it must be a nightmare, if it's like that from the inside.
:)
Literally, one in five days was spent working, the other 4 days spent fighting for the right to work.
As one example (an extreme one), a simple electronic phone book which took around 2 weeks of work - had (at the time of my leaving) been in discussion, revisitation etc for over a year.
I got made reduntant - why, not because I had no work, but because the work hadn't been approved to proceed after 4 years.
Anyway - enough ranting
PS: Around 30% of the staff were in the same or a similar pickle to me
EMail: 0110001101100010010000000110001101110010 0110000101111010011011100110000101110010 0010111001100011011011110110
IT operations are not independent. There is always a senior-enough management level that can decide to override an IT organization's policies. After all, if the IT group isn't meeting the business need, then it is failing no matter what the uptime statistics say.
It sounds like you are really running up against a difference in organizational priorities. For whatever reason, your side of the issue is deemed less important than the arguments advanced by the other group. It may be that there are technical issues you are unaware of. It also may entirely be a political issue: competition between groups for limited resources.
Here are a few of your options:
One of these will probably suit your temperament...
People can die when things go horribly wrong.
That being said, people can die when hardware and software changes aren't implemented immediately too. If a doctor can't perform a procedure because of some issue, it gets taken care of very quickly.
I'm still leaning towards the "culture" comment. I really think there are different ways to interpret laws that can result in very different workflows.
I once worked for a financial institution as a Software Engineer. It took them 8 months to install a debugger on my development machine. That's the way they do things. It had to go thrgough security, the IT heirarchy, and the change control group. The fact that I couldn't be productive without a debugger didn't even enter into the equation. If you want light on your feet, go work for a software company. I bet you don't worrry about whether or not they are going to pay you this week. With security comes overhead. Deal with it...
I am not sure what senior managment means in your organisation but if what you call "senior managment" can't get the IT department to change, then the organisation has bigger problems. What I suspect is that senior management actually could get the IT department to change but thinks it is best to do what the IT department says and tell you that they can't change anything.
If my suspicions are true then you should listen to the other posts and take a good look at your job and see if you would be best served by finding a new employer.
To make IT believe they spotted and fixed the problem. It's an ego thing; if you tell them the problem and the fix, they have nothing to do anymore.
Tell them your machine is really slow lately and the harddrive runs like mad. Sometimes you get a 'Not enough RAM' error, but you have no idea what that could possibly mean.
Chances are you'll have your stick within a day.
Alternatively, ask your cute receptionist to go over in miniskirt and take a few sticks of RAM; they'll never know what hit them.
'For we walk by faith, not by sight.' II Corinthians 5:7
I've been consulting for five years now and my firm has many clients in many industries and I can tell you with great certainty that this beauracracy exists almost everywhere. There are exceptions, but they are... exceptions. So, I'd say quitting your job is a bad strategy, because it is pretty likely that your next job will have stupid beauracracies similiar to your current job.
Here is my two cents:
You're better off adapting to the system and learning how to get stuff done in spite of all of the b.s. This usually means forming personal relationships with key individuals who can help you. You need code promoted to QA? It's way easier if the CVS admin is your buddy. Need to rebuild a test database? Helps to have a DBA friend. You get the idea.
"The bureaucracy is growing to meet the needs of the growing bureaucracy."
-Leonard Nimoy
I just did ... you can too.
It can be done in as little as 3 business days.
No paperwork required.
Just have all the indians say they will quit in the next 2 weeks.
IT STILL WORKS !!
Just to play devil's advocate, Enterprise-level IT has a lot of responsibilities. They have to manage physical assets, licenses, warranties/service agreements, as well as the "usual suspects" of keeping the network and PCs up and running.
My suggestions...
* Document it all. Yes, you've got a mountain of paperwork, but if you can make a business case for change, you have a better shot. If you can show lost productivity (due to server issues) so much the better.
Documentation is also a CYA in case something rolls back your way.
* Get to know the IT folks. For all you know, they could be seriously understaffed and simply unable to process requests in a timely manner. At the very least, buying lunch, dinner, or even a round of beers may help get you some name recognition.
* Think twice about rolling your own solutions. Being out of compliance with licenses can cost the company major money. Even if you're using a fully licensed "spare" PC from an empty office, there's no guarantee that IT won't be taking it away at some point.
Just because your current management approves of an "in house" solution doesn't mean the next managers will. Quick and dirty solutions can find their way into production...
* Leave. There's a lot of CYA paperwork going on, and you're not going to be able to fix it.
I work in the IT Department for my school district. Its unfortunate that even we have these same issues. I think it comes from the high up people that have no idea what a computer. When we mention a hardware upgrade to our server, they are like, "What, that server is only a year old. Its practically brand new." No matter that it runs 24/7 with over 5,000 users accessing it. I feel that if you want to work in the IT department no matter what position it is, you should know about computers. Even if its in the basic areas. All those suits sitting in the expensive leather chairs wearing Armani suits should let auctual Techy people do what they do best.
Click Click Bloody Click PANCAKES!
The CEO, the CIO, and the CFO. Seriously, stop toying around. The business is being eaten from the inside. Lost business opportunities affect the whole business, not just your department. When the executives hear about lost business opportunities because of some droids' managerial incompetence, they will want some answers. The CEO will make sure something happens, the CIO will know what to make happen, and the CFO will put concrete money figures on the effects.
I used to work for a big company that suffered from the same problem. In some production environments, the developers couldn't even look at the log files without filing a request with IT.
When it comes to dealing with projects. We'd mention that we'd complete development on 9.114.1.3 on July 1st, it'd be QA'd by the end of the month and then passed to IT for deployment.
Managment would then be the ones that took our deployment spec and dates to IT and pretty much force them to commit.
The lack of agility is very frustrating (and one of the reasons i left) but make sure that your management chain are aware that it's not your fault. We had root at one point, and a misguided developer managed to clean out / with an rsync typo.
OTOH our IT was stable as a rock. Our servers were rarely down for more than a few minutes a year. The UPSs and Gensets all kicked in properly whenever the powercompany managed to sever both the redundant powerfeeds simultaneously. In 4 years we were only down once due to hardware failure and once more due to IT's mistake.
Seriously. Buy your own server. Hell, start your own data center. IT loves it when some other department helps them out by getting things done themselves. I mean, as an IT guy, why should I have to do everything. Let the other departments get their own production systems if they like. When the time comes to integrate things, we'll figure it out somehow. And I love it when some group has spent big bucks for something that doesn't work! Gives me a challenge to figure out - like how to fix some now critical system that never did what is was supposed to - and do it without any budget. Whooo-eeee, I remember why I joined this group now! Hot damn!
Some of the suggestions amount to becoming what's part of the problem : people driving a personal agenda on company time, and damn the consequences.
While this is ok as long as it doesn't conflict with the interests of the company or screws somebody out of their job, it's what's at the root of a lot of efficiency problems within large organisations.
A handful of politically clever people tend to position themselves on strategic positions within the organisation, and purposefully create bottlenecks. They control the resources - be it knowledge, access privileges, hardware, funds, you name it.
This control is then used as leverage to further their own personal agenda. I'm sure most of you recognize or remember some of these pricks from current or past working environments. You need something, file the papers and after waiting a few days/weeks the bureaucratic treadmill stalls somewhere. It turns out you landed on some controlfreak's turf and he/she doesn't cooperate, simply because *you have nothing that can further their cause*. Doesn't really matter if it's their job to help you. If you escalate problems like these up, it tends to fly up so high that when the shit comes down again, everybody gets splattered, including you for stirring up trouble.
Usually these guys have big umbrellas too, because, you know, Bill's cool; when in trouble ask Bill, he get's things done. Of course he does, Bill is running the whole dog and pony show because he controls resources people need. Power.
I find it striking that these kind of shenanigans are still par for the course in most companies. I dare say it has to do with the prevailing attitude in (bad) middle management ? -- a lot of them fit this picture perfectly.
When you try to streamline business processes and workflow, introduce roles, responsibilities, accountabilities, and *gasp* add time windows for tasks/subprocesses, you'll notice that it's these guys that will most actively stir up the environment against these pagan, new-turk ideas. (It erodes their power.)
If you come accross one of these guys, you have a few choices.
- Surrender. Get some kneepads and chapstick. Brown your nose.
- War. Beware, they travel in packs. You probably won't win.
- Acceptance. Decide you don't care. It's just work, have a life outside it.
- Work your way around them. (I'd go with this one.)
How would you feel if the network guys just needed to hack a small piece of your code? Surely you wouldn't mind if they checked out your source code and fiddled around? Seriously though, you write that simply adding RAM to a development box is a simple task. You also write it's a huge network, which is more than likely more complicated to run than you make it out to be. Granted you probably could install the RAM, or perform other simple tasks, but what happens when you cause a problem?
I've worked network support for a while, and this type of tension cuts both ways. The stream of requests and favors is constant, and very distracting to the average net admin. If coders had someone interrupting them every 15 minutes, they would slow to a snails pace as well. If you really want change, you'll most likely have to get a new job elsewhere. You could try to convince the board that it's a real problem where you work now, but this would require your case to be fairly airtight. You do have evidence of the potential costs savings don't you?
I am sure you are highly qualified in your area. But if you are unable to change things yourself, and the folks in upper management don't believe your numbers, then you're out of luck.
the last guy with this problem got similar responses
filmcritic.com - Movie reviews on Internet time
And they always have numerous reasons why they cannot do what you want. I've worked in large companies where the scientists agitated and finally brought about an entirely separate IT organization. Traditional IT, along the lines you describe, was purely overhead. The scientific organization was more user-responsive, and was more involved in, you know, actually creating value. Look at Big Pharma for some examples of this. One way to start weaning yourself from IT bureaucracy is to first set up a "test lab". This is for testing new technologies in a location convenient for your end users. This can be gradually expanded, and used to sell your users on the idea of doing things, you know, their way. And like all empires, this can grow over time. You, in the meantime, can build a reputation for customer focus and innovation. Sure, it can take time. The other thing you can do with hidebound IT is to first document the business requirements of your users, and then ask IT to provide a solution which meets these requirements. I usually meet with the IT group's solution architects informally to see if we can settle on something mutually acceptable. At these informal meetings, I also explain the business stakes for the users of getting what they need. In addition, always include a few wildcard requirements, such as: "users will experience no degradation of service or performance as a result of the technical solution" Who would argue with that? So everyone will sign on. Yet I've been able to relentlessly use this to cut away at bureaucractic "requirements" to get my users what they need. There's more that can be done, but the key is to not approach this as a technical dilemma, but as a people problem. I'm curious to hear what other people think.
Management is powerless to stop Bureaucracy, they're part of the problem. Most CEO's are responsive to issues that have a direct impact on the bottom line. I would say that you take that exact posting that you have here on /. and submit it to your president anonymously if you desire.
If you don't hear/see a response, then post it for shareholders to see. Nothing effects major change like an overnight drop of 5% in stock price.
Good security is based upon reality and common sense. Common sense is a function of having common knowledge.
Dear Nedry57,
Thanks for your post. We have logged your request and have emailed a copy of this message to all interested parties. As soon as we reach your post in our reading queue, we will update you with a status. Once we receive funding for any and all requests you have made, we will proceed. Any further documentation you could provide about your problem would be helpful.
Sincerely,
Slashdot Reader
I can see your point of you that you want to deliver changes faster and I can understand your frustration with the IT department. However from their point of view their jobs maybe measured against stability not change and so they take such a risk adverse approach. Maybe your Senior managers could negotiate different terms of service for your servers to let them get on with their job and you with yours
You have two recourse:
Get the person in charge of IT to buy in on what you need. When you run into a bottle neck, email the pserson causing it, CC the person in charge of IT.
If the person in charge of IT won't Buy in, go to his boss.
You will need a plan of what you want, with the benefits. Such as:
It will save the company X, it will increase our profit with the same revenue(i.e. Bigger bonus for people whose title starts with the letter "C")
You other choice is to neautralize the stakeholder that is causing the bottleneck.
This can be done by getting upper management buy in and just ignore the complaints from IT. When the complain say "You need to take that up with the CTO", or whomever.
If you have problems getting access to the server room, go straight to the person who actually types in who gets access and say "Bob Smith wants you to give me access to the server room". When they say "Who?" you say "you know, Bob Smith, the CTO. The guy who approves your job."
OTOH, don't just toss out all process and accountability. There should be documentation whenever a system is changed, and rogue groups can do a lot of harm.
Break down the problem, find out how much time it takes to do something. Break it into FTE*. Then have a solution that also breaks it down to FTE. If you solution requires less FTE, but maintain reasonable accountability you should get buy in.
All this sounds tough, but once you get top level buy in, and you are successful, you will be a shining star. Hell, you might even end up in charge of IT.
As always, dress nice when you begin doing this. Yes it shouldn't matter, yes you will get your nice close messy. If you know you have a meeting with upper managment dress like they do. Theye where suit and tie, you where suit and tie.
Also, a proper fitting suit is not uncomfortable.
I should have to give dress cues, but this is slashdot.
*FTE: Full Time Employee.
The Kruger Dunning explains most post on
There are certain tasks (i.e. anything that happens in the data centers) that I don't have the access to do. Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration)
So, let me get this straight...
User is frustrated because request to make standard servers non-standard with a custom request in a datacenter requires paperwork and time. User is upset because IT has formal procedures for change control, service level agreements, and standard hardware configs. User doesn't get ram upgrade and posts rant to Slashdot.
User is technical, probably dual boots their desktop to non-supported OS, probably hacks computer stuff at home, probably very smart and capable of supporting five or six computers by him/herself.
IT department probably supports 1000+ machines, and that number has doubled in the last year or so while staffing has been cut.
IT department probably has 200 servers per admin and only maintans this ratio by with consistent server deployments that maintain standard configurations.
A good IT organization understands the company's business.
A good technology worker needs to learn to work with IT to get what they need. You would probably be able to request and justify 10 servers and get them in the time it takes to get a one-off upgrade
The lack of agility is maddening, because I know we are missing significant business opportunities.
Lack of planning on your part does not create an emergency on my part.
Learn how your IT organization works. Work with it.
we get no support from IT. Even senior management can't break through the barrier
Perhaps you don't see the big picture. Perhaps you don't see the corporate IT budget and where you/your team/your project is on the priority list for that budget.
I'm sure there are all sorts of IT departments, but the *good* ones understand the core business and know what's important to the company's bottom line. If your IT department doesn't understand that, then I'm afraid you're going to have to become the IT liason and teach them. Provide them with your requirements well in advance so that they can plan proper deployments. Work together so that you can understand the pain points of IT, and they can understand your hardware/support requirements and the *value* that this will provide to the company.
So called 'departmental' I.T. solutions, along with the developers and admin who support them, tend to be focused on line of business needs. Their target-locked on needs based designs. From their perspective, this means cheaper, faster to live, and more adaptive solutions that enable real world business. From the internal "CIO" direct reporting structure, these represent a problem because they tend to be built without regard to what the "strategic I.T. direction" may be, and their lower cost and faster results make the big fancy projects look bad in the short term.
So who's right?
Well, both are right -- and wrong. If the CIO's vision is really as good as he thinks, then clearly the departmental application is a problem. Mostly, the CIO's vision is corrupted so fully by the time it impacts departmental level projects that what's left of that vision is little more than an attempt to garner a larger 'empire' of control regardless of cost of feasibility. That's not even counting the "cool tech" projects which often make no sense.
I've definately seen both as an outside consultant (and one who really does try to be honest about his work). In the end, you have to follow your own joy. Do what are paid to do as best you can, and always remember to "dance with the one that brung 'ya" -- remember your loyalties and morals. If doing those things is too frustrating or is mutually exclusive, its time to find another job.
AP
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
It seems to me this is an increasingly common and distressing phenomenon, and I'm convinced that it's driven largely by the dramatic increase in regulatory compliance concerns. Case in point: in 2002, I collaborated with one person for two weeks and redesigned our company's website. I work in the marketing department, but I had a little bit of web design experience; my coworker is more of a hacker type who doesn't really have a title, but he's in I.T. We got the job done, great. Now, almost four years later, I'm still getting requests to make website updates, a procedure which has turned into an eight-headed monster due to the slew of 'systems' which have been put in place to regulate our public website content. This means that making a single, minor textual edit means filling out a Request for Testing form, logging into a semi-automated online QA application and submitting a second request for testing, making the change and updating the CVS repository in staging (which, incidentally, took me a year and a half to be granted access to, since I'm not 'officially' an IT guy), then sending a follow-up e-mail detailing exactly what was changed and listing all of the files affected. To top it off, we'd used a popular WYSIWYG HTML editor when we originally designed the site, and we opted to utilize all the cool automated site design features contained in the software. This resulted in a forest of static .htm pages (see, I knew little or nothing about server-side scripting way back then) with a ton of duplicated code. Needless to say, all of those automated features got tossed out the window a long time ago. Adding server side scripting is not an option, since all the pages would have to be renamed, links rewritten... which is straying from the point- which is, I.T. departments are running scared. Litigation is everywhere. Keeping things stable has become the sole priority in any company under any sort of scrutiny, since the stakes are higher than ever before. Nobody wants to pay fines or do jail time. Suddenly, the free for all has turned into a mine field. This is all the more reason that startups are likelier to gain traction in the marketplace. Agility is the key word- and it has been destroyed in all but a few larger organizations.
If it's any consolation, you're not alone.
The IT department exists to make sure they have regular, gainful employment. They do NOT exist to make your job easier, or anyone's for that matter, who does not have direct or closelly indirect firing power over them. There are mouths to feed, mortgages to pay, colleges funds to fund, retirement to dream about.
Cynical? Yes, but also very true. The above is the root of the issue. I'll put it in the terms that IT would:
ITs job is to keep the servers running, smoothly, with as little interruption to daily work as possible. As with any complex undertaking, different users have different priorities. CxOs come first. Period. Internal needs come next (see: "servers running, smoothly," above). High profile departments are next - marketing, sales, accounting. The last one is mostly because it comes under a CxO (F - you can choose what it stands for) who is intimitely involved with the month-to-month operation, and through which everyone gets their pay checks (including previously mentioned CxOs). Development is pretty far down, as you can see. You must understand - you don't bring cash into the organization (sales), nor do your efforts directly affect the price of company stock (marketing), both of which are of top importance to the CxOs.
That does not mean that you are not essential. But you are essential in a way that is ongoing - like the janitorial staff. If they lose development, things will slowly start to degrade, but it will be a while before there is a crisis. Either way, its an expensive mess to clean up, but if you throw some cash at it, you can bring things back to livable.
Now, lets look at the flip side. If IT goes down for a day, there will be hell to pay, and heads may roll. Every IT person knows this. Anyone who has dealt with complex modern systems knows that it's a house of cards. There are so many things that can go wrong. One failure, if not just costing your job, is certainly going to make for a long night getting things back in order. That would be uncompensated overtime, remember. Also, ten years without a single failure will not make you a hero, like landing a new sales client, or scoring a great marketing campaign which lifts the stock price or sales. It will make the company think you're reliable, but boring. Bonus aren't given out for boring. One failure, on the other hand, makes you a villain.
Now, if you've made it this far, how much value is there - for the IT professional - in helping you get your job done faster. In case you've skimmed, I'll tell you: none. It's like playing russian roulette for fun. Unless you just happen to like the life-or-death thrill, or have nothing to live for, it's a fools game.
I wish I had better news for you, but if you have a large corporation, than you have an ingrained corporate culture, and IT subculture. And they don't drift your way.
Oh, I've never been in IT. They piss me off 'cause I'm an engineer and just want to get shit done, and they want to worry about making sure the CEO's internet never goes down. I've learned over the years that, in effect, that is their job. I've stopped fighting them and learned to either (a) work with them or (b) work around them. The latter is done carefully to avoid stepping on toes. Just as they are under the thumb of uper management, they like to exert their power where they can. That would be against you and me. You don't tunnel under a mountain if there's a reasonable way to pass around it.
Is it just my observation, or are there way too many stupid people in the world?
Simple, newegg and a dept. charge card. Problem solved.
Adapt or die - and it's easier to ask forgiveness...
---
tjc
These are some known workarounds:
- Go into consulting. Many times, organizations will blindly accept advice and deliverables from a consultant than from within their own I/T staff. If your organization has a preferred consulting group list, then call the head shop guys at the consulting firms, explain your situation and that you want to stop working for the company but come back as a consultant. Either way, you can work on software projects at the pace your client wants and you have less interest in seeing them succeed/fail. If you run into a BOFH I/T shop when you're in consulting, you can just document this and then explain that your project is going redline because of it (missed deadlines, etc). Usually the consulting fees are a good incentive for management to quickly fix the problem.
- Battle of the White Papers. I had to do this one as a FTE working for a rather large insurance company. The I/T department or architects would say NO to something. In fact, this same company said NO to formatting IBM MVS VSAM datasets as XML. "We'll wait til version 2 comes out" I was told. I had to do some research to show how project managers at NASA, Lockheed, Progressive and a few other competitors were already using XML as a medium for communicating data between disparate systems, then showed the ROI in saved resources on the enterprise project I was on. Instead of sending this white paper over to the architects to get their input, I sent it straight to senior management, who then went back to the architects for further explanation. Two more rounds of this and they were finally trumped. You literally won your arguments at this company by who had the highest stack of white papers (they were never read).
- Let Projects Fail. Dare I say it? While you are waiting for requests to be completed by the I/T department, document when they were sent and how long they are taking to act. Report your findings and PERSONALLY go with your project manager to senior management. If nothing is still done, then just let the projects fail. It only takes ONE pet project from senior management to be seriously delayed before someone puts the grease on that wheel. Besides, it's not your fault, it's the company's. They might not see it that way, but you have documentation to back you up. Meanwhile, I would run over to dice.com and look for a smaller, more agile organization to work for.
- Go Political (only pull this move after you try Let Projects Fail). At some very large organizations during the Change of Century chaos, many project managers had cart blanche authority to create I/T "fiefdoms" within the larger I/T organization. At the same insurance company I used to work for, a seperate mainframe sysplex was setup just for this project and programmers had their own development playground, testbeds, networks and admins. They completely bypassed the larger I/T beuracracy. You can attempt to do this if the I/T department is seriously thwarting most of your projects as a whole. Be prepared to anger and upset the I/T guys as they will stop what their doing to defend their turf. Just be clear, concise, document (DOCUMENTATION IS IMPORTANT) the I/T staff's shortcomings and give solutions to the problem. Make sure you emphasize that your goals are completely in line with the company's. Don't leaving them thinking anything but. If you pull this off successfully, the BOFH I/T staff will look like bumbling imbeciles.
Above all, no matter what strategy you try, don't get emotional. Just present evidence, be convincing, and alingn yourself and your projects to that of the company's. If the I/T staff is withholding resources from you, then illustrate how this is damaging user productivity and stifling innovation within the company. Ask for a balance. After all, you have to get your work done, too.
Today I was told the IT folks would need to take my PC to the shop to install PcAnywhere.
They are in a building across the street.
No wonder people hate them.
GO to the CEO, tell him about missed opportunities, and tell him the IT department is hostile.
Tell him what needs to be done, and how you owuld do it. Get his buy in.
Then do it.
3 things can happen here:
1) S/He fires you. You were going to quit anyways.
2) S/He gets IT to start becoming more agile
3) S/He say "Do it". At this point you have to do it. Point all bottlenecks you can't deal with to him. In fact, when you have a people issue you can't solve, go to him and ask for help. You must be successful. When you are it could mean a good promotion, perferably over all those enemies you just made. Enemies are all right, you just have to deal with them calmly, and with authority(or assumed authority if you don't have any real authority.)
Hen you consideringf quiting anyways, you might as well take the risk and shoot for the stars.
The Kruger Dunning explains most post on
Sounds like working IT in the Air Force. Or a University. Good luck, it's bureaucracy. And because they're all the same, but in different ways, the only advice I can give you is keep on truckin'.
You can only create an I/T department that stifles, withholds and delays progress for so long before a change in senior management wipes them out. Very few organizations are immute. Just about anything can be outsourced or given to consultants.
I would remind those who aren't so mindful about the needs of your organization and are more concerned with protecting the environment that senior management may not see it that way and could very well outsource I/T functions in an attempt to speed up progress within the org.
Dear IT department,
I require this memory to be installed in my server so I can properly host a massive Natalie Portman gallery. I will, of course, provide you with URLs when it's done.
Wow, I didn't know that you worked at *myemployerhere* too! Actually, I read your article and thought it was amazing how exactly identical your situation is to mine. My solution? Well, I'm ready to leave if another comparable (benefits) job comes along working for a company which is smaller. Probably the only way to fix something like this is to make the company smaller.
Full-Featured GPL Web Hosting Control Panel
In a company that has an IT department that big, flexibility isn't very important to the bottom line. Stability counts for much more, and beaureaucracy creates stability, at the expense of productivity. The bigger the company, the stronger the preference to forego opportunities as a cost of achieving stability.
With flexibility comes the need for filtering... it's great to allow employees to try out new things, but most new things are really no good. Beaureaucracy provides a certain sort of institutional filtering. It's annoying as hell, but it does crudely serve a purpose.
Bottom line, if you're itchy and want to be innovative, stay out of big companies, unless you can find something like a research or product development setting.
Your IT department is no different than any other large IT department. They have to manage maintaining an infrastructure, and scheduling requests to make changes to that infrastructure.
It may look inefficient, but I would bet that it is more inefficient to have 100 software developers having a free for all with the network taps and the switches. Who wants to sort that mess out?
Out of this, evolved the bureaucracy.
The sad thing is, your management wants it this way. If they did not, it would get the proper funding to make things faster. Look around, what your management cares about, gets funded, what they do not care about, does not get funded. They are perfectly happy with the way your IT operations function.
Forgiveness is easier than permission ;)
But seriously, I don't make changes which could in any way impact production without proper clearance, but changes which can have no outcome but good (or at least are not going to effect users) I go ahead and make, that way I can iron out the wrinkles and then I seem to management to be a more "pro-active" employee...
There are places where I am not able to move at the speeds that I would like, that is ok, I simply let those move at thier own pace, and use that extra time to push other initiatives, of which I have more direct control (because I came up with them, etc).
Don't know if that helps at all...
Disclaimer: This advice is probably only applicable to small businesses. The maze of inter-departmental bullshit and bureaucracy that goes on in large corporations is usually the cause of their own problems, but, in most businesses:
The problem in IT is always money. Either too much or not enough.
Every IT department I've seen falls into one of two categories: spendthrift idiots with twelve copies of VMWare and 4 gigs of RAM on each machine. Or frazzled techs running an entire company on 8 year old PCs and unlicensed software.
And anyone but a competent, experienced IT person will have no idea how to determine which it is. Managers are especially bad. But some IT people can be even worse.
So, here's what you do. Hire an outside consultant, perhaps two, to come in, look around for a couple of weeks, and answer a simple question: are we spending too much money on IT or not enough?
Once you've determined that, figure out who the responsible member of IT is. Sit down with him, and with the consultant(s), and give him a budget. Base it on previous expenditures, future plans, and everything that IT is responsible for. Make sure it is easily accessible. And have someone oversee it every few months or so.
Then sit back and watch the problem solve itself.
"I assumed blithely that there were no elves out there in the darkness"
Beside which, what better way of making things better for those who stay? If management loses good people because of some problem, they just might address that problem.
Large IT departments have poorly designed feedback paths to guide their behavior -
r y=maggianos_versus_it_departments
http://www.realmeme.com/roller/page/realmeme/?ent
Somebody needs to hire IT managers with restaurant management degrees.
...So you are affirming they do exist? Or aren't you disproving that the inexistence of these situations has not been denied in no situation at all?
Victims of 9/11: <3000. Traffic in the US: >30,000/y
I am in the same situation at the moment working for, err... a major british telco. I've done it before as well, I spent 10 years working in the civil service. That was pretty tough.
I have a consistent approach to the problems of working this way: I do everything myself. If I need a server? I buy one, charge it, plug it in, install it, support it. If I need hosting? There are plenty of hosting companies out there selling services; I buy it and set it up.
I do try and keep the things I do clean and secure and "away" from the IT department.
I do try to point out, whenever I can, what a clueless bunch of losers the IT department are.
I do try to get people on my side by doing favours for them with "my" resources as quickly as I can. If you can save an important managers pet project by judicous installation then all well and good.
I tell as many people as possible about what I am doing; taking care to point out that if I relied on the IT department I would never have been able to achieve success.
Mostly this approach is getting easier. It's easier to buy powerfull servers that can host masses of virtual machines; it's easier to get the hosting you need.
Lastly, remember that IT departments are so swamped by their own dumb rules and ineptitude that they have very little time to concentrate on trying to stop someone who knows what they are doing.
This is the sort of complaint I hear constantly. So, speaking from the IT side of the house.... My job is to keep existing systems that generate revenue and enhance productivity up and running and secure. Downtime costs serious bucks in lost revenue. On top of that, I do indeed have an overwhelming bureaucracy to deal with, doubled in difficulty and complexity by Sarbannes-Oxley. The S-Ox auditors are not techies, they are accountants, which means a great deal of irrelevant detail has to be audited. Exceptions to existing application environments and frameworks are extremely expensive in terms of allocating dedicated hardware and dedicated people that could potentially be servicing ten times the resources, but those economies of scale are lost when we have to do special things for someone's one-of project. Handling exceptions is very expensive in a large scale environment. If we need something new, lots of planning goes into it, to make sure we can keep it up and running, and scale to much greater than anticipated load. If you want agility, you either have to find a way to achieve it within existing channels (in our environment, the turn-around for J2EE or Oracle apps is quite short), or you need to convince upper management of the value of a skunk-works type mini-DC for such "agility", with the understanding that anything successful will need to be reengineered to be robust enough for the main DC. Most of all, you have to have a value case. It's not enough to talk about lost business opportunities. You have to be able to quantify the projected value of the opportunity, and balance that against the cost of handling an exception in the datacenter. If IT cannot quantify the cost of an exception, bad on them.
From the IT side, you can't always respond to every change request ASAP. Simple things like adding more ram and stuff are all easy to actually do, but sometimes there are roadblocks, for instance who owns the system? If it's IT, they may not have budget to add more ram whenever someone asks for it. If it's you or your group, can you get a PO approved?
Does the system really need more ram? I used to get requests for more internet speed all of the time. It happens a lot less often since I started parading out the metrics to show people here that a) we are not using all of our bandwidth to the internet when their issue occurred, and b) I can prove that we can and do use up all of our bandwidth at times.
Policies can slow things down too, but to operate without them is a very slippery slope. I used to hate policies but as I moved up the chain in IT and we began to get requests for things that would create a great deal of work for very little return, or even more important, to deal with difficult HR situations, it became much simpler for everyone to be able to point a the policy that says "As far as the company is concerned, there is no personal data on that company supplied laptop, and you need to hand it over now"
From the non IT side it can be very frustrating dealing with IT some times. If they are really competent,and your requests are reasonable, they will get to your request in a reasonable amount of time. If not, well....
Here is all I can recommend if you aren't getting the service you need. Make the business case to your manager. Show him or her what these delays are costing the company, and allow them to take it up the chain of command with the data you have provided. In any well run company, showing how you can improve the bottom line should be enough to get things moving. Keep in mind that you will win some, and loose some. There may be issues you are not aware of behind the scenes (partnerships, politics between groups, etc.)
Leaving is an option, but save that one for when you are certain that you are dealing with real incompetence and you are sure that there is no way to fix things. If you think you have a good company, do what you can to make things right.
Eschew Obfuscation
I am in a somewhat similar circumstance, R&D activities in a more generic corporate environemnt.
:o)
In order to get anything done, I have had to learn how people work: Our outsourcer, their people, their roles, their procedures, their concerns: Our organisation, the management, the section that manages the outsourced contract, their people: Our immediate management.
That's the beginning of it. Then I have had to build relationships with all these people to show them that I am not some impulsive loon from a Dilbert cartoon. Pick smll victories to start and build from there.
Oh, and sometimes the best thing to do is to just get in there and do it anyway. Just be sure you have support from on high.
"Everything is adjustable, provided you have the right tools"
If "senior management" can't make a department behave that management is incompetent.
Quite simply the lack of responsiveness from IT should continue up the management structure until.
A. The issue is resolved.
B. Management decides that the level of support is acceptable considering their other demands.
If the current manager Doesn't select A or B, they should perform
C. Elevate it to the next level of management.
D. Give up because they have more important things to deal with.
D is always a bad solution and identifies bad management. If they're always too busy to deal with issues, they should hire someone else.
I used to work in a *very* bureaucrat infested research lab. This is the place that firewalled "new" ssh but let the "known protocol" telnet, out... Friend of mine was running very numerically intensive spectral analysis/matching on samples. Bung in sample. Get data. Process for about 8hr. So, do last thing of the day and you've got the results next morning. Until, in the interests of a uniform computing experience *all* boxes were required to have the same basic setup and were bolted down tight. This included everything. Including the screensaver that seamlessly blended from slide to slide of the company's publicity shots. Bingo! 100% CPU when the screensaver kicks in and the analysis runs can no longer work unattended. Bummer! OK so my friend takes an old mouse, a clamp stand, a magnetic stirrer and flea, and some epoxy. Glue magnetic flea to mouse ball. Clamp mouse over stirrer. Stirrer agitates mouse. Screensaver never gets to run. Once again work can happen!
Red to red, black to black. Switch it on, but stand well back.
Don't knock it until you've tried it...
I dealt with this frustration by quitting http://www.ibm.com/, the source of it all (for me anyway). Believe me, I'm not alone in my "exit strategy".
I work for a consulting company and I get the same thing. I always say it is because anyone with talent gets sent to work directly for clients, anyone still doing inhouse IT is uncontractable...
Funny, dealing with outside companies is often better, sometimes far better, than dealing with in house IT. Outside companies may be clueless but they have to get stuff done eventually. The in house people know that they have upper management buy in and they don't have to do anything. The delays and incompetence are staggering. Fortunately I don't have to deal with these people too often, or I would have a similar attitude.
Interdepartmentally the IT units charge ridiculous amounts, too. I can remember a senior IT guy getting furious that our price for managing our own servers was near zero (computer room and bandwidth already included in overhead, sysadmins already working on project, automated installs, additional overhead for new servers an hour or two to rack them and two minutes to run the install process (waiting for the install to finish is free)).
Any department with guaranteed work and no accountability stops being useful fast. Complacency breeds uselessness. Nine months sounds pretty crazy, but I'm still waiting for some stuff to be ready for servers that were installed 3 months ago (mostly details), so I can relate.
The only thing I can suggest is to follow up fast, document, and escalate problems. My servers would probably be ready if I had time to yell at the people responsible daily (no response to e-mails). You can often get a response if you send someone senior an e-mail chain with dates spanning months and silly responses to your requests. Of course if the department is crap the chain above them is probably crap because no one good wants to manage useless people.
You got me into this! You were the ideologue! I'm only a poor assassin! - Twenty evocations, Bruce Sterling
You work for a large corporation. Surely the company could afford hire a sys admin that could act as a liason between the IT department and the rest of the company. Some that was accepted by the IT department as one of their own, but could expedite getting things done for non-IT departments.
--
Laying low - trying to survive the remaining years of staying the course with W.
Couple of reasons for lack agility from my experience:-
- IT has different meaning in these organizations - it has almost nothing to do with software development. So called "IT Managers" are guys who have crawled up the corporate ladder the hard way or guys with no software background whatsoever, but who are supposedly good in managenment, can talk confidently and convincingly.
- Count the layers between actual users and "developers": Process team, Business Analyst team, Change Management, project management, release management and database administrators. Just imagine explaining a small piece of development to all of these groups - adding one field to an existing webpage could take months.
As a finance guy, here's my take. Make friends with your FP&A. Get all the data on how much time you and others spend on doing these unnecessary tasks. Take it upon yourself to gather all the data necessary to prove it's wasting money. Give that data to the FP&A in a reasonable and understandable format. Have him create a small presentation showing how costly this practice is. Have your senior management present that to the executives with the FP&A's blessing. You can't just complain about processes. Odds are they are there for a reason. For example, in the FP&A's analysis, he might discover that without those processes in the past, spending on memory in non-production units had a ridiculous growth rate. Sounds like all you've done is complain and not made a logical, dollars-based argument on why the system sucks.
...at the company I consult for. What a monumental battle.
I swore off imperious gate-keepers during the punchcard-FORTRAN days.
But they're baaaack.
My situation is very similar to yours. I am a tech person for a department outside of the formal IT group which sometimes seems to be inefficient and/or ineffective. However, I don't believe that IT is the enemy and tend to sympathize with them.
In my current position I've seen some of the worst behaviors (in system administration, application development, etc.) practiced by tech people outside of IT who then expect IT to automagically make everything work and clean up any messes the non-IT folk created. On the other hand, there are also times when our IT department really does drop the ball.
This has created deep-seeded animosity between some non-IT departments and IT, I think. The non-IT folk believe IT are bureaucratic obstructionists who don't know what they're doing; while IT believes the non-IT folk are disorganized, loose cannons who don't know what they're doing. Unfortunately, to some extent, I think they're both right.
That said, my best advice to you is to help IT help you. Try come to some agreement or understanding with IT and define what it is they they need in order for them to be more responsive to your needs. Respect their needs as much as you want them to respect yours.
Also, don't undermine or bypass policies and procedures defined by IT. It might seem like you can get around IT's requirements and do something your own way, but that just perpetuates the problem. If you think IT is being unreasonable with their policies, find out why their policies are the way they are. You might discover that there is a good reason for it.
Think of IT as a finite resource -- don't squander it. I've never met (or worked in) an IT department that wasn't overwhelmed with things to do. Keep in mind that any system you implement may require some amount of time and effort for IT to support and/or maintain it. And keep in mind that there is always a Y2K or Service Pack N+1 or something like that around the corner keeping IT busy. So, as much as possible, budget your IT-time wisely. And, of course, plan ahead.
You may already be doing all of this, which makes your situation a more bitter pill to swallow, in which case I'd suggest helping other non-IT departments do the same, if they're not already.
And, of course, doing all of this doesn't guarantee that your IT experience will improve. But, I think it's a case of "you catch more flies with honey than you do with vinegar" and "you who are without sin may cast the first stone."
may require much more than you realize. Case in point. A developer needed a single column added to a table, and we had done test and acceptance testing. He wanted the column added during the day, so we put it in with an alter - no big deal right? After 50 seconds or so, the alter timed out, and took down users all over the country with it!!!! And the alter did nothing wrong, but it needed exclusive access to the table - and could not get it.
We had to step back and put the alter in in the middle of the night on a Sunday. And with our usage, we can't even get that time every week.
Bottom line? Get over yourself. You would do better to go talk to IT and find out WHY things are the way they are, and work with them, rather than against them.
Sigh, this hits close to home. I hate our IT department:
"I'm sorry, your network won't be ready until we de-fabulate the mother-processors."
"Oh, well... Wow... That sounds complicated."
[...]
"It is."
I proposed that we eject the whole department into deep space, and hire several entertaining animals which would more or less perform the same function. Until then I'll secretly work on my personal laptop at work.
And in hope for us all: Don't quit! Fight the good fight and establish a productive environment!
Lease your server space from someone else, and have at it.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
First, the reason that IT organizations typically don't like technical folks outside of IT developing their own internal business apps, building their own infrastructure, or buying their own gear is pretty simple: we're the ones generally charged with ensuring predictability and security in the corporate infrastructure and we lose the ability to mitigate risk and provide reasonable levels of support with each bit of control that we give up. It's the same reason that the Legal department doesn't let you write your own contracts and why the Finance/Accounting department doesn't let you make journal entries.
I can't tell you how many applications, systems, and servers that my respective IT departments have had to inherit because the well-meaning business employee who developed or setup the system had either lost interest or moved on. When this happens we find that nobody left in the department knows anything about what is inevitably deemed a "critical app" by the department head (and is usually running on a server under a desk in a vacant cubicle). This scenario also applies to self-setup infrastructure of all kinds -- We regularly find rouge wireless access points, PCs and laptops bought and 'expensed,' application-ready mobile phones attempting to attach to our network and on and on.
The only way to deal with an increasingly technology saavy workforce wanting to do their own thing, in my opinion, is for IT to set clear policies and processes that allow for a certain amount of 'self help' but only within the guidelines of an IT ecosystem-friendly arrangement. We need to know about hardware and software you buy or make and we need to know where these systems and sub-systems reside, what data is on them, how that data is protected, who has access, and who is responsinble for maintaining them. In this day of increasing scrutiny (SarBox, etc), its more important than ever that we maintain some level of control.
Aside from all this ranting, I'll say that IT Leaders who do not realize that they are service providers at the end of the day are doomed to be loathed by business users. CIOs who stand fast with their arms crossed saying "no" to everything are obviously not familiar with the way a service organization is run. Unless a service-oriented culture is fosted from the top of IT, things will never change in your organization. The most successful CIOs that I know spend a lot of time with business department heads ensuring synchronization of priorities while also instilling in their IT employees a sense that proper, measurable internal customer sat is a standard part of doing business. Take to your CIO to lunch if possible and talk about this. I think you'll very quickly be able to tell if you have any hope of a culture shift.
How would you feel if the network guys just needed to hack a small piece of your code? Surely you wouldn't mind if they checked out your source code and fiddled around?
Sounds reasonable to me - this is the approach we use at my firm. It's a rare week when I don't get mail saying someone has fixed or enhanced my code, then pushed it to production machines. It's a rare week when I don't do the same to someone else's code.
Works well when you make the effort to only hire good people, treat them like adults, and reward them according to their contributions.
In my experience managers have a habit of shooting first, asking later. So IT needs to document everything they do in order to cover themselves when things go wrong. Like in court, IT departments need to have the documents as evidence that they did everything "by the book" so that they don't get nailed for things that aren't their fault.
"Oh dear, she's stuck in an infinite loop and he's an idiot" -Prof. Farnsworth (Futurama)
It really isn't clear from your posting, what it is that you do at the company. Obviously, you are not in the IT department. You are more than likely an engineer in one of the development or support departments who makes requests to IT? Not sure from the posting.
A slow IT department is the number 1 criticism I hear. Too much red tape, etc. As other posters have queried, what have you done to help mitigate these issues? Talking to your management(griping), doesn't help the IT department get things done.
Here's an example:
At one company I worked at, the IT department is often given a request. That request is then looked over and basically, it tells them nothing. The request usually reads something like:
We need a few servers to support an application that may need a database and a web server. How soon can this get done?
This is usually followed by weeks, if not months, of back and forth emails that culminate in a series of meetings to iron out the fine details of what is really being requested.
This usually ends up being one of 2 things:
1) The request could easily be handled by existing infrastructure, but the requestors weren't clear enough and/or didn't understand their own needs enough to produce a relevant request.
2) The request requires a significant amount of hardware and infrastructure to be installed because "may need a database and a web server" should have been written as: "We plan on rolling out a national product which will serve over 300,000 customers. We've already announced a start date and have pre-sold accounts to the new system. It needs to tie into existing infrastructure and will require the addition of a SAN to support all of the data being generated."
Both of these situations result from a lack of communication between the various departments and the IT department. This lack of communication and not bringing in the IT department into the early planning meetings results in further issues down the road.
Weeks and months will be spent getting various deparmtental managers to sign off on the work to be done. Work which commits their staff and resources to another department's project.
Additional time for project budget approvals, sometimes requiring the project to be added in after the end of the quarter, so as not to impact the current quarter's earnings reports.
Allocation of network connections(weeks to months from Telco companies) and data center space contracts(weeks).
Then, add in the weeks of testing of the implementation and the predictable arguing and bickering over how to best implement the system. There will be finger pointing and blame games between various departments and groups about which component is fouling up the system, etc.
From an outsider's point of view, the mass of individuals known as IT seem to be slow, but often times, it has more to do with a lack of clear communication.
This isn't to say that some IT departments aren't slow. There are quite a few organizations and departments, whose sole job is to pass the buck and not take responsibility. They take on a task long enough to pass it on to someone else. The work never gets done, but they get entries on their billing. These groups exist. But just try to get rid of them without a massive reorg/downsizing!
If your role isn't that of a decision maker and the environment is intollerable, then leave. Vote with your feet. Find a better job, if it is that unbearable. If it is something you can alter, then work to alter it. Learn more about the IT workings. Find out why something is taking so long. Run the approval tasks to the next rung on the ladder. It's amazing how quickly the "slow" departments can prove to be when properly motivated and/or stimulated.
I work in IT and provide support. I deal with IT and non-IT departments and believe me, there is plenty of blame to go around when it comes to a project or request taking forever and a year. People outside of IT, making requests may blame IT for being
Winged Power Photography
As an admin, where my job depends on stable, dependable code and well-documented functionality, I have a healthy paranoia regarding new features, system changes, etc. When there's a proper development process (with developer sandboxes, multiple test systems, a staging rig that lets admins beat on the code and get comfortable with it, and then _and only then_ updating production code) that paranoia is lessened if not completely assuaged.
If you have a system where developers gdb into mission critical code during peak live hours to hack around undocumented behaviors or bugs (I'm lookin at _YOU_, Dave...) the confidence level in your code (or whoever's code you inherited) goes waay down and paranoia rises in inverse proportion.
Also remember that admins are lazy, and good ones get away with it by getting systems running so tight they don't have to waste hours a day poring over logs or fighting fires. When they hear from users it's usually because something is broken (likely by the user or developer) or they want to make a change to the precious systems equilibrium. So the win for the admin is, what? Vs. the failure that might cost him his job? You think that having CD-R mailspool trail hasn't saved _my_ job a few times because of folks forgetting what they asked for or when, then trying to pin the blame on me?
Don't forget that pissing an admin off only puts you on his shit list. Admins have _loonnnnggg_ memories. And they can read your email.
I've been in a lot of situations like this. Sometimes I've tried to fix things through extraordinary effort. This has never, ever worked for me, though, and as I've gotten older, I've become much less interested in "rocking the boat". Why knock yourself out to help an organization that will only curse you for it? If your project is stalled because the idiots in IT are holding you up, well, there's your chance to learn a new language, tool, musical instrument or whatever will be useful when you move to your next job.
"Not an actor, but he plays one on TV."
A LONG time ago, I worked at a place that had similar problems, but anything that got product out the door always took precidence. So, if IT was dragging their feet, we did the work ourselves out of our own budget and cut out IT altogether. Since we could show that this got the job done way faster than IT could've we got away with it even though IT whined about it. In one case, they took a year and a half and lots of money to create a tool for us that didn't work right, so we never used it and put together a quick perl utility in a few weeks that did a much better job and that our guys could hack away at to add new features as people needed them. It was one of many useful examples we could point to whenever IT complained about our internal group. Unfortunately, IT had some power and annoying them had unpleasant concequences. Also, any project that required their co-operation was still at their mercy.
Several other departments also had their own internal IT groups just to keep the trains running on time. Eventually senior management noticed the trend. Of course the senior IT guy lobbied to annex these groups and make them part of his department, but management was smart enough to realize that these groups formed for a reason canned the senior IT guy and put in someone who was more co-operative.
Signatures are a waste of bandwi (buffering...)
we're also a massive US company with a tremendous infrastructure - this guy could be talking about my company although I thought we were a one quarter turnaround. However, emergency changes go in.
;)
Here's the deal - on one hand, it's a ton of paperwork - on the other hand, you're in charge of a massive infrastructure, and replacing one firewall somewhere may block ports to a vendor and all hell breaks loose - so they document the threats like it's going out of style.
Nothing worse then sitting in the NMC, seeing a site go down, and wondering "well, do I need a tech out there or is this a Sat. night change"
so sometimes the paperwork is a result of a couple instances where someone brought down something big - maybe that's what happens in some of these companies - I know we had a crackdown before I came to work for them cause someone sent all the network's routes out an ISDN interface due to faulty metrics - oops - several thousand sites all channeling to a 128K interface for routing
I just shrug - I have tried to move my position into as much 3rd level support as possible - I don't do installs, so I don't do paperwork, and if someone breaks something it's generally their responsibility as the change maker to fix it, though we usually get called -
every IT job is almost the same - 1/2 good stuff, 1/2 bureaucracy - and you try to avoid that part. My new boss used to be an engineer - the first day he brought down nerf-type guns for us to shoot each other with - it is a decent stress reliever, and something only an engineer boss could have thought of, so I'm hoping for the best with the new guy - I think if I can explain issues to him in an engineer POV - (he's highly certified, not just a paper tiger) that I can get more done.
but some days I just shake my head - LOL. The shit that goes thru that we don't know about - especially with telco maintenance -
RB
----------
ah honey, we're all resplendent - Bill Mallonee
What you do is just do the work without asking or waiting for anyone's approval for anything... you will get the distinct feeling of stomping forward with 5 people holding chains trying to pull you back. Thats who is the real producer. Your work will be thrown away anyways, but if theres nothing completed, theres nothing to save.
I'm an IT consultant that sells to the business side of our clients. We work with the business people to articulate an issue or opportunity, to craft a solution and to put together a plan. Then they go ask IT how much it will cost and how long it will take to get it done. Their quotes are often, I shit you not, 5-10 times what we will do it for on a fixed price basis. And- these are not nickle and dime projects - usually in the $1 million and up range.
I love big slow IT departments. Love them. Love them.
Please send a name.
There are some exceptions, but they'll charge you more money for the flexibility. That's the other way outsourcers make you money - precisely defining the scope of work and charging higher prices for anything outside of it. Sometimes that's a Great Thing - outsiders who want to charge money are often much more willing to do what you want than insiders whose reward structure is that they're a Cost Center incentivized to cut costs. But the kinds of bean-counters who outsourced your IT department on you are usually going to prevent you from getting the extra-value services if they can.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Simply break the machine in question. Be sure to inform the boss of the unfortunate situation. Not only will you get a day of light duty, but the problem in question will be IT's priority number one.
instead of just bitching on /.?
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
It's hopeless unless the CEO (or close to it) recognizes the problem. It's not only management, even though they're certainly at fault. IT infrastructure/tech support people seem to inherently have a kingdom-building complex. Their job, as they see it, is to protect their system from the clueless user community. And anything that deviates from their vision of a perfect system is not a priority. They have their own agenda of new features and releases and upgrades... user requests are an annoyance.
In 15 years of tech support work, I was always amazed at most of my colleagues' antagonism to "users". They seemed to think that anyone who didn't understand every detail of the system didn't deserve any support.
And I've certainly seen it from the outside.
Most people don't even think inside the box.
Take the hint and go start a company that does what your company does, but does it without the wasteful overhead.
You'll kill them by pricing your product lower, and they'll die and you can price your product where they priced it, and you'll get rich and your engineers will be happy.
1. Their computer problem is much more important than any other computer problem that might be on your plate on any given moment. Oh, and they are certainly more important than you going home to the wife and kids or to catch the latest episode of Veronica Mars or whatever you IT people do in your off hours.
2. Even though computers are mysterious things to them, they know that it'll only take you a couple of minutes to fix any given problems they have with them. So, you can get whatever you were currently working on done, if you IT people even really work rather than surf the net and play video games all day.
3. The words from the Veruca Salt song in Willie Wonka and the Chocolate Factory, "Don't care how, I want it now!!"
You may know the following things:
1. It will be very tempting to work on the most obnoxious person's problem first just to get rid of them. Even though that person's problem may be irrelevant in terms of the organizations productivity or profits, since they won't let you alone you may take your valuable time and use it to work on it just to get some peace and quiet.
2. There is nothing more fun than to be pressured into working late to solve some irrelevant problem because you are being pressured into it by some obnoxious co-worker who may be important in the corporation.
Face it, most of us need some sort of layer or wall between us and them so that we can work on our manager's priorities rather than J. Random Employee's priorities. When you waste hours on someone's project and your manager comes and yells at you for missing your deadline on your real project, you're not going to be happy about how little "red tape" is in the corporation.
"MIT betrayed all of its basic principles."
One place I worked (at dot com salary levels) I was to be administering servers in a remote site through a VPN. It took them 4-5 months to get the VPN up and working, so I did other (non-job description work) until it got up and running. Then a few months later, I ordered a cheap server box (around $2000) that was to be a primary project. My box was diverted to another emergency function, and I again sat around waiting for a new box. The box could not be obtained for a few months due to it's cost ($2000), so they would rather have me sit around at dot com salary levels doing little, rather than just buy the new box and get me busy.
At another place I did put a box in my office to act as a server, just to keep IT denial of service away.
You're the kind of jerk who gets what he expects to get from IT. When I run into someone like you it doesn't take to long to figure out what you think of someone in IT and you get treated exactly like you treat IT staff. I have a sneaking suspicion that you were one of the ones beating down the geeky kids and now you don't have the power and control you used to and you can't stand it.
Most IT staff I have known when I wasn't IT, and most that I work with now that I am in IT are decent people just like everybody else and if you treat them with the normal human respect you get normal human respect.
Sedennial's Corollary to the Golden Rule: On average you will likely have done to you what you project to others that you expect them to do to you. AKA: You often will get exactly what you ask for.
The Bureacracy is expanding to meet the needs of the expanding Bureacracy.....
All I can say is give me a break. A mountain of paperwork to get RAM added to a non-production server? If this were an efficienct company, wouldn't all of this be online? I can't think of a large company that doesn't have online change control these days.
My first step would be to implement online change control and then see how much is getting done and how long it takes. Too much of your story is ancedotal and you and your managers sound like a bunch of whiners. If you were doing anything remotely important, I am sure that your IT department would take care of you immediately. If I had to guess, you work in the department responsible to archiving old reports that only Bob in accounting still cares about.
Second, if you senior management can't get IT to do work for you then you have serious trouble. This means that your senior managers no longer have control of expenditures. Senior managers that don't have power over purse strings are NOT senior managers and are more likely team leaders, who report to a manger, who reports to a manager, who reports to the senior manager that attends the meeting with the VP and the CIO (Head IT dude).
Just take a chill pill. If your work is important to the company, then IT will get it done.
We have a similar IT department at my company, one of the "top 10" defense contractors. Here's what I do:
I break the rules.
I find guys in IT, and am nice to them. One guy, I call "candy man," and shcmooze him.
I do have to fill out the paperwork. After the job is actually done.
I game the system.
If someone in IT tells me something I don't like, I pause and count to 10. Slowly. I then say, "I am really unhappy with that."
You'd be surprised at how many people don't like to hear that while doing their job.
Or they can do something, like sneaker net me rule breaking stuff, and then I will rub their back with praise, thank them, and make their job easier.
I also play games with title power, and corporate speak. If there's a problem, I say, "speaking for [my division name], we don't like that." You know, as if I were the voice of the whole division. "We" speak, as in royal We, often makes the little small-minded paeons in IT just sort of pop on their jack boots and march to the corporate drum.
It works. I swear, it's true.
At least where I am.
Good luck to you,
C//
p.s., if these tricks happen to work for you, do be advised that there are, in any given limited time period, only so many bullets in your gun. Aim well, and count your ammunition.
"There are certain tasks (i.e. anything that happens in the data centers) that I don't have the access to do. Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork (no exaggeration), thus costing many times more than it should." The first thought is simple, It's not your job! The unfortunate consequence to working in a professional environment is excepting you job role. There are many instances of people working in one department yet having the skills of another. If we all moved in to some kind of independent work environment of "I can do that too, therefore why should I wait"then office anarchy would form and production would drop far below what you are facing now. Second thought is, if I do it for you I have to do it for everyone. You claim that a simple task of installing memory shouldn't take so long. Did you consider the responsibilities that the IT department might be faced with concerning an equal level of service. Or did you think of how it might fit into their budget. You work for a large company, therefore I have to assume you are not only one making that request. If your IT department bypassed any kind of procedure and simply gave everyone the ram requested. I would think the number of request would be constant regardless of the true need. This in itself would be financially irresponsible to the company. Not to mention most expected results for increased memory achieve very little job performance improvements, thus making the whole thing a true waste of money. Also, providing one person memory might sound cheep, but doing it for 100, 200, 1000 people get very expensive. So in the end who does the IT department pick when asked for increased memory? There has to be a procedure to show that they are being responsible when spending money, and they have to show that said money was spent wisely. In the end this equals paperwork and your hated bureaucracy. You don't know how many people requested ram, or the IT departments budget situation. If it took 9 months for you to get ram I would venture to guess that ether they couldn't provide it or you failed to provide a true need. In the end I think your pointing at IT as an overwhelming bureaucracy is short sighted and smells of inexperience. You need to look at the bigger picture and realize that every department has it's problems and responsibilities.
If they're really that bad (and I CAN believe it) AND you have management support, you have a few options. For Test servers, set them up in your own department on a private lan. Do what you want with them.
Talk with some of the actual techs in IT. Find out what the problem is. It could be truly miserable managers in their department, it could be upper management pressure on IT leading to the ultimate in CYA, perhaps a CIO with a tinfoil hat collection. They might be seriously understaffed and use the paperwork as a way to eliminate frivolous requests.
Or they could just be ID10Ts.
If none of that works, resign yourself to resigning. But before you do, order 100 Signetics 25120s. Be sure to fill out the paperwork in exacting detail. If they can't seem to find them, print the datasheet for them (page 2). Insist that any other part will allow Chinese companies to sniff all your data.
Then resign, QUICKLY
just outsource it
should solve your problems, and it would work out cheaper in the long run
---- Put Sig here:
First, let me state that I have never been in a situation as bad as you describe. But I have dealt with some similar types of problems. Some thoughts.
> I am...a technology worker...outside of the IT department in my company
This might be a part of the problem. IT personnel not under IT's department heads may not be considered as knowledgeable as the staff in the IT department proper.
> By far, the biggest challenge I face is getting anything done due to the
> bureaucracy that exists, within IT.
Usually, IT bureaucracy exists because (A) IT was slapdash, problems occurred, and management mandated heavier process, or (B) IT was pressured to be slapdash, problems occurred, and IT fought for and got heavier process.
The result is the same. In case A, your requests are being deflected out of self-defense on the part of IT staff. In case B, you are viewed as an outsider who doesn't understand why this is necessary. Making the connections with the people who can solve this depends on which case this is.
> There are certain tasks (i.e. anything that happens in the data centers)
> that I don't have the access to do.
If you aren't IT, then you shouldn't have access to most anything in the data centers.
> Even a simple task, like installing more memory in a non-production server,
> can take nine months and massive mountains of paperwork (no exaggeration),
> thus costing many times more than it should.
The problem is getting the non-production servers treated differently.
First, what do -you- consider a -non-production- server? If IT has to fix problems with it, if it's budgeted as IT equipment, If the IT staff gets called down at 3am to restart it, it's production. If any of this is true solely and only because it's in the data center, then you need a non-production area for this equipment.
> The lack of agility is maddening, because I know we are missing significant
> business opportunities. My management is extremely supportive and despite
> our excellent track record of success in creating robust/secure
> applications--our work has passed audit numerous times with flying
> colors--we get no support from IT.
This might be because you are competition. On the other hand, it might be that you are placing a burden on the IT staff that you don't realize. Creating the applications is only part of it. Deploying, maintaining, integrating, troubleshooting, etc. are part of the cost and trouble. Do you handle this or pay for this, or is it IT's problem?
> Even senior management can't break through the barrier.
How senior? Seriously, I don't know of any company where -no- senior staff can change the policies of IT. Now it might be board-level or similar, but -senior- management almost always can change things. How high are these senior managers? Division Managers? VP?
> I am very interested in hearing the experiences Slashdot readers have had in
> similar situations." How do you get your technology work done, when your IT
> department is more hindrance than help?
I have dealt with obstructionist IT staff. At one firm, developers outside IT had to go to training (offered once every six months) for 2 weeks before being allowed to use the (overly maintenance heavy) revision control system. I did an end run and installed CVS locally, but I mentioned this as a problem to my bosses, and said it really should be installed on a server or everyone on the main RCS. But until IT made getting on the main RCS reasonable, we were stuck.
I've dealt with firms where getting a non-production server set up was extremely difficult. But with persistence and a commitment to always be respectful of the IT staff and offer solutions to problems, it can be changed.
The one exception would be where someone has set up IT as his own little power base. Screw with him and he'll bury you.
I live in a regulated environment, and understand your issues with getting stuff done. However, there are generally two sides to every story. I will counter your discussion a bit.
You want RAM in a server. That company currently has over 2,000 server and they have a service level agreement that is currently not being met with the business. They also have people that use to take servers down to just do "one thing" and not document why they were doing it, then when someone else went to update the server later it was not in the state they believed it would be and it created more problems and thus the server was unstable after their upgrade or the downtime was far greater than expected.
So the I.T. department gets judged by the business on uptime and other service level agreements. They do NOT get charged on helping the business out. So they are very cautious on any change to the environment. They are so cautious that it has gotten ridiculous for any change to occur.
So what can be done? Well I would need a ton more information than you provided to make more suggestions. I will NOT believe that everyone in your I.T. department is a bunch of idiots and lazy. I bet that around 80% are average to good, 10% suck and the last 10% rock (Like every large company).
Now a few questions.
1. Do you have a CIO?
2. Where is the majority of your I.T. department located?
3. How does your I.T. department prioritize its' project?
Those are just the first three that come to mind. In short I need to understand the constraints on the department before any real suggestions can be made. It is far too easy to say "fire them all", and in most companies that would be a huge mistake.
Lastly, I can say that I have seen a company that making any changes to ANY router took forever. It flat out sucked, however the reason is that this company was part of a bunch of other sister companies and one parent company, and those same router guys use to make changes on the fly (quick), but then it would take down a sister organization for a day or so, until they realized the mistake they made. So because of the major impact to the other businesses those same router guys were not allowed to make a change without a ton of paperwork under the penalty of being fired.
The more I learn about science, the more my faith in God increases.
to ask forgiveness than to seek permission. Put the memory in the server while the redundant server handles its work. Then do the same for the redundant one. Screw the bureaucratic fuck-monkeys. Things in the IT shops where I have worked have very often happened mysteriously or by accident. "Whoa, when did this code get fixed? Who installed that memory? When did this port get boosted to GigE?" etc. etc. etc.
Other than this text, there is no discernible information contained in this sig.
I'd love to tell a few stories, but I don't want to get anyone fired. The short version is that when companies create bureaucracy that reaches down to technical workers, you read about their bankruptcies, restructurings, and layoffs in the newspaper. When they create no bureaucracy at all, they don't last long enough for you to read anything about them in the newspaper. When they shield technical workers from the bureaucracy and let them do their jobs in whatever innovative ways they dream up, you read in the newspapers about how they're making money hand over fist and changing the entire market. If you're a technical worker and you have to fill out more than one form a month that's not an actual part of your primary technical work or related directly to your employment and compensation, your upper management has screwed up.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
This reminds me of my situation at work. While our hardware guys kick ass (I ordered another GB of ram for my computer to bring it up to 2GB and it went from me ordering it, approvals up two levels, IT ordering it and getting the product in and calling me to install it in about a week - it would take me almost the same amount of time to order the part from newegg and get it in the mail).
So yea, I'm a software guy outside of the IT department (in the engineering department) on a team of five talented software developers (who all get along and work well together suprisingly). The five of us constantly embarass our IT department in terms of product quality, time to delivery, and a bunch of other factors, most importantly total cost and ROI. We've got such a good reputation that other groups within our company don't bother talking to the IT department about developing applications - they come to us in the engineering department. And boy does that piss them off. The group that is supposed to head up and be in charge of all software development hates us. And it does have reprocussions, since when we have to go work with them on a project, they drag their feet like nothing else, and we have to get senior management to poke them a few times with a sharp stick to get going (which is something else that is nice to have - one of the VPs of the company came from the engineering department and we're on very good terms with him and we can go to him with problems if someone is giving us a lot of grief). I just heard a funny story today - it turns out our IT software developers took so damn long implementing a major piece of CIS software, the version we put into production several months ago is 4+ years old and now the vendor wants a contract - in the range of tens of millions of dollars - to continue supporting beyond its original support timeframe. Do you know what kind of shitfit this is going to cause on the SMT? I hope someone loses their job over it.
For the most part, the IT group should not have as "robust" of a management structure as other parts of the organization - IT can be more flat in its management structure.
So in summary: make friends with the SMT and try to get in good with them by developing products that have a very high ROI; make sure they know the inferiority of the solutions the IT office provides (I'm sure if someone in the SMT has computer problems, they dont wait as long as you do); and finally circumvent a few noncritical IT rules - or if you're brave, use a program to get the password for the local admin account on your computer (and probably all the other computers in the company) and have fun with it.
The Doormat
If you're not outraged, then you're not paying attention.
We outsource to IBM. The price they charge for ANYTHING is rediculous. Want to save money? Don't outsource. Don't put contracts on your server that give free reign to anyone to charge you the earth to change anything.
Right now we have several web servers on our Intranet. NONE have the options enabled for dynamic languages to access small databases (ODBC). Guess why? You got it. We outsource to IBM. They support our DB2. Hence everything is either plain HTML or a Programm that needs what DB2 offers.
I can not get a single flag in IIS changed so that we can use ASP to connect to a simple database. I can not put DATA up to a production server (static HTML content!!!!).
Yes, this is due to contractual obligations. However.. the people who sign those contracts don't listen to IT. Our ZOS mainframe does 97% of our processing and costs the same (ball parking and rounding here) than the 3% of processing done on our midrange. Why? We outsource midrange and mainframe.. but each midrange box is worth $40,000 per MONTH. *sigh*. Just give it to me and I'll go buy a new box every month and live the good life ever after.
Don't even get me started on IT Helpdesk support (also outsourced to IBM).
We're also not going to discuss upgrading single workstations.
Nor are we going to talk about how much they charge to load a program / PTF onto the mainframe.
Outsourcing only saves money if you are interested in not paying redundancies / wages / etc.
Excuse me.. I've got to go, I need a coffee.
Dammit, I pasted in a fury. Let me explain this one:
... just so people can publish static HTML documents!!! Pure instanity. At one meeting I told the web team that I could have their Intranet running on linux, apacahe AND have ODBC/PHP/Whateveryoudamnwelllike running with their current content. They said it couldn't be so (lots of reasons. Main one is: no servers on the network if they are not managed by IBM). I gave them to the URL of the box where we were 'testing' this idea.
"I can not get a single flag in IIS changed so that we can use ASP to connect to a simple database. I can not put DATA up to a production server (static HTML content!!!!)."
What I meant here is:
"We pay IBM to upkeep a web server who's job it is to serve HTML content."
It's a file server that people access by typing in a URL. Basically. Yet, no one on our side has the authority to copy files from a LAN drive to a folder on that server.
So, what do we do? That's right. We zip the files up and email it to them. They then unzip it to the wrong place (usually), restore the main HTML dir from backup, unzip the changes to the right place and then people see the new content.
To combat this we are paying for a brand new server, a WCMS and a whole team of people
Anyway. Outsourcing really can suck. I guess it would be OK if the terms, conditions and relationship with the outsourcer were okay.. but usually you run into the program the outsourcers are in it to make money. They don't want to hand control of the program back to you, or to complete a project when there is little hope of them maintaining it.
Anderson Consulting / Accidenture (accenture) ripped us off of more than 5 million, in one hit, then went for me. *sigh* We fired more than 600 people, and are still firing people to make up for the loss of 12 million $$$. It's like a ghost town here now.
My company has both ends of the spectrum. At our biggest data centers, there are mountains of paperwork and ridiculous costs of internal billing you'll have to wade through to do anything.
... but they haven't "developed" anything since it was built and it runs live data), and the server room is filled with desktops-made-servers because some are too cheap to go out and buy real servers.
... where they proceed to plug a monitor into the new circuit while they daisy-chain 4 power strips off of one circuit. Never mind you can just walk up to the maintenance guys and have them do things correctly and help you figure out how many amps you're drawing on a single line (and they don't even need a ticket ... just walk in and talk to them). Why bother? Just do it yourself.
... but nothing goes wrong, either. Pick your poison ... carefully.
Then at the site I work at, we have almost zero costs to anything, production servers not in the secured server room ("administrators" circumvent the system by claiming it's in development
Then we have guys walking around the server room and turning on circuits because they need additional power
At my site things can get done quickly. At the main data centers nothing gets done
Winners tell stories while losers yell deal.
You can always try what we did at my last job...making friends with the IT guys. Part of my job as an applications developer for a call center included extremely informal tech support. The phone people would call us first, and we'd solve their problem if possible (about half the time), or call in their trouble tickets when we couldn't do anything ourselves. As a result, we saved the IT guys from having to deal with a lot of problems that could be solved by rebooting the machine. In return they sometimes did us favors... Me: "Think I could get an extra PC?" Him: "Sure, just take one from that empty cube over there." Me: "Hey, what can you do about getting some more RAM for this thing?" Him: "I'll see what we have in stock." Me: "Hey, can you turn on IIS for me?" Him: "Ok, just don't tell anyone." (We were developing web apps in .NET and weren't allowed to have IIS. Really)
If you split your IT staff, then no IT person from Marketing really would know what's happening in Development, or Accounting, or Procurement.
I would hate to see even a mid size (100-1000) user company go that route, since you would quickly have people putting devices on networks that no one knows about outside their department. A network cannot scale well without good, central management and a solid plan for growth. You can't have techs from 4 different departments deploying switches willy-nilly, creating VLANS that propagate through your enterprise, not to mention that the guy you have upgrading your harddrive or installing Photoshop on your machine is very often not the guy you want installing network hardware, building a sound fileserver or implementing security policy. Or even on more of a systems level, you'll have 5 guys each buying 500GB arrays from different vendors and with different manufacturers stuff. You'll have Netapp in Marketing, EMC in Accounts, Equilogix in Development, homebrew NAS in Customer Care, etc.
I CAN see the value in assigning, say, a dedicated desktop/internal support person to each department, to familiarize themselves with upcoming projects and keep up with trends to determine future needs for that department. But I would always want to have that person/group reporting back into a central IT structure to keep things rational.
I don't know how literally you meant that to be taken, but it seems that disbanding a core IT group and discontinuing central reporting and IT planning could get ugly in a hurry.
I like music
Contrary to the assumptions of some users and developers, some of what they see as 'bureaucracy' is in fact necessary to keep a stable production environment. We it is quicker to just edit the production code but there are good reasons why your IT department doesn't let you do this. Installing extra memory in a server means at least downtime and a reboot, and will take down any datbase connections.
I am also surpised noone has mentioned ITIL yet...
Regards, Martin IT: http://methodsupport.com Personal: http://thereisnoend.org
You have much to learn about BOFHdom, grasshopper.
So take control of your own computers. Put them behind a department firewall. Put big stickers on them saying "Not IT Controlled" or "Keep your hands off!"
And then IT sends someone over to simply turn them off and cart them away. Any attempts to question or complain about this are replied to with a copy of the company policy (signed by all of the top executives) that says all computer equipment is to be purchased, installed, and maintained by the IT dept. Said employees are reprimanded for failing to follow the established procedure.
Don't even try it a second time, or the lawyers will be all over your ass for putting the company at risk by violating SOX, etc.
Do you work at a smaller company?
Where I work, the IT department (company) must interface "at arms length" with "transparent visibility" due to FCC regulations. My customers are always complaining that we can't do anything for less than $20k in less than 6 months. The process that we are forced to work under for FCC regulatory reasons simply cannot support IT-on-a-whim needs. All projects need to have documentation available for competive review.
/PAE boot option to address it! What idiots - and they wanted to buy more RAM for those servers!
IT manages a Class 3 Tier IV datacenter with tremendous redundancy. Buying components off ebay doesn't hack it - that's usually why our customer comes with too little budget when they finally officially launch a project. Ebay pricing is usually 50-200% less than certified parts that won't void a warrantee. Or their "in-house expert" thinks that $200 buys 2GB of RAM for a 5 year old HP PA-8000 server. He didn't check whether he could actually upgrade by 2GB when the server already had 8GB (2GB more isn't ppossible), or whether another memory carrier is needed, or what the current memory density of the server is.
Don't get me wrong, 10% of the time, it really is - go buy 2GB more RAM and install it, but the other 90% of the time, it simply isn't that easy - or better, the non-IT application team thinks that more memory will make the system run faster, when memory isn't the bottle neck. I've come across entire server farms with 12 and 16GB of RAM, but without the
Before you bad mouth your IT organization, try to understand what the real issues are. Perhaps you aren't as smart as you think. I know I'm not.
Effectively your department has outsourced its IT to the IT department and look at the mess you have.
Our state government has outsourced its IT to EDS and that has made one hell of a mess. I support a little old DOS product that the Government refuses to pay to upgrade or replace. This however doesn't stop EDS from continually upgrading client workstations to incompatible configs - despite our detailed intructions on how to set them up.
Getting a new server takes months. Getting a test server takes months and while it is supposed to be the same as the new server, in fact the new server supplied was some dodgy old second hand box that wasn't compatible and didn't behave the same way as the test box, hmm that was all money well spent. And they charge a fortune just to have a box in the server room that they don't know how to maintain - they charge for maintaining it, and then I have to be called in to do it and I charge too, I don't work for EDS.
Most of the problem is high up management wanting the IT department to account for its spendig. After the 80s this is definitely a good thing but not when it goes over the top so everything has to be justified in triplicate by 30 different people for something that costs less than 5 grand. For something that costs millions I could understand but computer systems grow old and obsolete faster than these bureauracies move.
I needed access to the config of one pc that my client dept was using to bypass the EDS problem, and I phoned the help desk to ask them to give me access, and they said they could sort it in three days. Nice for them - that money isn't coming out of their pocket.
And then there is the other EDS site, where they won't restore stuff from backup because it will mean the whole system will have to be shut down for three days. For one folder of files. WTF - yes there are lots of people in there that don't know what they are doing and need a good kick up the butt.
And another EDS site - that didn't have any backups at all. Nice one guys. Though I will admit - EDS is not the only company that stuffs that one up.
I'm still doing migrations for servers from Novell and NT to Windows 2000!!! What year is it now?
-- it must be true, it's on the internet.
How can your senior management be supportive yet at the same time fails to mobilize the IT department to help get your job done? Am I missing something here? Maybe the senior management hasn't been supportive enough? Or is the IT department even more "powerful" than senior management? (I hope not...)
Having been based in Japan for almost ten years, I have had to deal with this kind of thing quite a bit. My recommendation: Have the IT department answer to matrix reporting(both functional IT manager and project manager). Give the project manager some say in the IT staff's review. Beauracracy seems to fade away when bonuses and promotions are affected....
Sorry to hear that, and please accept my personal apologies. I quit Accenture in disgust 1 year ago, for much the same reasons as you mentioned above.
If it's any consolation your's is far from the only company badly screwed by Accenture.
In their defense, please don't attribute to malignant intent what can be explained by incompetence.
(Anonymous posting, because their legal guys are somewhat more effective than their development teams, and slightly more "go-getting")
So let me get this straight...you named the company that you hated working for. You named the coworkers that you hated working with. You described the project that you (and two other people) worked on while you were there. Of those three people on the project, you admitted that you were the one running around offerring blowjobs to get the hardware to finish the project. And yet you post as Anonymous? I hate to break it to you, but anyone that would be in a position to know who you are won't need to see your handle to figure out who you are.
Comment removed based on user account deletion
Comment removed based on user account deletion
Any idiot can understand that a 9 month turnaround on a RAM upgrade is unacceptable. Get your bigwigs to talk to their bigwigs and iron this thing out.
"Avoid employing unlucky people - throw half of the pile of CVs in the bin without reading them." -- David Brent
I think a very good idea would be to outsource the entire department to some IT services company like TCS, Wipro, or Infosys. Give them the entire contract of all your needs (may be for next couple of years) and ask them to run the show for you. In my opinion your case is a perfect candidate for outsourcing. And not only it will help you to run the show in a better way, but also it is good for the American economy as a whole. It is reported that for every dollar spent on outsourcing to India, the United States reaps between $1.12 and $1.14 in benefits.
I swear this could be someone in my team. Hell, it could be me. Your screen name is my real name. Frightening. This is the same world I live in.
Some problem definitions:
Problem 1: Territory
Central IT has a valid objective of maintaining standards for the purpose of security, etc. They do not have a valid mandate to do all the work themselves, if they can't do it to your timeline.
Problem 2: Power and influence
Outside IT, it may appear that you don't have the necessary influence to force change on IT. e.g. I could save my company bundles by putting in a real middleware solution, but don't have to influence to push it.
Problem 3: Non-lean processes
Many beaurocratic processes don't add value. This is where we get frustrated. You needed your RAM now, but it got delivered 9 months from now. My IT department saved me 5% on the cost of dev tools -- after 10-20 staff hours of meetings (at >$50 per person per hour = $500-$1000) that wiped out all benefits.
Approach to problem 1:
Recognize that the IT department has a valid mandate to enforce security and other guidelines. Include them as stakeholders in planning, but reach a compromise where you can implement yourself. IT validates that product X is a good thing and adds their requirements. I'll go buy it and install it on my own, thanks. This can save months waiting for some bean pusher to remember to send a purchase order out.
Practice this yourself: never insist on doing something yourself, if someone else is perfectly capable.
Approach to problem 2:
Get sponsorship at the highest level you can. You'll be suprised at the results you can get. IT is a cost center. If you don't report to IT, you probably represent a department that earns the company money. Getting sponsorship at a high enough level, and your sponsor has more power than the IT guys.
Problem 3:
I don't have a good answer. Sometimes its necessary to just do it and ask for forgiveness - make sure you have an ally (preferably a boss) to protect to from the fallout. SLA's are also a possible fallback -- but it's more beaurocracy. To make matters worse, our IT reorgs every 4 months, and so the department that signed an SLA may not exist to honor it!
My motto: "A cat is no trade for integrity."
I have had similar experiences.
A first suggestion: Try to explain your requirements better. The IT support people are (too) often narrow specialists in entirely separate teams, with only a tenuous understanding of what the company actually does, and business goals that are more or less independent of it. If you are a technology person outside IT infrastructure, issues that seem obvious for you may actually be nearly impossible to understand for them. Take the time to explain your problem, repeatedly and in very simple terms, until you are sure that they understand. Even in the best cases it can be very trying; but getting an understanding of the problem is the first step towards cooperation.
If you have an IT department with the primary task of maintaining office infrastructure, then try convincing management to create a second IT team for the technology IT support. It should have an independent reporting line to management and its boss should not be someone who understands your work. The two tasks are so different that it actually is very difficult for the same people to handle both. A five-star restaurant and McDonalds both qualify (at least nominally) as food suppliers, but you would not (be able to) hire the same people for both jobs.
Sadly, I have to agree that IT departments harbour far too many control freaks with meticulous attention for every regulation that gives them a bit more power, and that over time their bureaucracy does indeed tend to get completely out of control. The first victims of this are of course the people in IT themselves, and that can be your lever to change the situation: You can often find allies within the group, if you look for them.
And if you really need to break through an obstructionist attitude, it can be very useful to do a reality check. That IT declares that security system X is absolutely vital to have, does not mean that you have to believe this, or that they have actually implemented it everywhere. Don't just believe what they say, check what they do. IT departments are often very absolutist and alarmist in their declarations, but as economic with the truth as any politician. Especially if their management is weak.
Paperwork is a fact of life. If you don't like paperwork you should have chosen another career. I'm serious. Defects, fixes, server changes, installations, backout plans, configurations... you name it. It needs to be documented. End of story, if for no reason other than; if for some reason you can't be doing your job in the near future how does the work get done? Not to mention there are legal implications all over the board now in the tech industry.
If you're burdened by your bureaucracy it's due to a failing in that process and not in the paperwork that needs to get done. Now if IT is slow to respond it's a bottleneck in their process and I'm certain a qualified analyst would be able to look at how things are done and understand the breakage in workflow. I'm talking process development, not software development now. Too often folks think they're the same when it's not. A solid process requires no tools to support it. I guarantee that the poster's IT dept has redundant data and double entry all over the map and no one has taken a good look at the workflow. Just my 2 cents.
Oops, how did this get here?
09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
We got a number IT refusals explained by the magical SOX. So I went and read what it actually says. It doesn't mention engineering, their desktops or their servers. SOX is only concerned with financials.
Of course, IT wanted to treat all users and all servers the same. However, faced with the letter of the law, IT had no choice but to yield.
Even a simple task, like installing more memory in a non-production server, can take nine months and massive mountains of paperwork
See if you can buy the memory yourself and/or pay for the installation yourself. The very act of asking in such a way may trigger change.
Table-ized A.I.
Vista:XPSP2::ME:98SE
I hate to say this, but from experience of being an IT worker outside the IT Departments and being one inside, I'm not always sure it is bureaucracy (unless you include company politics under the bureaucracy banner). The last company I worked for had timelines included for anything which needed to be requested, and I noticed most non-IT departments just ignored them. Like when I was asked at 4PM on a Friday to organise a PC for someone starting on a Monday morning. Like, how did they expect the IT Department to have that up and running in the 1 hour left of work time ... and on a Friday none the less when the PC guys were going to run off to have drinks.
On the other hand, I was also in a unique position when they wanted to upgrade the Network team from 5 to 8 to deal with all the work, but some bright manager decided to cut the team to two. The rational being that they could save money. Instead, we had over 100 projects which couldn't get done, as the two of us were fighting fires all the time. Yet, people claimed our department was often too bureaucratic and we were never getting anything done.
You're talking about a lot of paperwork being needed. I always found Paperwork to be important (not just for covering my butt, but also to ensure we didn't have any clashes with what work needed to be done. Nothing worse than removing a router needed for files coming in at 6PM when it could get done at 7PM with no problems). If someone is inventing stuff which needs to be included in paperwork just for the sake of it, then that's a problem. If its there for a reason, then unfortunately, its a nessecary pain.
I think your managements best bet is to get some people in there to do some time and motion type studies (find out where things are slowing down/stopping) and install better procedures. Also, better paper work should be created.
I used to create my paper work in the order of when things needed to be used or get done. Especially if it passed through the hands of a few departments.
Your memory upgrade problem sounds a little like a problem we had once. We ordered a Sun Ultra machine, and it was all worked out, specs, cost etc, and then a manager sat on it due to political reasons (he didn't want to make the WRONG decission in case he got fired, so he sat on the paperwork). One year later, we still didn't have the machine. Someone else pretended to do the specs again (and all they did was blow the dust off the old specs done by someone else), and they ordered the machine. By this stage, we needed a better machine, as some of the uses had now changed (with more peopel deciding they wanted to be put on that machine and everything), and it was underspeced for the job.
Then, there have been times work was given to some people (and it's sat in their intray with an tome of other paperwork) while other guys have been mucking about throwing balls around the room as they have no work. A simple misallocation of resources in a Department.
So, a Time and Motion study for the IT Department, and better procedures for them. Maybe moving people in IT from one area to another in order to better utilise them. More people in some areas to stop bottle necks with paper work. But, it will require the IT manaagers to be on board. Nothing worse than having to fight the Managers in order to get their departments to actually work.
Sure enough, the cow costume was hanging up next to the superhero outfit and sailors uniform. (S,Spud)
While you are still energetic enough for these sort of things to genuinely bother you, you should quit and join a start-up.
In Soviet Washington the swamp drains you.
In most large organizations, IT is risk averse, there are on average more managers than there are people who actually do things, and about 70% of the people who actually do things are so specialized that they would never go out and do a RAM upgrade. Then you throw in things like change control which was put in place to stop IT people from changing things willy nilly and upsetting people like you, and you find someone who can actually get the thing ordered this budget period, and it gets hard to get stuff done.
On top of this, people who actually do things like RAM upgrades tend to be lowly paid, poorly treated, and be the first people someone blames when something goes wrong. In a lot of cases their morale is so low they don't care about your problem.
You want nimble IT, fire your CIO, and senior IT management, replace them with new people, pay the people who do things a reasonable wage and stop whinging.
We hired our own local IT people for things that required fast turnaround, or groomed one from within the ranks. For small hardware upgrades under $500, like RAM, we'd do it ourselves with discretionary purchases that took under a week to fill, with our own management's approval. For group-wide upgrades someone would order the parts and blaze trails on how to do the upgrade, and send someone from their staff to do it. Basically circumventing the system whenever possible, set up your own systems when others aren't working, is a form of self-correcting situation. When someone calls to tell you to stop, upper management should have a prepared list of instances where they would have been put behind in the schedule / spent more money / gotten poorer quality had they not taken matters into their own hands. If told to stop doing your own after all that, prepare them when they take away your freedom that the schedule may slip, the quality of the product may suffer, and money may be wasted. If they're happy with that, accept that it's their company and not yours.
...with some bearing on your problem. Listed in no particular order.
1. IT folks will protect their work. If their work gets assigned elsewhere, they're out of a job. Of course, be too protective and you don't get much done, and you eventually get out of a job also.
2. IT is responsible for delivering applications to users. That means making sure things don't break, but it also means introducing new stuff. Sometimes one is in conflict with the other.
3. IT needs a mix of people to go forth and do new things, and people that maintain systems in good running order. It's tricky to get the right mix.
There's others, but I'm tired.
"If we pull it we definitely get hit on the service level agreement; if we put it in we've got a 50% chance of taking no hit and a 50% chance of an outage which we can absorb easily. Is this the best thing for the customer? No"
Your other points are correct, but this is wrong. It is the best thing for the customer. They don't want the outage, the SLA says so, so they don't want you to risk it either.
You are rebuking the guy for suggesting he treat the IT department with respect...so what are you really suggesting he do?
Stop treating IT like a cost center, and treat them as a money maker. Salesmen make a commission. Managers make a commission. The IT guy working his ass off nights and weekends makes nothing for his early grave.
You want to motivate them? Give them part of the cut that motivates you.
"I am in the somewhat unique position"
:)
Your position can be unique or not unique, but there are no degrees to uniqueness. Like pregnancy -
"I am somewhat pregnant"
There you go, now you've got it.
Environmentalism is the new Victorianism. Everyone ties on a green corset and pretends we're virtuous.
It's people like you in Banking that make my life a living nightmare. I'm grateful you've gone.
You're a web developer, you don't make the call on the significance of the system, the change, the potential impact or understand division of responsibility.
Hopefully your 'company' will grow and you will *eventually* learn what the phrase CHANGE CONTROL means and what it is for.
Im in a german research institute, and there the Problem is the same. ... (which would cost maybe 5 min working time) only to have the smallest service, even when your Experiment/Project fully quotes that this is mandatory for the Project.
We have to collaborate with plenty of other institutes in the World, but the IT guys (as an example) only wanted to allow access to code repositorys if someone has a valid login in our institute. It took 4 Months to tell them that this will not work, and the idea is stupid.
There are several other things where it normally takes 6months waiting time and at least a full week in writing mails, phoning people, paperwork,
In my case in the end, i just swacrifices 50 eur/month and bought a dedicated host at a german ISP and develop there all the services needed for the Project, and ignore more or less everything which they do inside the institute.
If this will not be possible in the future, ill just play hangman during worktime instead of wasting time to circumvent (mostly stupid) restrictions from the IT Group.
I don't have that sort of problem at all, because I am my company's IT department.
The downside of that is... That I am my company's IT department.
How are you going to present that at your next job interview then? Say you just quit without a concerted effort to improve the organization?
I wouldn't want to permanently hire somebody who cares so little for the place where they work. Try helping to fix things. If that fails, at least you can quit with one more thing to say at that job interview...
Irene KHAAAAAAN!
Oddly, we sometimes have the same kinds of problems.
For desktop boxes, there are stores with hard disks, RAM sticks, and so on. Spare boxes to replace those that blow up too.
For a server, there really are hoops to jump through. We don't carry spare RAM for those (as finance won't shell out for stocks of RAM; they're covered in 'support agreements' if they go wrong). Trying to get upgrades for a server is a nightmare.
We work out what needs to be done. Send it to Finance.
Finance sit on it for 3-4 weeks, then sometimes reject it as being an unnecessary purchase.
So, we re-submit it again, saying that if it doesn't go through, then we'll shortly be suffering big problems.
So finance sit on it for another 3-4 weeks until giving an answer. Sometimes, this involves putting things more strenuously again, and sometimes it'll get cleared, but it's rare to get things through the Finance department first time.
Oddly, they're the ones that expect things to be done yesterday when they even think they may have a problem, and the most highly overstaffed department here.
The trick to running an IT department is to keep the paperwork down (which means the managers actually have to work at giving real reports), and don't have them at the mercy of some monstrosity such as a Finance department where people don't have a clue what the purchase actually means (or at least have someone in Finance that actually understands what tech is generally about).
I've just placed an order for an additional 600GB of SAN storage out to one of our servers. I've been told that the recurring anual cost will be GBP63,000.00. There is just NO WAY that my client (internal, another department) is going to accept that cost, so I'm going to have to end up bringing some other solution online.
Well, if it takes $400 of MAN hours to buy a $100 printer, why do sales managers
go off to client dinner/lunchs and spend $800 on one night.
Who ever designs these business processes must have come from the soviet union, because
thats how long they take.
Best solution is to buy the equip your self out of your own cash, and then take that many hrs worth out as extra time off. Ie buy that $100 printer, then take 3hrs off out of the week, 40mins/day early. No one can see it happen as long as the manager says yes.
This is exactly whats going to kill USA, and china will kick your butts, because they will have the gung ho attitude.
Liberty freedom are no1, not dicks in suits.
Make sure you activate your mobile phones voice recorder or PDA voice recorder
as a backup!!!
If hes an ass!!, you can always email it to CNN. Or the BBC if the company has rouge cia agents at CNN.
Liberty freedom are no1, not dicks in suits.
Finding a dead machine or killing a crap machine, then moving its
ram to your PC.
Too many restrictions are like having to fill out paper work to ask for coffee or
to request some extra paper, or more post it notes.
Office supplies are a commodity and so should there be ITOfficeSupplies that
could be requested within reason quickly. This is not 1982 where ram costs 4 weeks salary.
Why waste 3x the rams price in paper work? This aint a govt department.
Liberty freedom are no1, not dicks in suits.
Deceit, power games and manipulation.
We tell them whatever we need to tell them to get them to do what we want them to do. We escalate as needed (we're a high profile project), and clobber them from above. We trick them (we know more about their systems they they do), and get them to do things that have side-effects we like (eg we once got control of the NIS system to add a large number of users for a tedious job, and added a UID=0 user that they didn't notice, so from then on, we owned NIS, and could get (some) things done easily).
We still struggle to get things done, and would have much better results taking a co-operative approach, but due to history, and the fact that everyone around me takes this approach, I can't actually change the culture here. It blows, but you got to do what you got to do.
Quidquid latine dictum sit, altum videtur.
There are certainly more factors than what is initially percieved. If it is not too late in the game (offending the IT department) befriend one of them which might lead to information such as why there is a long period of time before memory is added to a server or what have you. This can lead to several advantages over the rest of the department you are in. accurate information is more important than those that feed the gossip sow with assumptions and sour conceptions.
Having worked (and currently working) in the IT side of this equation, I can tell you why you're getting resistance. Your IT dept is measured against the 99.999% uptime stick.
That means they have 5 minutes, 15 seconds of downtime they can have during a year.
That's why IT managers are so nervous about changes in a data center. If you go over that 5:15 of downtime it's their bonuses that get endangered because they're not meeting their KPIs.
I know our dev department complains about it.
Min
On the whole, I find that I prefer Slashdot posts to twitter ones because I don't get limited to 140 chars before
It's hard to believe this kind of crap goes on. One would think it would be a huge competitive disadvantage which would force the company to change or go out of business. But the truth is most, if not all, companies are rife with these kind of inefficencies.
Does it word this way? You make the request for a RAM upgrade. The IT guys pass the request to the upper echelon for aproval, where is is held for at least two weeks, then it goes back to IT, which has to find supliers ... the suppliers have to send pro forma offers (at least three of them, from different companies), which takes another two to three weeks. The pro forma offers go back to management, where they are disscussed, and one of them is chosen (two weeks more). The "chosen one" goes back to IT and they discover that what they chose is not what IT gave them, but a fourth offer from yet another reseller of RAM, and it's incompatible with the hardware that has to be upgraded. One more week is spent trying to get a meeting with the upper echelon and then explaining them why their choise is bad. After another two months the right RAM is chosen from the right supplier. The request goes to Accounting and stays there for another two weeks, is rejected and another supplier is proposed. Two more weeks while Accounting and Management try to find time to meet and debate the issue.
About four month after the initial request was made the money are sent, and the RAM is on it's way ...
Then IT have to test the new setup ... lots of paperwork, documenting the change, documenting the impact on the performance of the system, imagining scenarios, getting ready a new compatible backup system to take over in case the one that is upgraded crashes, building a test machine to test the impact, doing tests and making statistics etc.: at least one more month.
all this for some RAM ... yep, it might sound incredible to somebody that expects the IT to "just works", and who thinks that "it should just work, or else ... ".
ups ... I forgot about the Legal department that gets called to review the warranty contracts etc. ...
...no one are willing to take the responsibility for any change... it is a virus wich have infected IT-depts., it came from management or/and accounting.
IT-depts. in the financial sector (where I work, outside US) have probably the biggest problem with this, We can't do anything because our big leader (the IT-boss) need/want to have his back clear.
--
I've met women who got more cojonez than most men have...
Hah I bet he's a government contractor -- I am, and this sounds similar to my situation. Only difference is, I'm IT! If we had a development group, even one with an excellent track record, we certainly wouldn't just give them data center access and control. That's the whole reason we're here. And yes, the paperwork is insane (though to be fair it's only gonna take more than a month if someone is stalling you) but there's no real way around it.
Browsing with classic discussion, noscript, at -1 and nested
no hidden comments and I only mod UP
Oh surely you're exaggerating just a little bit. The geological survey people would have noticed a paper mountian.
There are many ways to deal with an unresponsive IT department:
My general contentment with my job has always been inversely proportional to the size of the employer. This is regardless of whether it was a retailer, school, or non-profit. The college where I work now (I'm half of the IT staff) merged with a larger university recently, and it's gradually getting bogged down. The functions the university has taken over have become slower to respond, and now they're trying to integrate our activities into their change-management system, and it's going to slow us down as well. There are some legitimate reasons for why this has to happen (more complex systems are more prone to failure) but it's annoying as heck. (Fortunately our college has a large Mac population, and the univerity staff know next to nothing about them, so they don't interfere too much in that area.)
http://alternatives.rzero.com/
Most IT behavior I've seen is mainly the result of how the whole company is managed. If the IT division is a mess, chances are that the rest of the company isn't much better.
It's all about communication and cooperation. Many IT departments do not take the time to understand the other parts of the company they serve. The bosses' merit pay or bonuses don't use those metrics. So, neither do they.
If it's all about cutting costs, but not about investing, then the infrastructure will get slashed for no apparent reason and nobody will be able to make a decent business case for anything.
If it's all about strict heirarchichal bureacracy and justification of the need for resources, then there will be mounds of paper required even for the most ridiculously cheap devices.
If it's all about security or the lack thereof, then you'll see extreme behavior in both realms.
These problems are usually symptoms, not the disease itself. Fighting it is actually much more difficult than it looks because you're fighting corporate culture. If Managers want to know why their company can't have a good IT department, maybe they ought to look in the mirror and ask themselves what's wrong with the way they're doing their own jobs...
Nearly fifty percent of all graduates come from the bottom half of the class!
You say that your applications can pass audit criteria and that's a start. Create some measurement data that proves to the company:
- Your applications contribute measurably to the business
- You have high and improving productivity in development
- You handle product release and configuration control better than IT
- Your products have high customer satisfaction
- Your customers like working with you
If you don't have measurement data, it will be "he said - she said" and then the biggest pot of money wins.
Read "Managing Information Technology for Business Value" by Martin Curley.
"If all the American people want is security, let them live in prisons." Eisenhower
We had this problem at a company I used to work at. Very big company, very big IT dept. Nothing gets done. I was fortunate that I worked for the same VP that ran the purchasing dept. I basically created a satellite IT dept. for our dept. and for the purchasing dept. I maintained all our network and PC's and servers and accepted no support from the official IT dept. except for bandwidth, etc. I had to start out slow and work around the edges at first, but I was ultimately only able to do what I did because I had the support of my VP -- since he was resp. for all purchasing, including for IT, and since I kept *his* toast buttered, I was able to 'get away with it.' However, I had to keep a fairly low profile and not rub it in, if you know what I mean. We still bumped heads quite a bit, and I didn't win every battle, but at least I was able to keep my dept. going and vastly more productive than it otherwise would have been.
-pfs
The lack of agility is maddening, because I know we are missing significant business opportunities.
Is most assuredly not a 'technology worker.' That reeks of so much manglement BS (buzzword speak) that I can smell your shiny, diagonally striped tie from here.
Either that, or you had help generating your submission.
--
Sig arrêt
Group policy settings can block you from changing this. Let's not mention there's a way around them, but it's more difficult than that. ;)
...someone gets to be an IT worker outside the IT department. In my experience, usually this happens because some dept. head is unhappy with IT. They think if they have someone they can control directly, they'll get things done 'their way' instead of the 'IT people's way'. So they go and hire someone.
This poor soul then comes into the company without any knowledge of the wars that ended up by spawning his or her job and gets all surprised because IT is less than helpful to him or her. If you think their job's simple existence means IT lost that war it becomes clear why IT reacts the way it does.
But feelings and corporate politics aside, usually and especially in complex environments, there's reason for what outsiders perceive as bureaucracy in the IT dept. This is not to say that sometimes structures ossify and start abusing their powers, by no means. That does happen, but I believe most of the time that 'bureaucracy' is just IT trying to cope with absurd workloads.
Remember that IT depts have been hit hard by cost-cutting measures. There's never enough warm bodies to tackle all the projects and the backlog is usually huge. Remember that, even if the 'IT person outside the IT dept.' is absolutely flawless in their skills, mistakes and security vulnerabilities, especially done to central resources, will ultimately be blamed on IT and IT will be the dept. expected to correct the problem. Combine these two issues and you begin to understand why IT depts. everywhere are pushing for centralized controls. There's no other way to make sure of a lot of vital things such as: changes are logged somewhere so people know who did them, why and more important, how to undo them if they have to; proper testing has been done before changes are implemented; backups are being done and spot checks are happening so those tapes are actually useful if they are ever needed; all (sometimes thousands) Windows workstations are having security patches applied regularly and anti-virus definition files updated at least daily; etc, etc etc.
We have what I think is a good plan where I work (state university) - and yes, I work in the IT dept.: you administer, you support; you want our support, we administer. In other words, if you have the root/ administrator password, you are self-supported. Why is that? Because our team of 8 people wouldn't have time to fix everybody's computer if all our 8000 users had the freedom to download and install whatever they want.
Although I believe 9 monthes is way too long for adding memory to a server, if someone is trying to do it right it's also not a 10 minute job. In our environment we do have to cope with state purchasing laws and regulations, for instance. Yes, getting a memory stick from buy.com and sticking it into the server is appealing, but it's illegal and that's not the IT dept's. rules. Beyond that, we want to make sure we're buying a trusted brand, the vendor has proper warranty, the server actually supports the part, the server downtime will not create other problems down the line. Not to mention, in times of tight budgets, checking if the additional memory is even needed. Maybe trying to be a bit more efficient in your code or database design would save the company a lot of trouble.
I used to work as a sysadmin for one of the big 401k companies, and it was actually our departmental policy to have a large, 23 page form/mini-book that needed to be completed in its entirely prior to any server work being performed. This was intentional, to help minimize the bullshit requests from the important ones. We stayed busy enough trying to keep the infrastructure running, and to be constantly approached with new requests for new server builds, "minor upgrades", etc., while certainly justified in many instances, was just overwhelming. Management would not put more money in to hiring more resources, so we were left to come up with our own solution. If you truly had a server that needed an additional 4gigs of RAM, then you would spend the time to fill this form out, have various managers sign off, and then this would demonstrate to us that this truly was important, and that proper people had authorized it and were aware of what was going on. This was also a CYA tactic, as in any large organization, much of the time left hand does not know what right is doing. This process truly did help, and the CYA aspect actually helped as well in a few situations where one VP was bitching about a server being down for 20 minutes at 2am and wanting heads to roll, only to show him that another of his peers specifically requested the server be down for an upgrade/build out.
Don't mistake this for me trying to justify the type of hell people might go through to get this sort of thing done, but just trying to add a bit of perspective on it.
As I did- I simply took the bullet for my beloved departments and joined the IT staff in effort to advocate on their behalf. With any luck I'll be noticed for my efforts and grow beyond my present and former positions. Usually IT is really good at seeing "walls" between departments and if yours, IN EVERY WAY, is even a little untrusting of it they will know and thus the mountains of paperwork will appear. You can sometimes argue the service aspect of your needs and illustrate how allowing you security clearance will enhance the end result of their support for you but that's best done from within their toolbox so to speak. Become the useful tool they just can't put down! You are likely the main source of knowledge for your position that they can't duplicate and would snap you up in a second. They did for me. Good luck- it CAN be done.
I've been an IT person for several places, some of which had no rules whatsoever, and some that had the nine-month memory upgrade syndrome.
I've determined that once an organization grows beyond a "small business", there cannot be a "no rules" approach. If there is, lots of money gets wasted on hardware for people that self-approve their purchases, and critical apps go down in the middle of the day. The apps aren't fixable until the only person in the company who knows the system gets back from lunch, because he has all the info in his head.
The other side can be worse. My last job was for a company that got the whole ITIL religion. Absolutely everything had pages and pages of documentation attached to it. Service requests got routed through several levels of helpdesk before they got to us. We had a full-fledged project management office that made us spend more time in status meetings than working on actual projects.
There must be a happy medium. Period.
I feel your pain. I survived a Anderson infestation in the 80s. It always amazes me that anyone would hire them considering their track record. I put them in the same moral category as spammers and 419 scammers.
The obscure we see eventually. The completely obvious, it seems, takes longer. - Edward R. Murrow
1st week: Help desk ticket assigned to someone that doesn't work there any more!
2nd week: Assigned to queue for "first person who is available" to take it. Translated: "They cherry pick jobs that are easy out of the list"
3rd week: Assigned to someone that's on vacation.
4th week: reassigned to someone who is present, they email me at 4:45pm telling me they "need more information" and they're going
on vacation starting at 5pm.
5th week: They come back from vacation and work with me to install
part of outlook so DTS jobs on the sql server can send email.
Woo hoo!
-- Programming with boost is like building a house with lego. It's a cool but I wouldn't want to live in it
You gotta use bright shiny trinkets to get their attention.
I'm in the same situation. Last week I was asked to review an IT proposal for a project. They wanted $30K to implement it. I read the spec, wrote the code, tests, documentation, UML diagrams and demo site in about 6 hours. It's modular, and flexible enough to be used generically in a variety of situations.
Every time I do this, I call it a "prototype" so that the IT people can save face. We are doing the final customer testing now and it will be in production in about a week.
Last time this happened, the customer asked me how long it would take to write. It was just about Noon, so I told them that if they would go down to the cafeteria and get me a sandwich, I would write it over lunch on my own time.
IT provides a valuable service to most companies. But in every company I've worked for, IT has always been a dinosaur. They backload their costs into every project to hide their inefficiencies. My best advice is to learn how to play the game and work with them to make them look good while keeping their burden low. In my case, they can handle having a "prototype" in "extended production testing" since it does not add to their workload and it really isn't in production (it's just been running as a customer acceptance test using real data for the last 3 years).Well, I don't work in IT anymore for this very reason. During my 10-year stint with servers, routers, PCs and the like I did meet one guy who could really get things done. He ended up being the Cisco monkey at the last company I worked for and this was his advice:
"It is better to beg for forgiveness than to ask and be denied."
Bascially they would find a way to get something done and have it finished and in production before the sluggish controls could kick in. He'd get called in for a reaming and just say he was sorry and agree that he was wrong. Eventually, the management decided he was just the man to upgrade the entire enterprise network over only 3 months. Maybe you can't do this, but usually if you put out good stuff the mediocrity-moderating red tape won't apply if you just go bad ass enough to risk it.
For inspiration see "Sharky's Machine" with Burt Reynolds, great flick about this very thing with a little extra evil so you don't fall asleep. Lastly, it helps to not have some arsonist waiting in the wings to burn down your machine to forward their own goals. Loose lips sink ships.
It's amazing how this discussion has degenerated into IT people bashing non-IT people, and vice-versa. From my point of view, the real problem here is the beaurocracy itself.
I have worked with many IT departments in many companies, large and small, over the years. Most of the people I have communicated with are helpful people who are willing to help me out as much as they can. Sure, sometimes I am not a priority and what I want to have done ends up on the bottom of the queue, but usually I am told this up front and am given a timeline that I can plan around.
The real problem, I find, is the upteen layers of beaurocracy that have nothing to do with IT that still are a major part of how the IT department works. Someone in finance wants everything justified in triplicate from every member of the company in order to get something done. Many of these beaurocrats don't understand the necessity of a timely response to X request/procedure/whatnot (which isn't their job), but they're not willing to take the recoomendations of people who have been trained in such things either (which is their job). They justify it with not wanting the IT department to pull a "fast one" with their budget; I understand that there does have to be accountability, but it does become ridiculous.
In my experience, most of the time it's not the IT people who are to blame for it taking 6 months to get a new stick of RAM approved. It's the fault of the beaurocracy itself. So here is my solution:
1) Document the timeline and costs, including man hours, of getting a non-essential piece of equipment/software/whatnot replaced.
2) Present this data to the bean counters.
3) Suggest that all of this be able to be authorized in retrospect within a certain time limit. i.e. IT person has new RAM ordered, installs it, THEN the majority of the paperwork can be done.
4) Point out that a low-level peon or temp could be hired at a much smaller wage than yourself or an IT professional to do the necessary accountability paperwork, hence saving the dept. even more money.
5) Repeat all steps as necessary.
It will take perseverance (squeaky wheel gets the grease, yadda yadda yadda) and probably some time, what with administration shuffles and how long policy changes can take. But once the policy has been changed, it will be well worth it.
I have worked on both sides of the fence, IT and MIS, but my experience in IT has shown me that most MIS people are clueless morons who couldn't plan their way out of a paper bag. Suddenly their lack of planning is my emergency and their lack of inclusion of IT into their planning forces IT to perform Herculean tasks on miniscule times.
Examples:
Like a previous poster said: "Your lack of planning does not constitute an emergency for me".
When business people stop looking at IT as an impediment and a drain on corporate finances and start looking at IT as essential for the company to function and as a partner, we would be glad to assist you. Until then, go pound sand.
Yes I see this a ton in my line of business (Gov. Contracting) and it is absolutely offensive when you know more than the life longer with the company on the other end of the phone in the tech support department. Or have to explain basic Network Security concepts to 20+ year Network Admins!!!! You woulnd't happen to work for the big dog of gov contrating would you? LOL
More than this I find that the older elitist are discriminitory against young employees bring fresh ideas to the table to make small changes. I am not sure if they fear the change or are insulted that a young employee is basically suggesting that there way isn't the best any longer.
I believe a path to resolving the bureaucracy resides in understanding the source of it from both an IT and a Business perspective. Let me start with the Business side. Business users (oops, now you know I am in IT!) get frustrated when they have IT needs that they cannot seem to get accomplished. In a large number of cases, there is no process for even getting a project "into the hopper". How many times do you ask for something and are told to complete a project request form that defines the value of the project, only to never hear back the status of your project from the IT department? Probably more often than not.
Now for the IT department. Requests for IT resources are coming in from all facets of the company, not necessarily being filtered in any way. Add the sales force and other field employees and now you are talking about a very overwhelming number. Not to mention, IT has their own "technical" needs (upgrading server software/hardware, training, etc.) that is in addition to the other requests flooding in. Sometimes it seems like people feel if they bug IT enough or go "up the ladder", their projects will get done quicker. Unfortunately, it is much easier for the people at the top of the ladder to tell IT to help the business user out than for them to be involved in justifying the value of a project.
The business users want their IT projects completed for the benefit it will provide their respective area and the impact that can have on the company while the IT department wants to work on the projects that have the highest value to the company with the least amount of complexity. What this spell? P.R.O.J.E.C.T. P.R.I.O.R.I.T.I.Z.A.T.I.O.N.!
A formal project prioritization committee can greatly reduce the perceived bureaucracy within a company. By defining and agreeing a methodology, the emotional aspect and uncertainty are removed from the project list. Additionaly, the estabilished priorities can facilitate resource scheduling.
Without priorities clearly established, IT may end up producing nothing because the constant changing of priorities is causing an enourmous amount of overhead. Due dates begin approaching rapidly and analysis/design phases are cut short. This leads to higher maintenance costs and therefore less resources for the IT department in the future.
Some recommendations:
...turning the IT Dept off and on again?
Lower rung? If you consider being one rung down from director "lower."
z antine
In a simple heirarchy, sure, your place on the totem determines how much crap you have to put up with. In a Byzantine bureaucracy like an academic institution, the whole point of the exercise is that your place on the totem has little to no bearing on how much b.s. you have to endure because the bureaucracy itself encourages sociopathic behavior as people assert the sovereignty of their little kingdoms. Even the executive director would biatch about this crap when he was battling for resources between institutes. Whether you win or lose these battles is inconsequential. The problem is, you have to fight them, no matter how high up the food chain you are.
See:
http://en.wikipedia.org/wiki/Derogatory_use_of_By
I am an IT worked in a very large organization. And yes, often I am embarrassed by my own IT organization. Many of the ~5000 of us could do things better, plus, clearly the processes are what are most frustrating for anyone on the receiving end of our services. I would claim that all of us, providers and customers need to step up and become accountable for the numbers associated with our work. At which point we tie these metrics to our performance reviews and increases/bonuses {wouldn't it be cool if the Feds did this too}. Eventually we will see a turnaround. Keep talking about it, it will get better. As my famous neighbor says, "I guarantee it".
-- "I have something stupid and ridiculous to tell you." Alfred de Musset, 1833
The HELP DESK?! Dude. I was in a proxy-war between two directors.
Yes, I got what I wanted. Yes, there was a business case. I had to load $100k+ of server software on this danged thing because I was going on about $30k of travel to several cities around the country within a couple weeks and needed to have my project components with me with the assumption of no (or slow) VPN connection.
The guy was just letting my director know that HE was in charge of IT purchases and he didn't approve of a couple previous _major_ purchases that successfully got around his gridiron. So, he gave me the screws over nothing just to make a point not to me, but to my boss.
This had nothing to do with money and everything to do with politics.
Sigh. As a department head, the department and its budget is yours. Of course, I also presume you would also tell the VP above you that you're doing it.
We do this at our work for some machines. We would have to go out of business otherwise. We are not a small company, we're a division of one of world's ten largest corporations. I remember one instance when a newb IT flunky got delusions of godhood and wiped and unplugged a machine that wasn't under IT control. He's not here anymore.
If there's a silly policy in the way of doing your job, get the policy changed. An ordinary employee won't be able to do this, but a department head does have the ability to apply some pressure. Unless you like sitting around with your thumb up your ass for nine months waiting for RAM, that is.
A Government Is a Body of People, Usually Notably Ungoverned
This doesn't sound much like an IT-specific problem to me at all, but rather an organizational one. IT management needs to be taken to task, simple as that. It would be no different were this a manufacturing bottleneck, or a procurement bottleneck, or a legal bottleneck, etc, etc, the only difference is that people tend to think less of IT concerns than the more traditional and understood concerns of these traditional departments. The problem is always in the management, not the people. In a word, escalate. If that doesn't work, escalate more. If your CEO can't fix it then your company's broke so either deal with it, work through it or move on to another job. Everyone else here who is complaining seems to be talking rubbish since you haven't really stated what the specific holdup was. For all I know, this non-production server is sitting next to/on top of/sharing UPS with the most mission-critical server in your organization. Or it could be like it is in my company, an IT department that has been cut to the bone and is struggling with 2 people doing a 5-man job.
If there is a log in the road, you go around it. That presumes you have the skills to do so - otherwise you are stuck.
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
Nearly 1000 developers. We're Fortune 500.
Viewing the current date is only for the Administrator.
That's what you get for using Windows.
It seems you work for a business embedded IT department that develops applications, and you have to request work from a centralized IT infrastructure services division that serves multiple business units. The frustrating experience you have shared is a well known side effect of this type of set up. The centralized IT unit is often allowed to operate as a cost center, where they get a bag of money at the start of the year. The amount is not based on the actual work they are expected to do that year, but rather it is an arbitrary percentage of the profits made by the company as well as industry standards. As you can imagine, when the supply of money is unrelated to work, the centralized IT unit has little incentive to pay heed to business urgencies. Instead, the unit continues to try and make optimum use of the said bag of money. Very often, the centralized unit will have a system of "charge-back" and it would appear that they charge for their services, however, this system is merely an accounting convenience and it is largely used for justification to the shareholders and auditors. The good news is that because the problem is systemic, there are fairly simple ways to get around it. The system can be made to work, even taken advantage of, if you know how it works. 1. Establish personal relationships with the group that does data center work (like server installs and upgrades) for you. Use common vendors as a channel for introduction. Take them out to lunch. After putting in a formal request for work, follow up through friendly, personal meetings. Avoid email exchanges, especially ones that are copied to too many people. 2. Build a case for how much delays cost in terms of opportunity cost. Use specific examples. Ask a senior manager in your division to share the document with a senior manager in the centralized IT unit. Ask for a written response from them. Persistent, written demonstration of flaws will eventually penetrate even the thickest bureaucratic walls. 3. Quickly escalate if things are not going your way. Quantify the opportunity loss (dollars/day), and send out an email every morning to management on both sides. In the escalation messages, focus on the business loss rather than wrong doing of groups or people. Managers use fires as a methodology to prioritize work, so go create the biggest one! I hope this helps. Please keep sharing your experiences with us. Good luck!
the HR IT department within the company I work for is down right surley/arrognt towards it's clients. I work in a well known financial company. It takes months & months of haggling with them to get even simple business requirements accomplished via the "project managers". The mountains of paperwork needed defies common sense and any "best practice" standards. The majority of the IT staff has resorted to making up stuff to put in the "required" documentation that it is demanding. Oh Yeah... that's effective...? Management is supportive to the constant frustration in working with the IT department, but nothing seems to change or work. Hence we have learned new ways to avoid any contact with our HR IT department unless absolutly neccessary, but this is not a long term solution, but what other options are left??
But a RAM change is not necessarily simple.
In datacentres where you may have 30 or 40 machines in a cabinet it is a real risk to allow people to fool around changing some RAM.
It should not take 9 months, but it should not be allowed to happen in "automatic" approval mode.
IANAL but write like a drunk one.
.... to get lost and follow the procedures.
The rest of my users deserve that I weight carefully their concerns and needs and act farily in consequence.
IANAL but write like a drunk one.