Slashdot Mirror


The New IT Crisis

Matey-O writes "You've succeeded in delivering 5 9's, your server farm is a well oiled machine, the helpdesk lines lie dormant. No? Well then how do you get credit for the work you do, when all that's noticed is the downtime? When the IT budget has to be justified, and you're overworked, undermanned, and you have to apply three patches to 100 servers before Close of Business, what has to change in IT before we melt down? Marc Andreessen has an interesting article on what has to happen to IT next."

69 of 309 comments (clear)

  1. I always just "look" busy by SteweyGriffin · · Score: 5, Insightful

    Well then how do you get credit for the work you do, when all that's noticed is the downtime?

    I feel that if I work hard (and smart enough), then I deserve free time every once in awhile. After all, I earned it.

    But, managers don't understand this. So, I relax by reading The Onion or Freshmeat at work, but always make sure my hand can quickly hit ALT+TAB to get back into my work window (usually Emacs).

    1. Re:I always just "look" busy by LostCluster · · Score: 4, Insightful

      If you're doing an IT system properly, whoever's assigned to supporting it should have some of those "Maytag Repairman" moments where they have nothing better to do than surf the web just to stay alert. This isn't because he's goofing off, it's just that he's done everything on the preventative maintenence checklist, and during his time alloted to responding to crisis, there is no crisis to deal with. The system is working, nobody is complaining of a downtime or slowdown.

      What you're really doing is not wasting time, you're sitting in an "on call" position waiting for your phone to ring with the next big crisis. If that call never comes, then you must have been doing something right when you were working hard.

    2. Re:I always just "look" busy by WCMI92 · · Score: 3, Interesting

      "My old approach was to say to the boss "What ya gonna do? Fire me? I'm too low paid and I claim slacktime NOW." Wierdly, it usually worked and I got my slack off time. Turns out he needed me more than I needed him."

      I usually just try to always look busy. I have such a reputation for hard work, that in the (rare) slack times I can usually screw off and get away with it.

      But you get to that point by ACTUALLY working hard when there is work to do.

      Thing is, when you are a network admin for an enterprise, there is ALWAYS work that you can be doing.

      --
      Corporatism != Free Market
    3. Re:I always just "look" busy by ergo98 · · Score: 3, Interesting

      Mind you virtually no one is fooled by the alt-tab: When everytime you walk near somebody their screen flickers as they swap windows, or they fervently hit the keys whenever someone nears, you know that they are slacking, but to many people the thought process will be that they're looking at porn or playing a game.

      If you aren't willing to do it as "research" (for instance as funny as it sounds Slashdot can be a work-related website for many tech firms. It is "putting your ear to the ground" in a sense), then I recommend what another recommended which is that you browse in a manner that is inconspicuous and requires no screen swapping: Lynx, or grahics-less.

    4. Re:I always just "look" busy by Dysan2k · · Score: 4, Interesting

      God.. I pity you peeps..

      Seriously, any time I go to work for a new company, the first 2 months are spent automating everything across the board. I'll generally get no sleep quite a few nights writing software, and when I'm done, I just tell management that because of errors in the past, all work orders will be processed through this new interface. I give 'em an internal web addy and then go play Quake/EQ/etc. for the rest of my time there (which has never been less than 1 1/2 years which I get vested options from). The websites handle all the dummy checking, logs all processes to another system to check periodically to make sure all works well, and performs whatever request they want.

      I order spares for about every piece of equipment in the building including spare switches and 1-2 spare servers for the occassional *frying* motherboard. If the servers are set up in non-redundant fashion, I make sure load balancers (or happy cisco 6500) are ordered, every server has a backup, all backups are automated and working properly, and put in the DR (Disaster Recovery) report JUST incase they want to go that route.

      Frankly, I have MAYBE 1 hour of downtime a year, and that's usually attributed to my tripping over a cable that some numb-nutz (who's gonna get chewed out for an hour) layed outside of the wire maintenance tray. Only reason I move jobs is because I just get WAY too bored. Admining is easy for those who aren't inept.

      --
      -What have you contributed lately?
  2. We should all follow Marc's example... by PissingInTheWind · · Score: 3, Interesting

    capitalize on other's work!

    It's the easiest way to success.

    Details here.

    Quote: it's not as if he needs to sit down in front of his screen and busy himself with the notoriously arduous task of hacking out a few lines of software, which, astonishingly, is something he has never done in the short but spectacular history of Netscape Communications Corporation [...] Not a single line of computer code. Never.

    --

    A message from the system administrator: 'I've upped my priority. Now up yours.'
  3. Do what everyone else does . . . by Gabrill · · Score: 3, Funny

    Blame it on management. If that fails, then blame it on the budget. If that fails, then blame it on HR. If, by that time, everything comes back to your fault, then you should have had enough time to land a new job.

    --
    Always going forward, 'cause we can't find reverse.
  4. solution for one of the problems.. by MoceanWorker · · Score: 5, Informative

    and you have to apply three patches to 100 servers before Close of Business

    Not flaming, but a fact.. Microsoft SMS will get that done for you with a couple of clicks.. literally..

    And we're expecting to receive a beta copy of SMS 2003 (incl. Active Directory, the big feature).. so i'm kind of looking forward to that..

    I have to say.. even though we run 2000 on our servers.. SMS is a dam good product and a timesaver!

    --


    "The ones who dont do anything are always the ones who try to pull you down" -- Henry Rollins
    1. Re:solution for one of the problems.. by mcrbids · · Score: 5, Informative

      Another solution? RHN!

      Red Hat Network allows you to do the same - a couple of clicks, and away you go!

      The advantage is that RHN is here now, and works very, very well. I've never seen a Red Hat Network update break anything. Ever.

      And they are quick about getting updates ready!

      --
      I have no problem with your religion until you decide it's reason to deprive others of the truth.
    2. Re:solution for one of the problems.. by LostCluster · · Score: 5, Interesting

      Why don't we just go back to the mainframe way of doing things?

      When users had dumb terminals on their desk, they had the illision of a full power computer, but it was actually a small box with few moving parts that was linked to high-powered computers (or cluster of computers) who were actually doing the heavy lifting. Since all of the functional components were in the Computer Room, there was rarely a need for tech staff to touch the dumb terminals, and the tech people could work in their own distraction-free environment.

      What's more, failures could sometimes be abstracted away from users. Hard drive failures happen, that's a fact of the technology. However, if a HD fails on a user's desk, it means that user has lost the use of their computer until it's fixed. If an HD fails in the datacenter, there's usually a backup of the data which can be put into play immediately by mounting the backup on a good drive that's already spinning. To the users, the disk crash can be practically invisible.

      There's already tech technologies such as X-Windows and Windows Terminal Service with which to create GUIs on a dumb terminals. Why does the common secretary need a full-powered PC on her desk anyway?

    3. Re:solution for one of the problems.. by Catbeller · · Score: 4, Insightful

      As a guy who got his first start programming on an IBM mainframe, I just want to break down in hysterical laughter.

      Early nineties: everything was to be decentralized. Get rid of that antique iron! Go Intel/Microsoft! Can the tech Luddites, bring in the Bright Yunguns, who will show us the way.

      Mind you, I was one of those calling for the company to modernize on a client-server model.

      But I've watched for, what, 20 years since this all started. I watched the expensive mainframes go into the landfill, and the new servers take their place.

      I've watched those servers get more expensive. I've seen the communications hardware become massively expensive. I've watched the new Data Priesthood come into being. I've watched companies being wiped out converting to, and maintaining, the new paradigm.

      Not that it would have been good to stay on the old iron. BUT...

      We've come full circle. The hounds are baying for simpler solutions, and most importantly, for the elimination of the new mainframers, the admins.

      I predicted it over ten years ago. The PC's will cluster into data centers, pseudo-dumb clients will spread onto everyone's desks and laps, the services will automate, and the present IT industry will convulse. A shakeout is coming. Time to get that teaching career started.

      As a sidebar: in my experience, far more money was spent on executive games and perks than was ever lost on IT spending. Cost accounting is applied where management wants application.

    4. Re:solution for one of the problems.. by pmmay · · Score: 3, Informative

      They make Red Hat Network Enterprise, Proxy and Satellite services just for this. https://rhn.redhat.com/purchase_info.pxt

      I have no clue how much it costs. I assume if you have the need for it, then it isn't that much.

  5. My 2 cents by WetCat · · Score: 4, Insightful

    Create presentations, monitoring
    systems with a lot of 3d and 2d diagrams
    and histograms.
    Bosses love that.

    1. Re:My 2 cents by fredrikj · · Score: 3, Funny

      You only get two cents for that? Indeed, the situation for IT workers is getting worse...

    2. Re:My 2 cents by Mr_Person · · Score: 3, Informative

      Well then you should try Nagios (Used to be Netsaint). Does all sorts of monitoring and 2D maps of systems (and their status) or a 3D VRML map! Doesn't do histograms though... In all seriousness though, it really is a neat piece of software and very handy.

  6. Am I missing something? by silvaran · · Score: 5, Insightful

    Since when do telephone systems maintain themselves? Last I heard, my voice was still running along wires and branching about in switching stations. Unless I'm mistaken, they still require maintenance. Granted that the operating costs have been reduced (fewer operators = fewer wages), but you could take a similar stance on IT. What about self-help forums where you can search a knowledge base to find answers? These can replace a lot of man hours of technical support work. There's always going to be some kind of human element to whatever equation. We're never going to find empty power plants that can generate their own electricity indefinitely, because there's always going to have to be human intervention.

    Some businesses demand complex solutions, and I fail to see how these complex solutions are going to be met by turnkey solutions -- where a manager can go out, purchase a server, turn it on, and have it run his business for a year without any kind of customization whatsoever.

    1. Re:Am I missing something? by mickwd · · Score: 4, Insightful

      "Since when do telephone systems maintain themselves?"

      It's not just telephone systems.

      Think about almost any complex engineering system. Many are computer-controlled nowadays.

      But almost all of them, even those that aren't computer-controlled, are taken for granted by everyone. Transport, power generation and distribution, warehousing, running industrial plants, you name it.

      When was the last time you praised the people that keep all these services running ?

      But when they go wrong - the power goes off, there's no produce on the shelves to buy, roads are closed or flights are badly delayed - there's millions of people jumping up and down and demanding the situation be sought out straight away.

      When IT isn't a hobby or an "entertainment medium" it's nothing more than a tool to get a job done. I know how difficult it can be to work in IT, but I've got to look at some of the responses here, compare them to the jobs of people working in some other industries, and think: "what a bunch of whingers".

  7. Meatless drivel by iceT · · Score: 5, Insightful

    Ok. Just becuase it Marc Andreessen doens't mean that it's news.
    This is an editorial, not an article.

    How about talking about Grid computing. Or Organic IT. Developing systems and monitoring capabilities that go beyond telling you things are down, or when they're too busy. They actually add capacity on the fly through virtualization, taking from inactive systems to cover for the active system.

    I think Marc once had a vision, but I'm not sure how strong of a visionary he is these days...

    --
    -- You can't idiot-proof anything, because they're always coming out with better idiots.
    1. Re:Meatless drivel by sql*kitten · · Score: 5, Informative

      Ok. Just becuase it Marc Andreessen doens't mean that it's news. This is an editorial, not an article.

      You mean Marc Andressen of Loudcloud, who sell server and datacentre management services, and who are desperate for revenue? That Marc Andressen?

      It isn't even an editorial, it's free advertising disguised as a story. I hope they're paying whoever owns Slashdot this week for it.

    2. Re:Meatless drivel by selectspec · · Score: 4, Interesting

      No doubt. Andreesen is trying to fabricate a crisis so he can save his worthless company, Opsware (LoudCloud), from insolvency and his career from the where are they now file.

      --

      Someone you trust is one of us.

    3. Re:Meatless drivel by HamNRye · · Score: 3, Insightful

      s/Meatless Drivel/Total Bullshit/;

      Aside from this essentially being an advertorial, there remains the fact that he presents a problem, (kindof) and then presents no solution.

      Auto Mechanics, tired of only being appreciated when that car breaks?? Try fixing it before it breaks! Don't ask how.

      Firemen, tired of only being appreciated when someone rams a plane into a building?? Try just driving around town hosing everything down.

      Now, lets talk "DataCentre Management tools". My experience is this: You need to hire two extra geeks just to get it running, It never really works that well, and spending that money of some clustering and a good perl guru would have a more drastic effect on your uptime. I personally don't have a Unix box that has not maintained 5 9's in availability over the last 4 years. I have a few NT servers that can boast the same.

      Products like these are only supposed to appeal to the shirts in a business. You know, the same guys who get freaky about "free" software.

      And to think MS once feared this guy....

      ~Hammy

  8. Opsware? by RoyBoy · · Score: 5, Interesting

    Ok seriously, does anyone RTFA anymore? How about the comments? This is a clear PR stunt aimed at producing more leads for Marc's new company. And ZDnet, that fine bastion of even-handed IT reporting, has once again saved us all by printing only the relevant facts. Just once I'd liek to believe that one of my old IT heroes didn't sell out and become a corporate whore (can you say RMS anyone?).

    --
    -- People who think they know it all, really annoy those of us who do!
    1. Re:Opsware? by AtariDatacenter · · Score: 3, Insightful

      Glad you mentioned it. Yes, this is just Mark pushing his own product (Datacenter management tools). In fact, he already has a few big customers. And this is the same direction Sun, for example, is going with N1. But it is going to have problems.

      You can't abstract away problems. And software is designed to be customized more than plug-in and exchange like lego blocks. Further, the 'leaky metaphor' problem hits these kind of systems big. (An example from a previous article, "UDP does not guarantee data delivery, and certainly not in the correct order, but TCP/IP does." So where is that guarantee if I cut a 1 foot break in the ethernet cable?)

      Software is going to have to get a lot more dumb for this sort of thing to work.

    2. Re:Opsware? by AtariDatacenter · · Score: 3, Interesting

      > Having a 20 GB HD on everybody's desk allows them to save their data locally intead of on network
      > shares where it is easier to back up. HD failures are much easier to abstract when their is a RAID
      > system running in the back room instead of a HD on every desk.

      Great. Now you've got to abstract the network reliability and performance problems. Abstract the lack of privacy, too, while you're at it.

      I think these 'magical solutions' try to avoid the appearance of trade-offs. They give the false impression that whatever is the method used is going to be perfect. And local storage vs remote storage is a good example. Assume a magical datacenter management tool gave me an option between the two. How would it abstract the weakness of one vs the weakness of the other?

  9. Nothing by halo8 · · Score: 5, Insightful

    what has to change in IT before we melt down ? Lots

    What WILL change? nothing

    for every burnt out admin thats going to quit theres 5 more waiting to take his place

    good companies keep good employees
    eather your not a good employee
    or your not working at a good company

    --
    The More Knowledge you have the Luckier you Get- J.R. Ewing
    1. Re:Nothing by Jerf · · Score: 3, Insightful

      for every burnt out admin thats going to quit theres 5 more waiting to take his place

      I've seen that comment several times but it finally occurred to me what's wrong with it: KM. (That's "Knowlege Management".)

      When an admin leaves, he takes a tremendous amount of knowlege away with him or her, which must be painstakingly re-aquired by the next admin. This can easily take in excess of a year, depending on how well the original admin did his job. (It takes less if everything's broken; well-oiled parts of the machine may not require attention for a long time, so those take longer to learn.)

      Changing admins is far, far from free, and business will eventually notice that. (In fact, they are starting to, but my impression is that KM is still a "kooky" field, not yet mainstream. Corrections from those closer to that community welcome.)

  10. It's not the work you do by Helpadingoatemybaby · · Score: 4, Interesting
    I have a lot of experience in this matter.

    I've worked in a few large outfits and in my own small business, and I can testify there is almost zero correlation in a large office between the work that you do and people's perception of what you do.

    The people who have the most problems, the ones who have a terrible catastrophe which just always seems to happen to them, are seen as the problem solvers. Despite the fact that their own lack of organization, incompetence, or laziness often brings these things upon them. No matter, they can proudly trumpet how they once again "saved the company" and worked 30 hours straight. The ones like me who prevent the problems, who organize their day so that nothing exciting happens if it can be avoided, and quietly solve problems on their own without assistance before people notice them, are seen as either invisible or lazy.

    And no, I don't work a 30 hour day. Ever. I don't need to. I'm not bitter... gak!

    --

    The baby's fine -- please stop sending business cards.

  11. Quit by Rai · · Score: 4, Insightful

    Well then how do you get credit for the work you do, when all that's noticed is the downtime?

    The very reason I left. When something crashes, who gets blamed? When users forget their passwords (which are usually something as simple like their friggin' username), who gets hasseled? When admin lays down an impossible time table with ridiculous performance expectations, who gets told "make this work or else?" When the company starts loosing money due to poor business decisions and/or the economy being in the toilet, who's the first dept to get cut?

    Not as appealing as those tech school commercials make it out to be, huh? :)

  12. If you are, so am I. by __aaahtg7394 · · Score: 5, Interesting

    I think a better point to critique on his phone analogy is the implied point that the phone system isn't held together with "bailing wire" or "chewing gum." Nope, it's all pretty standardized, well-integrated equipment. Why is the phone service so much more "professional" than IT services?

    Because phone service is a relatively well-defined, consistent, limited problem domain. Internet servers, dynamic web sites, and local security are loosely-defined, constantly shifting, open-ended problem domains. They're very different, and you can't compare one to the other.

    However, for certain applications, there are well-defined standards, and well-defined practices. Still, for a lot of IT, it's a matter of custom engineering and architecture. For example, online content management: you can buy one of the management engines off-the-shelf, which will probably do most of what you need in a structured manner. For CRM, well, there's about a dozen of those. These packages are well-behaved in that it provides a well-defined interface, but that's not always an option (i know, i used to do data migration for small- to medium-sized businesses. at the low end, when you change systems, you'd better damned well know perl or some other text processing language to massage the data--that is, you need to be good with your bailing wire).

    In the future, this situation will hopefully be better with standardization (mostly using XML it seems, even though the actual encoding doesn't really matter.. we could have standardized years ago, but nobody saw any benefit then). Having done data migration in the past, i'm all for keeping things disparate and non-standard, but that's because the work pays well and is fun ;^)

    A better analogy might be a pool of corporate autos. Except that you don't have to interconnect the cars to get them to share load dynamically, or access content generated on one to form a report on the other, etc. A lot of IT is like trying to drop a big old hemi into a metro, or getting a suburban to go anywhere with just metros to provide power (two in front and one in back, it might go up a hill!).

    Overall, I was not impressed with this article, but I'm afraid it's going to carry more clout than it should. oh well.

    1. Re:If you are, so am I. by wfrp01 · · Score: 4, Insightful

      Posolutely. I couldn't agree more.

      I'd even take this a step further. Let's just say, for the sake of argument, that we reach Marc's promised land of "it just works" hands off IT administration. Then what?

      Marc is missing the point. IT is not a necessary evil, it's a competitive advantage. I've sometimes told people who spend too much time attributing their problems to their computer that they should stop using it then. If you can do your job better without the computer, then by all means do.

      I have yet to see anyone take me up on this challenge. They know, we all know, that despite their occasionally infuriating peccadillos, that computers make us more productive.

      You cannot remain competitive by sitting on your hands. Marc's world of "it's all better so we can rest now" is pure fantasy. The problem with this article is that it portrays IT as a problem, rather than a solution and a competitive advantage. Who wouldn't like to compete against the company that decides it's achieved all essential IT objectives? You want to get your ass kicked, then stop trying to figure out better ways of doing things.

      --

      --Lawrence Lessig for Congress!
  13. Automate system administration by Soft · · Score: 3
    Marc Andreessen proposes automating systems administration, arguing that humans are already at their limits, so scaling up the network require that the machines take care of themselves as much as possible. I fully agree. Basically, he just rediscovered what the Infrastructures.org people have been saying: that a network should be thought of as a single infrastructure, not a number of individual machines.

    I recommend the paper, "Bootstrapping an infrastructure."

  14. This ignores two problems... by starseeker · · Score: 5, Insightful

    a) The phone industry had one task to do - get data over wires from point A to B, and however they did that was fine - users wouldn't have to be retrained if you replace old cables with fiber optic. That task is relatively simple, compared to the complexity of what is running and being distributed over the internet today. Automation works best when the target is static and clearly definable. I'm not sure either applies with servers/IT/internet.

    b) A significant amount of trouble with maintaining systems comes from having to figure out lots of different pieces of hardware. Lots of random equipment makes IT support a great deal more difficult. There are two solutions:

    1. Standardize all company hardware on a small number of systems/components, say one type of desktop, one type of server, and a few special purpose machines, and then only support those. Tools like VASystemImager then can make tasks like upgrading and bug fixing vastly simpler.

    2. Use inexpensive thin clients interfacing to some powerful central server, ala Largo, and only have to maintain that central machine and swap out cheap, dumb clients. Also simplifies things tremendously.

    People will no doubt point out that you have to run different types of OSs for different jobs and so on, but you can still use the central server/thin client approach and just make the connection to whichever OS you need transparent. It takes thought to set up, but once it is working you don't have 4000 individual support headaches to deal with. Only a few machines to upgrade, support, etc.

    Unfortunately, this won't happen. First, you would have to have a truly MASSIVE infastructure upgrade, which replaces a working system. Riiight. Second, you need to have management willing to try something new and be patient to wait for the long term results. That's not how they think - they think next quarter profits. There is also sheer mental inertia to contend with.

    It would be much easier for new companies to adopt this idea from the get go, than for older companies to adopt it. That may be where new, useful IT principles get applied.

    The only way current companies will do something is if the system BREAKS, and I mean just totally stops functioning. Thats when they will wake up to the fact that significant changes are needed.

    --
    "I object to doing things that computers can do." -- Olin Shivers, lispers.org
  15. This is a huge problem where I work by WCMI92 · · Score: 3, Informative

    My company does IT outsourcing for several local companies, inluding one that has several remote divisions.

    We are NEVER allowed enough time to do more than barely keep things running. Which always leads to things breaking that could easily have been prevented had someone been there to do routine maitenance.

    New deployments (like a Linux server that does remote access for terminal emulation) that I do are compromised by the fact that I'm never given more time than is needed to barely get it working. I never truly get to finish a job to my own high standards.

    My employer views this service contract as "nonbillable hours" despite the fact that they pay us $9K a month for it. The boss wants myself and our other engineers working on other "billable" projects that bring in far less than that $9K. That money doesn't get this company a SINGLE dedicated contract employee (despite the fact that our whole tech staff's monthly salaries don't equal $9K)

    Not only that, our company recently fired our most talented engineer without cause (the week before Thanksgiving) becuase he could hire a "paper tiger" (ie: MCSE) for $5K a year less, and a H1-B who he brought in and treats like a slave.

    It's definately the dark side of IT oursourcing, and something that companies considering doing this should think about.

    If I were going to outsource an IT department for a company with multiple locations and servers, I'd keep at least ONE in-house guy and use the outsource company soley for the "it's broken" crises which need more manpower.

    --
    Corporatism != Free Market
  16. Did he say anything? by IGnatius+T+Foobar · · Score: 5, Insightful

    Ok, I read through the article and came away with absolutely no information. He says some things we already know: data centers are expensive, IT people are overworked, and the rest of an organization only notices the technology folks when something breaks. So, what should we do about it?

    Well, here's where you expect an innovator like Andreesen to come up with a brilliant idea that's going to begin the next IT paradigm shift, but all he says is that we need to find some revolutionary way to automate our own stuff -- basically, to automate the act of automating things. And how? Well, he doesn't really know. He makes some vague reference to sending out automatic updates to hundreds of servers at a time, and that's it.

    Real bright there, Marc. Automatic patches and updates. As if that's the answer. In the real world, you don't have a huge farm of servers that all run the same patchlevel of the same operating system. I've got a few hundred boxen behind the glass, for example, that are a mix of Linux, Solaris, FreeBSD, Windows 2000, and Windows NT. And I'd guess that at least 50 percent of them would experience some sort of problem if we were to just push updates out to them unattended -- different applications require different patchlevels and break on others.

    Let's not forget the fact that there's more than just servers. There's infrastructure such as routers, firewalls, and switches. And of course there is the dreaded desktop, which is probably the source of 90+ percent of IT headaches. Until the IT world wakes up and gets the hell off local desktops, the maintenance nightmare will continue. Seen what Microsoft is doing lately? Their vision of the future is one in which applications are loaded through a browser and executed in a local .NET environment. It's basically the same as Java applets, but they call it "Smart Clients" to give you the impression that it's something they invented. Sounds a lot like Network Computing to me -- which simply means that Network Computing is a good idea after all! And now that Microsoft has "invented" it, the idiots who make up most of the world may finally start to adopt the idea. Make the desktop a stateless device like it was 20 years ago when we all had dumb terminals on our desks, and IT overhead will drop like a rock.

    The other trend you're going to start to see is outsourcing. People are realizing that it's expensive to build and run a data center. Fortunately, you don't have to. All you have to do is run your servers at a hosting center that knows how to do outsourced IT (as opposed to just hosting web sites, like the first generation of centers like Exodus did).

    There are ways of streamlining IT after all. Unfortunately, Marc Andreesen didn't touch on any of them. I give this article a "C minus."

    --
    Tired of FB/Google censorship? Visit UNCENSORED!
  17. Moderated Articles by md17 · · Score: 5, Interesting


    I am not typically supportive of /. bashing, but recently the number of quality articles has gone way down. To go one step further than just plain old bashing, I have a suggestion... Can we start moderating articles themselves so that I can browse articles at +3 on a normal day, +4 on a busy day and +5 on those insane days?

  18. Have you tried: by Hubert_Shrump · · Score: 5, Funny

    Going Postal?

    Yes, going Postal. It's hot, it's new, it's... Satisfying.

    "Cool and smooth, I don't know how I dealt with those bastards that kept asking 'this mouse?' without going Postal" -- Satisfied customer.

    And now you too can try it, for the low low price of a single aluminum hardball bat. Spun aluminum with a non-slip rubber grip means never having to say "For all that is good and holy it does not optimize your connection!"

    --
    Keep your packets off my GNU/Girlfriend!
  19. What a pointless article... by sterno · · Score: 5, Insightful

    Wow, that was insightful. So you mean IT is overworked and that the it'd be nice if IT could become a more automated process? Wow Marc, you've really stunned me with your insights!

    DUH! I mean c'mon, the reason why IT is such a mess is because all the IT staff are being expected to do more with less and do it faster. They are overworked fighting fires which means they aren't given the resources to do advanced planning and put together systems that would really get things right. He makes it sound like this is the responsibility of the IT people, but it's really the responsibility fo the business as a whole to have some foresight and help these IT people out.

    Overworked sysadmins do what has to be done to keep something working RIGHT NOW. Sysadmins with some free time will spend their efforts writing scripts, automating tedious tasks, and making sure fires don't happen. When fires do happen, they have the time to deal with them effectively because they've had a chance to automate a lot of the other tasks.

    I have known many a sysadmin and I have never met a one who wasn't constantly pissed off because he lacked resources. A friend of mine was sysadmin for a company that wanted to have 24 hour uptime for their systems. He was the ONLY sysadmin. That sort of crap happens constantly in the IT world because the other members of the company have the wrong atttitude about IT, that it's an expense. If they looked at it like they look at factories and buildings, as an infrastructure investment, then you'd probably see a lot more happy IT managers out there.

    --
    This sig has been temporarily disconnected or is no longer in service
  20. Without reading the article.. by defile · · Score: 3, Informative

    Marc Andreessen has an interesting article on what has to happen to IT next.

    I'm going to hazard a guess that this will forecast the overworked, underpaid endgame of IT gruntwork and usher in a new era where companies with CUSTOMIZED SERVICES and NEXT GENERATION TECHNOLOGY come in and automate IT and drastically reduce every admin's workload. He happens to run just a company that provides these services.

    I will now read the article and be amazed if that isn't exactly what he says.

  21. Why IT is not (yet) telecoms by melonman · · Score: 3, Interesting

    Telephony is a mature technology that doesn't completely change the way it works every five years. For 80 or so years the way the signals got routed didn't change much at all. Then exchanges went digital, and, for the transition period, it was all bailing wire like the article says, except that the telephone companies had - dare I say it - telephone number budgets to pay for the changeover.

    By comparison, the rate of change in IT is still very high. We've gone from mainframe to micro, from thin client through peer networking and back to thin client, from standalone to the Internet, we've done dial-up, ADSL, wireless...

    ... and one of the main reasons for all the bailing wire is that no company can afford to throw away all it's infrastructure every 24 months. If telephone systems stored data, and if handsets ran bespoke software, there would still be a few manual exchanges in use for backward compatibility.

    --
    Virtually serving coffee
  22. Management... by Chicane-UK · · Score: 4, Interesting

    One of the biggest problems as I see it is that management dont appreciate how important IT has become in their company. Looking at my company, I think they still relate the IT department to the same IT department of five years ago when to be fair the technology was a little easier to grasp and there were much less computers in offices.

    Take networking for example - it used to be in our place BNC and the occasional run of UTP cable - all attached to relatively unintelligent devices. Now its all Cisco switches, fiber and cat5e - and it really is a full time job in itself managing a large network with so many 'intelligent' devices.

    Also taking into account the addition of so many more servers (SQL, Mail, Finance stuff, Student Records, DNS, Proxy..) - the list is endless. Again, these systems have really bloomed in the past 4 or so years, at least for where I work.

    I guess they dont see how much goes on behind doors when it comes to this business..

    --
    "Hey! Unless this is a nude love-in, get the hell off my property!!"
  23. Sound design wins again.. by defile · · Score: 4, Insightful
    "...and you have to apply three patches to 100 servers before Close of Business..."

    No sweat.

    # for i in `cat servers`; do scp *.rpm $i:~/; done
    # for i in `cat servers`; do ssh $i rpm -Uvh ~/*.rpm; done
  24. Credibility by Alethes · · Score: 3, Insightful

    I'm really not trying to troll here, and even though he makes some interesting points, it's very difficult to trust the business opinions of a man that has one major dropped ball in his experience and is now trying to push his automation software as the Next Big Thing in IT. I don't honestly think he has any clue what the NBT is, but neither does anybody else.

  25. Useless Drivel by hemp · · Score: 3, Informative

    LoudCloud sold all of the data center hosting business to EDS. His company now develops software to 'manage more efficiently' those datacenters they couldn't make any money on. Obviously, the business model is 'if I couldn't do it and make $$$, then no other company in the world can either and they must be willing to pay me lots of $$$$ to allow them to'.

    --
    Skip ------ See the latest from http://www.anArchyFortWorth.com
  26. Comment removed by account_deleted · · Score: 3, Interesting

    Comment removed based on user account deletion

  27. Comment removed by account_deleted · · Score: 3, Informative

    Comment removed based on user account deletion

  28. Big words, little substance by mrobinso · · Score: 3, Informative

    Marc Andreessen sounds an awful lot like a lot of IT managers these days. You know, the people who say lots and know nothing. Boardrooms are filled with the notion that IT people are an automaton army that needs to be micromanaged right to the last char and nanosecond. Andreessen sounds like the propogator of this notion.

    In two paragraphs he pronounced to the world the basically all IT infrastructure and paradigms are broken. He later suggests that in order to keep up and succeed, things will have to change.

    Duh.

    The article is so short on details of the failure and possible solutions, I don't know why he wasted the space and bandwidth to deliver this most elegant piece of fluff.

    Fortune 2000 and enterprise in general has been raping its IT payroll for well over a year, probably close to two years. In that space, technology has changed, service delivery ramped up to top up the cuts-enhanced bottom line, media reinvented itself 4 or 5 times, and customers got a whole lot pickier and smarter.
    So the one piece of non-fluff in the article was the mention that a lot of data centers are being held together with spit and string. Well, this is what happens when you whip 5 people to do the work of 20. Seems like thats ok to do as long as the victoms have ballpoint pens in their shirt pockets and hornrimmed glasses perched on the noses.

    The biggest problem in the IT field these days is entrenched in the problems Tim Perdue experienced at SF. Every time an achievement is approached, 45000 know-it-alls with 6 digit incomes glom on, take credit, and micromanage at the DNA level. The suits, blissful in their ignorance and trusting of middlemanagement, believe the stuff that spews out of these ninnies mouths. The solution is for upper management and grassroots IT people, the folks in the trenches, to get together.

    Upper management, in order to be able to do this, needs to be sensitized to the machinations of IT people. They need to know what makes us tick, or they risk finding out what makes us ticked off. They best be doing this quick too. The downturn will end, business will pick up, and a lot of these companies will be up a creek with their infrastructure decay and miniscule overmanaged IT budgets.

    Revenge of the nerds indeed.

    --
    -- Karma whore? You betcha. --
  29. Re:@#$%! H-1B's by WCMI92 · · Score: 4, Insightful

    "Not only that, our company recently fired our most talented engineer without cause (the week before Thanksgiving) becuase he could hire a "paper tiger" (ie: MCSE) for $5K a year less, and a H1-B who he brought in and treats like a slave.

    "Don't get me started about this H-1B thing. You will get my britches in knots again."

    The "paper tiger" MCSE/CNE, I object to. Not the H1-B. He was hired 3 months before the boss let my friend go (who was my mentor, and largely respobsible for why I'm as good as I am). The H1B in question is bloody brilliant. One of the hardest working and smartest people I've ever met. He remembers anything I teach him after showing it to him ONCE... I can only WISH I were that smart.

    I object to H1-B more because it VICTIMIZES those who come here on that kind of visa more than I object to skilled immigration. The USA is better off letting people like him into this country, especially when it deprives an opressive country (Iran) of a brilliant mind. But scumbags like my employer shouldn't be allowed to bring him in on a H1-B "slave" visa...

    --
    Corporatism != Free Market
  30. The problem with the hacker ethos by jjohnson · · Score: 5, Insightful
    In the 90s, I saw endless screeds about how technology professionals were a different breed, and how the best thing you could do was get a hacker and leave him alone. Things like The Hacker FAQ fed the self-image of geeks as lone gunman who were above petty business concerns.

    This is what feeds Andreeson's IT crisis today: the fact that technology professionals took their apparent suffering and feelings of being not understood, and used it to isolate themselves. They refused to act like businesspeople in an organization that lives and dies by its profit and loss statement. They complain about how management doesn't appreciate them, but how many learn to do a cost analysis that will show the business reasons for buying software X or hardware Y? In other words, the glorification of the geek in the 90s gave a lot of geeks the idea that they didn't have to learn the language of business to survive there. That's why they're underfunded, underappreciated, and harassed.

    I've had the benefit of a boss who demanded a business analysis for any significant technology initiative at the company. He doesn't get computers, but he understands ROI. He understands a well-presented business case for anti-virus software. We have a wireless network in our new facility in Texas for a real-time inventory management system for one reason: my cost analysis showed that the implementation costs would be recovered within a year because of labor saved from eliminating batch-mode downloading, and that the cost over five years of our wireless system was ~15% of the batch-mode system.

    When geeks figure out that they have to speak the language of the business, then the IT department gets properly funded, properly respected, and properly treated.

    --
    Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
    1. Re:The problem with the hacker ethos by sawilson · · Score: 3, Informative

      There is a problem with this mentality. It's kinda
      once sided.

      "If these damn smelly hippy UNIX admins could learn
      to act more like good corporate lapdogs, we'd make
      a bundle"

      Not to say there isn't some validity in there. But
      the other side of that coin is:

      "If these retarded, non-tech savvy idiots gave me
      a boss that understands technology, didn't let
      the morons in marketing make promises they didn't
      check if we could keep. If HR didn't have so much
      power they effectively ruin the company..." and
      so on adnauseum.

      The real problem stems from the "us" versus
      "them" philosophy. "Gotta have a strong manager
      watch over them there technogeeks or they'll walk
      all over us". And the "You don't need a technical
      person to manage technical people" mentality also.
      It has to do with the level of fear a
      non-technical person has about technical people.
      Because technical people "know things". And they
      are probably "trying to trick you". So the ball
      starts in their court. Most of the time upper
      managment fumbles that ball all the hell, and
      deserves the treatment they get from their
      overworked, underpaid wage slaves that they don't
      listen too or appreciate. That's just my opinion
      after 14 years in the business from both sides
      of the fence.

  31. Nagios by Dunkirk · · Score: 4, Interesting

    If you haven't checked into nagios yet, you owe it to yourself to do so. Now. It's a monitoring application that can take action on problems. That's the first step to automating things in the datacenter. It's open-source, and it's highly useful, if a little tricky to get working.

    I moved out of a group running a lot of big Sun machines (I set up an E10K for them) because of managerial issues. Before I left, we had a budget item for about $250,000 to set up a monitoring and job-scheduling application. It was going to take *another* Sun box to run, and we were being told that it would take 3 months to get it all set up and configured.

    With Nagios, I can do everything we they were talking about implementing. I spent 3 weeks, and it cost me nothing. I employed a dual PII 266 that was collecting dust. (I also used an old P166 as a dedicated kiosk for showing the web page.) My boss and my co-workers think it's great. I'm dying to show it to my old group...

    --
    Acts 17:28, "For in Him we live, and move, and have our being."
  32. machine? by BreakWindows · · Score: 3, Funny

    your server farm is a well oiled machine

    Pretty weak-ass server farm, if you ask me..

  33. You forgot support costs.. by mindstrm · · Score: 3, Insightful

    It's not the up-front cost of the machine, it's the ongoing support costs. Thin client has a much longer working life, and takes much less time to support.

  34. You Americans need to fuck people back. by CrystalFalcon · · Score: 3, Insightful

    If one of my co-workers had been fired for that reason, I'd have quit on the spot. And so would most of my colleagues. Why? Because of loss of faith in the leadership (boss not understanding morale concept, and not knowing who'd be next, leads me to prefer to dictate my own future).

    The boss would be left with an empty department to explain to his VP and justify his bad decisions. Not that it would happen - I have the total faith in my boss and the morale is good, people help each other out.

    You Americans need to learn to fuck people back and not take shit from nobody. The SINGLE reason managers (in IT and elsewhere) can treat people like this, is that the people being screwed react with bending over and asking for more.

  35. Automation nation by wytcld · · Score: 4, Interesting

    Back in the early 80s (when Marc was in what grade?) the business press ran articles about how software was getting so good that soon we wouldn't need programmers, because writing software would itself be handled by software.

    I predict that we'll have software that can write major software at just the same time as we have software that can write convincing novels. In both cases you have the task of putting together language that respresents a broad swath of messy reality.

    Now, systems administration may be more like writing a good technical manual than writing a good novel. Ever notice how many good tech handbooks there are out there? You haven't? Maybe it's because novels are easier. Good systems administration is about leveraging people strengths with machine strengths, and vice versa. Automation without the human element is as uncompetitive as, well, the human element without automation these days.

    --
    "with their freedom lost all virtue lose" - Milton
    1. Re:Automation nation by sql*kitten · · Score: 3, Insightful

      I predict that we'll have software that can write major software at just the same time as we have software that can write convincing novels. In both cases you have the task of putting together language that respresents a broad swath of messy reality.

      Actually, we do already have software that can write software, it's called CASE, Comupter Aided Software Engineering.

      Unfortunately, it's takes about as much effort to explain to CASE what you want your application to do as it would to write the application yourself anyway!

  36. Automation isnt possible today. by miffo.swe · · Score: 3, Interesting

    Why?

    Well because the software we use today doesnt lend itself volontarily to extencive automation. You can automate patch installations, users added to all relevant systems at one click of a button, backup and all such things. The problem that some people seems to have a hard time grasping is that software sucks mostly. It is ridden with faults that make any automation fail randomly no matter how well it is implemented. Thats where most IT staff is doing their job, straighten out faults in the software and installing it. Support is also very hard to automate.

    Before any automation can be used on a daily basis software must get much better and have much less bugs at shipping date than today. Its a very wrong approach to go backwards and automate fixing of faults related to bugs. Fix the bugs instead.

    --
    HTTP/1.1 400
  37. Re: laptop users left out? by King_TJ · · Score: 3, Informative

    The way I usually see a Citrix deployment handled, the laptops are all configured with modem dial-in access to the Citrix server (via VPN). They also have a full installation of the most-used applications (MS Office, for example?) that they'd want to work with "offline".

    Then, it's just a matter of training for people to understand that they need to copy their completed work up to their shared disk space on the server when they get a chance. If you keep the versions of software on the server and on the laptops the same, all should go pretty smoothly.

    (Of course, this also undermines some of the supposed cost-benefit of Citrix, because you're still buying full software licenses for each laptop, plus all your user licenses for everyone potentially connecting to the Citrix servers.)

  38. Stop trying to sell us something Marc by gelfling · · Score: 3, Interesting

    Your Loudcloud was supposed to do this for us and it failed.

    Why did it fail?

    1 No one really knows how to do it
    2 The infrastructure is too expensive
    3 Customer requirements are too dissimilar from one another
    4 No one has the balls to tell customers their requirements are crazy and impossible
    5 Transition costs are poorly understood
    6 Exeutives are measured by overhead and customer satisfaction and not doing the right job the best way
    7 People are not a resource they are an overhead item

  39. supporting open source: radmind by clunis · · Score: 4, Informative

    My team runs just under 6 dozen web and database servers ( Solaris and Linux ) for the University of Michigan using an open source system management suite called 'radmind' and I can't say enough good things about it ( I'm not one of its developers, so I can get away with this ): fast, secure, stable, standards based, and makes a little thing like patching several dozen servers a breeze ( though ... what kind of freak patches in the middle of the day? ).

    Incidentally, the CTO of loudcloud ( a.k.a. opsware ) is Tim Howes, of LDAP fame and formerly of the UMich RSUG ( the same group that has since developed radmind ). small world.

  40. The best sys admin by ellem · · Score: 5, Funny

    is the one you never see. That idiot running desk to desk doesn't have any idea what they are doing. The one that shows up at 8 and leaves at 6 and appears to be reading screen after screen of pr0n. That SA is the bomb.

    Worship the SA, do not replace the SA with a small shell script.

    --
    This .sig is fake but accurate.
  41. Sounds like you've picked some cherry companies by jobugeek · · Score: 5, Insightful
    Aparently you have done a pretty good job, finding companies with fat budgets and no politics.

    Seriously, I don't get the money to buy redundent Cisco routers. I don't get to buy extra switches and motherboards.

    And the little internal web addy, that has been in place for 2 years? Some tool of a manager who is having a hissy fit, just walked into my office and asked why I haven't fixed his problem that he never told anyone about.

    Please......

    Whenever you are dealing with people who have almost as many deadlines as you do are involved. Things are never that easy.

    Now where I will agree with you are scripts. Learn them. Use them. Love them. But do you have an IDS system? Who verfiys those alerts? Who checks your server logs for descrepencies? Admining is rarely easy.

    --
    I'm not drunk, I just have a speech impediment. And a stomach virus. And an inner ear infection.
  42. There's something in what he says by scottme · · Score: 3, Interesting

    It all sounds a lot like what IBM's Sam Palmisano was preaching back in October, about "eBusiness on demand".

    The idea that impressed me then was the thought that nobody would seriously consider generating their own electricity now that it's a utility. But back in the early days companies and communities did just that. Same thing today with computing, but tomorrow...

    It strikes me there's a shade more to IBM's vision than there is to Andreessen's, though. Check out the IBM version here, with links to some more in-depth material.

  43. Yeah, but I always WAS busy! by The+Spoonman · · Score: 4, Insightful

    I know exactly what he's writing about. I was just fired from my job of two and a half years. The reason? Check my sig. That's right, we started a personal website in our own time, with our own reasources, and because the CEO didn't like it, we were out. "Decreasing company morale" was the reason we were given. They fired me, who was the Sysadmin, the webmaster, and the only PC technician.

    Forget the fact that for the last two and a half years, I haven't had a real vacation, because I got a call EVERY FREAKIN' DAY about some minor issue.

    Forget the fact that I worked a minimum of 2-3 hours every night on company stuff, 'cause I wasn't allowed to make ANY changes during the day. (The night before I was fired, I spent 3 hours writing a script to fix a problem that was affecting only 1% of the users. No real problem, but I didn't want them to have to deal with ANY issues.)

    Forget all of that, just get the fuck out. I take some solace in the fact that two days after they fired us they sent out a global e-mail of "Please bear with us, it'll take up to five days for your calls to be answered." And, e-mail was down for almost a week because no one but us knew how it was setup!

    IT needs to get the respect it deserves. In this era of decreasing budgets, the only way companies will be able to make any money is to increase efficiency, and that means automation.

    --
    Which is more painful? Going to work or gouging your eye out with a spoon? Find out!
    http://www.workorspoon.com
  44. I'm busy... by phorm · · Score: 4, Insightful

    Patching servers, fixing machines, crawling under desks. If you want a job with a fair bit of action, work at a school. If there's anyone who can find a way to screw something up, or do something unexpected, it's a 15-yr-old with a keyboard and mouse. That, or it's my 57-yr-old grandparents that somehow magically manage to screw things up that should be unscrewupable.

    Combine that with archaic computers, underfunding, etc etc... and it's an exciting job.

    1. Re:I'm busy... by buffy · · Score: 3, Insightful

      Allow me to add to that...work at a university. The "abilities" of tenured professors and the sort are fairly impressive.

      Of course, by abilities, I'm referring to them in their destructive and or disruptive potential. :(

  45. Where to start by jjohnson · · Score: 3, Insightful
    • Ask your non-geek boss how to make a business presentation on the business merits of a technology initiative. Ask her about cost analysis. Ask her about justifying new software to her bosses (what they're looking for).
    • Pick up a textbook on managerial accounting (which is different than what accoutants do): it'll go into great detail on how businesses analyse costs and plan with them.
    • Look at your company's profit and loss statement and figure it out. I started at the company as a cost analyst, and now I know exactly how the business is financially structured.
    • Try doing it yourself: start a spreadsheet. List all the costs of something, say, upgrading to the latest version of Office. Include every cost you can think of: retraining, troubleshooting, labor spent installing, licence fees, costs to convert all the documents on the server. Don't just list dollar figures, list your calculation: 30 hours at $20.00/hr (but don't forget to include the cost of benefits and other expenses: an employee paid $10/hr costs the business $12-14/hr). Then list all the monetary benefits: time saved with zippity feature X, etc. Compare the bottom line: will the business benefit financially from the upgrade?
    At it's heart, understanding the internals of a business means understanding it's programming language: finance. You don't understand how a business works until you understand the flow of money through it. Figure that out, and you'll be a respected geek who can talk to managers on their own terms.
    --
    Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
  46. Details, details, details....... by zerofoo · · Score: 3, Insightful

    And IT must start taking steps forward, if it wants to be an enabler of growth

    Mr. Andreessen,

    I must applaud the points you've made in your article: "Welcome to the new IT crisis". I agree completely with everything you've said.

    I'm tired of working long hours, hand-holding my users with software they should already know, and applying countless service packs, patches, and firmware upgrades. My staff is over-worked and under paid...but we are happy to have jobs in this economy. I loved the article, but it left me with one question....

    HOW THE HELL DO WE DO THIS?

    -ted

  47. Who is Marc Andreessen? by llywrch · · Score: 3, Insightful

    Isn't this the same Marc Andreessen who once responded to a criticism of all of the new bugs in Netscape with ``Hey, we don't have the time to do it right"?

    And if you use software that is written to ``do it right", won't uptimes of 5 nines happen by itself?

    I'm just one guy who spent a year trying to get Andreesen's software to work on hundreds of computers. I really don't have any opinion about this multi-millionaire.

    Geoff

    --
    I think I see a trend here. Maybe for them it really would be easier to muzzle the entire internet than to produce p
  48. Ah yes, automation. Nice plan. by stinky+wizzleteats · · Score: 3, Insightful

    There's just a few gotchas. These revolve around consistency.

    Are your servers consistent? Are your OS's consistent? Are your applications deployed the same way in each of your remote offices? Do your users have the same applications on their computers?

    Or is your IT infrastructure a giant ad hoc hairball resulting from IT decisions being made by non-IT personnel? Do you use anything because "that's what everyone else uses, so it must be good enough, therefore opinions to the contrary are wrong"?

    Have you implemented things like network management and application/workstation management only to find that the investment is worthless because your organization cannot adhere to a set of standards long enough to make such systems effective, resulting in such an expanse of policy variance that the "management" systems themselves become a net drain on your management resources?

    If so, then you've come to realize that the IT problem does not have a technical solution, rendering the entire premise of the article false.

    The problem is cultural. IT people are not trusted to make IT decisions, such as meaningful policies with regard to how technology is implemented and what resources are required to deliver a given level of service.

    What all of this means is that your Great Answer may not come from the vendor/consultant of the month, but by simply asking your people what is going on, listening to them, and giving them the leeway to make things right. And yes, this was written by a consultant.