Pleasing Google's Tech-Savvy Staff

An anonymous reader writes "Douglas Merrill, Google Inc.'s chief information officer, is charged with answering that question. His job is to give Google workers the technology they need, and to keep them safe — without imposing too many restrictions on how they do their job. So the 37-year-old has taken an unorthodox approach. Unlike many IT departments that try to control the technology their workers use, Mr. Merrill's group lets Google employees download software on their own, choose between several types of computers and operating systems, and use internal software built by the company's engineers. Lately, he has also spent time evangelizing to outside clients about Google's own enterprise-software products — such as Google Apps, an enterprise version of Google's Web-based services including e-mail, word processing and a calendar."

Comments

All Credit to Him

[Avohir]

I've had to do IT work for tech companies before, and it's like being the caterer at a chef's convention, they always think they could do it better. That he's managed to do it with a relative degree of success at a place as eclectic and high profile as google is impressive. I think the approach is novel too, although I'm not sure how well it would apply outside of their unique company culture.

Re:All Credit to Him

[zappepcs]

It always applies to other companies. The thought process it takes to create software services is what I believe should be the approach to network services. If each little group of employees is walled off the basic network, and their access outside that playpen restricted to what they need, any major error inside the playpen is less likely to corrupt the whole network. Much like a city's services are configured. Everyone needs water, electric, sewage, trash service, roads etc. If you trip the breaker in your office, the next office building is unaffected just as they are normally unaffected if your toilet overflows. In that way each can do pretty much whatever they like and all remain unharmed. I'm not saying that your hobby of cultivating anthrax is going to fly for very long, but short of that... well, you can (more or less) grow what you want in your window-box garden. You can walk down the street to the park, just not through everyone's backyards.

The idea is not to restrict people, but restrict damaging elements from hopping around your network.

Re:All Credit to Him

[Kelbear]

I think the kinds of people Google hires are less likely to run executables and install toolbars from seedy and irreputable niches of the internet. Other companies probably can't assume the same of their employees.

Even smart people can make errors of ignorance or naivetè with regards to their computers. It's nice that they've cordoned off the system to prevent them from torpedoing the whole network at once, but you still have a mess on the other side of the wall to clean up. Most of the important stuff is probably saved where they're regularly backed up(Google sure as hell isn't going to have problems with storage space) but there's definitely going to be downtime involved.

It's probably not worth the cost and risk for most companies. If someone wants or needs something on their system, just having them ask first is a reasonable approach.

Re:All Credit to Him

[TheLink]

That's fine if the walls are 100%.

If you allow some employees access through those walls to other networks, and a hacker manages to get their credentials it can start to get quite nasty.

Even if the isolation between networks is good there's also the possibility of _work_ being secretly tampered with. I'm sure there are hacker who would want to tamper with GMail or Google Desktop.

Or confidential information leaking out.

Re:All Credit to Him

[Anonymous Coward]

Maybe the chefs feel that way because it is true that the chefs who can make the food from scratch can do a better job than the caterers who simply reheat and serve it.

Re:All Credit to Him

[hummassa]

I've had to do IT work for tech companies before, and it's like being the caterer at a chef's convention, they always think they could do it better. That he's managed to do it with a relative degree of success at a place as eclectic and high profile as google is impressive. I think the approach is novel too, although I'm not sure how well it would apply outside of their unique company culture. The fact is: if you are the caterer at a chef's convention, probably (1) 80% of them would do it better than you and (2) the remaining 20% wouldn't, but they do think they would.
So, all credit to him for making them cook their own meals, which was more intelligent anyway and less reputation-damaging.

Re:All Credit to Him

[Anonymous Coward]

I think the kinds of people Google hires are less likely to run executables and install toolbars from seedy and irreputable niches of the internet. Other companies probably can't assume the same of their employees.

Exactly. IT security at most companies is designed around the belief that the average clueless user will find a way to screw something up if given too much freedom. So we lock them down in order to minimize the damage that they can do.

That's less of a problem with more technically inclined users. At my organization, we keep most of our users locked down but give our development group freedom similar to what is described in the article. They're a competent lot, fairly trustworthy and they're right across the hall. So we let them do whatever they want on their workstations, within reasonable limits.

Re:All Credit to Him

[nschubach]

Considering myself a technically inclined user (being a senior developer) I lock my machine down myself. I know it sounds backwards, but I don't want rogue applications running on my machine when I'm testing. Not even the ones used by my company to keep the system "inventoried."

Re:All Credit to Him

[element-o.p.]

Exactly. That was my first thought when reading this article. It's fairly safe to assume that the employees at google are tech-savvy and motivated. It is *not* safe to assume that the customer service representatives, accountants and other non-IT workers at most other companies are equally knowledgeable about what is and what is not a good idea on company computers.

For that matter, even IT workers can be pretty adept at shooting themselves in the foot. At a place I used to work, one IT staff member was having problems with his computer in our sandbox network. Another IT staff member who was helping him troubleshoot the computer suggested they bypass the router and switch for our sandbox, and plugged the problem computer directly into a core network switch. Unfortunately, the problem with the computer was it had been infected with a virus...which then spread to (and hosed) most of the corporate network, rather than being restricted to our sandbox. Oops...

Re:All Credit to Him

[Instine]

However, its not the impression I got when I was there. In and interview i was told that "we don't use IDEs here" and "we use EMACS or bla...". My fondness for VS was definitely frowned upon!

Re:All Credit to Him

[nschubach]

I guess that would depend on how separate your production data is from your development environment. If you give your devs full access to production, you're asking for leaks and trouble. In your scenario, with a proper separation of development and production, you shouldn't really have a rogue hacker be able to get to sensitive data on your protected production network. Well, that and you wouldn't have a developer push something to production without proper testing causing Google.com to go down... I'm assuming they have a proper separation, and most likely a testing environment that mimics production.

Re:All Credit to Him

[Anonymous Coward]

Which is exactly my point. We trust our devs to be security conscious without us forcing it on them to the extent that we do our regular users. I didn't mean to imply that they might be running wild with scissors. So far, this policy has worked pretty well for us.

Re:All Credit to Him

[rtb61]

Really all google are doing is getting their tech staff, to research, trial and test applications in their own time and at their own expense. When it works they do limited deployments in the company, interesting certainly, a tad exploitative definitely.

At the end of the day, it is all about a psychologists endeavouring to manipulate the greatest possible productivity out of the work force until they burn out. Google is a marketing company through and through, hence they use every marketing tactic available to recruit and manipulate the tech staff.

Re:All Credit to Him

[bit01]

It's probably not worth the cost and risk for most companies. If someone wants or needs something on their system, just having them ask first is a reasonable approach.

No it isn't. You've just created a catch-22. How the hell is a user able to know whether an application is useful to them without installing and testing it?

I've worked in far too many places where people didn't install what would have been useful and productive software because it was just not worth the hassle. That by itself is an indication that system admin regards their control as more important than the needs of the company and the individual users. An admin does not know what is useful to each user and the company and they are arrogant to automatically assume they do.

The correct approach is to provide a sandpit so that users can test what they need in a safe environment, or even allowing controlled installs in the regular environment, make sure they know what they are responsible for (e.g. keeping licenses legal and not wasting anybody's time) and provide simple tools to rollback as needed. If it turns out to be useful and worthwhile then allowing limited hassle while installing it in the regular environment to limit impact on other users may be justified.

I've administered this way on 1000+ station LAN's and it works very well. Work smart, not hard. Not only did the users like the environment a lot (I was perpetually getting positive feedback) I learned from my users a lot about the huge variety of software out there and business processes as well.

More succinctly; helping users to help themselves is a far more productive use of an admin's time than treating the average user as the enemy.

---

DRM. You don't control it means you don't own it.

Re:All Credit to Him

[rmerry72]

That's less of a problem with more technically inclined users. At my organization, we keep most of our users locked down but give our development group freedom similar to what is described in the article. They're a competent lot, fairly trustworthy and they're right across the hall. So we let them do whatever they want on their workstations, within reasonable limits.

Oh, that's nice of you. You LET us developers do our job on equipment provided to us for that purpose. Thanx.

I'm being argumentative I realise. But your chosen phrases belie the general infrastructure attitude I've come to know and dread of the infrastructure / desktop support guys being here to protect everyone - even developers - from themselves for the benefit of the company. I'm sick of being "allowed" to install some port analysis or debugging tool or other.

Now some developers are clueless sure (well, have you seen the market - only idiot, time watching, anagram laden, script monkeys are getting hired by these moronic IT managers) but its our job and our responsibility to develop software and to protect and use our tools as we see fit. We know our tools, our preferred environment better than you (mostly) and I'd hoped that eventually the default position would be - "Hey, they are developers, they probably know their shit. Let 'me install version 1.2 if they like." instead of - "Watch 'em carefully. They are obviously two-bit cowboys and there ignorance of the default packet length of TCP/IP will bring us all down. No, 1.0.3 is the approved version, and we'll upgrade you when 1.0.4 has been tested with Bob from accounting. "

After all, it was probably a bunch of us developers that wrote all the software tools you use to lock us out.

Re:All Credit to Him

[TheLink]

The dangerous hackers aren't those that cause things to go down. So how are you going to notice those? It's hard for a hacker to bring the whole of google.com down, especially since it's "sharded" - you could just have different teams in charge of keeping their respective shards up and give bonuses based on uptime (factoring out externalities beyond their control) - then regularly adopt the best practices and ideas from the top groups.

Good luck with the IDS/IPS when your employees also use encryption (ssh, https etc), leave jobs running overnight, remotely come in to do work after hours etc.

I personally think that Google really aren't doing anything that wonderful with respect to security. They just haven't been unlucky so far (or maybe they have been unlucky, and just nobody knows about it yet ;) ).

You see lots of people who obviously aren't good drivers, but they manage to avoid crashes _most_ of the time.

I've seen companies with pretty crap security (I used to be in the IT security line), but in most cases no hackers brought anything down there either.

Re:All Credit to Him

[Daniel Phillips]

the problem with the computer was it had been infected with a virus...which then spread to (and hosed) most of the corporate network, rather than being restricted to our sandbox. Oops... So you got rid of all those dangerous Windows boxes and installed virus-proof Linux? Just asking.

Re:All Credit to Him

[element-o.p.]

Don't I wish!

The problem at that job was that we were sys admins in the ISP department. The company had a separate internal IT department to manage employee desktops, except for ours. We were free to install what we wanted, as long as it was properly licensed. The two sys admins who infected the network were Windows guys; most of the rest of us (including me) used Linux desktops. Interestingly enough, the laptop that caused the problem was issued by the internal IT department, a fact that I found greatly amusing.

Re:All Credit to Him

[Anonymous Coward]

I've had to do IT work for tech companies before, and it's like being the caterer at a chef's convention=, they always think they could do it better

Congratulations on being the one first one to use a good metaphor correctly in the history of Slashdot (well, since 1999, when I started reading anyway). I'm going to immortalize you in a 1000 word blog entry, but until I do, let me take just point out this fact to everyone. Remember this sentence, people!

Nice approach

[the computer guy nex]

Unfortunately it will take only one mistake by one employee to ruin it for everyone.

Re:Nice approach

[SanityInAnarchy]

I'm not really sure how that works.

Other than leaking source code onto the Internet, I don't really see what problems this could cause. I work at a small company with a similar philosophy -- the company buys your hardware, and certain software if you need it, but you can use whatever you want so long as you're not fighting with it on the clock.

But think about it: Spam botnets can be blocked by killing port 25 outbound. Data loss can be managed by the fact that everything's on version control, which is backed up. Traditional spyware and viruses will at worst take a machine down, at which point, it's the responsibility of whoever owns that machine to fix it -- or maybe they try to spread over the local network, at which point, staying patched and/or running a personal firewall will pretty much stop it.

The only real danger would be if we got big enough to be a target for deliberate attacks, and someone stole our source code. Google is arguably this big, but I've never heard of a leak from them. TFA does mention a possible strategy:

We have antivirus and antispyware running on people's machines, but we also have those things on our mail server. We have programs in our infrastructure to watch for strange behavior. This means I don't have to worry about the endpoint as much.

So what mistake could one employee make to ruin it for everyone?

Re:Nice approach

[the computer guy nex]

So what mistake could one employee make to ruin it for everyone?

Your logic is faulty.

Traditional spyware and viruses will at worst take a machine down

Google is not targetted by 'traditional' viruses/spyware. The first hacker to take down their network, either internal or external facing, would be infamous.

Re:Nice approach

[Brian Gordon]

It will take 1 mistake to ruin that one computer he's working with, but Google can well afford to just buy another if it keeps its engineers happy.

Re:Nice approach

[orclevegam]

So what mistake could one employee make to ruin it for everyone?

Installing your entire warez collection on your work computer. Sure you'd get fired when you finally get caught, but if the BSA raids the company before you're found out it could be major fines the company is responsible for. Yes they could go after you in court for it to pass on the cost, but that's even more overhead dealing with the legal system. Even barring that, there's lots of ways to misplace license keys, and the BSA won't cut you any slack unless you've got damned good records.

Re:Nice approach

[Anonymous Coward]

sheesh ... people make mistakes ... get over it!

Truth is if you can't handle mistake, you can't manage.

Re:Nice approach

[pongo000]

Unfortunately it will take only one mistake by one employee to ruin it for everyone.

Only in an organization run by an IT staff that doesn't have a clue. In any other company, said employee would simply be put on a very short leash, or shown the door.

Re:Nice approach

[Anonymous Coward]

It'll take too long to replace anyway, so they'll just turn it off and set up a new one next to it.

Re:Nice approach

[TheLink]

"So what mistake could one employee make to ruin it for everyone"

Get pwn3d and:
a) Commit GMail/etc code secretly backdoored by a hacker.
b) Leak out the search ranking and antisearch spam methods/algorithm google uses. Google's search results are already not as good as they were years ago.

Re:Nice approach

[Anonymous Coward]

I thought that their algorithms were designed to maximize spam results...if not, they were never very good.

Re:Nice approach

[bishiraver]

a) I'm fairly certain google employees would review each others code before commits. TFA mentions they have automated scripts that check security of code.
b) I got nothin', though I'm willing to bet the search algorithm is one of those things that not many people get to see/tinker with.

Re:Nice approach

[jd142]

Well, let's say that an employee downloads a piece of software with a license agreement that allows the software manufacturer to monitor all the data the users produces, what websites the user visits, and gives the software manufacturer the right to keep that information in perpetuity. By installing the software on google computers as an employee of google, google is now bound by that license. So sensitive company information ends up being stored on the software manufacturer's computers in perpetuity. And if the license gives the software manufacturer the right to read the information you've got a really nasty can of worms.

Or how about an employee who downloads a piece of software that is only to be installed on the employee's personal computer. The employee installs it on a work computer, thinking that it is the employee's computer and is only using it for personal use. That's wrong and suddenly Google gets audited and sued for illegal software usage.

Or even better, the software manufacturer makes the legal venue the laws of Lichtensteinavania, where the user has no rights at all.

I know, I know, the slashdot response is switch to gpl, but that isn't always an option.

I've actually run into all of these software licensing issues at my job.

Re:Nice approach

[argiedot]

I would think that such licence agreements wouldn't stand up in court, as in there are some things you just can't agree to[1]. I am not a lawyer and have no knowledge of law whatsoever, of course, but I would think that this would be one such case.

[1]You can't accept a licence that makes you a slave, I think, or that says that you can be killed, of which I'm sure.

Re:Nice approach

[somersault]

The first hacker to take down their network, either internal or external facing, would be infamous. He'd also be killed in less than 24 hours by an army of angry geeks who want their porn back

Re:Nice approach

[jd142]

Just out of curiosity, which of the 3 examples were you thinking wouldn't hold up in court?

The first example, about monitoring all communications and tracking is pretty close to Google's own licenses.

The second example is close to one we ran into where the license said for non-commercial use only. The software's writer said he meant that to be interpreted as a personal computer at home, not a registered non-profit entity. We probably would have won if it had ever actually been adjudicated, but we just found other software to do the job.

There are lot's of contracts and licenses that bind people to a jurisdiction. Makes it easier for the big corporation if there's a problem. Credit cards do this all the time, really any time a company does business across state lines the contract outlines which state laws will control the adjudication of disputes. Dealing with international companies just means your dealing with country laws instead of state laws.

The only things you can't contract for are things that are illegal, like the murder example. You could make a valid legal contract that said you agreed to paint a person's house for $1, slave wages. You'd be stupid, but the contract would probably be ruled valid.

Re:Nice approach

[forgotten_my_nick]

I think it has less to do with a hacker and more to do with litigation. IBM for example is extremely anal about what developers are allowed access when creating applications and have to account for everything they do. Because when your a large multinational with lots of money people will try to get it from you.

Re:Nice approach

[argiedot]

...monitor all the data the users produces, what websites the user visits, and gives the software manufacturer the right to keep that information in perpetuity...

It just sounded like a horrible thing in entirety, and I'm sure some company could stick it in their EULA and get away with it. Would that make it legal, because that sounds insane. All sorts of users just 'know' that they should check the I Agree box [1]. But then, what if someone makes software so awesome that you would actually pay that price to use it. So frankly, I don't know. And as for what I think wouldn't stand in court, I do not actually know the law, just remarking that such a thing is dangerous enough to possibly be disallowed.

I'm pretty sure that the $1 wage thing doesn't work though, wouldn't minimum wage law apply? Anyway, interesting stories, thanks for sharing. Funny that about non-commercial, I suppose one should make sure the terms mean what you think they mean when you write out a licence :)

[1]I'm pretty much an idiot too, I use GMail, but I haven't read the privacy policy through. I reckon I wouldn't understand all the implications even if I had.

Re:Nice approach

[element-o.p.]

Data loss can be managed by the fact that everything's on version control, which is backed up. Traditional spyware and viruses will at worst take a machine down, at which point, it's the responsibility of whoever owns that machine to fix it -- or maybe they try to spread over the local network, at which point, staying patched and/or running a personal firewall will pretty much stop it.
That's a great theory, but more often than not, that *isn't* the way things really work. I've seen sys admins really bork config files that were using RCS. I've seen a virus take a network down for two days despite updated and running A/V and firewalls. Anyone who has worked in IT for very long is forced to admit that you can make it really, really difficult for your users to shoot themselves in the foot, but nothing you can do can guarantee security. The best firewall, the best anti-virus and the best revision/version control will give you some measure of protection, but it won't be 100% effective. Ever.

Re:Nice approach

[Anonymous Coward]

By installing the software on google computers as an employee of google, google is now bound by that license.

Just one of the many retarded things about EULAs. Your average worker doesn't have the legal authority to engage into a contract which would allow these things, no matter how many times he presses "I agree".

The software company wouldn't stand a chance in enforcing this. However, you've still got spyware on your machine.

Re:Nice approach

[TheLink]

1) The human process could require review before commits, but is there anything preventing a person from making extra commits after a review? It might _eventually_ get spotted, but by that time it might have been exploited already.
2) Automated scripts that check security of code. I think those only work for _nonmalicious_ coders - e.g. they detect common "oops" conditions. I doubt you could have automated scripts that can detect malicious backdooring of code. Unless Google has Skynet online already ;).
3) Of the few who know the search ranking stuff, what are their controls? Are they allowed to discuss it amongst themselves on Gmail?

Google's track record on security isn't very good. It's not something they are good at.

Thing is though most companies aren't good at security, most still don't get pwn3d regularly, even the big juicy targets.

Re:Nice approach

[ampathee]

Have software clickwrap licences even been tested in court yet?

Are you *sure* google would be bound to an agreement clicked-through by one of its employees? Sounds unlikely to me, but of course IANAL.

Re:Nice approach

[SanityInAnarchy]

Your logic is faulty.

Show me how.

The first hacker to take down their network, either internal or external facing, would be infamous.

And traditional viruses/spyware won't do that.

The trouble is, modern OSes are reasonably secure at this point, and you can bet the external-facing IPs are going to be locked down. Same with internal services -- some random developer's desktop might be open, but the service is going to be secure. So what you're talking about is someone actively making a "hacking" attempt at something that, to my mind, looks pretty much impenetrable.

The only other option is something more insidious -- set up a website which exploits some browser flaw, then hope someone at Google sees it. Or sit around a wifi hotspot, praying that someone logs on with a laptop that's vulnerable, infect it, set it to phone home, then pray it is actually able to phone home, and that Google doesn't take a peek at exactly where it's phoning home to.

And a successful variation of this is still just going to give you the one insecure machine. It's not going to give you the entire network. It's probably got less of a chance of doing that than if they were extremely anal-retentive in their security policy (and refused local-admin rights, etc), because it's going to be a heterogenious network.

But then, you did just provide the perfect counterargument: The first hacker to take down their network, either internal or external facing, would be infamous. Therefore, people are trying. It's not working. Therefore, whatever Google is doing for security is working.

Re:Nice approach

[SanityInAnarchy]

Sure you'd get fired when you finally get caught, but if the BSA raids the company before you're found out it could be major fines the company is responsible for.

Seems to me they could just as easily turn around and sue you for abuse of company property. I'm sure you signed an agreement with what you're legally allowed to do (or not do) with company equipment. And if it was your own equipment, it's even less their problem.

Even barring that, there's lots of ways to misplace license keys, and the BSA won't cut you any slack unless you've got damned good records.

Well, they did mention places you could go to grab software. I imagine just about any software you need to do your work, you can get through official channels. Thus, any software that's not kosher could fall back on "Well, we provided all this legitimate stuff for the employee, it's not our fault they did something different."

Disclaimer: I don't actually know how this would work. I am not a lawyer. I don't even play one on TV.

Alternate disclaimer: We were talking about security, not licenses, right? Is this offtopic?

Re:Nice approach

[SanityInAnarchy]

Commit GMail/etc code secretly backdoored by a hacker.

That seems a bit absurd. Aside from code being reviewed on commit, or periodically, do you think they'd be able to actually deploy it to gmail.com without it being caught first? I really hope they're testing things separately and internally before making them live, even if it is a "beta" product.

Re:Nice approach

[orclevegam]

Slightly off topic with regard to the article, but not this thread. The question was, what's preventing other companies from adopting an approach similar to what google has. The answer is that other companies would have more concern over their employees violating licensing issues. The way the legal system is structured, the company that owns the PCs is liable for their contents. They can go after the employee after they pay out to the BSA to recoup their losses (that is, file suit against the employee in response to the BSAs suit against them), but it's an entirely different lawsuit from the one the BSA would bring. They can't just turn around and tell the BSA "Hey, that guy over there did it, sue him not us.", it just doesn't work that way. Whether you're google or not, the security aspect is kind of hard to argue with, although even there google would tend to have less issue than other companies simply because their employees would tend to know what sorts of behaviors are suspicious, but because they've attempted to harden their infrastructure even a compromised machine should be relatively simple to isolate.

Re:Nice approach

[V for Vendetta]

Have software clickwrap licences even been tested in court yet?

Yes. At least in Germany. Here, you, the purchaser, need to able to reed the EULA/ToS before even buying the software.

NO TFA

[Coraon]

tried to read TFA, much to my surprise it isn't there...someone got the story?

Re:NO TFA

[orclevegam]

Reload the page, it worked for me. Looks like their server is having a minor case of slashdotting.

Re:NO TFA

[snib]

What are you talking about? The link is right in the summary, as always.

http://online.wsj.com/article/SB120578961450043169.html?mod=googlenews_wsj

Re:NO TFA

[The Mighty Buzzard]

You tried to RTFA? You must be new here.

Re:NO TFA

[Coraon]

...I was looking for pictures...I'm in lust with the google building.

Not actually a big deal

[Anonymous Coward]

Even Microsoft let's its engineers download software, pick their hardware, and install an alternate OS. It's not remarkable at all in the software business.

Re:Not actually a big deal

[Danny Rathjens]

But do they have a PR guy savvy enough to advertise that fact and the related "enterprise" products by getting a mention in the WSJ and submitting the story to /. anonymously?

Re:Not actually a big deal

[evilklown]

I'm guessing that Google gives more of a choice in OS than Vista Home Premium, Vista Business, Vista Ultimate, and Windows Server 2008.

I wish our IT was like this.

[dangerz]

With all the restrictions on tools and languages, it seems like our IT holds us back more often than pushing us forward.

I recently built an application for my group that started off in PHP/MySQL. The customers were using it and loving it, but IT said they're not interested in supporting PHP and we weren't allowed to stand up a server. After months of talk with them and compromising, it was rewritten into JSP/Oracle. Then they said we're not allowed to do that either, so we agreed on C#.net/MS SQL. I rewrote it to that and after a month, they again came back and said no way. Getting ever more frustrated (I now had the same program in several languages), I ended up in C# Desktop Application instead of web/MySQL. They've been complaining again, but we have more leverage there in that my entire group was stood up to build desktop apps. I'll probably have to switch it to Oracle, but that shouldn't be a big hit.

We wasted lots of time and money rewriting what was already done all because of politics. I always thought IT was meant to *support* rather than hinder.

Re:I wish our IT was like this.

[Anonymous Coward]

Sounds like your manage is a little bitch and didn't get them to do their jobs.
My view is that situations like this are what managers are for. They are there to traverse the politics for you to get your php application up because that's what needs to be done. They also have more leverage when talking to the IT department's manager, or when talking to the Department Manager that the IT manager probably reports to, which is good.

Re:I wish our IT was like this.

[pongo000]

With all the restrictions on tools and languages, it seems like our IT holds us back more often than pushing us forward.

Beware of any job where IT support calls the shots. That is an incredibly inane and inefficient business model. IT support is exactly that: They are there to support development efforts, not to hinder them with brain-damaged policies usually written and enforced by CTOs that don't have a clue and administered by low-paying drones who substitute authority for what they lack on the pay scale.

Why even bother working for a company like that? With the upswing in IT, you sound like you've got way more than enough experience to find a job elsewhere.

Re:I wish our IT was like this.

[filterban]

Wow. Did you bother asking them what they would support before writing the application? That seems like the better approach to me.

If they're only willing to support a specific language, then you need to work in their requirement (generally speaking).

Re:I wish our IT was like this.

[mc900ftjesus]

Does IT make the company money? No, not a dime, they're a money sink-hole like electricity and phones. They don't call the shots just like the maintenance man doesn't call the shots. IT departments need to be enablers. When IT crosses the line from preventing you from installing tons of crap on your desktop to killing the rollout of a platform that generates revenue, someone in management should have been fired on the spot, no questions asked. IT should never dictate a product, only internal policy.

Re:I wish our IT was like this.

[filterban]

What if the IT department was doing code-level support and their staff only was trained in supporting a specific language and infrastructure?

I agree with you in principle, but it sounds like in the original comment that there was no communication between IT and the developer in question.

Re:I wish our IT was like this.

[dangerz]

This app started in PHP before I was here. When I came in, I rewrote it in PHP to make it more efficient and strip out some of the fat. There were emails with IT on it and they didn't seem to care. It wasn't until the app got popular and used that it became an issue.

My management did their best to fight it, but IT has a strong pull here I guess.

Re:I wish our IT was like this.

[Tetsujin]

Wow. Did you bother asking them what they would support before writing the application? Well, from what he said it sounds like:

- Initially, no - they wrote the thing in PHP just 'cause (maybe it was a prototype or maybe the devs were just experimenting and found they'd come up with something people wanted)
- In subsequent rewrites, yes - they agreed on C#, for instance, and then IT changed their mind after the thing was rewritten again in C#...

Re:I wish our IT was like this.

[houghi]

I feel with you. The several IT departments I wored with have the same attidute of not wanting to change anything and forbid everything that could hinder them.

The worst I have seen was where I requested an email to be send from a a system. I knew it was possible. What was even worse was the fact that they had bought the CRM package for a LOT of money, because it was able to do so.

So when I asked if it would be possible to implement it, the answer was that I needed to fill out a request. I told them I could only fill out the request if I knew how much money it would cost.

Catch 22. The procedure on how to do things was written and nothing could change that.

I have seen IT departments that were unable to remove certain rights from people if they would not need them anymore, because there was no procedure for it.

I myself had, due to human error, access to each and every place in the building. More then anybody else. When I mentioned this, they told me that because I got it, somebody must have OKed it so I have the right to it.

IT departments just LOVE procedures. Basicaly because they are so easy to put in logical yes and no questions and answers. They should start with some debugging of their procedures and realise that the real world is more then if, then, else.

It seems that the person at Google has done just that.

Re:I wish our IT was like this.

[kiwimate]

After months of talk with them and compromising, it was rewritten into JSP/Oracle. Then they said we're not allowed to do that either, so we agreed on C#.net/MS SQL. I rewrote it to that and after a month, they again came back and said no way. Getting ever more frustrated (I now had the same program in several languages), I ended up in C# Desktop Application instead of web/MySQL.

What am I missing? You had discussions with IT and agreed on whatever platform. What happened when they said "no way", and you waved the sign-off sheet in their face and pointed out they'd already agreed to this? If this is a pattern (which it obviously is), I think it's crystal clear you get an agreement in writing.

Re:I wish our IT was like this.

[Anonymous Coward]

I'm in finance/IT and I'd just like to say: *all* large financial companies are like the one described by pongo000.

Why not switch to a company like google ?
Simple: they pay me so much money that this form of light torture / kafkaesque work environment is still more attractive to me. The banks I work for pay me approx 4 times more than google would - this way, I can retire when I'm 40 years old (and spend time doing interesting/creative IT stuff instead of having to be chained to a corporate entity).

I work to live - I don't live to work. As hordes of clueless MBA's have had 20 years to surgically remove all the fun and creativity out of corporate IT, I have decided that I prefer to take the route that enables me to be *truly* free once I'm 40.

Re:I wish our IT was like this.

[Anonymous Coward]

We wasted lots of time and money rewriting what was already done all because of politics. I always thought IT was meant to *support* rather than hinder.

Not if it's Microsoft. Then the 'IT' department is working against you. Sure you pay them, but their goal is to further the agenda of their political party. It's got stock and it files with the SEC but sure enough some kind a political party.

If they can't force you to toe Bill's line, they do their most to throw sand in your gears to see if you'll give up.

Re:I wish our IT was like this.

[poot_rootbeer]

See, after the first time you spent a month rewriting a working application to satisfy requirements agreed to by the IT Department, only to have the delivered work capriciously rejected, that's when your department should have gone to the CEO/VP/any bigwig with a sympathetic ear, and the director of IT should have gotten a chewing-out.

Re:I wish our IT was like this.

[mungtor]

"Does IT make the company money? No, not a dime, they're a money sink-hole like electricity and phones."

IT is a cost, but if they are doing their jobs correctly they can also work to save the company money. Most software engineers have no clue about what technology would be best to implement their products on, they only know what got touted as the best/fastest/newest thing on ./ and therefore they *must* have it (otherwise IT is "blocking" them, of course).

Generally, there's just too much ego involved from both sides. Everybody thinks their right and are more willing to play office politics to try to "prove" it than to just get the fucking job done.

Re:I wish our IT was like this.

[Culture20]

Sounds like there weren't any security impact meetings and that one group in "IT" didn't care, but once the app was up and running, another group (whatever your security group is called) _did_ care, a _lot_. IT isn't just a "utility" through which the money makers provide services to the clients, it's also a buffer that protects the whole company from security flaws that can lose said clients.

Re:I wish our IT was like this.

[dangerz]

Normally, it's an amazing job. I work on several fighter plane programs so I get to see and be part of some awesome technology. It's just dealing with IT that sucks.

Re:I wish our IT was like this.

[Anonymous Coward]

We wasted lots of time and money rewriting what was already done all because of politics. I always thought IT was meant to *support* rather than hinder.

I Hate programming in I/T shops. Most managers are like Dilbert's always seeking the most complex error prone way of software design and coding. They repeat their disasters time and time again. If with they would measure management effectiveness like code, defective decision per hundred made. They might find, management, not programmers need the most help.

If you really want to program and do it well, you make for a company that develops software as their #1 business and has set good practices and culture. Far too many I/T shops are far too out of control to be any good at real software development.

Re:I wish our IT was like this.

[afidel]

Money is nothing but a measure time and value, if IT is doing their job they are saving the company so much time that when multiplied times the average salary in the company they are in fact making the company TONS of money in reduced labor costs. That's why I restrict my users, because I get them the tools they need to do their job efficiently and I keep those tools up and running and performing well enough so that they aren't the bottleneck in the organization.

Re:I wish our IT was like this.

[VENONA]

Users v Admins is yet another category of religious war, and has been for at least 30 years. It's further complicated by the fact that the role of IT can (and does) vary from org to org. Sometimes it follows a role somewhat like you'd find described in a college's curricula listing, but they sometimes absorb more MIS-like functions, etc.

One large factor that keeps the war burning brightly is that the relative skills between various user communities and an administration community is also all over the map. I've seen developer groups who were purely code-monkeys, and made some very bad calls on software that they would then have thrown over the fence for an admin group to support, no matter the (large) impact on that support group, if someone from an admin group hadn't been able to do some basic sanity checking. OTOH, I've seen groups of users thrashing about trying to accomplish even the simplest thing, because some bit of software they needed had been wedged in the IT approval loop for several months.

Another factor is that admins often have little concept of what the developer has to deal with on a daily basis, and vice-versa. In my experience, this one doesn't get enough attention, and it often leads to people from different groups talking past each other, instead of helping each other.

Better communications, and a bit of experience on both sides of the fence, often helps people find some commonality of experience. I know I've usually had buddies (and people I didn't were too clueful) in both broad groups, in any org I've worked with. If nothing else, you can always band together with admins in mutual hatred of Roving Bands of Managers, thereby moving the religious wars to a different level.

I don't mean to deprive anyone of the pleasures of a religious war. If the two groups could somehow band together, but somehow not against Roving Bands of Managers, all is not lost. Developers can always fight other developers in the language wars, etc. Well, actually *both* sides can do that, so never mind. But admins can always fight the MTA wars, or similar, amongst themselves, while developers can argue about the One True Way to do IPC, etc.

As far as I can tell, it's turtles all the way down.

Re:I wish our IT was like this.

[Anonymous Coward]

Decision tree for approving an IT implementation.

IT QUESTIONS

1. Does a similar service already exist on the network?
        Yes - Not approved (use the existing resource...)
        No - continue

2. Will the implementation harm the company/network? (security flaws, license problems, bandwidth hog, etc.)
        Yes - Not approved (try again with a redesign...)
        No - continue

FINANCE QUESTIONS

3. Does the budget exist to cover the initial costs? (HW/SW purchase, developer time to write code, etc.)
        Yes - continue
        No - Not approved (come back when you have the $$...)

4. Does the budget exist to cover ongoing costs? (license renewals, maintenance contract, support staff, etc.)
        Yes - continue
        No - Not approved (come back when you have the $$...)

MANAGEMENT QUESTIONS

5. Is there a business need?
        Yes - Approved
        No - Not approved

Re:I wish our IT was like this.

[silas_moeckel]

I think your experienced divided shops to much. Try a silo approach where teams do full life cycle with admins involved from the specifications phase (things like can it work with our existing SSO engine CB evals on adding new platforms to the mix etc) and the devs involved with long term support (write a bad app you gets 3am calls, feedback loops weed out bad calls and blind passing up). Once your done a few projects that way it often leads to a much better integration.

My favorite MTA fun is a large insurance company that the programs use the email system as a message passing queue between systems think 10 million internal emails an hour. I see this all the time since oracle and the like seem to support email and ftp as built in functions so they are what get used. Scale the system up and it's a nightmare vs using the right technology's.

Re:I wish our IT was like this.

[Fastolfe]

IT departments just LOVE procedures. Basicaly because they are so easy to put in logical yes and no questions and answers. They should start with some debugging of their procedures and realise that the real world is more then if, then, else.

I call this the Principle of Least Work. If you have a department that has or can easily justify some form of authority over another (and any services-oriented department can do this by virtue of controlling access to the service, in this case IT services), without sufficient oversight, that department will always create policies and procedures designed to push work onto the clients. In IT departments, you see this as complex processes and forms that require far more detail than is usually necessary to communicate the request, because why spend 30 minutes trying to understand a request that the client took 30 minutes to prepare, when you can spend 20 minutes on a request that the client needed 60 minutes to prepare? You've saved 10 minutes of work! Ka-ching! Good job!

Re:I wish our IT was like this.

[Anonymous Coward]

I always thought IT was meant to *support* rather than hinder.

Mod parent +5 Funny!

Re:I wish our IT was like this.

[Anonymous Coward]

Horseshit. Spoken like a true Punky Codemonkey. The sysadmin is not there to give you a goddamned pony every time you lame for one; the sysadmin is there to ensure that the systems and network operate continuously, can be supported, and that both goals can be achieved with a minimum of cost. The sysadmin is there, in short, to provide adult supervision to people like clueless lusers (and yes, this includes most "IT-skilled" people in a user role) and attention-deficit developers, the end.

I'm sure you don't agree, and think you have a valid perspective on this issue. I propose that you spend a couple of years doing sysadmin work before you shoot off your mouth; wiping somebody else's ass for the twentieth time, and pulling 36-hour days because shit's broke and needs fixing are the sorts of things that might provide the impetus to dislodge your head from your ass.

It always looks like the mean sysadmin is out to get you, until you've spent some time on the meathook he spends his days hanging on. Then he doesn't seem like such an asshole.

Re:I wish our IT was like this.

[Firehed]

Did it serve its purpose? If so, why the hell would they care what language it's written in? Were they in some sort of strange denial that a PHP/MySQL app can be used in a useful/large-scale/production/"real" environment, or was there a legitimate need to recode it into a MS-centric solution? The sales people where I work talk about .NET as if it's magic powder that you can sprinkle in your server room and have everything start working (while not knowing a damn thing about it, other than our product uses it - can't blame them, but it's a pretty uninformed and inaccurate statement). Of course they're sales people so they can't talk down about the tech that powers our product, but there's plenty of un/misinformed nonsense that goes around.

Re:I wish our IT was like this.

[VENONA]

I'm dead-on with you for all of your first para. That's the better world, and not seen remotely often enough.

Re: the second para, using email as a message-passing interface. I've an example that backs up your first para.

I once created such a beast, horrible as it is, in concept. The data source was commercial software that couldn't emit anything but warning emails, and the demands upon the system I needed to create demonstrably wouldn't grow beyond a message count in the low hundreds in any 24-hour period.

Several years later, no scalability tweaks have had to be added, as it's still operating well within original constraints on message counts, and meeting the need. The admins have a firm handle on it, etc. A nasty concept, from a pure developer POV, that proved to be a solid win.

Here's the interesting bit--I wrote it as a developer who was then working in an admin group. More focus on clear commenting than clever code, etc. I knew the people that would have to maintain it after I left, wrote it for that maintainer audience, left an overview and design doc, etc. No way was I going to subject them to 3AM support calls--these were a group of people that had all helped me out, when I was the one getting the 3AM call as a newbie admin, and who I might be having a beer with after work. I'm funny about that--anyone who bales me out of a jam at 3AM, when they really didn't have to, is someone I'm likely to have a beer with. I couldn't care less if they're a Microsoft or Linux fan, from the developer or admin side of the fence, etc.

It's amazing how well even some bizarre (and using an MTA as a programmatic messaging interface certainly meets that description) concepts can work quite well, if everyone is actually on the same side, and talking.

I've other examples, such as convincing a developer of the Microsoft persuasion who was responsible for several internal Web apps that using IE-only extensions would hose some people that did Unix development, and rarely even looked at their Win machines. We ended up investing something like half a day's time swapping some ideas and code around, and that was sorted. Neither manager involved had a problem with it, when it went into our weekly reports. My boss just expected that sort of thing out of me, the LAN app developer got a kudo out of it, and another interdepartmental bridge was built. There was no downside anywhere; everything was either neutral or up. That's a win, by my definition.

Religious wars are tons of fun on Slashdot, where we can mod each other as trolls, etc. This is a voluntary playground where we can all happily fight amongst ourselves, and no harm done. Religious wars carried into the workplace tend to suck, with rare exceptions.

Re:I wish our IT was like this.

[vbraga]

Plumbing is a cost, but if they are doing their jobs correctly they can also work to save the company money. Most Sewage producers have no clue about what technology would be best to implement their products on, they only know what got touted as the best/fastest/newest thing on ./ and therefore they *must* have it (otherwise Plumbing is "blocking" them, of course). We, software developers, know what the f*ck we're going to do (I have only experience on companies that software is company bread-and-butter, so, my POV is engineering-centric). If Development finds that it should use a tool it's IT duty to support it. If I need another telephone on my desk, it's maintenance duty to put other telephone on my desk.

Simple as that.

I know I'll be flamed. But putting my humble mode on, why shouldn't be this way? On a software development company why should IT say what I (a software developer) should or not run. It's not acceptable to me. And I can find a reason to things to be this way. I know what I'm doing. If I want to use Buzzword 2.0 on Monorails with Web 3.0 Beta flavors, I have a reason for that.

Re:I wish our IT was like this.

[Krishnoid]

I always thought IT was meant to *support* rather than hinder.

I'm certain this man would completely disagree with you on that point.

Re:I wish our IT was like this.

[rmerry72]

I recently built an application for my group that started off in PHP/MySQL. The customers were using it and loving it, but IT said they're not interested in supporting PHP and we weren't allowed to stand up a server.

And at that point I'd talk to my manager to talk to IT's manager. If nothing happened in a month go above him and explain the effect of IT's policy to the company's bottom line. If that then doesn't work - walk out and start your own company running your own servers with your PHP script. Obviously they don't need your ideas and you might as well satisfy your customers demands in a way that provides monetary compensation for you.

IT is there to support the business, and the business should allow IT to do that. If not, customers get screwed and low level employees get squeezed.

Re:I wish our IT was like this.

[rmerry72]

That's why I restrict my users, because I get them the tools they need to do their job efficiently and I keep those tools up and running and performing well enough so that they aren't the bottleneck in the organization.

Are you qualified to know every tool that a user might want / need for every job, specifically software engineers? Do you know how they should be doing their job efficiently and so not be a bottleneck? I doubt it.

You're there to support your users not dictate to them. If you are qualified and experienced enough to do every job in the company then what the fuck are you doing in IT support? Surely you can make more money doing your user's jobs for them.

Re:I wish our IT was like this.

[natmsincome.com]

It generally doesn't start like this but they end up like this one step at a time.

To start off with IT is really helpful but then different departments start to abuse the IT department:

• Sales ask IT to help them with charts, report, power point slides
• Department ask them to rush through a project
• Managers ask them to make a minor change.

Generally the IT department starts off without any power and they do what they are told until something goes wrong:

• To many IT staff are required because they are all making charts, reports, power point slides
• Projects that a rushed through fail.
• Minor changes break the whole system

To stop these problems from happening procedures are put in place by management:

• A request needs to be submitted before work is done
• Project have to be tested which takes time
• Ever minor little change needs to be submitted as a request

Often IT can't change the procedures unless they file in paperwork and even then because they are a service department they are often not allowed to. They might have to convince someone else in another department to fill in the paperwork to change the procedures that they have to follow.

I'm not saying that it doesn't suck but it's generally not because they are jerks. It's because they are covering their buts just like everyone else.

Re:I wish our IT was like this.

[afidel]

A) I only support a small handfull of developers and they all have local admin and are expected to maintain their own boxes.

B) Even if I WAS qualified to do every job in the organization not that many of them would be more highly compensated. Due to the multiplier effect of my efficiency gains I am more valuable to a company making lots of people productive then I am doing some singular job. As a wise man once said, pick up a nickel and you have a nickel, have 10,000 people pick up nickels for you and soon you're talking real money.

For the vast majority of my users they need communications, productivity software, and financials. They do not need to install random weather plugin accompanied by spyware. I have a list of over 150 software packages that we support, that's quite a bit for a company of ~1,000 employees. Heck just for financial reporting we have like a dozen different packages we support.

Re:I wish our IT was like this.

[silas_moeckel]

Unfortunately thats a world that CTO's have to build. It's very easy to segment things I've even done it during my time as a CTO. Ultimately to build silo teams has to be a management choice and it involves giving up some control. But then again being able to give up control and instead lead is when management gets fun (and keeping your guys out of office politics and getting them what they need to be successful but thats another thread) granted I'm a CS major that manages because it makes my projects successful so my views might be different from the MBA types.

On the email bit it's all about the design scale a couple hundred emails when thats your best choice is one thing designing an entire enterprise architecture and extending it is another.

Re:I wish our IT was like this.

[VENONA]

LOL. Figures that when I do a post that mentions Roving Bands of Managers, a CTO would be reading the thread. I've always been lucky that way...

OTOH, I wish CTOs who were also CS majors would post in a lot more places. For example http://www.theregister.co.uk/2008/03/20/motoring_offences_clampdown/
has the math wrong. They forgot /2. A CTO/CS major might be able to write directly to an editor, and have some effect on editorial policy.

I guess I just wish CTOs would encourage their people to post in public fora as a Good Thing. You would, in general terms, encourage employee membership in USENIX/SAGE/ACM, right? What those orgs do is disseminate knowledge. I wonder if corporate officers couldn't have more an effect, with a minor bit of employee encouragement, than USENIX/SAGE/ACM combined.

I'll likely be wondering for some time, as I don't see any way to develop metrics via postings. You'd have to send talent to a college or something if you had to have metrics. So it's not likely to happen. OTOH, maybe just advertising that some percentage of time is devoted to edu would grab some Google-like good will, and good will can go on the balance sheet...

The question is...

[adpsimpson]

From the article:

"How do you run the information-technology department at a company whose employees are considered among the world's most tech-savvy?"

Re:The question is...

[Jellybob]

The way we do it at the place I work (a mid-sized ISP), is that the first thing you do when you start is pick an operating system, and install it on your workstation. From that point forward maintaining your desktop is your job - IT support are there to manage the network, the internal file servers, and to look after the non-technical departments Windows machines.

This works remarkably well, but that's because our floor is about a 50/50 split of software developers and sysadmins, and we all know our way around a *nix install. If you do have any problems you can't fix, odds are there's somebody who can fix it around the place.

Re:The question is...

[Quattro Vezina]

This is exactly like the company where I work.

Well, we usually don't have to install our own workstations--new employees usually get a machine that used to belong to someone who left. They can reinstall it if they want, but no one bothers. Engineering desktops are an eclectic mix of Debian, Ubuntu, and Fedora. The actual machines are usually old HP, Compaq, or Gateway PCs, but we also have several System76 boxen with preinstalled Ubuntu.

For engineers, we maintain our own desktops, though the company-issued laptops are maintained by IT (but nearly everyone in engineering has a Linux desktop they maintain). The layout of the labs is the responsibility of the department using the lab (development, QA, or customer support depending on the lab); IT only handles the basic network infrastructure. File servers, build machines, etc. used by development are maintained by developers (and I'm one of the two developers who usually maintain these machines). IT just gives us the hardware we need and makes sure we have connectivity. The only servers IT directly maintains are general servers like the mail server, the company directory, the core gateway, etc.

If I have a problem with something on my machine, I'll always ask one of the more senior developers. Our IT guy (there's only one; this company is tiny) is a great guy, but the developers have the most experience with Linux workstations. I love working at a startup.

Re:The question is...

[Jellybob]

Well, we usually don't have to install our own workstations--new employees usually get a machine that used to belong to someone who left.

I think people *would* do that around here, except one of your last day tasks is to cause as much damage to the OS as possible, so that the next person has to reinstall ;)

You should see people's faces when they realise there's no shell installed on their desktop anymore, and ps is doing funny things.

How?

[Anonymous Coward]

Okay... Sounds interesting, but how exactly security and proper licensing is maintained? Could other companies emulate it?

Re:How?

[orclevegam]

Okay... Sounds interesting, but how exactly security and proper licensing is maintained? Could other companies emulate it? Maybe. Depends a lot on the company I imagine. Part of the reason it flies at google is because of something mentioned in the article. Almost everyone is an engineer of some type, and they all have security training. The security bit isn't as important, but as far as licenses go, most of them should understand you can't for instance bring your copy of MS Word in from home and install it on your company system. At companies with less technically inclined individuals, they may not see the problem with installing whatever software they can find on their company systems (talking from a purely licensing standpoint here, not talking about security). Essentially if Google got raided by the BSA they'd probably fair pretty well, but some other non-IT centric company might not fair as well with a similar IT policy. Of course, there's no reason for any company not to implement a similar policy for all their technical users at least.

Re:How?

[Anonymous Coward]

In my experience even most software developers don't really have enterprise-level security mindset/knowledge embedded in their mind. No one can blame them, it's just not their job, concern. And it is certainly not their responsibility. It would be interesting to see, what would happen at Google in case of a major security breach, caused by uncontrolled environment. Where would the buck stop?

As for licensing... I really don't know if engineers have any higher standards in this area than people at accounting.

The question if this is a model that can be followed by other companies remains.

Re:How?

[bishiraver]

I'm willing to bet that any licensed software is freely available from internal google downloads, along with the legal license to said software. Google has the money to, after all.

Re:How?

[orclevegam]

As for licensing... I really don't know if engineers have any higher standards in this area than people at accounting. Not really a question of standards, it's more an awareness of what your doing on your computer. Many people just know you click next a bunch of times until the software gets installed, and the licensing issue doesn't really cross their mind much. They may even think that the license they have for their home system can be used to perform the install on their corporate system (in some cases it might, but you actually need to read that license to know for sure). To a great many users software is just something you put on the computer to do things and they don't give a second thought to where it comes from, or what license it uses. Engineers (or developers or system administrators, pick your IT title) usually have a better grasp of software, and a more refined understanding of where things come from, and the implications of clicking that "I accept" button.

Mostly fluff

[orclevegam]

Not much to this article but there are a few interesting tidbits. A lot is in the summary, so not much need to go to the actual article, but something interesting not in the summary is when he talks about googles security environment, and why it's not really a security risk to let people install whatever they want. What it boils down to, is that the old style security of locking down the endpoints (that is, peoples workstations) makes people sleep better, but doesn't actually provide much in the way of security. Instead they focused on securing the infrastructure, such as running AV software on the mail server, and intrusion detection software that monitors the networks and servers, plus one would assume properly configured firewalls. He also mentions that being a search company they already had really tight security in place and that few people had access to customer data, so adding security to support outside enterprise data wasn't a big leap.

Re:Mostly fluff

[Anonymous Coward]

There is a huge flaw in that argument. By protecting the infrastructure you have prevented any malware form hitting the outside or using that infrastructure to spread. But any nodes internally are wide open to be compromised. A simple mail bot could flood your entire local network segement, even if it never got past the gateway router to send to the internet. It could also send to any mail servers accepting SMTP sessions from internal machines.

Security people have preached a layered approach for a long time for a reason. Ignoring one of those layers is foolish at best.

that having been said, the userbase at Google is probably 100x more capable of dealing with an issue should one arise than a typical company. That alone explains why this work in my opinion. Its like giving car keys out. You would never do it in a preschool or Elementary school, but a highschool you might and a college you definatley would.

Question:

[Anonymous Coward]

Mr. Merrill's group lets Google employees download software on their own, choose between several types of computers and operating systems, and use internal software built by the company's engineers. Do they use Google to search for torrents, or The Pirate Bay?

It all comes down to this....

[Itninja]

Mr Merrill: "....We use automated tools that check every engineer's code."

So who writes these 'automated tools' and who checks those? I sure hope they have a human in the security audit mix somewhere....

Re:It all comes down to this....

[ccguy]

So who writes these 'automated tools' and who checks those?

Most likely they use those tools to check themselves, pretty much as you compile (most of) a compiler with itself, debug a debugger, and so on.

If you are interested in how these recursive tools work, check valgrind's documentation (interesting because it relates a bit how some design decisions were made so that valgrind could be used on itself) for example.

Re:It all comes down to this....

[tkw954]

So who writes these 'automated tools' and who checks those?

Most likely they use those tools to check themselves, pretty much as you compile (most of) a compiler with itself, debug a debugger, and so on.

Yeah, but then you become vulnerable to a checker that (by malevolent design) overlooks a security fault in both itself and other programs. Something like Ken Thompson's "Reflections on Trusting Trust" in which the compiler inserts a backdoor into the login program, but also inserts the same code whenever it compiles a compiler.

Enterprise-software?

[Bromskloss]

Is that a synonym for "software"? The sentence would seem to make sense then.

My company does this too!

[That_Chubby_Kid]

Nice people email me free software samples. I even get more free email now. My computer runs really slow now. I think it's because it can do so much more work now.

Great! I'm tired of the servant being the master.

[Anonymous Coward]

Why shouldn't the system adapt to the people instead of the people adapting to the system?

I fail to see the "tech-savvyness" here...

[pongo000]

...the last few gigs I've worked, there have been little to no restriction on what we could download on our Linux/Windows servers and workstations. We were tasked with a job, and granted the level of trust and discretion needed to get the job done.

Why would I work at a company that expects me to play the game with my hands tied behind my back?

As usual, another non-story about Google framed as an earth-moving event.

Not uncommon in tech-savvy organisations

[Bertie]

I also worked at a very big company which let us do this. Not company-wide, just the couple of thousand people that worked where I did, which was probably very similar to Google in terms of the sort of people who would work there. We were considered to be bright enough to stand on our own two feet. We weren't the sort to bother tech support unless it was a problem with, say, networking - applications we'd installed were our problem, and besides that we'd be more likely to know what we were doing with those applications than the average techie. It meant that if we needed a particular piece of software or equipment, we didn't have to wait weeks to get sign-off from God Himself - we went and downloaded it and our manager found the money for it if it had to be paid for. We were trusted not to buy stuff we didn't need, and by and large it worked. Treat people like adults and they'll behave like adults, mostly.

More than once I got hold of an oldish spare computer and installed Gentoo Linux on it, and the only justification I had for doing so was that Windows got on my nerves. Not much of a business case, but as far as they were concerned I was a big boy and could look after myself, and it was no skin off their nose as long as it didn't take up tech support's time.

The only thing that made us different from the tied-down masses elsewhere in the company was our level of knowledge about what we were working with. I maintain that the best security system is user education. Obviously that's not to suggest that you should throw caution to the wind, but clued-up people generally won't get you in trouble. So clue them up.

Right now I'm in a much more locked-down environment and it's incredibly frustrating. Something as simple as connecting to a printer is a nightmare because I have to go through some tech support clown who invariably knows a lot less than I do and bumbles around randomly prodding things till it works. I don't have admin rights to my own machine, and useful things like the command line are blocked. It drives me mad, and it holds me back in my work, but hey, some IT goon has an easier life because of it, so it's all fair enough, right?

Google is full of smart people, and the people in charge are clearly smart enough to treat them as such. I wish more companies would follow this example.

Re:Not uncommon in tech-savvy organisations

[KiltedKnight]

Not for nothing, but back in its heyday at AOL, you supposedly had some of the best, brightest, and most innovative developers... yet a lot of them were NOT email savvy at all. People would just download and open attachments from random, unknown people without performing a virus scan or anything like that.

Just because you have some brilliant techies doesn't mean they are all security conscious as well.

Re:Not uncommon in tech-savvy organisations

[Anonymous Coward]

some IT goon who probably has to manage 999 other desktops for people less technically inclined than this one code monkey has a [manageable job] because of it
Fixed

Programmers make marginal sysadmins, and vice-versa. Let the goon do his job.

Re:Not uncommon in tech-savvy organisations

[Bertie]

But that's my point - basic computer common sense training should be mandatory, so that we can have a bit more freedom with our machines, and tech support can have a quieter life. I'm sure they'd rather not have to do mundane crap like authorising me to use certain printers when I've got the wit to do it myself, and I understand why they won't let me, but if time was taken to show everybody the ropes, it'd be one less thing they'd have to worry about, because they could assume a certain level of knowledge for all users. Right?

It really bothers me that people can spend all day every day working on a computer and not only not know much about how to work it, but think that this ignorance is something to brag about. If it's the main tool of your trade, you should know how to operate it, and by that I mean something beyond finding your way around the common applications you come into contact with day-to-day. I hate to use a car analogy, but it's like driving without knowing how to change a tyre.

(I'm not a code-monkey, by the way, just someone who knows one end of a computer from the other)

Re:Not uncommon in tech-savvy organisations

[DanQuixote]

Wow, no admin to your own machine? No command line? WTF?

Are you required to get potty passes as well?

When you are denied access to the required tools for the job, you are selling yourself short.

Get. the. hell. out!!! of there!

It's kinda like drugs... a fun ride, but you really don't want to be stuck there. Just say NO!

Re:Not uncommon in tech-savvy organisations

[element-o.p.]

I think the only way to really teach people what you call basic common sense is to make them work the IT help desk for a month before they start their real jobs, and I don't see that happening any time soon.

Once you've had to clean up the mess that someone else made of their computer because they didn't understand nearly as much as they thought they did, you start to realize why the rules are the way they are. But until you've been there, it's very, very easy to just assume that IT is on a power trip, they are just trying to get in your way, they are just playing office politics, etc., etc., etc.

Ehh.

[bytesex]

This is /special/ in IT ? Well, I be darned - it's never been different in any way for me, at least.

Could someone post the article text?

[Anonymous Coward]

Apparently WSJ's web site is so broken, you can't see the article text if you use Noscript to prevent them from executing code on your machine.

standards-compliance

[PigleT]

The reason this works is because he's a sensible fellow who knows standards-compliance. both in network protocols and data formats, is more important than the mere name of the OS or application issuing them.

Re:standards-compliance

[BenoitRen]

I wish they would get the HTML that their search engine outputs standards compliant already. It doesn't even have a DOCTYPE!

Quick Story

[Cytlid]

I've actually experienced this type of thing in the last two jobs I've had. Allow me to explain.

  I moved from my job in NY as a System Admin for an ISP. I won't name names, but our major tech we used was Cisco, Solaris, Linux and VMware ESX.

  My family and I moved to SC for the nicer weather ... I landed a job as Sr Network Engineer for an ASP. I thought, ASP, can't be too different. Well 800 miles away, some things are the same, some are different. I'm a command-line, CLI type guy. The ASP is an MS Gold Partner and takes advantage of Citrix. All the network gear is Cisco (which is where me and my team come in). I thought, oh great ... I don't belong here (except for the Cisco stuff). For the record, we do have *some* Linux hosting and colo.

  But I setup a few smallish vmware servers and I'm happy. I have my Linux-in-a-box. I've done a bunch of grepping and typing and scripting and such this morning, and I found some new issues that I didn't see before without seeing the "big picture".

  So back to my point. I'm very picky about the apps I use and whatnot, so it's hard for me to "conform" to an IT ruleset about what can and cannot be run on company machines. The ISP I worked at was very flexible in this manner, for some reason I expect this out of the new job.

  Our business model is we sell these published apps and hosting to our customers. We run a large private MPLS network and connect many smaller places to us. They can run Office 2007 from a website.

  Then it hit me. Things have been getting really optimized in the last year or two, so we're using our own stuff. My office apps "live" in a website. The revelation came that now, when it comes to my laptop (or desktop), I can do whatever I want. Notice this is typically a nightmare for common IT shops, but many of our smaller customers think IT is a pain and will be happy with published apps and thinclients. For someone like me, who is tech-savvy, I can format my machine and install Linux (some of the other guys have already done so). Because there's a Citrix web client for Linux (I use it at home). Involve virtualization in the mix, and our datacenter becomes one giant network, one giant machine that we manage and the apps are just floating around inside. We manage all the security and whatnot, and keep it running.

  So in a way, you really can have it both ways. We're not a Web 2.0 shop, but our method is definitely Another Way to Do It.

Last Adopter

[salesgeek]

IT departments are typically the last adopters of anything. They typically roll up to the CIO, who typically is not a real C level executive. The CIO typically works for the CFO and is an advisory member of the executive committee in most companies. Information Technology generally has two crucial corporate functions: automating accounting functions and managing corporate communication platforms like phones and email. Everything else that happens on a computer - i.e. productivity applications, intranets, etc... are side effects of putting general purpose computers on desks and are secondary functionality. IT Departments have generally claimed fiefdoms over all things computerized so they can have bigger budgets, more resources and are harder to fire and outsource. It's ugly. But true. Most IT innovation starts in some department, and goes like this:

• Kid in sales writes really cool web app that sells product automagically on MySpace.
• IT finds out about it, can't integrate it with accounting, tries to kill it.
• Kid freaks out because someone who is three managers over him is calling him asking what he's doing.
• Kid's boss freaks out because CIO is calling his employee.
• Project is killed when Bosses Boss finds out about it because it doesn't make sense to him OR - Bosses Boss intervenes and tells IT to stuff it, and counts money from sales from web app.
• IT is forced to support web app because CFO now needs to book revenues for month or quarter.
• Kid is transfered from sales to IT and leaves company one year later to start company that sells MySpace widgets and goes on to become millionaire.

Re:Last Adopter

[narsiman]

Moral of this story is - if the company had kept the kid happy, she would have been just that - a programmer with a simple salary. Instead she is an employer. Long last the boring IT.

Re:Last Adopter

[mythosaz]

Very, very true.

I work in the IT "automation" department of a company where I help support 30,000 desktops and 2,000 servers. Nearly 5,000 of our desktops now run a shell replacement that was designed only as a way to prevent a small number of machines from ever having access to their printer settings. Someone at the top liked it, and now our tiny widget is a desktop standard.

Re:Last Adopter

[Anonymous Coward]

"really cool web app" kids are rare. "time wasting, torrent dl'ing, IM chatting" kids are pretty common. Lock-down works well for 95% of the kids, therefore lockdown is preferred. :(

Re:Last Adopter

[photonrider]

Missing a couple steps in there: - cool web app runs on kids spare pc under his desk - cool web app uses data mined from big oracle database and processed every night with Access macros - cool web app is not backed up - cool web app dies one day and kids boss franitcally calls in IT for help - IT looks at smoking mess and asks where the backup tapes are - IT spends a week rebuilding cool web app - IT spends $$$$ getting cool wep app on supportable hardware - profits from cool web app up til now.....gone Then there are some variations: - cool web app data gathered by Access 97 macros - cool web app can't be upgraded from Access 97 to Access 2008 - cool web app is running box stock IIS - cool web app is running on unpatched Windows OS

Re:Last Adopter

[soulfury]

I notice that your username is "salesgeek."

Re:Last Adopter

[salesgeek]

I'm long past working in sales - I own my own company now and have been demoted to chief toilet cleaner and director of dirty work.

Re:Last Adopter

[mrscott]

When did you work in IT? I am a CIO (in the "C" level sense in that I do not report to the CFO and am a full member of the executive team) and your view of the purpose of an IT department is... interesting and a bit outdated.

Re:Last Adopter

[salesgeek]

I have continuously been involved in IT since 1986 in either a vendor, consulting or senior management role, but I've never worked in an IT department. Most often my role has been ramming new technology down the throat of risk and profit adverse technology managers who are busy protecting their precious integration plan, vendor allegiances, security model (usually doesn't work) or sacred cow application, not realizing that they no longer are defining the problems experienced by the corporation.

Outside of SMBs (up to $500 million in revenue) your situation isn't as common as you think. As most companies grow and become more risk adverse, decision making and pressure from wall street diminishes the importance of the CIO. In a few companies, the CIO and IT department stay relevant by becoming more innovative and even more supportive of business operations and internal entreprenuership. Unfortunately, many CIOs don't get this and end up being a business impediment department that basically says no to anything that doesn't come with an IT budget. In these companies, CIOs are simply the lightest weight C level executives in a company (C level in name only), and as a corporation grows, they become less and less important as the rest of the Executive team is more focused on M&A, financial performance and less on the next great productivity enhancement. IT no longer delivers competitive advantage the way it did in the 90's and 80's.

Not really...

[Per Abrahamsen]

If you read behind the lines, there are security measures in the network to prevent problems from spreading, and there are networks within the network, so really sensitive information is only available to few people.

It sounds like a superior approach, that probably will only work if you have a superior IT staff. So I'm not sure it is something that will scale to the rest of the industry.

federal regulations

[narsiman]

Google does not have to care about is federal regulation like SOX since IT is its internal process.

Much of what is being done in other firms are due to accounting than core IT. If this guy has a process that can be easily accounted and audited, fortune firms would jump in. Otherwise it will be just another starry view at Google.

Re:federal regulations

[wsanders]

Huh? GOOG is a public company - it is subject to the same rules as everyone else.

They are just doing things are are so weird, and so different, from anyone else, that their IT functions are more tightly coupled to the rest of the company than most places.

Wow

[Teflon_Jeff]

Reason's number 62,459-62,468 why I wish I worked at Google. Letting you choose your own machine and OS? No limits on software? I'm in heaven.

Re:Wow

[lintux]

I'm actually surprised that things are different elsewhere. I admit, I haven't worked at many big companies, but I'm used to being able to install whatever I want on my PC and to a fully unlocked Internet connection. Those are things I need to do my job properly, so employers harm themselves as much as they'd harm me if they introduce retarded IT rules and limitations.

Confessions of a former IT employee...

[Anonymous Coward]

One of the developers downloaded a demo of an IDE and developed a small project using it. Somehow his code made into production even though he hadn't acquired a license for the software. A few years later, we had to fix a bug in his code. The guy was gone. The software he had used was no longer in the market and a lot of the stuff he did was stored in proprietary binary formats. Result... they had to re-write everything from scratch. At least we avoided the law suit since the supplier never found out that we had broken the license agreement of his demo software.

This is a true story and it happened in a company listed in the Forbes 500...

...just like at IBM.

[Anonymous Coward]

Its exactly the same at IBM.

You can lock the machines and internet connection down, but in a company that practically invented modern computing and half a million hugely, hugely technical employees its not going to stay locked for very long!

Its all about trust. If you cant trust your staff to look after their computer, maybe they aren't as high calibre as you thought they were?

Re:...just like at IBM.

[Raineer]

The problem IBM has (which I'm sure is prevalent at so many other companies) is the internal software management programs, which I'm sure are maintained by your region. The garbage *required* to be on any inventoried system is just ridiculous.

I would understand if the reasoning was for an actual needed level of security, but they are stupid "helper" apps which amount to not much more than something one would receive by clicking a context ad on a google search. Most departments around me are still running on older-than-shit laptops, which are dramatically hindered by such software.

Of course, with the coming move to Infoprint, we'll see if Ricoh has anything better planned :)

This is a higher education IT model

[nadahlman]

I work for a small liberal arts college, and in many ways this is the IT model we have, in many ways because we can't have control: students come with whatever computer they like, and the faculty (who have a lot of power here) can't be hindered from writing their own programs, collaborating with other faculty in other colleges with other software, etc.

So we secure the infrastructure, we lock down the administrative systems and keep it behind a massive firewall, and we do our best to make sure all Windows users have antivirus and antispyware (while not blocking Linux and Mac users). Students, faculty, and staff can download and install concurrent-license copies of Photoshop, etc., and we have very low academic prices for Microsoft Office and now for Leopard and iWork.

If someone calls us because they can't get something academically-related done, we do our best to give them permissions to do so as soon as possible, and try to fix it so similar people in the future won't have that problem at all (unless it does lead to a real security issue). If they ask us, "How do you do this?" we have standards and recommendations for them, which makes it easier to support, but we won't stop people from doing it another way. For example, we're on Exchange, but we support Thunderbird and Mac Mail as well as Outlook/OWA/Entourage. And we still have some Eudora and Netscape Mail and Pine nd who-knows-who-else users, and so we have IMAP setup instructions on our web site and good luck to them.

Google bases a lot of their company on higher education, so it's not really s surprise to see this -- but it's an example that other companies may want to emulate, even if not all their employees are engineers. Many of our students barely know computers beyond word processing, web surfing/social networking, and IM, yet network security and monitoring for sudden traffic surges, combined with good free (for them) antivirus, means it's very unlikely they'll get themselves into trouble -- and if they do, tech support sweeps in and cleans it up ASAP. That's a model where the vast majority can figure it out for themselves, and it would help businesses to trust their own employees enough to do so.

the best way to please a professional

[wikinerd]

First, computer and software experts are professionals and not "staff" or "workers" (as long as they know what they are doing). The best way to please a professional is to let them work at home or at their office and carry the work in any way they want at any time they want (as long as this is possible depending on the nature of the work). However, working as an employee means you have to go to a specific place for no apparent reason (teleworking could work just as well) and do the work according to rules prescribed by the employer rather than you - the professional (even though the work could be done more efficiently if you were allowed to use the tools of your choice: for example it's just silly to see employers allowing only MSIE on their PCs for no reason when there are better browsers around). It's just plain impossible to be happy as an employee unless you have no other option. It's no surprise to me that the majority of high-calibre professionals (and natural-born entrepreneurs) become consultants and start their own company, never to return to the job market again (just like what I did), if they ever passed from it in the first place. With this in mind, it can be understood that for a company to succeed it should prefer to work with consultants in long-term projects rather than with employees. Simply put, the pool of people who send CVs asking to become employees does not have the same quality as the pool of people who accept contracts as freelance consultants or independent businesses. In the eyes of a manager the consultants may seem expensive, but in reality they cost less than employees when you take into account the greater quality of the resulting work and all the complexity and inflexibility of hiring and firing employees (even for at-will employment the employer may be held liable for wrongful dismissal or discrimination - but companies working with independents do not have such risks): Accounted for the long-term, companies primarily working with independent professionals end up creating more value for their customers and thus able to extract greater profits from their transactions.

Fringe Benefits?

[bill_mcgonigle]

You can write in at least three languages/DB's, desktop and web apps, and you're working for these tools?

What, do you get a free car every month? Free sex? Do you work at the Vatican?

These are about the only things that would motivate most people to put up with it.