Slashdot Mirror
Ask Slashdot: Advice On Enterprise Architect Position
dave562 writes: I could use some advice from the community. I have almost 20 years of IT experience, 5 of it with the company I am currently working for. In my current position, the infrastructure and applications that I am responsible for account for nearly 80% of the entire IT infrastructure of the company. In broad strokes our footprint is roughly 60 physical hosts that run close to 1500 VMs and a SAN that hosts almost 4PB of data. The organization is a moderate sized (~3000 employees), publicly traded company with a nearly $1 billion market value (recent fluctuations not withstanding).
I have been involved in a constant struggle with the core IT group over how to best run the operations. They are a traditional, internal facing IT shop. They have stumbled through a private cloud initiative that is only about 30% realized. I have had to drag them kicking and screaming into the world of automated provisioning, IaaS, application performance monitoring, and all of the other IT "must haves" that a reasonable person would expect from a company of our size. All the while, I have never had full access to the infrastructure. I do not have access to the storage. I do not have access to the virtualization layer. I do not have Domain Admin rights. I cannot see the network.
The entire organization has been ham strung by an "enterprise architect" who relies on consultants to get the job done, but does not have the capability to properly scope the projects. This has resulted in failure after failure and a broken trail of partially implemented projects. (VMware without SRM enabled. EMC storage hardware without automated tiering enabled. Numerous proof of concept systems that never make it into production because they were not scoped properly.)
After 5 years of succeeding in the face of all of these challenges, the organization has offered me the Enterprise Architect position. However they do not think that the position should have full access to the environment. It is an "architecture" position and not a "sysadmin" position is how they explained it to me. That seems insane. It is like asking someone to draw a map, without being able to actually visit the place that needs to be mapped.
For those of you in the community who have similar positions, what is your experience? Do you have unfettered access to the environment? Are purely architectural / advisory roles the norm at this level?
I have been involved in a constant struggle with the core IT group over how to best run the operations. They are a traditional, internal facing IT shop. They have stumbled through a private cloud initiative that is only about 30% realized. I have had to drag them kicking and screaming into the world of automated provisioning, IaaS, application performance monitoring, and all of the other IT "must haves" that a reasonable person would expect from a company of our size. All the while, I have never had full access to the infrastructure. I do not have access to the storage. I do not have access to the virtualization layer. I do not have Domain Admin rights. I cannot see the network.
The entire organization has been ham strung by an "enterprise architect" who relies on consultants to get the job done, but does not have the capability to properly scope the projects. This has resulted in failure after failure and a broken trail of partially implemented projects. (VMware without SRM enabled. EMC storage hardware without automated tiering enabled. Numerous proof of concept systems that never make it into production because they were not scoped properly.)
After 5 years of succeeding in the face of all of these challenges, the organization has offered me the Enterprise Architect position. However they do not think that the position should have full access to the environment. It is an "architecture" position and not a "sysadmin" position is how they explained it to me. That seems insane. It is like asking someone to draw a map, without being able to actually visit the place that needs to be mapped.
For those of you in the community who have similar positions, what is your experience? Do you have unfettered access to the environment? Are purely architectural / advisory roles the norm at this level?
What they're offering isn't out of the norm, though I might negotiate with them and ask for read-only access (non-root for servers) at least. I've been a network architect for a few years, and one of the things that comes with: loss of enable access to the routers and switches. Mind you, I was a data center network engineer for a whole bunch of years so I know my way around them. But the organizations would rather I "look, but don't touch". The great thing about it is: I can't be called for an on-call issue because there's nothing I can do to fix it. :-)
Welcome to needing to think strategically. Take what they're offering as a compliment and run with it!
Jason Van Patten
An architect is someone who designs, implements and oversees the day to day progress of large(r) scale projects. You get to define who/what to buy and how to realize your vision. But no, you don't need access to the systems but you do need an overview of the entire infrastructure, you're an architect, not a builder/maintainer/owner, you get to see the site, you define future upgrades but you don't maintain the system(s), you surround yourself with others that do that job.
If you can't get a full picture of the network and systems without full admin access, your underlings are doing something wrong and it's time for you to kick them out or go on a major discovery/documentation project. If I were an architect, I'd make sure I have plans, diagrams and documentation on the entire picture first before embarking on a next project.
Custom electronics and digital signage for your business: www.evcircuits.com
With 60 hosts and 1500 VMs I would certainly expect separate roles for enterprise architecture and system provision/admin. If you were talking about a a dozen hosts and 100 or fewer VMs then a sys admin with architectural responsibilities would be quite common. The main thing here is what can you do? My guess is with your experience they could use you overseeing a system admin team and doing EA, moving more to the latter as the team's experience grows.
I'm one of several that play the EA role for an 8,000+ employee firm and for the most part I have no special rights to the systems, domain and network. I can see read rights to everything your describing being reasonable, but just like we don't give developers rights to the production app, the architects shouldn't have the ability to change production either. The person designing and advocating for the solution always thinks they are right. But having to put it through a good change control process helps reveal assumptions and ill considered ideas. That isn't to say you should be able to see into production either directly or be able to request any data you need to design better solutions. And you're not powerless, sounds like the last guy had enough power to screw things up, use the same bully pulpit to start guiding the changes that make it better.
The role of an enterprise architect is to work with stakeholders, gather requirements, create time lines and then hand their work over to another team to implement and continue to provide governance. At best you might be lucky to get access to some sort of test environment. I am TOGAF certified and like you before I started didn't understand what it was before I started. The trainer I had described it as creating cartoons for executives. I still got the cert but realized it really wasn't for me. I will say that I think the role is very important and as an implementor is designed to answer the questions I often have when building something like number of users, availability requirements etc.
The architect is the "big picture guy", he should be able to design it and explain it. But he sure as hell shouldn't be running it.
The architect is most decidedly not the sysadmin, he's there for strategic and long term planning, but not day to day stuff.
If you want to be both the architect and the admin, you'll do a piss poor job of both, and likely cause more problems than you realize. I've met a few architects who thought they should still keep their fingers on the switch, so to speak ... and as they generally made a hash of things because they didn't have the time to be good admins, and though they knew everything at all times.
An architect who thinks he's ad admin is someone who has delusions of being able to do everything, and ends up doing everything badly.
Your management is right, it's an either or thing.
If your organization is small enough you can do both, you're not really an architect. If it's large enough to need an architect, it also needs a sysadmin. It doesn't need some guy who thinks he can do both.
Lost at C:>. Found at C.
I would say that the best enterprise architects are the ones influencing the business, listening to their pain points, identifying the risks with the security manager. Set the boundaries or constraints of your organisation. No point wanting to use public cloud if you hold TS information. Do you suffer more pain for not technologically being able to spin up things automatically? Or do senior management have drivers to meet compliance this year because they now got to a new $ value and have more auditing responsibilities. They will be the ones with the money to let you execute your roadmap, so you need to show the global view of how IT enables them. Don't be precious around opensource vs Windows, Oracle vs PostGres. Only if you want to analyse the costs and direction of the company as a whole, might you say that you have an organisational direction to set. Expect possible resistance from the person who takes your job. Negotiate, document, communicate, review and sign-off.
Develop roadmaps, ISSP's and ways to get there that are easy for people to follow. Use the operational staff from your new level to derive the information on your behalf. Do your work on it, and then communicate how you want to tackle problems. If you have not had clear leadership in the organisations technology aspirations space, then take advantage of being the EA/CA and the position it generally enjoys closer to the senior management. For all sides though, you need to bring ways forward, budgets expected to be consumed, tech sets to be used, tech sets to be retired. Read up on TOGAF or other architecture methodologies. See how an EA can slot into the surrounding processes such as programme management, business planning. And fully expect those people and processes to be needing help too.
I can say as a 6+ year infrastructure architect, I've only rarely hopped onto a read-only view of a vCentre, or into an Azure/AWS config. Moreso for my own training than anything. Same goes for spinning up VM's, playing with controllers - only for training and keeping across what the market has to offer. You will find very quickly it is hard to stay abreast of all of that, especially with the ton of work mentioned above. Embrace that, do a good job and try not to upset the alphas on either side ($$ and priorities on management / business side, and tech arguments on IT side can bring out worst in some meetings).
I don't want leadership, management or anyone with any kind of oversight responsibility making changes on the live gear. That's the entire point of having an operations staff.
However, I see absolutely nothing wrong with read only access. The ability to change things - Not Good. The ability to gather information, that I would deem to be necessary for someone who's going to handle the care and feeding of the system going forward.
It also sounds like you need to clean house if your ops staff is pushing back at designed changes, however. Putting in a competent staff that will follow your dictates and provide you with the information you need would go along way to making access to the actual gear unnecessary
I'm an Architect. Also with a long technical background. Similar size organisations. It's not normal to have admin access. Largely because that level of detail can overwhelm you. It's also easy to get dragged back into your old job if you can be dragged back. In one org I worked in where the Architects did have access (before I was one...) one of our vendors develops the habit of finger pointing when mysterious issues occurred that looked like unauthorised change. We stopped that with some config monitoring software that notified us of any settings change - but I mention it to show what can happen.
One of the hard concepts to grasp is what is Architecturally significant. Mostly that's big block level stuff, but sometimes certain details can be significant too. Working out which without looking at every detail is where your experience comes in.
Most of the time the team members doing the design and implementation work can show you the detail when you need to see it - and by asking them you can discuss what you're looking for and why. This builds up trust that your solutions aren't just ivory tower creations from some distant figure but things they're connected with.
If you must have some ability to see every little detail you could always try asking for read-only access. It might be a reasonable compromise.
This has been a bit of a rambling post, but I hope it has something useful....
If you are logging on to boxes, you are getting too close to operations and too far away from architecture. You get the admins to pull reports and logs you need, but you don't really need logins to the entire infrastructure. What on earth would you do with it that you can't get from the admins? I'm an IT architect for a DR outsourcing company; I wouldn't even have the least clue HOW to login to the gear I'm buying by the truckload (much less do anything useful with it), so obviously I don't have the ability to do so either.
An architect need not have admin rights or even the knowledge an admin must know. (And likewise, it's not important for an admin to know things an architect must.)
P.S. Errr, 1500 VM's for 3000 employees? I sense that a lot of these (and whatever massive amounts of stale data they are attached to) sit utterly disused.
P.S.S. And your historical analogy isn't even valid. Cartographers are generally not surveyors.
An architect (and one that is trying to be forward thinking and implement all sorts of fascinating new gear) is wasting time learning the admin interface for every box he/she specifies.
And if an architect is having trouble getting away from daily ops, not having any access to the boxes at all will help with that transition. (Not to mention that the architect will inevitably get pulled into ops problems, leaving less time to do the actual job.)
It would be funny if your comment had some context.
With 60 hosts and 1500 VMs I would certainly expect separate roles for enterprise architecture and system provision/admin..
This statement is quite right. Apparently the OP doesn't deal with auditors at all in his job. Lucky him. I do in mine and I have something like a Linux system admin job. For the product I work on, and I work for a Fortune 500 company that sells a lot of software products and services, I am the main contact person every year for auditors. Since the OP works for a publicly traded company, he should know that audits are required by US law. Every year I have to answer the same questions from the auditors about separation of responsibilities on the product I support. Honestly, I don't know how the OP doesn't know that getting that kind of access for an architect is going to raise all kinds of red flags in an audit that have to be explained. If I remember correctly, we have exactly 4 people who have root access to our servers who don't work on my team. They're software developers who've worked on the product for years and need that access in an emergency if we have a software related disaster that impacts customers. We have to jump through a lot of hoops to justify this on the audit. In fact, we've actually had our access restricted from some activities we used to do that fall outside of traditional system admin tasks just because it's easier for auditing purposes for us to not be able to do it anymore. In my job my group also doesn't have access to the storage, network or virtualization layers except as users/clients and all changes have to be done by others. Sometimes it's a pain, but at auditing time it makes my life easier as I can tell the auditors "We don't have the ability to change that, so you have to talk to group X on that one".
I think you may be overlooking one thing...eSOX or equivalent policies.
Company assets have to be partitioned. You can't have people that are not trained and/or not accountable for data/hardware messing with stuff. Auditors for the Government and various Standards organizations (ISO this or that) look for these things. For instance, as an "Administrator" for our Manufacturing software, I can change master controls and permissions. But I cannot actually use the software to do anything like create POs, print Invoices, etc. and that is it should be.
I am also in the approval chain for granting access to shares...but I ultimately do not have access to any of the shares (except mine). What's in them is none of my business and outside the scope of my duties, so eSOX and similar policies say I should be locked out.
If you have someone who can do anything they want to anything they want, you are setting yourself up for a disaster.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
Thank you everyone who took the time to respond to my question. Reading the responses has been very insightful and a bit humbling.
I appreciate those of you who called out my tone, pointed out that I'm a whiner and even insinuated that I am not qualified for the position. What would an "Ask Slashdot" post be without one or two snide comments along the lines of, "If you have to ask slashdot, you're obviously an idiot."?
I came to the community as humbly as I could because I realized that my own ego was likely getting in the way, my understanding of what the position is might be skewed, and needing a reality check. I got it.
There were way too many questions and comments along the way to address them all individually. (tl;dr feel free to skip the rest) I will try to respond to most of them here. I hope that by providing some background about my professional experiences and how I got to where I am, others who are on a similar path will gain some insights.
A lot of people had questions about the company itself, its size, the VM to user ratios, infrastructure and other questions. Without spending all day writing about it, the company is included in the Russell 2000 Index. That makes it "medium" sized here in the States. It is a consulting company and we frequently bid (and occasionally win) jobs for the same organizations that KPMG, Deloitte and PriceWaterhouseCoopers go after. My five years at the company have been spent working in the legal technology segment. We provide electronic discovery services to some of the largest organizations in the world. Most of the VMs are application / processing VMs that churn through large batch jobs. (Think producing TIFF files of tens of millions of emails, Office documents, etc. from a large corporation involved in a dispute. Think Enron. Getting caught rigging LIBOR. Creating MBS products that send the economy into a recession...). We also have a number of SaaS solutions for that market.
The IT organization has an ITIL compliant change management process. I deal with auditors frequently. Due to the nature of litigations we are holding onto reams of personally identifiable information, confidential information, privileged information. We deal with large financial sector clients who are subjected to all of the regulations. We deal with health care clients who are subjected to all of the regulations. As irksome as auditors are, I have found that they truly do help us elevate our operations and we have been able to use audits to get capital for systems that we otherwise would have never been able to justify on our own.
When I say that the IT group was traditionally internally facing, they were. They deploy laptops, manage remote offices, keep Exchange running. Their customers are internal to the business. The prior CIO (who was moved out a few years ago) failed to properly size the "cloud" (kill me now for even using that term). Our operations completely outstripped the resources available and required millions of dollars of additional investments in storage (primarily) and compute resources. It was such a large investment that there were even rumors of the business divesting itself of the practice entirely rather than spending the money.
Before I got to my current company, I was a consultant in the (truly) small to medium sized business (SMB) market. (1-250 employees) In that life I was the primary IT resource for small companies where I did everything from design to deployment to operational support. I worked with everyone from architectural firms, to city governments, to waste management companies, 501c3 non-profits, air freight shippers, restaurants, manufacturers (things are still made in America?!?) ... a very diverse client base. I have been working with IT systems professionally since 1996 and using and building my own computers since the early 90s. (The first computer I built myself was a 486DX2/66. I am not as grey bearded as some here, but old enough to have used a 2400 baud modem and