Slashdot Mirror
More on AT&T Wireless's Bungled System Upgrade
An anonymous reader writes "CIO.com has posted a very in-depth article on the recent failings of AT&T Wireless that resulted in the state of the company today. What's fascinating about this article is the sheer amount of accurate information gleaned from former and current employees on the company's bungled attempts to follow FCC mandates on local number portability last November, the inside story on outsourcing efforts, and terrible executive management decisions that ultimately led to its demise. Ironically, the scathing and sometimes highly sarcastic commentary at the end of the article from former employees makes this read even better."
[huge grin] wonder how he got *that* nickname
Seriously, someone who has new/wonderful management process *can* help a company tread a dangerous path, but you *need* the domain expertise to be well-represented in a solution to a technical problem. Imho, at least. From the reports, it seems the process wasn't that wonderful, either
You can expect a certain level of bitterness in ex-employees, especially after a disaster, but there ain't no smoke without fire, and when there's lots of smoke, start looking for the towering inferno...
Simon.
Physicists get Hadrons!
Including wireless internet access to write this on Slashdot, and I've had no pro
For some reason I just cant feel sorry for a company not being able to rally its workers and threaten the workers of off shoring their work. I believe that AWE got exactly what it deserved - number portability was nothing new and they should have been able to get the job done. Yet AWE insisted on moving towards outsourcing instead of figuring out what needed to be done. I have seen similarly situations where no matter how much cheaper labor you look for, if you can't devise the plan, no one will be able to follow it. Good riddance to AWE and I wonder if Cingular is going forward with the outsourcing.
- a young blogger
Siebel is horrible and this isnt the only company that this has happened too.
Take a look at Telus Communications.. When they implemented a Siebel based system customer complains skyrocketed.. The system was unstable and basically useless.. You couldent get any information to people on what was happening..
Papers had a field day on how much customer service sucked.
God have mercy on anyone who has to implement Siebel 7 in a large enterprise enviroment.
As the go-live date neared, former employees say that Deloitte and Touche project managers relaxed testing requirements for various pieces of the system.
The Big 5 (or however many there are now - I mean Arthur Andersen, Ernst & Young, Price Waterhouse, Deloitte & Touche, etc) charge hundreds of dollars an hour for "experts" that aren't experts at all. They're usually just one page ahead of the client. They even charge over $100/hr for wet-behind-the-ears college grads.
We've all dealt with them before, they are usually intelligent people but have no expertise or experience in the task they are being paid to complete.
Yet again and again, despite all their failings, they are being hired by big corporations for major projects.
I'd like to know why.
I don't give AT&T Wireless any credit whatsoever. For God's sake! They can't even figure out how to properl set the time on their towers.
In the Boston area, they reset the time for their towers by setting the clocks forward one hour at Daylight Savings Time (as opposed to properly setting the "Daylight Savings Time flag). Now whenever you use Cingular's network, you get the proper settings, but as soon as you go bact to AT&T, it puts you an hour ahead on wintertime hours.
having worked for attws for a year now, and having been on the receving end of all the angry bitter customers when seibel 7.5 f-ed up, I can tell you that this was the very worst transition I have ever seen. 100 million dollars? between 50 and 200 thousand customers lost to curn? WTF? and it's still a joke on the inside.
Posted ac for my job...
I had AT&T wireless 5 years ago in college and they were no better then. Bad coverage areas, customer service people without a clue, and screwed up billing.
Say what you like about Verizon's CDMA technology. There is more to a phone company than the technology they choose. If the company can provide robust coverage, provide wireless broadband, and treat me right - they will succeed.
As for the CDMA detractors: Try getting 500k/sec. on ANY GSM system now....Verizon's testing it in DC and NY and will soon roll out nationwide.
AT&T wireless didn't fall apart because of their technology choices...they fell apart because they treated their customers badly.
-ted
Yeah, it is. 3G is based on CDMA. GSM is evolving through EDGE into WCDMA and current CDMA systems are evolving into CDMA2000. It has absolutely nothing to do with the US, the major players in mobile phone standards are all outside the US.
She started out by developing the industry standards, and then learned all to quickly to play Government Fat Cat. When we look back at the contributions to science that came out of Bell Labs, both in communications and computer science; it is easy to see that this was once truly an industry giant.
But like all giants, when you get used to playing 800 pound gorilla, you stop thinking and just keep throwing your weight around.
Even after it became public knowledge that Ma Bell was holding back technological advancement for their own purposes and profit, as long as the lobby on the hill kept a few important palms crossed, the tyranny continued. Finally, after a couple rounds of public humilliations and rebukes, the government was forced to order the split-up.
But very deeply imbedded in each and every part of the baby Bells was the crippling notion that they were the best and only company and that the thought of changing their behavior neven even had the slightest possibility of beginning to cross their tiny little corporate brains.
To make a long story short, their corporate egos never evolved back to being lean mean compedetive machines. If there ever was a company that should get back to it's roots of research and innovation this would have been it; but the chance is gone.
My local baby Bell, for example, relies on their internet customers to have their error checking turned off, when they visit the customer service website. As a developer I keep mine turned on and get about a half-dozen errors when each page loads, and a few more with each and every control encountered. Why is it that they still behave like the customer doesn't matter? Because in each division there is at least 1 fat cat who is more concerned with their own well being than anything else; and someone who profits by their actions does their level headed best to keep them there.
Whatever happened to quality of service?
"Can there be a Klein bottle that is an efficient and effective beer pitcher?"
personal development. it takes training to become good in your field. they have in-the-house training.
Spoken like someone who has either (1) never hired or dealt with a Big 5 consulting firm, or (2) someone who works for one.
The partners in these firms will sell anything. They will claim expertise in any thing they must in order to start billing. They send out green college recruits and charge over $100/hr for them.
As for "in-the-house training", the only experience most of them have is a series of failed PeopleSoft deployments (or Ariba, insert your own "enterprise" software here). Many of them have degrees in something completely unrelated to the project. Yes, the English major from Duke is very intelligent, but she doesn't know anything about the project at hand.
-they pay hard on their mistakes. Cash. When you buy outside your organization your excpectations are higher
Well, they may pay for their mistakes, but they certainly don't pay cash. These guys will absolutely bleed a company through continued billing regardless of how successful the project is.
While this is a sad story--especially about the poor guys with Indian "consultants" following them around asking a zillion questions about how to do their jobs--it's worthwhile to remember where the article appears: CIO magazine. CIO is focused on the needs/wants/interests of the guys in ties in a corporate IT environment--and in general a lot of CIOs think that outsourcing/offshoring is a hell of a good idea. The general tone of this article is "look at how these yobbos bungled the implementation of Siebel CRM." What they didn't mention at all is, "look at how these geniuses totally misunderstood their business, and pissed away roughly $40 billion in stock capitalization in just three years. And therefore died the death that they so richly deserved."
It's the technology, stupid...
There are companies, even in the 21st century, that can ignore cutting-edge technology. You don't need to be e-commerce enabled to be a plumber. But if you're in the wireless telephony business, in the midst of a headlong rush into a blizzard of new technologies, the core focus of your business isn't marketing or sales or re-carpeting the executive suite. Your core focus MUST be on the technology--and as soon as you lose sight of that focus, your competitors will consume you.
And these geniuses decided to offshore 3,000 jobs. And were doubtless shocked--shocked, I tell you!--to hear that employee morale about the developers was down.
I'm no techno-protectionist
I remember discussing the inevitable introduction of competition from overseas back in the late 1980s, and debating the possibility endlessly while working in Japan in the mid-90s. There will be companies that decide that, in their businesses, in their business models, IT work is a cost, not an investment. They will decide that they want to minimize that cost. They will focus on maintaining existing systems (with marginal, incremental improvements) and eschew major new developments. They will find that that approach may make it feasible to hire developers in the Third World. But those businesses that do so are making a conscious, deliberate decision: we're not going to focus the company on technology. We're going to try to minimize the company's dependence on technology. IT is a cost--it does not contribute to revenue.
For a wireless telephone company to take this position is simply insane: they are in the technology business. They are smack in the middle of a global technology race--one of the few technology races with competitors from practically every part of the northern hemisphere. They need to be faster to market with new products; the new products must be faster, better, more efficient, and more effective; and they have to have a world-beating customer service experience. Instead of fleeing from technology, they should be driven by it. They should be absolutely focused on it. They should be actively recruiting talent to build their strengths....
Because that's what every other company that's focused on technology is doing. Subcontracting out your technology--in a technology business--is sort of like farming, but buying all your crops at the supermarket.
I am not a lawyer...
But I am an engineering team leader at a U.S. electronics company that leads the world in our industry: lighting controls. We export electrical and electronic equipment to countries around the world--including Japan, South Korea, China, Taiwan, Singapore, Australia, and every country in Europe--because we focus on five core principles. And Principle #4 is "Innovate with high-quality products." In other words, we're in the technology business, so we focus--relentlessly--on the technology.
Once upon a time, AT&T did too...
AT&T Wireless was spun off from AT&T--but the corporate heritage is obviously there. And AT&T, once upon a time, ruled the world--literally chan
I am the CTO for a large enterprise software company (>$1B).
I spend about 30% of my time in front of the IT departments of the largest companies in the world, all of which are household names. They almost all tell me two things about our software:
1. It is heavily modified (they all have source)
2. They wish it was not
The fact is that these large customization projects, particularly ones which involve the Big 5, are over budget and late by factors that would boggle the minds of most mortals. It is not uncommon for these companies to spend >$100M for a software upgrade ON A SINGLE SITE. These companies have hundreds of sites.
As a contrast, another $9B electronics company I met with a few weeks ago can install a complete factory, including financials, manufacturing, logistics, scheduling, human resources, and reporting, all in less than 6 weeks. They have done it over 100 times. How do they do it? They have the entire cookie-cutter system burned on a DVD. Literally no customization is allowed at the plant level.
The only way to be successful at these kind of projects is to use an axe, not a scalpel. AT&T Wireless tried to use a scalpel. They should have thrown out all that junk and started over.
I would also point out that if you read the CIO's biography, he is an advisor to HP. Notice that they also chose HP as their outsourcing partner!
Can you say "conflict of interest"?
About six years ago, I was a contractor at ATTWS in Paramus, NJ, working on the deployment of their point-of-sale cell phone activation app.
Without a doubt, it was the most dysfunctional office of a fortune 500 company I had ever seen, and I've seen a few. There were about six absolutely brilliant people there, who I would be glad to work with anywhere else, and a few hundred that I wouldn't trust to flip the proverbial burgers without putting someone's eye out.
Just one example: there was a pointy-haired middle manager there who liked to gather about $2k/hour worth of consultants into a conference room twice a week and just expound at length upon his management philosophy.
I was required to attend meetings on "planning my career path at ATTWS". I was a *contractor*, and I had work to do that didn't include making busy work for HR drones who didn't grok that ATT was a CUSTOMER, not a CAREER for me.
I even went to a meeting (again, mandatory), to hear some blithering bureaucrat tell us about ATTWS's process for developing processes. (I swear, I'm not making this up.)
Thank Judge Bell for opening those clowns up to competition. Somebody had to eat their lunch, and I wish every other cell phone company the best of luck.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Has anyone ever checked out the AT&T wireless coverage maps? Your 'local calling area' (the places where you don't roam') are like a shade of orange lighter than the 'roaming' area. So to many males and people with less than good eyesight, it all looks the same.
Check it out on your own at http://www.mlife.com
I'm convinced that some companies just have a dysfunctional corporate culture that's immune to real reform. Their only hope is that things get so bad that all the top idiots lose their jobs -- and they're very, very lucky in choosing their new management. (That's basically what saved IBM.) But AT&T's so far gone, not even a total shakeout can save them.
namely, denial. the only piece that works is the old long lines department, now ATT business data. everything else with the "death star" logo is useless. outsourcing the people who are supposed to save you is the latest ATT lunacy, capping a string of them all the way back to divestiture.
thank god the baby bells got freed from that mess. all the folks vying to lead ATT in the 90s -- joe nacchio, mike annunziata, leo hindery, c. michael armstrong -- turned out to be a shitspread at their respective next stop in employment at the top of the tower. "little mikey" in particular broke up and sold his company down the river in several stages, then left it to hide out at comcast and count his money. "joey nachos" almost killed qwest, a fiber startup, and USWest together after he merged them to bleed the treasury at USWest. annunziata and hindery rode Global Crossing into the toilet, and hindery got into another telco startup and crashed with it.
moral: if you want to invest, check for former ATT execs on the board of a company. if you find any, flee in terror.
if this is supposed to be a new economy, how come they still want my old fashioned money?
HP used to make extremely well-made products. Then they shifted their focus, from serving customers to serving stockholders. Now they produce garbage. I was in Fry's today and overheard a lady returning a cheap HP printer she had foolishly spent money on. "Their customer support is horrible and they're all from India!" I almost burst out laughing. Sucker! NEVER buy anything from HP. They've been coasting on their name for years and they're slowly grinding to a halt. Soon "HP" will be synonymous with crap and they'll have to change their name to something like "Claria" the way Gator did.
If it weren't for the printer ink racket they're running, they'd have gone under long ago. What a sad end to what used to be a great American company.
Did you read the quote you posted? The planning had begun, and people saw the signs... while they were up against an immovable deadline on number portability. No doubt D&T can share the blame generously, but frankly, when you're counting on your employees to complete a very difficult job is not the time to be plotting to fire them all and replace them with cheap knock-offs.
Don't you wish your girlfriend was a geek like me?
I'm waiting for a new book along the lines of "The Japan That Can Say No" (sans the jingoism) -- about a new breed of CIO and IT manager that can push back when told to accomplish the impossible within an unrealistic timeframe.
Part of the problem is, I think, the New Success Story psychology. CEOs are so brainwashed into thinking that accomplishing the impossible is what defines you as a successful CEO, that they push their people to do absurdly difficult things in the most miniscume timespans. It's not doing the impossible that's a hallmark of a good CEO, it's doing the possible well and doing the impossible when you HAVE to -- not because it'll win you bragging rights.
(Of course, the whole question of what constitutes a "have to" in this case is probably open-ended.)
Honorary Member of Jackie Chan's Kung Fu Process Servers
"Big government" is one of the political cliches I get really tired of. Anything you dislike about what the government does you can conveniently label as "big government". If the government won't let you burn your leaves, and you think that's dumb, it's "big government". But if you care about air pollution, it's government doing it's job. Your necessary program is my "big government".
You're entitled to criticize what the government does (indeed, it's more or less your obligation as a citizen!). But if you hope to actually accomplish anything, try to make your criticisms based on specifics, not vague, subjective terms that mean whatever you chose them to mean.
"Yes, AT&T Wireless (now part of Cingular) had its share of problems. There's no doubt about that. But at they made the right decision to choose GSM. Verizon and Sprint PCS chose wrongly, and so they are destined to fail."
Sprint PCS and Verizon use CDMA2000, hereby referred to as "CDMA".
Right. CDMA must be worse because it handles more users per cell, right? Or is it because it doesn't have hard cell-size limits? Or... how about the fact that it copes with noise better. Or the fact that it uses less power to go the same distance. Or is it that the voice quality is better?
Look, GSM has some advantages (worldwide standard, SIM, cool phones), but CDMA is fundamentally the better technology. That's why the new GSM (UMTS) system uses CDMA technology.
Verizon Wireless is doing great. So is Sprint. Verizon is 2nd to ATT/Cingular. And most of ATT/Cingular's customers are still using IS-136 (D-AMPS). Sprint has captured a stunning market share with there relatively new network.
"There is only room for one mobile phone technology in this world, and it's not CDMA. I know the US government is behind it, but they cannot force us all to use it."
Right. Just like there is only room for one operating system. Just like there is only room for one political opinion.
The US government *is* behind the usage of CDMA in the US. But it's not because they mandated CDMA. Far from it. In Europe, GSM *was* mandated. Mandating GSM had some advantages - Europe had a fully-digital system with good coverage far before the US did (part of that has to do with population density).
But not mandating GSM also had advantages in the US. We had competition between formats. CDMA was developed and implemented because carriers had the ability to choose the best standard.
GSM is, realistically, not the right standard for the US. GSM cells are too small for rural areas - much smaller than AMPS cells. The carriers who have deployed GSM in the US have learned the hard way that covering Wyoming or Kansas with GSM cells is extremely difficult.