Stanford Learns a Software Lesson
Nick Irelan writes "In 1994 Stanford set aside $60 million to aquire the latest financial and management software from PeopleSoft and Oracle. However, the upgrade that was planned years ago is still not complete. Stanford has even begun outsourcing! 'Those who can't do teach :)'."
'Those who can't do teach'
As if the computer science professors at stanford are the ones that set up the financial and human-resources systems.
Three Stanford professors serve on Oracle's board of directors, and CEO Larry Ellison has pledged $10 million to the university as director of the Ellison Medical Foundation. Across San Francisco Bay behind a range of hills is PeopleSoft, which has been fighting Oracle's hostile takeover attempt for the last year.
Seems like there is a bit of a conflict of interest on all sides here. Big surprise that this is an expensive bust...
"Those who can't do, teach"
Last I checked, faculty was not generally responsible for doing IT software upgrades.
"Sometimes I look back and wonder if this wave of ERP software ... wasn't a collective hallucination," says Stanford CIO Chris Handley
That would have been Berkeley then, no? Home of LSD and UNIX IIRC.
Nothing disgusts me more in normal conversation than this sort of bullshit parading as wit (its similar to 'kill all the lawyers' being invoked as the wisdom of Shakespeare, with everyone forgetting that the line is a description of the first step in installing a tyrant).
Those who can do, do. Those who teach are doing! You think you learned everything you know on your own? Go tell your parents, your teachers, your professors, your bosses, your friends, etc.
Pardon the vulgarity, but grow some fucking common sense.
"Stumble before you crawl"
how dare you suggest that Don Knuth cant "do"
The One Rule Of Chess You'll Ever Need: Don't play someone who carries a kit in their bookbag.
Another PeopleSoft SNAFU is at the University of Missouri. They have been working on their project for > 5 years and are STILL using their old COBOL-based mainframe system. Millions of dollars down the drain because the pointy-headed academic administrators can't lead their way out of a wet paper bag.
-JT
about being able to do partial rollouts of various systems, keeping loose coupling between them and planning a migration path that doesn't require changes to everything all at the same time. The problem with the "business software" and the required customization, however, highlights the problem with packaged, closed-source software. Open Source software does not require you to be at the latest and greatest version. However, software vendors are often only willing to support the newest versions and discontinue support for older versions.
There will be a great market for companies who specializes in supporting older versions of software that the original software vendor no longer supports.
I mean, after all, it's not like John McCarthy wrote the Oracle financials package.
Seastead this.
'Those who can't do teach'
And those who can't teach, teach gym.
While those who can't teach gym, teach college.
Surely the same institution that came up with a distributed computing software project such as Folding@Home can handle a menial financial and record-keeping software project. If they made their own, using the GPL, then other universities could adopt it as well, and contribute to its development.
I don't know any PhDs, let alone proffesors, who specialize in the pro's and con's of individual applications. Most of them are far more focused on the science behind all of this stuff. They tend to leave the details of implimentation to the folks in industry...
and yes, I do work for a university.
"I'll have a Guinness, no wait, make that a Coors Light" -Grad student I work with, who shall remain anonymous...
Those who can't do teach...
/., but none as ignorant as this. Teaching is one of the most admirable things a person can do as it gives back to the community in every way, shape, and form. Those who 'do' learn from those who teach.
I've read some ignorant things on
As a student I actually think that it is much more true that "those who cannot teach 'do'" rather than vica verca. Get some common sense before saying somthing extremely STUPID like that.
In my own experience with PeopleSoft at a major university, let's just say it can be rather frustrating. Yes there's lots of useful functionality BUT, the forced upgrades are more trouble than they seem to be worth. And some processes that ran perfectly on the old systems are glitchy as all hell now. And there's not much we can modify - just have to wait for the next so-called "upgrade".
"We are the first generation to influence the climate and the last generation to escape the consequences." - John McCain
Those who can't teach, do.
Many of those who teach can in fact do, and what the heck do you think teaching is? Is it not doing?
However many that can do, can't seem to teach. Which is why they pretend that those who can't do, teach.
The DC public school system has had a similar project going on for most of the last decade that does not work yet. Also a large database, management, and payroll system. They are actually being advised to give up on it since they are now out of money and the citywide system will do a better job. But they don't even have the money to join the citywide system now. A lot of it stems from unnacountable and incompetent administration for large .edu and government projects that change specs often and insist on a lot of customization which then has to be redone every time they change the specs. They are also only interested in the latest buzzword instead of what works. The companies are all too happy to take advantage of the situation. In the DC case and in some other school districts they purchase systems well in excess of their current and future needs because they refuse to hire competent people for project planning and administration. In most cases the needs fulfilled by these systems could be done with very little customization and be planned and implemented in less than 2 years. Consultants can cost a lot but its a lot less than the cost of buying something that never works. One more reason why colleges are always so behind the times.
I proposed this idea to Clarkson University -- that it should become the first university to commit to 100% open source in five years. The president (Tony Collins) gave me the warm fuzzies and then dropped the idea like a hot potato.
-russ
Don't piss off The Angry Economist
Yup. If he could start with something as mundane as typesetting and come up with TeX, just imagine what he could do with ERP :)
Humoring the author here, what about the professors that do both? Plenty of my professors teach during the day/at night and work at JPL or other research firms in the LA area. Not sure where your ignorance is coming from, but it's quite unfounded about the teaching community, in general.
Those that can't make the news, submit the news!
I just wasted your mod points! HA!
A failed or struggling ERP implementation is no an IT issue. Implementing new financial and business software is very difficult, especially in organizations that require multiple methodologies to manage money. Success requires that nearly every employee change some facet of their job... and when you look at a university that is a staggering number of people.
Fast moving private corporations struggle with ERP implementations - some even go out of business and blame it on the software... when in reality the problem was millions of threads holding gulliver down.
-- $G
The admin people (accounting, personnel, admin data
processing) never talk to the academics. It is just not done.
After a number is major systems at the U of O (over 27 years) I can tell you,it doesn't happen.
The academics may not even be aware a system is changing until their secretary can't log on( or more likely is gone for training).
The biggest problem today in business with respect to software is that people in business don't understand that the reason you have software in a business at all is to make the processes of that business more effective.
Instead, there is a notion that "well, our competitors have it", or "we have to have it or we'll go out of business".
If you're just playing catch-up with your competitors, you aren't. There's certainly no innovation going on in your company, and beyond that you have no competetive advantage. That would be "stuff that makes you DIFFERENT".
So -- there's a fundamental perception problem. Since transitioning from a relatively advanced-thinking commercial development shop to an insurance company almost 10 years ago, I've been seeing this problem.
Given all of this context, the quote toward the beginning of the article by the Stanford CIO shows that Stanford also doesn't get it:
"Just buying the software does not solve the problem. You have to change the institution, and that's something Stanford struggled with."
No. You write (or buy/obtain if it's commodotizeable, like word processing or web servers) software that works to make the processes that you have more effective. Sometimes you need to make adjustments to have them work together. One case where you'd need to change is if you had a team of 50 people that did nothing all day long but go and pull index cards out of the card catalog in response to user requests -- putting in a database would require them to change this task. But overall, the process would be much more effective. Looking for a book (in this case) would remain functionally the same sort of thing.
The problem with software of this nature, or any "black-box-off-the-shelf" core business software is that it always comes with its own agenda regarding what the core processes of the business should be. To implement, the business has to change the way it does business in order to map into this new set of processes. AND often pay millions of dollars for the privilege. So, the business has just lost some of its competetive advantage (distinctiveness), AND has to pay a BUNCH per month. Plus they all come with maintenance fees now. On top of the original ridiculous price tag.
Why don't these businesses just write their own, you may be asking? Sadly, the answer is rather simple. In order to find out what you need the software to do, you need to get the users together and find out from them what they do.
First, this will take time. Generally, in a business, if you stand up and say "I have time to be able to do this extra thing" it translates as "because I don't do anything anyway", which is managerial for "I am an expense that produces nothing, fire me". So people don't like being put in that position. Second, it's human nature to not have a good idea what it is that you are doing. Go read about contextual design for discussion on this subject, and ideas on a method of getting around it. Suffice to say, people don't give good information when just asked -- they need to be watched. Which is time intensive (see 1 above). So, even if you get volunteers, unless you use the special tricks, you get bad information. Which leads to an incorrect product. See the last 20-30 years of "the software problem" for references here.
Sounds like a bottomless pit. The way out seems to me to be to get the users educated as to why the software need exists in the first place, then once they're educated, get them motivated to work together to discover what the software needs to do.
Easier said than done. Here're your shovels, get digging!
The University of North Texas is about 60% through our own migration from mainframe to Oracle/PeopleSoft and I have to say that the transition is going quite well so far. We are already done with financials and inventory and many other parts of the system and are going live with registration this coming fall term. Projects are being completed mostly on-time and with relatively few problems. Now, our team did a tremendous amount of research before getting into this and knew much about the problems at other universities. It seems the problem is not the software, but the tendency of these organizations to continue doing business in their old ways. They try to force the new software to behave much the same way as the legacy systems they are trying to replace. From what I can tell, the problem is not with PeopleSoft or Oracle, but the universities themselves.
Ouch! The truth hurts!
I've heard about many ERP nightmares, both with Peoplesoft and SAP. Even when they work, the projects are always incredibly expensive.
What's so frickin' hard? I am a programmer, and I know how hard programming is, but (correct me if I'm wrong) the goal of ERP is to use a single integrated program to do tasks that have been written a million times before: accounting, payroll, inventory, etc.
I can't help but believe that the problem isn't on the technical side but the business side: each organization has an idiosyncratic way of doing business and believes that it's cheaper to write custom software, or expensively adapt ERP software, to its specific goals, rather than doing things in a standardized way that can be assisted cheaply by standardized software.
When you bring a program like Quickbooks into an office, you're expected to do things its way, because "its way" is a collection of well-understood accounting principles. The more you try to customize it, the more likely that it is you are simply doing the wrong thing.
ERP is, to my understanding, a scaled up version of the same thing. The scaling will always make things difficult; large organizations are going to be more different from one another than small ones. It also presents performance and reliability issues.
Still, I've heard of so many failures costing tens of millions of dollars with these programs that I start looking to blame something other than the software and software developers.
He [Handley] notes that every ERP package he's worked with--Oracle, PeopleSoft and SAP--has a single ship-to address in the purchasing module. That's great for a company like IBM, which is organized around a central receiving unit, but ...
No, it's not even great for a large public company. It's unbelievably stupid. These vendors are getting the big bucks for massive ERP products containing everything but the kitchen sink, but when it comes to shipping and receiving, they typically just tack on a ridiculously simplistic toy module so they can add "Shipping and Receiving" as another bullet in their marketing materials.
Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
This seems to be a normal thing... Three large Norwegian universities (the universities of Oslo, Trondheim and Bergen) signed up for a brand new personell management and whatnot system from IBM 5 years ago. It's still not working and has caused a lot of trouble for the universities.They were actually at one point unable to pay their employees.
Eventually they found out that IBM had stopped development and sold the product to another company, without telling any customers. I understand that they're mad.
The whole project ended up in one large lawsuit where the universities sued Big Blue for NOK 50 million (approx. $7 million). IBM ansvered with a counter-suit for NOK 5 million in damages. The case ended with a NOK 20 million settlement.
Ironicaly it seems they have gone for an Oracle-system after this...
Link to an article about the case, and one about the settlement (both in Norwegian) for those who are interested.
-- If ignorance is bliss, why aren't there more happy people?
Change an institution to match software? Why not change the software to match the institution?
the board of trustees since 1999 has been asked to approve $93.4 million in capital expenditures for applications and infrastructure . The trustees had approved $60 million in 1994 to overhaul Stanford's entire administrative information systems, a project they expected would take five years, even though controller Susan Calandra says some of the projects in the original plan were never started.
For $60,000,000 they should have a custom system that works with anything. Hell, they should have as much for $5,000,000. Now they want 93,000,000 more?
The delay has been caused in part by Oracle itself, which helped Stanford customize the software so heavily?changing Oracle Financials to accommodate the way Stanford redistributes overhead costs across its grants, for instance?that together they broke continuity with future versions of the software, rendering portions of what they put in place unusable.
I can't imagine something so poorly modularized. What's going on here?
The university must cope with what Handley calls "version upgrade gridlock"?installing Oracle v. 11.5.9 requires changing PeopleSoft v. 7.6, upgrading to PeopleSoft v. 8 requires changing Oracle v. 11.5.9, and so on.
Oh, now I see they should have used free software from the get go and done it themselves.
Friends don't help friends install M$ junk.
ERP systems implementations fail due to people and organizations, not due to technology.
Give a university administrator a system she doesn't know or like, and she's not going to put any effort in to making it work.
Give an IT department a mandate that they don't feel they had an adequate role in bringing about, and they're going to blame the technology, no matter what the real problem is.
Slap down a system made for a sane business in front of a university and tell that university to behave like a sane business in order to make the system work... well, it won't work.
Having seen PeopleSoft and Oracle Financials implementations from several angles, I firmly believe that the technology is fine - nothing spectacular or earth-shattering - but fine. The problem lies entirely with the organization implementing.
How to fix this? There's the ten million dollar question. A hint at the answer is this: look at Oracle Financials and PeopleSoft implementations in organizations with strict heirarchical (read militaristic) management. Success rates?
One thing that was very clear to everyone who was thinking, even back then, was that an ERP would not pay for itself and therefore had to be bought on the basis on making life easier. Another thing that was clear was that you had to have a clear idea of how it would be used, and how much it would cost to use, otherwise it would never get used.
I saw the same blindness when i was working for a company that sold custom websites. Mostly we took a cut of advertising, and I suppose paid salesmen commission based on what we all now know is mark to market. At that time the advertising market was dying, and all the tech people, and even some of the managers, knew that the deals would result in no money. However that truth was not useful for the salesmen who wanted large commissions or the upper management that wanted large sales. So deals were put together that cost the company hundreds of thousands of dollars to honor, with customers that made not commitments whatsoever. Of course all this came crashing down.
So, having worked in small business, corporate, and academia, I would say there is little difference in the ability to be blinded by greed and the smooth talking salesman.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
When you repeatedly hear stories of companies having problems installing ERP packages, why should it come as any surprise that an educational institution for which the package is not designed has problems with it?
Personally, I think the person(s) responsible for specifying off-the-shelf software with some customisations should be shot.
I've worked on ERP implementations, heck, I've worked on ERP software development. It's all about providing a sophisticated accounting system with cookie-cutter business modules around it. Everyone has customisations on it, how large those customisations are depend on how far away you are, or want to be, from the template the ERP provider offers. Education is well away from what those templates offer. Probably so far away that you cannot justify the cost of the migration and customisations. That leaves you wondering if someone recommended the migration because it would look good on their CV.
Where's the Kaboom?
There's supposed to be an Earth-shattering Kaboom.
In my opinion, they should use more of their 'in-house capital'. I'm sure they have business and IT related corses there.
By having graduate students have as a final project something like that, they can save lots of bucks on things like that.
Why shouldn't they ? I know my university does. It also shows in a way that if you prepare students in those fields, you are confident of their capabilities, e.g. the level of 'education' your own university provides is good enough for they big companies you are training them for to work later.
It's a 'eat your own dogfood'-kind of thing.
All your Student Info Are Now Belong To India!
Remember this when someone from abroad starts ruining your credit with SSN data stolen from outsourced student records.
Pray that East India doesn't treat you the way Dow treated those East Indian citizens during that little chemical accident a few years back...
--- Grow a pair, liberals... stop letting the Republicans bully you!
They must have thought it would cost too much. Anyone who objects on those grounds should be shown this $150,000,000 vendor nightmare.
The nuclear power plant I used to work for had spent $5,000,000 building custom software for itself with Powersoft tools. It worked beautifully. The administration types thought that it cost too much and fired their programmers with the bone headed attitude, "we are an electric company not a software company." Now they are putting in a fifteen million dollar commercial package. I'm not there anymore, but I'm sure it's going to be a dissaster. You have to wonder if they are going to fire their engineers and clerks because they are not an engineering firm or a filing company.
Just think of how much money everyone would have saved had they switched over to free software in the mid or late 90s.
Friends don't help friends install M$ junk.
A lot of places have had similar experiences. The University of Minnesota (which has one of the largest campuses in the country, though the overall statewide system isn't extraordinary) began switching over to a PeopleSoft system back around 1997. I'm not sure if it is complete yet, but I guess I haven't heard much about it for a few years (but then, I graduated a year and a half ago).
One thing to keep in mind is that productive research faculty tend to be very adept at avoiding committee assignments that have little potential upside and are primarily administrative, such as one overseeing adminstrative computing in a case like this one.
I remember someone who was a reasonable faculty member who had been doing a good job as department chair, who agreed to become chair of a university committee that was overseeing a tranistion to PeopleSoft, in fact. I tried to talk him out of it and it did in fact become the huge morass with fingerpointing that I was worried it would become, but when deciding to do it he was sure this was a straightforward ticket to moving up the administrative food-chain to dean and so on. In my experience, research faculty tend to work much better in environments when the success is primarily determined by their own efforts, and being in a situtaion where you are depending upon an outside entity (particularly one from another (non-scientific) universe, like PeopleSoft or other huge corporate entity) is a recipe for disaster.
The point is that a university is a community and in general, people end up in different roles, perhaps at different times in their careers. Some faculty are effective researchers throughout their careers and would be unlikely to ask or be asked to serve on what I would think of as a "committee from hell," whereas others who are not contributing research-wise are often the ones who feel obligated or are asked to shoulder more of the adminstrative burden. Remember that faculty generally have no particular preparation in adminstration, and it is pretty random as to whether or not anyone works out well.
It's psychosomatic. You need a lobotomy. I'll get a saw.
I was on the Stanford faculty from 1983-1994. There was very little relationship between administrative computing and academic computing at the departmental level. (There was a centralized "academic computing" facility, run as I recall by the same people who ran the administrative stuff, that continued to be used for a while by the older-fashioned people in some non-science departments as others adopted PCs.) Administrative computing centered on an IBM dinosaur that ran a lot of locally developed software. Migration away from a system like that can be pretty rough, with data tied up in peculiar local formats, and a lot of the staff get very invested in it.
Stanford was also rather prone to central decision-making. Around 1983 they decided that every faculty member should have an IBM PC and arranged a cheap deal. (As I recall we paid a modest amount and the machines eventually became ours.) Later, they made a sweetheart deal with Apple and only wanted to support Macs. They were very slow to support Unix systems, even though when I got there in 1983 there were about 150 Vaxen, two running VMS, the rest Unix, and soon after that Suns, Microvaxen, and HP Bobcats.
Administrative computing was a different world, one from the past. Logging in to the admin system was kind of like "Voyage to the Lost World". I can imagine that the decision to go to outside suppliers reflects a lack of confidence in the ability of the internal administrative computing people to do the job.
Ten years for a glorified accounting and payroll system? That's just insane!
Oracle's licence model was (and as far as I know still is) based on number of users, number or CPUs, speed of machine, etc.
So putting oracle onto even a workgroup sized SUN box (E450, V880) can run several hundred thousand a year.
Given the size of Stanford, the requirements for redundancy, many users requiring different database access, I would imagine that the licences alone between 1-5 million a year. That's 10-50 million over the last decade.
There are support costs, need for table locking, performance issues for a large database. Who get's called when the database doesn't come up at 3 AM after it crashes and the system won't roll back?
So 60 million doesn't sound out of line. The customer needs a database that can reliably handle billions of dollars a year, tens of thousands of payroll changes a year as students and faculty change, take on jobs, contracts, etc. Being a university and getting public money (Grants, contracts, etc.) their probably are requirements to maintain financial accountability. And of course, there are privacy laws to limit access.
The 10 year rollout seems quite excessive though.
These ERP implementations fail because each and every part of the existing process is not defined and documented. If the current processes are clearly documented, then they can be compared to the proposed ERP solution to see if it makes sense.
Our company licenses Oracle's complete system. During the latest upgrade to 11i, I looked into the possiblity of using an Oracle module for tracking prototypes in our developement lab. I submitted a complete process definition along with flowcharts and process diagrams. After about a month of communicating with various Oracle departments, they finally admitted that they didn't have anything that would fit.
A clearly defined process saved us from trying to convert our existing in-house system to something that wouldn't come close to meeting our requirements.
Someone mentioned the PHB problem. No doubt. PHBs don't understand the "make it work" step. I bought something, I'm done, right?
Paul.
You are lost in a twisty maze of little standards, all different.
Last I checked, faculty was not generally responsible for doing IT software upgrades.
You must have missed this in the article: Stanford CIO Chris Handley, a former psychology instructor who joined Stanford from PricewaterhouseCoopers in 1999.
Granted, he's not an instructor now but he surely is responsible for fixing the mess and has been for five years:
Handley joined Stanford in November 1999 as executive director of administrative systems. Previously, he directed the national PeopleSoft Practice for Higher Education at PricewaterhouseCoopers. Before that he held positions at the University of Toronto, where he enjoyed a 14-year career as a psychology instructor before taking responsibility for university systems there.
No mention of a CS degree or any technical background, just an affiliation to PeopleSoft? Is this why Stanford has been screwed around by their vendors for so long?
The plot thickens, he's spoken at Open Source conferences! He should know better. I'd love to know what he said.
Anyone known anything else about Chris?
Friends don't help friends install M$ junk.
Yup, spent over 200 million as a capital expence to install SAP to replace our old IBM mainframe apps. Took about 6 years to do, during this time we had to run both systems. The "old" IBM way cost maybe 5 million a year including personnel to run and maintain it.
SAP may be nice, but it ain't 200 million nice. It does pretty much the same as the old system, with little "value added".
As a Stanford employee working for ITSS, here's my 2c on this article. 1) It makes no mention of the Opensource software solutions in place that have saved the university hundreds of thousands of dollars (For example, implementing OpenLDAP as our directory service) 2) The account of outsourcing to India fails to mention the fact that they (a) failed to meet their last deadline (b) Recent deliveries had issues (c) major security concerns about data The article glosses over a lot of real issues, but that is understandable, given the source they talked to.
I'm currently a grad. student at Stanford, and they don't even begin to document the Peoplesoft bugs. People getting incorrect paychecks, all sorts of stuff. The rumor that I heard why they switched in the first place is that the admin/creator of the system wanted to retire, and no one else knew how to run the thing. It's too bad, because the old system was far superior.
Regarding other schools. I know that in Canada the University of Alberta, University of British Columbia, and the University of Western Ontario (all 20,000+ student schools) all went through hell when "upgrading" to PeopleSoft. Is there any alternative to their software that isn't open-source?