Slashdot Mirror
$10B Annual Tab for Spreadsheet Errors?
theodp writes "According to PWC and KPMG, more than 90% of corporate spreadsheets have material errors in them. With each error costing between $10K and 100K per month, one expert estimates corporate America loses in excess of $10B annually through the misuse and abuse of spreadsheets." From the article: "The key point about spreadsheets is that you need to know which ones are critical to your business, which ones are merely important and which ones you do not have to bother too much about. Once you know that, you can start to apply appropriate policies depending on the criticality of the spreadsheet involved."
That's a $10 Annual Tab for Spreadsheet Errors. Misplaced a decimal!
Documents have typos. Film at 11.
people make mistakes.... it costs money.... next please?
This article only brings up losses and problems with the spreadsheet format. I'm sure, however, that there are inaccuracies in company word documents, e-mail, and other forms of communication. How should spreadsheets be any different?
6x9=42
Spreadsheets aren't costing money, any more than pencil and paper costs money. It is the bad math that costs money.
I'd bet a LOT of money that fewer mistakes are made with spreadsheets than by people who think they can do perfect math in their head, or perfect long division or multiplication on paper.
Many people say that the whole idea or spreadsheets is fundamentally flawed because a single error can propagate itself throughout the whole spreadsheet so a miscalculation early on tends to expand exponentially down to the rest.
411 Y0UR 8453 4R3 8310NG 70 U5!! -NSA
One small problem with spreadsheets is that people sometimes use them instead of databases, I guess because the interface seems simpler than making a properly developed interface to a database from the getgo. Then you get locked into the solution, etc.
This was definitely a problem at my old job. They wanted to create a payroll sheet to keep track of hours, and the easiest way to do it was via a spreadsheet. I was the most programming-savvy person there (heh, you can already smell their doom), at so, not having any database training, I created a really suped-up spreadsheet that handled it for them. It was GREAT, until we had a work situation in which some people worked past 12:00 at night. At that point, people's total shift hours came out negative. We got it fixed eventually, but it involved some really nasty calculation, and it was a problem that could have much more easily fixed if it'd been done by database from the start.
abuse of spreadsheets
Is it just me, or is that just a wee bit breathless, from an analytical point of view? I doubt that even "misuse" really even has the right connotation, here. More like, misuse of math.
Don't disappoint your bird dog. Go to the range.
If I undercharge due to a spreadsheet error, then I'm out $N, but someone else is $N better off.
If only there were some consulting company, someone who I could call to help me implement some best practicies, to help me avoid these tragic errors. Do PWC and KPMG know anyone who can help?
I run both perl and excel on my windows computers. Perl versions for windows have been out for a long time.
The scary thing is the suggestion that the IT department should take over spreadsheets. Many people use a spreadsheet for applications that would better be served by a database with the appropriate front end and back end, or a dedicated software application. This article mentions managers specifically, but lots of employees whip up a spreadsheet and throw in some macros then find that the spreadsheet grows to a point of some real usefulness.
It's when the spreadsheet becomes useful that people realize it's not scalable (maybe they don't use that word, but I do) and can be tough to maintain.
Not to single out IT departments in particular, but I think the reason that these spreadsheets start up and grow is specifically that it's often difficult to get someone in another department to understand your needs well enough to make the tool that you really need.
Today managers can't fund a good solution because their budget doesn't allow for the necessary development. Tomorrow they won't be able to afford to get the support they need to get a spreadsheet done.
I don't have a great solution outside of better training for people on how to make spreadsheets that serve their needs.
more of the same on Twitter.
So what you imply is we should overcharge on EVERYTHING and hope the people we're selling XYZ to catches it.
As long as we make mistakes, in OUR BENEFIT, we're ok, right?
ESA has a similar facility, as does NASA Goddard. And from what I've heard contractors like Boeing have experimented with the same kinds of ideas.
I fixed the problem in the spreadsheet, and then dug through all the existing orders that were about to be filled and corrected them. The problem had cost GE about $300,000 and was about to cost them another $120,000 in the next month. The interesting thing, is nobody had really cared to do anything about it until an intern came along, and dumped it on me. They just don't see $10,000 as a whole lot of money in the grand scheme of things, so I'm sure stuff goes on like this all the time.
Although spreadsheets can contain costly errors, so can programs written in any language. I would argue that spreadsheets are a very powerful IDE for a wide class of small problems and can more easily create software with lower rates of errors than other "language oriented" approaches to software development.
The reason spreadsheets provide superior debugging versus language-based software is that they instantly display the intermediate results of the formula every time the inputs or formulae change. Change one number in the inputs and the programmer can instantly see the intermediate and final calculations and do a visual sanity check on the results. In contrast, language-based software creates several impediments such as a manual edit-compile-run cycle, manual/isolated debugging statements, and few easy ways to visually monitor all the values of all the intermediate variables.
Don't get me wrong, spreadsheets have some severe limits. First, they can provide too much power to developers with too little experience/competence. If the developer is an idiot, they are more likely to be able to create a spreadsheet than a program, but just as likely to create (and not find) serious logical error. Programming languages, to some extent, create a barrier that blocks morons (not always). Second, spreadsheets don't scale to large/complex problems very easily. Some of this reflects the monopolist state of the spreadsheet market -- the lack of competition for Excel means that it has not substantively improved in the last decade. (e.g., why is Excel still limited to 256 columns?!?!?).
Two wrongs don't make a right, but three lefts do.
This one has been known for a while, but perhaps the FUD associated with a number like "10 BILLION DOLLARS" (said in appropriately Dr. Evil-ish fashion) could get some attention.
Spreadsheet functionality enables people to bury calculations and they become legacy tools within departments. They are like some of the worst spaghetti code. Someone who may be a serious spreadsheet jock develops a neat tool and it gets implemented in his/her department. The jock leaves, but the tool stays and continues to be used, despite the fact that no one left really knows how it works. Even assuming that there are no errors in it, as circumstances change, the spreadsheet might not produce the "correct" answer, but everyone accepts the answer produced by the legacy spreadsheet because "that's the way we've always done it." And, should someone attempt to modify the spreadsheet, they could get bitten by buried or misunderstood calculations.
Also, spreadsheets enable executives to embed assumptions and play "what ifs" with their forecasts, which is good. But then they use the scenarios they like best to get their pet projects approved using some rather suspect forecasts that "must be true because that's what Excel says the results are."
Spreadsheets are valuable tools, but, like any tool, you can get bitten if you don't really understand what you're using.
John
"The plural of anecdote is not data."
This is money that is "lost" or "gone".
Assuming these errors are uniformly distributed, there are roughly equal numbers of errors in the positive and negative directions. The idea that such money is just vanishing from our economy is flat out wrong.
And even if the errors are heavily biased in one direction, the money is still somewhere, it's just being less efficiently distributed.
How do spreadsheets cost companies money?
... which gets munged or erased due to a
Just about everything in many companies is tracked on spreadsheets. Expenses, costs, estimates, budgets, projects, etcetera so on so forth.
Often times, employees will use spreadsheets when a database (even Access) should've bene used. As soon as the spreadsheet becomes 'mission-critical' and contains information that is used to run the business and cannot be lost, you'll start to see employees whose sole job is to feed, maintain, and munge that spreadsheet. When data's in a format like Excel that can be shakey, you can see data errors start to build up when one page is dependent on another page which is dependent on another page which is dependent on some figure buried back in cell DA256 on Page 5 of the workbook... which is dependent on some other figure
And the worst part is that it's usually impossible to trace these errors back because there's no way to take a step away from it or a debug tool.
(How do I know this? I write custom software for small businesses that realize that they can't continue doing business the way they're doing it.)
--
Vote for your hopes, not for your fears - Vote Third Party
>It is the bad math that costs money.
Wrong, it's not bad math it's wrong use of math. It's more the case of using wrong models to solve problems.
>fewer mistakes are made with spreadsheets
That should read, more mistakes are made faster with spreadsheets. Take a simple example like a spreadsheet to calculate the cost of some project. Lots of places they use a template, filling in some values and the spreadsheet does the rest. Small mistakes in the template can become seriously expensive when all is accumulated.
As I understand it, it's a slow news day can mean time to post things from the totally unsubstantiated category that's always chock-full of stories thanks to the the only way I'll beat the deadline is to make something up effect.
It's simply bad journalism. The author names PWC as a source of the 'over 90%' figure, but PWC in turn was citing some professor from Hawaii who had looked at 54 spreadsheets and found errors on 49 of them. 54 is a sample so small as to be absolutely meaningless and everyone responsible for the story finding it's way here should hang their heads in shame.
Making the world a better place, one psychotic episode at a time.
In other words, the major factor in your switch is your laziness to google for perl+windows?
Rule1 - Never! Never! Never! put your busness on a speadsheet.
Rule2 - refer to rule 1
There is nothing wrong with speadsheets and they are very usefull for analysis but what happens is people try to use them for everything and eventually you have a spreadsheet that is used as a company database.
Yes I am aware you can lock a spreadsheet but how long before someone (usually a manager) makes a "special" change and before long all sorts of "special" changes occur and things start to get rapidly out of synch.
A simple analogy is how may people have ever seen simple Unix groups work really well, now take that one step further to ACL's and it starts to get interesting. This is particularly true when you have many people wanting to make changes. The poor Sys Admin can only duck and run for cover.
>... $165,000 times 9 times 500. That amounts to just shy of three quarters of a billion dollars. And is that anywhere near realistic? No. It is probably safe to say that corporate America, for example, loses in excess of $10bn annually ...
Can You Spot The Spreadsheet Error?
Cell B1 = $165,000
Cell B2 = 9
Cell B3 = 500
Cell B4 = B1*B2*B3
Cell B5 = $10 BILLION
--- Attorneys Assisting Citizen-Soldiers & Families -
The big problem with spreadsheets is that they are increasingly being used to implement software, but that the architecture they provide (a matrix of expressions) makes it almost impossible to validate the code.
If the CFO of $COMPANY produces a spreadsheet demonstrating that all is well with the company finances then it is difficult to prove him wrong.
This may be what went wrong with companies like Worldcom. They could have had one spreadsheet for insiders and another for auditors.
http://michaelsmith.id.au
The big problem with spreadsheets is that they are increasingly being used to implement software, but that the architecture they provide (a matrix of expressions) makes it almost impossible to validate the code.
Looks like they could have found a new line of business: give professional advice how to solve the problems generated by their professional advice, whoops... this is the old consulting business model!
Andrea
I made a couple spreadsheets recently, and here's what I found:
Fragile references
They're still referencing virtually everything by [A-Z]\d+. This is beyond GOTO considered harmful - when Dijkstra made that claim, we at least could do "goto blah" instead of "GOTO 2050".
Excel has a couple "solutions", neither of which are good:
Massive code duplication
In my spreadsheet today, I ended up with whole columns of formulas like this:
I would have much rather made a function FedIncomeTax(AdjustedGrossIncome) that applied that same bracket logic. Once. And called it the N times necessary. You can define VBA functions, but I didn't see a way to reference cells from them. (Probably because it doesn't have a reliable way to do the dependency/error tracking seamlessly. I can think of how I'd accomplish that in Python...but Python is a very flexible language.)
Unreadable code
There's no way to put longer bits of properly-indented, commented code in there. Certainly related to the above; you're trying to cram way too much stuff into a cell (or group of cells) that's massively repeated, so no one even thinks of doing this.
Poor layout
The result looks poor in a couple ways:
Poor charting abilities
It didn't have much support for charts with confidence intervals. (Don't tell me there's no use for these in finance! They may write everything out to the nearest cent, but that doesn't mean they don't made wild-ass estimates when talking about the future.) If you want to do something like a box-and-whiskers graph, you have to do elaborate tricks. Even basic error bars have weird defaults; to get a meaningful confidence interval, you have to do custom stuff with ranges. The friendlier check boxes end up with the same-sized error bar for every point, which is worthless.
Overall
Just using Excel for my small needs was frustrating, and it's not because I don't know how to use it. (I can read Help files.) I can easily see how people would screw up badly with them and not notice.
It'd be so much better if there were a more free-form document (no overal grid) you could throw 1-dimensional lists and 2-dimensional tables into. With support for formatting, referencing, and summarizing them well. (There shouldn't be [A-Z]\d+ references at all; the concept shouldn't exist.) Including the PivotTable stuff, of course. (Excel's one good point, though it could be better.)
It also should have support for referencing external data easily - a RDBMS or CSV/
Just boggles the mind.
"So how much money is the Fortune 500 wasting annually? It is a simple sum: $165,000 times 9 times 500. That amounts to just shy of three quarters of a billion dollars. And is that anywhere near realistic? No. It is probably safe to say that corporate America, for example, loses in excess of $10bn annually through the misuse and abuse of spreadsheets. That's a big number: it suggests a problem worth managing."
Translated: If I take the actual numbers I have, it's a $750mil problem. By some magical feat, however, we can just assume that it's a $10bn problem because it makes my article seem MUCH more important.
How can you possibly just increase the number THIRTEEN TIMES just to suit your needs. Show some integrity.. seriously.
Turn s60 photos into awesome videos with mScrapbook for all S60 3rd edition phones!
A programmer would be chastised for: cutting and pasting code; for not using revision control; and for obscured or hidden behaviour. Yet spreadsheet users commoly have all 3 problems.
The problem is that spreadsheets have become increasingly complex without a corresponding adjustment to formal specification. Imagine if you were using a programming language that doesn't have formal type declarations.
The are tools that assist businesses using visual modelling (much like scientists use) that can help detect and prevent errors in spreadsheets. Have a look at...
XempleX or their product sheet Xemplex Product Overview.
p.s. I'm not affiliated with this company.
The ease of spreadsheet creation is the problem, not the solution. Yes, it allows non-computer-literate managers to create an analysis of a particular problem... but that analysis is often flawed, and it is nearly impossible (for any non-trivial spreadsheet) to figure out where the problem really lies.
I have met several people who claimed to be "computer experts" based solely on their Lotus 1-2-3 / Quattro Pro / Excel expertise. It's all well-and-good to create a spreadsheet; but just like computer programming, you need some sort of development and quality control methodology. Too often (like, in say 99.999% of the cases) there is a single user creating a single spreadsheet that eventually controls some aspect of the way a business is run. There is no quality review; there is merely a, "yeah, that number looks right" phase.
I've seen it too many times. It's endemic in business. I'm not surprised with the results of the study ("Spreadsheets considered bad"), though I'd rank the monetary valuation right up with the report I read 2 years ago, "Slow modems cost US businesses $4B yearly!").
Microsoft is to software what Budweiser is to beer.
Anyone employing Excel for any statistical calculations should get a nice chill from reading any of B.D McCullough's papers on Microsoft's egregious (and mostly uncorrected or corrected badly) errors in this area.
Click here for a link to one of his recent critiques entitled:
On the accuracy of statistical procedures in Microsoft Excel 2003
Here is a nice quote from the above paper:
"...persons who wish to use Excel for statistical purposes should exercise extreme caution...Persons desiring to conduct statistical analyses of data are advised not to use Excel 2003."
"Excel is like a bikini. What it reveals is suggestive, but what it conceals is vital." -- apologies to Aaron Levenstein
This is not bad journalism. It is a serious article about a serious problem. Take people who have never written a program and have never heard of program-design-101, give them a huge collection of poorly documented functions and tell them construct a large complex program to calculate a number. Just how much would you trust that number? It is not hard to imagine what the resulting spreadsheet looks like. Now suppose that number is, say, the value of a mine, or road, or company. If your number is too low, you miss a great opportunity; if your number is too high, you buy the asset and subsequently lose a lot of money. In either case, the loss is serious money- hundreds of millions or more.
I have spent the last two and a half years auditing spreadsheets for (1) complex financial transactions and (2) models for large public infrastructure projects. I work with a dozen other rocket scientists and actuarial types who specialise in this. My experience is consistent with "some professor from Hawaii", namely Ray Panko who is the world expert in the field. Almost every worksheet of every model I have audited, has been riddled with potential and actual errors- and these spreadsheets are written by professionals and have been already reviewed internally. Auditors like KPMG and PWC are interested in whether an error is "material", i.e. big enough to effect the client's ultimate decision on whether to proceed at a given price. The sample size of 54 is large enough to give overwhelming evidence of the large number of errors, and of the proportion of such errors which are "material".
All software has bugs when you write it. Reviewing your code, peer review, formal testing, code reviews help you reduce that. Even with this, how much released software is genuinely free of errors? I think perhaps TeX is. With spreadsheets, it is hard to write clearly and simply, it is hard to review, it is hard to test and you have no comments. There are going to be mistakes, and lots of them. If you are not seeing them, it is only because you are not looking. To make a spreadsheet (or any software) without errors you need to approach the problem like NASA. This, of course, requires a budget like NASA or a horde of open source zealots, and so PHBs and accountants need to decide when the cost of detection balances the risk of error.
john@xq.se
The Aristotelian sage is not free from emotions: he moderates them. -Montaigne