Slashdot Mirror

I double majored in applied math and computer science, so I feel quite confident that I am not speaking from a position of ignorance when I agree BasilBrush: The Windows calculator is wrong!

The correct answer is zero, in all of the number systems involved, and no amount of pure math hand waving will change that basic fact. There is no other correct answer for the expression 4^(1/2) - 2 in base-10, binary integers, or any of the standard binary representations of floating point. Fortunately, it's easy to determine the magnitude of the error in this case: it's the answer the calculator gave.

Now, I will agree that an error on the order of 10^-39 is probably not worth worrying about, but it does make me question the algorithm they chose for calculating the root, and I'm going to be leery of using Windows Calculator in the future, especially given Microsoft's less than stellar history with math algorithms. If nothing else, the existence of this bug exposes an ignorance of machine precision, which is a pretty fundamental topic in applied math. After all, if they can't even account for machine precision in such a simple case, how can they determine if their algorithms are numerically stable? Indeed, I seriously doubt the programmers responsible have any idea that numeric stability is even a thing!

PLEASE ACTUALLY READ WHAT YOU LINK TO.
MODERATORS: LOOK AT WHAT YOU ARE CALLING INFORMATIVE.
YEP, I'M YELLING. DEALING WITH STUIPIDITY IS FRUSTRATING.

Excel and other spreadsheets suck at stats:

That is one camp of thought. There are others. Every package has it's limitations

* Burns, P. (2005). Spreadsheet Addiction.

Doesn't talk about never using statistics. Talks about misusing them by pressing them past their limits. "I know there are many spreadsheets in financial companies that take all night to compute. These are complicated and commonly fail. When such spreadsheets are replaced by code more suited to the task, it is not unusual for the computation time to be cut to a few minutes and the process much easier to understand."

* Cryer, J. (2001). Problems with using Microsoft Excel for StatisticsPDF.

Focuses on poor charting in the Excel 95 era. Title should be problems for using Excel for graphing. The article is a decade old. Excel has had several refreshes.

* Pottel, H. (n.d.). Statistical flaws in Excel. PDF

Another article about Excel 97 and 2000. Decade old software. Many flaws since addressed, and new flaws added. Clearly Excel bashing was popular around 2000.

* Practical Stats (n.d.), Is Microsoft Excel an Adequate Statistics Package?

This one suggests it's just fine for the submitter's purposes.

"Excel’s limitations, and its errors, make this a very questionable practice for scientific applications. For business applications where questions might be simpler and precision not as necessary, Excel may be just fine"

* Heiser, D. (2008). Errors, faults and fixes for Excel statistical functions and routines

For a more comprehensive and technical discussion, see the papers by Yu (2008); Yalta (2008); and McCullough & Heiser in Computational Statistics and Data Analysis 52(10).

Gets very technical, and I bet some of those remarks are valid, but if it's important you become aware of and work around the problem. If it's not, there is no problem. If you don't understand what you're asking Excel to calculate and why it might be wrong, it doesn't matter.

The more you go into this, the more it requires specialist training. The idea that just replacing one software package with flaws and features you don't understand with another geekier more difficult product with flaws and features you don't understand is ridiculous. As is moderation on slashdot. The comments are being moderated by monkeys practicing to type up Shakespeare..

This is an awful advice which ignores everything the submitter asked for.

http://www.practicalstats.com/xlsstats/excelstats.html

READ THE DAMN ARTICLE BEFORE YOU LINK TO IT. YES I AM YELLING AT YOU!!!

"For business applications where questions might be simpler and precision not as necessary, Excel may be just fine."

This is an awful advice which ignores everything the submitter asked for. http://www.practicalstats.com/xlsstats/excelstats.html

Excel and other spreadsheets suck at stats:

* Burns, P. (2005). Spreadsheet Addiction.
* Cryer, J. (2001). Problems with using Microsoft Excel for StatisticsPDF.
* Pottel, H. (n.d.). Statistical flaws in Excel. PDF
* Practical Stats (n.d.), Is Microsoft Excel an Adequate Statistics Package?
* Heiser, D. (2008). Errors, faults and fixes for Excel statistical functions and routines

For a more comprehensive and technical discussion, see the papers by Yu (2008); Yalta (2008); and McCullough & Heiser in Computational Statistics and Data Analysis 52(10).

It is no good idea to do statistics with Excel:

* Burns, P. (2005): Spreadsheet Addiction.

* Cryer, J. (2001): Problems with using Microsoft Excel for Statistics. (PDF)

* Pottel, H. (n.d.): Statistical flaws in Excel. (PDF)

* Practical Stats (n.d.): Is Microsoft Excel an Adequate Statistics Package?

* Heiser, D. (2008): Errors, faults and fixes for Excel statistical functions and routines

For a more comprehensive and technical discussion, see the papers by Yu (2008); Yalta (2008); and McCullough & Heiser in Computational Statistics and Data Analysis 52(10)

"Excel is the one app that MS got very right."

Except for the statistics functions

Excel by itsel really is nothing special.

- Microsoft didn't invent the concept (visi corp did, with VisiCalc)
- Excel is full of bugs that have never been fixed.
- Excel is adequate software for doing some simple-minded accounting, printing some somewhat pretty graphics and producing some extremely simple applications with the help of an intern or two, all in a neat package for not that much money.

It should never ever be used in the following situations :

- in lieu of even a simple database
- for doing proper accounting
- for running ever so slightly complex simulation
- for crunching real numbers that have meaning and significance.
- especially, NEVER, EVER for statistics.

I wouldn't touch numbers coming out of Excel with a 10-foot pole. From personal experience, due to the fact that Excel users in 99.5% of the cases are people simply clueless with data.

Moreover Excel's features (and bugs) have been duplicated in Gnumeric, Openoffice, Siag and whatnot for a very long time.

The only reason Excel is in use everywhere even in situation where it shouldn't is simply due to Microsoft bundling it in office. In my experience people require a word processor first, purchase Office to get Word and then often use Excel because it's there.

On another note, you might not want to do too much statistical analysis in MS Excel. (Maybe not in OOo calc either) http://www.practicalstats.com/Pages/excelstats.htm l/ http://www.csdassn.org/software_reports/gnumeric.p df/ http://www.daheiser.info/excel/frontpage.html/

If you just want to do that sure, Excel will do fine. However I've been in many business settings when the available data needed to be massaged a little more, e.g. find averages, medians, extrapolate to next year, find a model that fits, explain away a few outliers, etc.

Quickly Excel is no longer up to the task. If you want to do real statistics you need to be careful with Excel too.

Many people think that basic stats are easy to understand and their data is simple to manipulate. Nothing could be further from the truth.

Excel charts are generally horrible- the default values tend to include extraneous "chartjunk" (to borrow a word from Tufte). It is tedious to get a nice-looking chart from your data, and seems to be very difficult to produce any even mildly-complicated graphics. I use R for charts, and I'm familiar with several linux charting apps of varying degrees of complexity, but I'm not sure whether there are any good OSS apps for Windows.

A much bigger problem, though, is Excel's lack of statistical quality! This website provides a quick overview, with links to some more detailed references. Excel is occasionally accurate for simple analyses, but why on earth would you use an unreliable program for _anything_? The only way to be sure that Excel did your ANOVA or whatever correctly is to redo it in better stats software, and at that point I don't see the advantage.

This is an issue that comes up regularly on scientific mailing lists. Lots of people seem to take the path of least resistance and use Excel for both their analysis and presentation. Ick!

Except Excel is notoriously unreliable for statistics, and MicroSoft seems to be uninterested in fixing the known (for years) problems with distributions, linear regression, etc.

this has an overview and citations.

If you need particular kinds of stats, a tweaked Rweb server might be one option - set it up to run "canned" analyses on user data. You'd have to have a web server available, obviously, plus enough knowledge of R/Splus to set up the analyses necessary.

I have something very much like this set up with a database interface to allow my collaborators to run canned analyses (or any, if they know the R needed) on the most current data.

Okay... having gotten way afield from where I meant to go... be very, very careful using Excel for any kind of stats!

Does Gnumeric have better implementations of Excel statistical functions?

While MS Excel may have an extensive array of features it is somewhat lacking on the accuracy front. At least as far back as Sawitski (1994) various scientific analyses have been critising Excel using phases like "can be judged inadequate" and "it can be deduced that Excel uses an unstable algorithm". However as McCullough & Wilson (1999) note Microsoft has done little to address these concerns. The problems Sawitski found in Excel 4 were still present in Excel 97 and Excel 2000 for that matter. In fact critisism of the accuracy of Excel 2002 and XP in the scientific literature continues e.g. McCullough & Wilson (2002).

To quote the The Gartner Group, "Enterprises should advise their scientists and professional statisticians not to use Microsoft Excel for substantive statistical analysis". Of course if you do not need to do accurate statistical analysis then these problems will not effect you but given that Microsoft knows about and has largely ignored these problems and scientists are the people most likely to check that a given piece of software really does what if claims to do rather than using it blindly, it seems quite possible that similar problems exist in other parts of Excel but have yet to be exposed.

Rather than blindly copying Excel, the Gnumeric team might do better by trying to bring on board some of these scientists who have been testing and critising Excel in order to improve the accuracy of Gnumeric, so that not only does Gnumeric beat Excel on features but also, and far more importantly, on accuracy. See the following links for more info on the problems with Excel, 1, 2, 3, 4.