Slashdot Mirror

Next time, instead of C and gnuplot take a look at R.

It is easy to do analysis and plotting of scientific data from databases. All you have to do is actually use the right tool for the job. Do you have gobs of numerical data that you want to crunch, analyse and plot? Then you probably want Matlab or R. Is that data in a database? Then you'll probably be using Matlab's database toolbox or R's database connectivity to import and export data (based on whatever database query you care to use - you can import whole tables easily if you like). Once the data is imported (which is trivial) you have no shortage of tools to process, munge, analyse, and plot to your heart's content.

Jedidiah.

Another alternative is R, which is much more powerful than anything MS Office has to offer, is Free, and runs on most platforms.

And you can even use R within PostgreSQL with plr.

I kid you not:

http://www.r-project.org/

Take a look at R language.

It has a command-line interpreter, a lisp-like underlying language with *nice* syntax, available on both Linux and Windows (and OS X, AFAIK) and also has some nice plotting facilities.

Surprised there is so little quality statistical software that will run natively on Linux. I know there's R and PSPP, but I'd really like to see SPSS or even Statistica on Linux.

Matlab, faugh. The right tool would be R http://www.r-project.org/

I use GNU R for my heavy lefting. If Excel isn't meant to do statistics right, then maybe it shouldn't have statistical functions! That Gnumeric and others can get much of this right is a sign that spreadsheets can be decent at this--maybe not as good as SPSS/Stata/SAS/R/S+, but still better than Excel.

If people complain about OO.o Calc or Gnumeric not importing their Excel spreadsheets (as OP had done), they are almost certainly using Excel beyond the basics.

I'm not complaining that my hammer can't handle screws. I'm complaining that MS sells toolboxes that claim to be able to deal with screws, but that none of the tools that appear to be screwdrivers actually work. Only it is worse than that: if I had screws, I'd know they had failed--I would see they were stripped. When Excel @#%@#$s up, you haven't a clue.

Gnuplot. Gri. R w/ gnuplot. Octave w/ gnuplot. Asymptote with LaTeX. etc. etc.

I produced many, many, many data analyses and so forth along with heavy scientific charting requirements using tools like that finishing up my chemistry degree. (Gnuplot and octave in particular I got a lot of mileage out of.)

Most of those should be able to export the graphs from your analyzed data into something like a png, eps, etc. that you can then embed in your word-processing program's report/paper document.

Frankly, as a scientist, Word kind of sucks, and Excel is a really shitty platform for data analysis for anything more complex than sophmore-level undergrad labs. At the least, using a dedicated analysis and charting tool or set of tools is like a breath of fresh air after dealing with Excel's cramped, business-oriented data toolset.

Anyone relying on Excel for real statistical work is brain damaged. If it's your management, I'd put them out of their misery. If you need it to integrate it with excel, integrate it with something else.

If you were a real "statistics geek" (whatever that is) you'd probably use a real statistics system like S-Plus or R http://www.r-project.org/ instead of a spreadsheet.

Look< son ;/patronising>. PHP may be fine for some things, but for out and out management of data, as messy or as clean as you like, nothing beats perl. Unless it's a statistical programming language OK? Mind you, those cunts^H^H^H^H^H at NVIVO could make an effort and get off the microsoft juice too.

If these people do any kind of statistical analysis or modeling, they would love the R Project software.

http://www.r-project.org/

Its open source. Its got loads of examples. In runs in every environment (I've got it for Mac in a .dmg, Windows in a .zip & Linux [lots of download options] and it works fine.)

This is a great idea! It's going to help technical application like RKWard (http://rkward.sourceforge.net/, a GUI for the R statistical language: http://r-project.org/), integrate informations from wikipedia. In the field of statistics, help is a big issue. It's quite difficult for F/OSS to compete with SAS or SPSS. KDE/Wikipedia is certainly the way to go to fill the gap.

R is a language and environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories (formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be considered as a different implementation of S. There are some important differences, but much code written for S runs unaltered under R.

The open-source statistics package R makes all kinds of graphs, has all kinds of great extensions for sophisticated statistical manipulations, has a following among professional statisticians, and is in active development. It is also pretty fast for large datasets, too. The language is a bit of a pain to use at first, though.

Pirates like it, too.

GNU plot, as several people have suggested.

If your doing stats stuff, seriously look into R.

ePix looks good, although I haven't tried it.

asymptote is very powerful, although you probably want to do some tool hacking or scripting to make use of it.

Probably the ultimate tool for such things is Mathematica. Costs money, but the student version is feasible and it's a lovely for all this sort of thing.

And, of course, you could try searching freshmeat --- there are many many other tools there.

I usually input all of my data into PostgeSQL, use R to do an analysis and insert the new data into the DB, then use ploticus to pull directly from the DB and create PNG format plots. Couldn't be easier once setup, makes writing conference papers and whitepapers (relatively) easy. If you are regenerating the same style of plot lots of times, ploticus is well worth the effort of setting up the first time.

http://www.r-project.org/
http://ploticus.sourceforge.net/

MATLAB started as a Fortran library it seems. As handy as it is, MATLAB has some dire limitations: its performance and syntax. While certain vectorized operations can be speedy, many folks I know end up recoding in C because MATLAB just crawls for everything else. On top of this the language itself is just plain ugly. It's reminicent of BASIC with random bits of bash scripting and other oddities thrown in to make a patois that is decidedly disgusting.

I myself have switched over to using R for statistical computing mainly because it's nicer to look at than matlab and has some really great statistical facilities built right in. For more analytical stuff, Mathematica or its open source cousin Maxima are definitely a better choice.

What excites me about Fortress is that it's both cleaner looking than MATLAB and has some neat features like traits. I'll be curious to see how it pans out.

R packages (i.e third party plugins) are available from CRAN http://cran.r-project.org/. There might be some packages on other web sites, but these are the ones that have passed the QA tests.

A lot of us can't run it because... - the software support isn't there. No CATIA, no ProE, no etc. Can't be an engineer using Linux alone.
I dont think "a lot of us" need CATIA or ProE.
But it's slower for getting things done because double-clicking an icon is easier than typing /usr/share/baoeu/otehu/ -x -die.
You need more doubleclicks to find the directory alone.
Pressing a flurry of keys might feel faster, but it isn't actually faster.
.. once you discover the [Tab]-Key it actually is.
They've focused on making an OS that's sane to use while attempting to make it stable. And, lo and behold, it works.
Although they are drowning in bugfixes now because of this approach and have fewer ressources left for implementing new features.
Hate to break it to you, but as a long time xfce4 user, XP is still faster. still haven't gotten octave working with gnuplot.
So you have a few things gone wrong with your installation. How often did that happen in Windows? BTW: http://r-project.org/
And so, until linux developers wake up, linux as an engineering tool will not progress.
It might progess, but only after open source has taken some inroads on standard office desktops, which it probably will: http://it.slashdot.org/article.pl?sid=05/03/06/214 1225&tid=185&tid=218

Two other wonderful, and interrelated, open source packages are R, the open source statistical software project, and Biocondutor , basically a set of R functions.

statistics:
R http://www.r-project.org/
WinBugs http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents .shtml

symbolic mathematics:
Maxima http://maxima.sourceforge.net/

numerics:
Octave http://www.octave.org/
Scilab http://scilabsoft.inria.fr/
g95 (Fortran 95 compiler)http://www.g95.org/

Easy to install and test out, and great package tools are onboard.

Recent versions are outperforming Matlab in number crunching benchmarks.

For a symbolic package, I personally prefer Maple to Mathematica.

Two excellent open source programs with a lot of mathematical and statistical power are R ( www.r-project.org) and MacAnova ( www.stat.umn.edu/macanova). Both are used in college level teaching of elementary and advanced statistical topics. R is almost a clone of S-Plus and MacAnova is in the same family but is not closely related. Both are extendable by writing functions or macros.

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

• R, including several add-on packages (such as tseries, RODBC, coda, mcmcpack, gtkdevice, rgtk, rquantlib, qtl, dbi, rmysql), out-of-the box support for the powerful ESS modes for XEmacs as well as the Ggobi visualisation program;
• A complete teTeX, TeX, and LaTeX setup for scientific publishing, along with TeXmacs and LyX for wysiwyg editing;
• Perl and Python with loads of add-ons, plus ruby, tcl, Lua, and Scientific and Numeric Python;
• The Emacs and Vim editors, as well as Gnumeric, kate, Koffice, jed, joe, nedit and zile;
• Octave, with add-on packages octave-forge, octave-sp, octave-epstk, and matwrap;
• Computer-algebra systems Maxima, Pari/GP, GAP, GiNaC and YaCaS;
• the QuantLib quantitative finance library including its Python interface;
• GSL, the Gnu Scientific Library (GSL) including example binaries;
• The GNU compiler suite comprising gcc, g77, g++ compilers;
• the OpenDX, Plotmtv, and Mayavi data visualisation systems;
• it includes apcalc,aribas,autoclass,

As several commentators have suggested, R is a terrific platform for statistical computing. Here's a link to a blog post that, in part, contains more information about R, in particular links to some of the textbooks (both free and commercially published) that use it to do statistics. R is one of those open-source projects that's absolutely first class but doesn't get so much exposure in the mainstream because it's a bit specialized.

For statistics software that's free, you're not going to get much better than R. R is an implementation of the S language; so is S-Plus, but that will cost you. R is awesome for many things.

If you have data of any type and want to easily prepare graphical summaries, R is good for that. For beginning students in statistics, it can look up critical values for all the distributions so you don't have to use the blasted tables. It also has functions for everything you'll see in an intro class (regression, ANOVA), although I'd consider learning those first by hand so you know what the computer is doing.

There's also advanced packages for everything and anything statistical. There's an entire package built with R for analyzing bioinformatics data.

I use R daily for lots of different things, it's really a handy tool. However, if you don't know statistics already, I'd suggest a book called "Introductory Statistics with R" by Peter Dalgaard(sp?). It will get you up and running in no time.

Finally, R is also a programming language which is very Lisp/Scheme like, and makes it really fun and easy to write your own statistical functions. If you have to (or want to) take statistics, just get R!

For graphing, I use gnuplot. It can do 2D/3D graphing and is quite easy to learn/use. For statistics/regression, I use R

There is R
R Project

Have you tried R?

First, I loaded your data into R from The R Project for Statistical Computing:

> ohio
county reg.voters precincts evoting turnout.2004 turnout.2000 bush.swing
1 Adams 17696 35 FALSE 65.94146 60.77620 -0.00219
2 Allen 68174 139 FALSE 69.60278 65.05813 -0.03396
3 Ashland 34847 65 FALSE 69.36322 69.49464 -0.01306
4 Ashtabula 62926 127 FALSE 70.18720 60.81940 -0.01259
5 Athens 45100 69 FALSE 60.49002 53.53627 -0.06889
6 Auglaize 33094 39 TRUE 66.97891 70.44227 0.01753
7 Belmont 44452 83 FALSE 73.18231 60.26522 0.03944
8 Brown 28922 35 FALSE 67.55411 62.55611 0.00865
9 Butler 238117 289 FALSE 67.58022 64.26633 0.07879
10 Carroll 20076 26 FALSE 68.34529 65.92923 -0.01509
11 Champaign 25376 29 FALSE 71.65826 59.84996 0.01343
12 Clark 89683 100 FALSE 75.00641 65.74651 0.03348
13 Clermont 125823 191 FALSE 69.15429 62.39119 0.08463
14 Clinton 25092 32 FALSE 71.21393 63.96370 0.02330
15 Columbiana 78536 103 FALSE 61.24070 60.96343 0.01846
16 Coshocton 22679 43 FALSE 70.03836 68.79806 -0.01573
17 Crawford 29591 46 FALSE 71.95769 62.60209 0.00060
18 Cuyahoga 1005807 1436 FALSE 64.51397 58.06637 -0.43531
19 Darke 38290 43 FALSE 66.68060 65.90556 0.02968
20 Defiance 25847 42 FALSE 68.48377 64.42229 0.00557
21 Delaware 100676 123 FALSE 78.19937 69.83352 0.04064
22 Erie 55517 62 FALSE 69.65614 64.24870 -0.01385
23 Fairfield 91498 118 FALSE 72.54585 67.34156 0.00302
24 Fayette 16093 38 FALSE 71.24215 64.46000 0.00296
25 Franklin 845720 788 TRUE 60.27633 61.26558 -0.68834
26 Fulton 28561 35 FALSE 75.42103 68.82543 -0.00806
27 Gallia 23567 35 FALSE 57.31744 60.89664 -0.00163
28 Geauga 65393 96 FALSE 75.73899 68.72101 -0.03420
29 Greene 105079 142 FALSE 72.50735 67.70133 0.03101
30 Guernsey 27129 37 FALSE 59.59306 64.84132 0.00374
31 Hamilton 573612 1013 FALSE 70.88328 65.58803 -0.54742
32 Hancock 49607 62 FALSE 69.09307 66.81487 -0.00663
33 Hardin 18921 38 FALSE 68.23107 61.67072 0.00914
34 Harrison 11769 24 FALSE 69.18175 66.77524 0.00746
35 Henry 19685 33 FALSE 75.16891 69.13808 -0.00666
36 Highland 28243 31 FALSE 63.31834 63.88105 0.00927
37 Hocking 18369 32 FALSE 70.15080 65.36343 -0.01329
38 Holmes 18089 19 FALSE 60.37371 59.26876 0.00001
39 Huron 37436 55 FALSE 66.53221 58.05025 -0.01538
40 Jackson 23997 38 FALSE 57.92807 55.87854 0.01179
41 Jefferson 49655 91 FALSE 71.61615 64.12859 0.02110
42 Knox 36971 56 TRUE 71.10979 61.14969 -0.00844
43 Lake 160165 217 TRUE 73.72772 67.60981 -0.05749
44 Lawrence 41424 84 FALSE 65.30514 57.18568 0.03291
45 Licking 111387 122 FALSE 69.52517 64.26959 0.03209
46 Logan 29406 52 FALSE 70.48902 61.72690 0.00504
47 Lorain 196601 239 FALSE 69.30941 61.55434 -0.05374
48 Lucas 302136 495 FALSE 70.92137 62.36231 -0.03023
49 Madison 23477 44 FALSE 72.45815 64.42444 0.00847
50 Mahoning 194673 312 TRUE 66.50537 65.10254 0.02792
51 Marion 43323 84 FALSE 65.14092 60.71360 0.02260
52 Medina 118330 149 FALSE 70.33212 66.17253 -0.02282
53 Meigs 15205 27 FA

Let's not forget the indirect costs to the workers. People have Windoze machines at home, and they take stuff home to work on it there. They would be using different programs / interfaces at home and work. The program I work with the most (SPSS) is not available for Linux (or Macs, or anything else). (I have made attempts to switch people in my department to the open source R http://www.r-project.org/ as as alternative, but when they saw the interface, they laughed.

The program is GPLed, and shows that excellent documentation is possible, if enough people.

The program is GPLed, and shows that excellent documentation is possible, if enough people.

The program is GPLed, and shows that excellent documentation is possible, if enough people.

The program is GPLed, and shows that excellent documentation is possible, if enough people.

"The R Project for Statistical Computing"

http://www.r-project.org/

This is an amazing stat program that is open source with a lot of documentation backing it up.

Some of the documentation you can download:
An Introduction to R (approx. 100 pages, 650kB)
The R Reference Index (approx. 2300 pages, 12MB)

Also, under their contributed documentation section, they have the documents sepearted by "Documents with more than 100 pages" and "Documents with less than 100 pages".

If I could run it 100% Windows Free I would, but the University I'm due to transfer to next year insists on everyone running Windows 98SE/ME/2000/XP, Office2000 or XP, and SPSS. Aside from SPSS I'd be totally fine being Windows Free but so far I have found no way to either run SPSS in Linux (neither Codeweavers nor the regular WINE project can do it at this point) but there are SPSS workalikes that are Linux native. Hopefully by the time I have to deal with stats I'll be able to convince my math prof to let me use either RProject or PSPP instead. I am so ready to ditch Windows once and for all.

LaTeX produces the most visually attractive documents out there--there's no reason not to use it.

Let's not forget Donald Knuth for TeX which powers it all, and Leslie Lamport for the LaTeX macros. And of course, Bram Moolenaar for my preferred authoring environment.

Also cheers to the folks behind EMBOSS and those behind the R project. Wayne Rasband for ImageJ, and all responsible for SciLab. Thanks to everyone for making science (more) fun. :)

Let's not forget Donald Knuth for TeX which powers it all, and Leslie Lamport for the LaTeX macros. And of course, Bram Moolenaar for my preferred authoring environment.

Also cheers to the folks behind EMBOSS and those behind the R project. Wayne Rasband for ImageJ, and all responsible for SciLab. Thanks to everyone for making science (more) fun. :)

Anyone comfortable with scripting languages should be able to use the Gnu R statistics package and the GGobi visualization package to get the same effect in a cross-platform, free-as-in-speech way.

Not that Excel has any graphing capability worth shit either, for that matter.

Personally, I use mainly R, and sometimes Grace when I want a nice and simple to use GUI thingy.