gridMathematica Announced

simpl3x writes "Mathematica for grids was announced at Comdex. It offers support for the usual platforms--Windows, OS X, Linux, and Unix--and offers the ability to use heterogeneous OSes. I haven't used the product in years, but cool nonetheless. Does an off-the-shelf software package, which is scalable as this is provide competition to custom packages--is it easier to add machines than develop custom programs?" And just when you thought Comdex was good and dead.

Comments

distributed functions

[BadDoggie]

According to the Features section, it includes:

• Support for multiprocessor machines, heterogeneous networks, LAN, and WAN
• Support for scheduling of virtual processes or explicit distribution to available processors
• Support for virtual shared memory
• Support for synchronization, locking, and latency hiding

It looks like they took a few pages from the various distributed projects (United Devices, distributed.net). I can see this being used for universities and some research and scientific institutions, but still, with processor power today, even Mathematica representations of 10-dimensional Calabi-Yau spaces can be rendered in minutes.

woof.

Re:distributed functions

[orcaaa]

Actually, Mathematica, in the GUI mode, has a lot of processing overheads. And performing complex calculations can take a while. I use Mathematica regularly, and a class assignment took me 6 hours to "compile" on Mathematica. (It was about modelling proteins and showing how they evolve with time). If it indeed takes such time, then this kind of grid computing can be a boon. I can see this being used a lot in the Universities, where usually, there are a large number of computers not being used simultaneously.

Re:distributed functions

[f00zbll]

Back in 95-96, these types of calculations for large complex proteins were typically sent to UCSD. In fact, most of the grad students working with these types of calculations all sent their jobs to UCSD supercomputing facility (if they were lucky). Some of the ones that I've seen took 1 week to finish. Not that I understood what I was looking at, but gridMathmatica should make it easier for grad students to use resources on their campus. Getting time or slots at UCSD's supercomputing facility took some serious butt kissing and wrangling. Atleast that's what I saw first hand. It may have changed since the 90's.

Cool stuff... but I thought research labs

[f00zbll]

already have their own cluster, and grid systems? This should mean some small junior college or state college w/o tons of government research grants may be able to even the playing field. With the reduction of cost, it begins to make it easier for smaller research labs and schools to build grids. I remember assisting graduates studens prep processes so that it could be sent to UCSD's supercomputer. Now more universities will have their own system and be able to utilize their computer labs as grids at night. Atleast in theory.

Re:Cool stuff... but I thought research labs

[Lars Arvestad]

I thought research labs already have their own cluster, and grid systems?

Many do, but they are somewhat hard to utilize sometimes. For instance, in the lab I am at now, they have a well-sized cluster which in its first incarnation used double-CPU nodes. It turned out that usually, only a single CPU on each node was used. This was because the applixation that people mostly were running didn't support using more than one cpu.

In this case, it has probably (I am note a user myself) been a headache running Mathematica scripts on multiple nodes. I guess it means writing small Mathematica scripts that are then distributed across the cluster and a perl script or something collects the results and merges them. Being able to write the whole logic in one environment must be a big step forward.

One of the cooler things here though is the heterogenecy. In the organisations I have been, there have been enourmous computing power in administrative PCs running windows that no-one has been able to really take advantage of. At least without a big effort. gridMathematica may actually provide an easy way of tapping into that resource.

Re:Cool stuff... but I thought research labs

[capt.Hij]

It is not uncommon to find a cluster of networked computers acting as a distributed system. They are usually implemented using things like MPI or other message passing protocols. The programs tend to be written "in house" and are generally written for specific applications.

Grid Mathematica is different because it is a commerical application, can be used for general problems, and is used for symbolic computations. This makes it much easier to implement an algorithm on a distributed platform. The results may take longer than a program written specifically for a particular application, but distributed environments can be exploited by many more people with this product.

question : OSS/free project in this space

[wfmcwalter]

Is there an open-source or free-software product in the Mathematica / Matlab / Maple etc. space ?

How to the free solutions, if they exist, compare with their (darned expensive) commercial bretheren in general, and in particular is there anything like grid support?

Re:question : OSS/free project in this space

[starseeker]

Probably the most advanced open source project competing with Mathematica et. al. is Maxima. It's a spinoff of Macsyma, which was the first symbolic integrator. Originally developed by MIT, it's got a lot of features the other programs have, and a few they don't. It's got some bugs, but is under very active development.

It's major weaknesses currently are in the GUI and documentation department. TeXmacs can do a decent job of providing a nice interface, but it still won't measure up to Mathematica, which can handle 2D input and output. The default interface is a Tcl/Tk program, which is OK but pretty basic. My prefered way to use Maxima is through emacs - it has a very good emacs mode called emaxima.

As far as grid support goes I'm not aware of anything. The project isn't really to that stage - it's currently working towards a stable 6.0 release which fixes all known mathematical bugs. Then comes feature extensions and new GUI work. That would probably be the point to start thinking about grid support - basically someone would have to decide they wanted it enough to impliment it. The usual open source thing.

Re:question : OSS/free project in this space

[Cyclops]

Hi, you can always checkout octave which is provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab.

Re:question : OSS/free project in this space

[bezza]

There is no reason why a Matlab clone couldn't be made (it is just a interpreter with some built in numerical functions that have already been developed). Anyone can write functions to evaluate expressions numerically.

Maple on the other hand is the most amazing piece of software I have ever used because of its ability to deal with variables etc exactly like a human can. I am studying for my finals right now and I use it to do some of the more tedious work so my study is more efficient. Calculating the exponential of a matrix is tedious at best but Maple does it with ease. I don't believe a product like this could be made in the open-source evironment...a massive amount of research would have to be undertaken and this would require a heap of money, as no methodology could be taken from the Maple team itself.

I am not trolling but the open-source community is much better at creating a (usually better) alternative to existing software with an obvious algorithm or method rather than investing money into computing theory like a clone of Maple would need.

A similar example would be linux desktops...take KDE or Gnome...great desktops, but most ideas have been taken from either Windows or the Mac OS'es, who have sunk millions of dollars into market research.

As I said, I am not trolling, and am open to be proven wrong. Prior similar examples etc.?

Re:question : OSS/free project in this space

[starseeker]

If you don't like the current gnuplot you might want to try cvs. The pm3d mode makes 3D plots look much nicer, and cvs also allows interactive rotation of 3D plots with the mouse (in Linux, anyway. These features haven't been translated to Windows as far as I know.)

Re:question : OSS/free project in this space

[starseeker]

Sorry to reply to my own post, but one thing I should add - while I don't know quite how this relates to grid support per say, there is a lisp interface to the pvm library at http://www.symbolicnet.org/ftpsoftware/cl-pvm/ which help with getting Maxima to do this stuff.
It is free for noncommercial use - I imagine you could contact the authors to discuss other uses.

Part of the readme:

The CL-PVM package consists of a set of Common Lisp functions that interfaces Common Lisp (KCL, AKCL, or GCL) to the C-based library of PVM. Generally, there is one CL interface function to each PVM C library function. The CL function calls its C-based counter part and relays data to and from the C function. This interface is complete and allows Lisp-based programs to take part in a PVM arrangement and thus facilitates the combination of symbolic, numeric, graphics, and other useful systems in a distributed fashion.
CL-PVM also offers a set of tools to help use it effectively with Lisp and MAXIMA tasks. Documentation, on-line manual pages, and examples are also included.

Re:question : OSS/free project in this space

[call -151]

The most impressive open-souce computer algebra system is Axiom, which has a homepage here.

The compiler Aldor for Axiom is available for download. Axiom was developed at a number of places, including IBM, and it being released as open-source is something that has only been finalized in the last couple of months, so the distribution is just beginning to be more widely available.

Re:question : OSS/free project in this space

[rutger21]

I haven't seen this being suggested here yet, but R for statistical computing (link) (GNU 'S') is not only open-source, but also used a lot in several scientific fields, such as statistics and machine learning (books have accompanying source code in R). It has loads of packages which allows you to do all kinds of stuff.

Re:question : OSS/free project in this space

[bcrowell]

Can you imagine if someone invented a computer language that was 100% proprietary, and there was only one vendor for the interpreter? How many Slashdotters would use it? Well, that's what Mathematica is.

Wolfram recently wrote a book claiming that he could explain the whole universe using cellular automata. The problem is that it's all expressed in Mathematica's notation, so at least one reviewer I know of ended up saying essentially, "It looks like crap, but I don't feel like learning a proprietary computer language just so I can check out all the details."

My personal experience with Wolfram has been a pretty good example of the abusive relationships you can get into with proprietary software vendors who have no morals. I paid for a copy of Mathematica, which stopped working when I upgraded to MacOS 8. Tech support's reply was that if I wanted to keep running Mathematica, I could pay for an upgrade to a newer version of Mathematica.

Stuff like this makes me really grateful for Maxima.

Students

[Omkar]

Although this would be expensive, couldn't Wolfram set up a subscription service? Students who need temporary access to the power of Mathematica (I'm thinking of doctoral theses) could but computing time.

On an unrelated note, Integrals.com is one of the most useful high school math sites ever (up with Ask Dr. Math. It ended two weeks of misery by telling integral(sqrt(1+x^-4)dx) is not an elementary function.

Re:OT: Not to be anal-retentive, but...

[BadDoggie]

From the Supported Platforms page (first link on the Specifications page):

• Windows 95/98/Me/NT/2000/XP
• Mac OS X
• Mac OS 8.1 or later
• Linux (Redhat 7.1 or equivalent) or later
• PowerPC Linux (Yellow Dog 2.1 equivalent or later)
• AlphaLinux (Redhat 6.2 or equivalent) or later
• Solaris 8 or later
• Compaq Tru64 Unix 5.1 or later
• HP-RISC HP-UX 11.00 or later
• IBM RS/6000 AIX 5.1 or later
• SGI IRIX 6.2 or later

If the call is for "Redhat 7.1 or equivalent", I can't think of any reason that a distro with kernel 2.4.2 or later wouldn't work.

woof.

bah

[grub]

No matter how much horsepower I put behind Mathematica, it still gives me errors when I divide by zero. My employer didn't spend zillions of dollars on SGI Origins just to get errors. Can't Wolfram include some sort of Clippy helper? /sarcasm

Re:Yes, but...

[BabyDave]

No, but if you scale up enough, it can emulate a cellular automaton using the Universe ...

discount

[sstory]

wonder if i'll be able to get a $130 gridMathematica for Students version. :-)

free alternative

[g4dget]

You can get a copy of Numerical Python and run PyPVM or PyMPI with it for distributed computing.

I think for numerical computation, that's technically actually a better environment than Mathematica. And, while I'm not usually one to harp on the fact that free software also doesn't cost money, given the steep price of Mathematica, in this case, the money saving aspect really does matter as we..

sure: plenty

[g4dget]

For interactive symbolic manipulation, Maxima is an excellent open-source alternative. For numerical applications, Numerical Python and its associated packages beat both Matlab and Mathematica in my opinion. For 3D visualization, you can get VTK, which also has Python bindings.

Maxima is also used occasionally as a rapid prototyping language, but it's proprietary and it has a lot of rough edges. You are probably better off using one of a number of open languages with similar features, like Scheme, OCAML, SML, Prolog, or Haskell.

Don't forget about C++, however. In many ways, C++ nowadays allows you to write numerical code more naturally than any of these other languages (yes, better than Matlab and Mathematica), it has by far the best libraries available for it, and it gives you excellent performance. And you can even do symbolic mathematics in C++, with the right libraries (though it's not interactive, of course).

The main problem is LICENSING

[jki]

The company I work for has some competence in grid computing, and we have a platform that we have tailored for some customers. There would be a gazillion of END-USER companies interested in utilizing grid/meta computing but a high percent of them are faced by the same problem: LICENSING methods which do not take the need for gridcomputing in account. Even if a computer only works as a processing slave providing computational resources, for many types of software from many vendors the end-user company still needs to purchase similar licenses as when all the features of the software are used. This makes integrating existing software with a gridsolution just to enhance performance less favorable than buying huge amount of memory and CPU power for a single node: because then they survive with less license costs. A license for such software can today cost $20 000 per license for example. In my opinion, this practise should be changed and maybe the end-users should combine forces to make these changes happen and put some pressure on the software vendors. There are cases where companies could do their simulations hundreds of times quicker if they could just afford the licenses.

It is good that atleast mathematica has altered their licensing methods a bit. Maybe this licensing scheme could be used also when utilizing mathematica over 3rd party grid architecture. If someone from Mathematica is listening, I don't mind you contacting me. :)

NeXT did this with Zilla a decade ago!

[mbourgon]

At Texas A&M, one of the local CS students was doing his Thesis on some sort of large math problem in HRBB. He had two choices for writing the code to provide the solution. He could either write it in Fortran and use the Cray Y-MP we had (which, 10 years ago, was a fairly big deal). OR, he could use a high level language and use Zilla and our NeXT lab to solve it. He chose the second.

Amazing to see - you'd tell the Cubes to run Zilla in the background, feed it the problem, and away you go. We had 6 computers in the lab, and you could tell he was working on it when you first logged in - it would be a bit sluggish. IIRC, he later took over the NeXT lab downstairs, which had 30 NeXT "pizza boxes". Not bad, especially for 1991-1992.

And this paper says:
Parallelization: with a NeXTstep application called Zilla, multiple Mathematica sessions may be invoked on networks of NeXT computers to allow the simultaneous solution of different parts of a large problem.

BTW: anyone happen to know if they're doing Zilla on OS X? I remember reading something about an easy way to cluster Macs for performance, but I forget the details. It just involved running a client on each workstation.

Quick Poll

[ackthpt]

For mathematical modeling I use:

Mathematica

LPL

AMPL

I code it all myself in assembler, thank you very much!

Fingers and toes

Another method

CowboyNeal works it out for me on his abacus

Capitalize on the hype

[pridkett]

It seems like this attempt to market something as "gridMathematica" is really a little deceiving. In reality it is more distributed Mathematica. Grids involve virtual organizations, authentication, etc. For more information see Ian Foster, Carl Kesselman, and Steve Tuecke's paper The Anatomy of the Grid.

There are other packages which do very similar things and have a for a long time, such as NetSolve and Ninf which allow you to do cool stuff with most any application that needs computational power.

There is also a Commodity Grid Kit (standard interface to Globus services) for Matlab that should be out soon, more info can be found here.

So for now, I'll just consider this more someone wanting to capitalize on the hype around Grids at SC2002 than anything else. Unless I'm missing something obvious.