Slashdot Mirror

Telling the troll to spend all the money on defending onself seems to be the best bet to win.

Perhaps you can cultivate a sense of DIY by showing how you can make a microscope using an optical mouse.

I recommend that you visit this course wiki. It contains free material relevant to participating in an open source projects. The course, "Open Source Software Practice" is a taught at RPI. http://public.kitware.com/OpenSourceSoftwarePractice

They have other things. MIDAS, for example, is really spiffy and under a BSD-ish license, and is probably part of why they got this contract:
    http://www.kitware.com/MIDAS/resources/software.html
"MIDAS integrates multimedia server technology with Kitware's open-source data analysis and visualization clients. The server follows open standards for data storage, access and harvesting. MIDAS has been optimized for storing massive collections of scientific data and related metadata and reports. MIDAS is available under a non-restrictive (BSD) open-source license."

More on the sorts of issues any FOSS-oriented progressive company may struggle with (by me):
    "Re: [Open Manufacturing] Open source manufacturing social organization"
        http://groups.google.com/group/openmanufacturing/msg/6819187b74f4b7db
        http://groups.google.com/group/openmanufacturing/msg/fa4459793c6b7ed3
    "Jobs at Materialise 3D in the Ukraine; thoughts on social change"
        http://groups.google.com/group/openmanufacturing/msg/04fbdf60ad463dbb
    "Beyond a Jobless Recovery: A heterodox perspective on 21st century economics"
        http://knol.google.com/k/paul-d-fernhout/beyond-a-jobless-recovery

But I might just be saying this because I live not too far from them and maybe I'll need a job there someday. :-) And they might have the contacts and social infrastructure to get this project better funded: :-)
    "The need for open source sensemaking tools (Score:5, Interesting)"
    http://slashdot.org/comments.pl?sid=1746980&cid=33177866

By the way, Kitware also develops CMake, the build system used by Second Life and KDE.

For a real nice cross-platform build system with more of a C++ bias, check out CMake + Dart. I've recently been forced into developing on a Windows desktop rather than Linux. I've got everything running smoothly under cygwin, including nmake VC++ (one of CMake's targets) builds from within emacs.

I'm not sure how directly relevant it is, but you should definitely take a look at:
http://www.dgp.toronto.edu/people/stam/reality/Res earch/pdf/ns.pdf

(or better yet, find papers which cite this one)

As others have pointed out, multi-grid is a fine way to go, but there's some interesting work on using a combination of finite-element method with particles:

In a nut-shell, your finite-element grid holds the smoothly-varying components (those with c-less-than-1), while the particles hold the discontinuous aspects (those with c-much-greater-than-1). In the sonar case, the particles would correspond to the wavefront(s), while the grid would hold pressure and velocity.
(Stam describes how to re-inject the energy lost to damping in the (c-much-less-than-1) case, but this may or may-not be desirable depending on your problem domain)

One simple trick is to switch to a (symplectic) verlet leap-frog scheme to ensure energy is conserved: Update the grid at time={0, 2, 4, 6, ...) and update the particles at time {1, 3, 5, 7, ...}. ( http://artcompsci.org/kali/vol/two_body_problem_2/ ch01.html )

If your particle density is higher than the grid cells, you can get away with simple splatting (easy!). Otherwise, you need to use something more complicated, radial-basis-functions are great, but some kind of 8-way smearing operator might work too..

For visualization, see also: http://public.kitware.com/VTK

One last thing, if you're using a regular-grid for the finite-element, try and use an irregular representation for your terrain, or else you may find the directional components in the FEM influencing the outputs in the terrain..

Good luck!

In what ways can you code plots of surface charts using a z(x,y) function or a cloud of points?

These are really two separate questions. Plotting a surface for z=f(x,y) requires evaluating the function over the domain and constructing a tesselated mesh with vertices at each point. Once you have the mesh, you need to render it, which involves issues like hidden surface removal, shading, perspective and so on. Not a small undertaking to write all that from scratch.

There are several algorithms for rendering clouds of points, depending on the nature of the data. First you have to cluster the points, then you have to mesh them, then render. It's not trivial by any stretch. Have a search through ACM SIGGRAPH for a few of the top algorithms.

I'm looking for a tutorial that explains this that doesn't use DirectX or OpenGL libraries (the language I'm using cannot use either framework anyway).

The vast majority of books that would cover this kind of thing will be either be: a) practically oriented and use OpenGL (or equivalent) for the rendering, or b) be highly theoretical and written in dense mathematics. It is unlikely that you will find a book that fits your description. The best you can probably hope for is to find an OpenGL book and abstract away the algorithms you need (after all, OpenGL is really just the lowest rendering stage).

(Now of course, one must ask the question: why are you restricting yourself to using a language or environment that is clearly not appropriate to solving the problem at hand?)

Others have suggested the classic Foley & Van Dam book, which is excellent and provides much useful material. But the one you really want is the Visualisation Toolkit book. It describes the algorithms and implementation behind VTK, and covers precisely the algorithms that your are asking about (clouds, surfaces, etc etc). VTK is implemented using OpenGL, but the book focuses on the algorithms, and includes comprehensive citations to the original papers where these algorithms are published. It is one of the best books available on Viz and 3D.

• The Visualization Toolkit: An Object-Oriented Approach To 3D Graphics, 3rd Edition - Softcover textbook, 520 pages, printed in full color, and comes with the VTK 4.4 CD (ISBN 1-930934-12-2)

(It can also be ordered from Amazon, but they're on backorder.)

Now go read it, and tell me you still want to re-implement all that yourself from scratch.

How is the 3D mess generated and how can the 2D contour plots be generated as well?

Contour plots involve yet another class of algorithm, producing isocurve from data cells. Others have mentioned marching cubes, there's also marching squares and a few others that are used. Note that marching cubes is patented, so you need a license for commercial use! (Others who blindly suggested this might also want to take note.)

I'm assuming once I know that I can also use it to make torus plots as well.

I wouldn't. What the hell is a torus plot?

Remember, I'm asking for the explanation of the underlying math and an example code that does that. The GNUPlot gallery has some examples that I find helpful, but are there similar examples out there? (Remember, I am writing the plotting code as well)? Can anyone help or point me in the right direction?"

The VTK book will provide all the maths and algorithms you require, if you insist on going forward and reimplimenting all this yourself. But why? Methinks you are drastically underestimating the complexity of the task you are asking. Especially if you aren't already experienced in the areas of 3D rendering, vectors, linear algebra, (and it appears you are not) you are going to be climbing a massive

VTK has also spawned off two other opensource projects, DART and Cmake.

I have to add another vote for VTK. It's not ideal for 2D plots, but then there are so many packages that can do a good job of those. If you have to do any complex data representation in 3 or more dimensions however VTK is amazing, both in it's ability to produce great looking interactive plots, and in its incredible flexibility in how you trnasform and represent your data. Nothing else I'v used comes close.

Jedidiah.

ROOT and VTK

Can someone who knows say how OGRE compares and contrasts with vtk as a 3D engine?

If you want to implement one of these, check out Dart from the guys at Kitware.

I've seen their in-house dashboards and they're quite impressive. These guys eat their own cooking.

Sorry, but I fail to see what Blender and the GIMP have to do with real scientific visualization. Blender is for 3D modelling, and the GIMP is for image processing.

If you're looking for complete, open source scientific visualization and data analysis packages, try VisIt, which supports dozens of input formats and runs on Linux, Windows, and MacOSX. Pick it up at http://www.llnl.gov/visit, or get the latest binaries from FTP here.

I have less knowledge of ParaView, but it is also free: http://www.paraview.org.

Both of these are also developed in part by the national labs; they can run parallel to handle terabytes of data, so if you've got small dataset they should be smokin' fast, and if you've got your own cluster you should be able to visualize some huge data.

If you're looking for just a toolkit to build your own application, try OpenDX or VTK.

If you are looking for a more generic build/test runner dashboard than cruise control, take a look at DART. Its free, open source, and makes very few restrictions on those projects which wish to use it.

Here are some example dartboards.

Here is my dart board for the Mobius Project I work on.

If you are looking for a more generic build/test runner dashboard than cruise control, take a look at DART. Its free, open source, and makes very few restrictions on those projects which wish to use it.

Here are some example dartboards.

Here is my dart board for the Mobius Project I work on.

Hi Bruce,

The case in question refers to *use*. The university was sued for continuing to *use* patented inventions they had no license to. I believe this is still OK to re-implement a patented technique for the purpose of comparison with a novel technique that one has developed or that one plans to develop. The article you reference is short on details but I seem to understand that the university had no intention to work on the inventions to improve them. The article seems to imply that the inventions were used in some kind of production environment.

This is the whole idea of Patents, to disclose how an invention works so that others can improve on it. The purpose of patents is to foster creativity, not stifle it.

I work in image analysis research, and I occasionally do read papers in areas that are heavily patented, such as Bayer filter interpolation (to remove colour artifacts from digital camera due to the peculiar arrangement of the colour sensors) and motion estimation. Patents are commonly cited as references and authors readily show their improved results compared to a variety of patented techniques.

It is a common practice in my field to patent some technique and then publish it. The expectation is that people will re-implement what is disclosed in the paper and everyone seems to expects that this is legal.

There are freely available libraries like the VTK that provide reference implementations of famously patented techniques such as the Marching Cubes algorithm (to transform a cube of voxels into a triangulated surface, used everywhere in medical imaging). These algorithms are in a special "patented" directory. You can compile them and presumably compare your results to what these techniques do, but you are not allowed to *use* these techniques in any production enviroment, be it for or non-profit (e.g. a public hospital) without a license.

This state of things doesn't seem to be contradicted by the referenced article, at least that's my reading of it, and I'm not a patent lawyer.

Anyone interested in doing powerful 3D data visualization should make a mandatory stop here. It's an open source visualization toolkit written in C++, but with bindings for Java and Python as well. This is a very powerful and very impressive system, and ought to be rated as one of the great open source projects. It doesn't seem to get much attention - I'm not sure why.

Have a look, and look at what it is actually capable of doing. If you want to do any sort of 3D visualization, it really is worth your time to learn a bit about VTK.

Jedidiah.

It's not "false savings" in my case. We compile ParaView frequently during development. It's over 1200 C++ classes and takes at least an hour to compile from scratch on a fast machine. Even small changes to a single header file can cause large numbers of files to be recompiled. And we like to run continuous, experimental, and nightly builds (with DART) for testing purposes. Since we run ParaView on clusters, it's nice to be able to build in parallel, too. With the demise of pmake, it's good to see some open source tools for distributed compilation being developed. distcc isn't ideal (cpp and ld must be run on the head node and there can only be a single head that passes does load distribution), but it's a lot better than being stuck on a single node.

What's the problem with distributing patented technology in source form? I believe this is legal. As an example VTK distributes the marching cube patented method (among others) with no problem.

Unisys never had a problem with any of the LZW implementation in source form. They never asked for them to be pulled out of any site, and neither could they legally. What they asked is if you were using this technology for anything other than research and study (i.e. if you really wanted to compress some file with it for redistribution) *then* you needed a license from them.

The use of patented methods for research and study is legal, this is the whole point of patenting technology. Patenting is a publication process, in exchange for exclusive control of the technology *in applications*. The idea is that other people can study this technology and improve on it.

If you as a user take some source code floating on the net implementing some patented technology, and add it to some application, be the application free or not, you are responsible for obtaining a license from the holder of the patent, but AFAIK the author of the code is in the clear, and so are the distributors.

If you're interested in something a little more complex than basic plots and charts I very highly reccomend VTK a visualisation toolkit that is unparalelled for putting together complex 3D visualisations of data. It's all in C++, is open source, and has Java, and Python bindings.

I generally used the Python myself - and the python API is very nicely done - a pleasure to use, and a great way to do complex 3D data visualisation.

Jedidiah.

Where I work (an environmental science organisation) the emphasis is on getting the job done, whichever tool is approiate for the job, or whichever tool is familiar to the scientist gets used. (I'm not saying that this is necessarily the best policy, just what happens here.)

There are a range of programming competancies from people who take code written by someone else that they have always used and feed data through it black-box style, through to scientists who write their own F90 or C++ code with MPI and BLAS libraries for running on the HPC clusters.

We have OpenDX, Matlab, octave, IDL, Statistica, S, SPlus, R. Programming in F90, F77, C++, C, Perl, Python. The platforms include Windows*, Linux, FreeBSD, Solaris, Tru64. Some instrument makers use labview. You name it, someone here will be using it somewhere.

Quite often one person will develop using a particular package, then anyone s/he collaborates with will have to use the same package to read the files or run the programs.

I've been mucking about with python and Vtk.

For interactive symbolic manipulation, Maxima is an excellent open-source alternative. For numerical applications, Numerical Python and its associated packages beat both Matlab and Mathematica in my opinion. For 3D visualization, you can get VTK, which also has Python bindings.

Maxima is also used occasionally as a rapid prototyping language, but it's proprietary and it has a lot of rough edges. You are probably better off using one of a number of open languages with similar features, like Scheme, OCAML, SML, Prolog, or Haskell.

Don't forget about C++, however. In many ways, C++ nowadays allows you to write numerical code more naturally than any of these other languages (yes, better than Matlab and Mathematica), it has by far the best libraries available for it, and it gives you excellent performance. And you can even do symbolic mathematics in C++, with the right libraries (though it's not interactive, of course).

Check out The Visualisation Toolkit

An alternative to autoconf is certainly CMake , an open source cross plattform configuration and make tool developed by Kitware . This is a great tool, easy to set up and it has already successfully been used in large projects. This could reduce the effort needed for your task considerably.

An alternative to autoconf is certainly CMake , an open source cross plattform configuration and make tool developed by Kitware . This is a great tool, easy to set up and it has already successfully been used in large projects. This could reduce the effort needed for your task considerably.

2. For volume rendering i recommend VTK the visualization toolkit. A bit high on overhead required to use it, but quite powerful when you learn. In addition to volume rendering, supports isosurfaces (via marching cubes), segmentation algorithms, and many other image classifications. Delauney triangulation, and many of the elements needed for production visualization pipelines.

3. BLAS and LAPACK are absolutely essential. Basic Linear Algebra and Linear Algebra subroutines for everything from optimized matrix-matrix operations to Singular Value Decomposition and cholesky factorization of band diagonal symmetric yadayada. I use this stuff daily and the LAPACK subroutines would be one of the first things I would compile in a new environment. The LAPACK subs call BLAS subs. Note that I have these in fortran but called from C/C++. I dont know if they have been ported to C yet.

4. Stay away from those fancy "data explorer" deals. Complete waste of time. Chances are with a little more work you can do a better job, in a smaller package, with a *ton* less overhead by writing a bit of code. Learn a command line parser (you could prolly use getopt) and write your own library. I recommend brewing up 1D 2D and 3D storage classes that are reusable via C++ classes. For 2D/3D we use files with an ascii header and binary data, and have written utilities to do math on or between them. We also spent the time to write our own plotting software direct to postscript, so I have not had to struggle with the crap that is the freely available plotting software. GNU plot is simply pathetic. And if you pay for something like NCAR it is at least as bad but costs a hell of a lot more.

5. As far as those fancy environments go, I have used AVS, KHOROS, IBM explorer, and the SGI IRIS explorer. One of these that was free and probably the easiest to use is now not free (khoros). The IBM data explorer is also free now, but it is a total piece of crapola in my not so humble opinion.

6. Finally. Get the numerical algorithms book for your fav language. You wont regret it.

2. For volume rendering i recommend VTK the visualization toolkit. A bit high on overhead required to use it, but quite powerful when you learn. In addition to volume rendering, supports isosurfaces (via marching cubes), segmentation algorithms, and many other image classifications. Delauney triangulation, and many of the elements needed for production visualization pipelines.

3. BLAS and LAPACK are absolutely essential. Basic Linear Algebra and Linear Algebra subroutines for everything from optimized matrix-matrix operations to Singular Value Decomposition and cholesky factorization of band diagonal symmetric yadayada. I use this stuff daily and the LAPACK subroutines would be one of the first things I would compile in a new environment. The LAPACK subs call BLAS subs. Note that I have these in fortran but called from C/C++. I dont know if they have been ported to C yet.

4. Stay away from those fancy "data explorer" deals. Complete waste of time. Chances are with a little more work you can do a better job, in a smaller package, with a *ton* less overhead by writing a bit of code. Learn a command line parser (you could prolly use getopt) and write your own library. I recommend brewing up 1D 2D and 3D storage classes that are reusable via C++ classes. For 2D/3D we use files with an ascii header and binary data, and have written utilities to do math on or between them. We also spent the time to write our own plotting software direct to postscript, so I have not had to struggle with the crap that is the freely available plotting software. GNU plot is simply pathetic. And if you pay for something like NCAR it is at least as bad but costs a hell of a lot more.

5. As far as those fancy environments go, I have used AVS, KHOROS, IBM explorer, and the SGI IRIS explorer. One of these that was free and probably the easiest to use is now not free (khoros). The IBM data explorer is also free now, but it is a total piece of crapola in my not so humble opinion.

6. Finally. Get the numerical algorithms book for your fav language. You wont regret it.

• VTKis a very good package for scientific visualization.
• Maxima is a Free computer algebra system, a bit like Mathematica. It can solve equations, do calculus, plot things, produce TeX output of what you've done, and lots more. Incredibly useful for long tedious bits of algebra.
• gnuplot is a versatile graphing package (2D and 3D, but maxima or VTK are IMO better for 3d stuff). As well as graphing, it can try to fit arbitrary functions to your experimental data.
• LaTeX -- it's very hard indeed to typeset equations better than LaTeX can.

2. For volume rendering i recommend VTK the visualization toolkit. A bit high on overhead required to use it, but quite powerful when you learn. In addition to volume rendering, supports isosurfaces (via marching cubes), segmentation algorithms, and many other image classifications. Delauney triangulation, and many of the elements needed for production visualization pipelines.

3. BLAS and LAPACK are absolutely essential. Basic Linear Algebra and Linear Algebra subroutines for everything from optimized matrix-matrix operations to Singular Value Decomposition and cholesky factorization of band diagonal symmetric yadayada. I use this stuff daily and the LAPACK subroutines would be one of the first things I would compile in a new environment. The LAPACK subs call BLAS subs. Note that I have these in fortran but called from C/C++. I dont know if they have been ported to C yet.

4. Stay away from those fancy "data explorer" deals. Complete waste of time. Chances are with a little more work you can do a better job, in a smaller package, with a *ton* less overhead by writing a bit of code. Learn a command line parser (you could prolly use getopt) and write your own library. I recommend brewing up 1D 2D and 3D storage classes that are reusable via C++ classes. For 2D/3D we use files with an ascii header and binary data, and have written utilities to do math on or between them. We also spent the time to write our own plotting software direct to postscript, so I have not had to struggle with the crap that is the freely available plotting software. GNU plot is simply pathetic. And if you pay for something like NCAR it is at least as bad but costs a hell of a lot more.

5. As far as those fancy environments go, I have used AVS, KHOROS, IBM explorer, and the SGI IRIS explorer. One of these that was free and probably the easiest to use is now not free (khoros). The IBM data explorer is also free now, but it is a total piece of crapola in my not so humble opinion.

6. Finally. Get the numerical algorithms book for your fav language. You wont regret it.

The OpenSource community has adapted in various ways to address the issue of coexistance. To use a rather crude analogy, BSD is bisexual whereas kits (see VTK where patented stuff can be compiled out or purcahsed separate) is like having a mistress (or 5). Embedded manufacturers are homosexual in that they enjoy all the priviledges of the CCC toolchain but reserve the right to go the otherway at the last instance. Is there a happy marriage which represents a middle ground? RMS would clearly like the code to be virgin but unfortunately his philosophy of choice does not jib well with a command and control mentality (which if you understand military hierachies is inevitable). While there is some comfort that in the long-term the market/community will evolve solutions, your situation needs to be resolved in the short term. Some suggestions for negotiations:

- suggest to the military-industry perplex (contraction of person and complex) that the GPL form the demo/training version which will reduce the cost of controlling sensitive/unstable components.

- reduce to core + open when only the core is restricted. This reduces the support costs of the ancilliary parts which in a larger OpenSource world there are alternatie suppliers of programmers/expertise.

- emphasise the reduced maintainence costs (OK rather difficult in a government bureacracy which has little concept of fiscal discipline) as you will benefit from the reduction of bit-rot and early obsolescence.

- encourage the development of policies which progressively open up closed source due to the risk of a key personnel leaving. The military understand the value of redundacy (at least if they expect combat casualties) and you want to reduce the risk that a crazed terrorist might decide to nuke the dev-team (OK outlandish but this is the paranoia state) and cripple the critical nexus. OpenSource (GPL) distributes the expertise so the risk of system catastrophe is much less.

- try to understand the client's mindset and address their concerns. GPL/OpenSource is not a universal antidote to world poverty, there are situations where it may not be relevant. If the worry is support/responsibility, suggest mechanisms such as a trouble-ticket. Establish the criteria for it not being GPL (and avoid the generic bullshit about national interests and spy vs spy). Most code is for mundane purposes like logistics and task-automation. Suggesting that tax-payer subsidies of $xxx * 500% per line-of-code will not go down well with politicans.

- read stuff from HOW2-advocacy-Linux to understand how a consultant should think. Your primary concern should be to best meet the needs of the mission. For techniques on critical thinking, read up suff on 6 Thinking Hats to get logical rationals (pos,neg) and reduced bias as to the selection of the licensing/distribution/usage. And for gods sake document it so the next perplex doesn't come along and changes the rules of the game (yet again).

- understand the long-term impact of licensing by using market examples ... the Sun Community License as a defensive chastity belt (once you get in you can't check out), university (MIT/CalRegent) condom is disclaim any warrenties, the tactical role of Apple's dildo to try and screw the media control plans of the opposition, However I would suggest using more ... ummmm ... reserved language.

In short, rather than covering your butt, be prepared to stand up for what you believe is the best for your users in gaining access to software that doesn't suck. Sure you make get knocked about a bit in the early stages but at least you won't be screwed for life by being forced to use software that everyone hates.

Good luck,
LL

#include <IANAL.h>

Patents cover the use of *methods* of doing something. And, in many cases, you need to use it commercially in order to infringe. OpenSource refers to a particular *implementation* of that method. It is possible to keep a method patented, and still release an OpenSource implementation of that method. People can get the software and play with it all they want, but they need to license the *patent* if they want to use the method embodied in that patent, regardless of whether they use the OpenSource implementation or roll their own. One example of this is the Visualization Toolkit (VTK) software library. Some components of this toolkit contain methods patented by GE Corporate R&D (where VTK was originally developed) and Kitware , a company started up by two of its original authors. The vast majority of VTK does not involve any patents, and all the code is OpenSource, but if you need to use one of the pieces that implements a patented technique, and you want to use it in a commercial product, you need to license the patent from whoever owns it.

#include <IANAL.h>

Patents cover the use of *methods* of doing something. And, in many cases, you need to use it commercially in order to infringe. OpenSource refers to a particular *implementation* of that method. It is possible to keep a method patented, and still release an OpenSource implementation of that method. People can get the software and play with it all they want, but they need to license the *patent* if they want to use the method embodied in that patent, regardless of whether they use the OpenSource implementation or roll their own. One example of this is the Visualization Toolkit (VTK) software library. Some components of this toolkit contain methods patented by GE Corporate R&D (where VTK was originally developed) and Kitware , a company started up by two of its original authors. The vast majority of VTK does not involve any patents, and all the code is OpenSource, but if you need to use one of the pieces that implements a patented technique, and you want to use it in a commercial product, you need to license the patent from whoever owns it.

While object oriented techniques make sense for real software development, if you merely want to run a particular algorithm on a lot of data points, any structured language will do.

Well.. numerical simulations are getting larger all the time, and structuring the code and enabling evolution and replacement of components is becoming more and more important.
Although I've never used Fortran, one associate professor I've talked to, claimed that Fortan was unsuitable because of this.

Believe me; there's more to numerical simulation (and visualization) than to run a particular algorithm on a lot of data points.

I took a course in numerical simulation and visualization last semester, and we used C++ with Diffpack and VTK, and although many students did not take advantage of C++ by programming OO, I did it to keep my sanity.. and it paid off. Big time.