Slashdot Mirror

Anecdotal evidence: I work in remote sensing, mostly image processing apps using data from airborne sensors. The sensors are mostly experimental, and the image processing stuff is very researchy. We use MATLAB all the time- I honestly think if we used C++ (or pretty much anything other than MATLAB) during the algorithm development stages, it would take us literally 5-7 times as long to do anything, and in the end we would not have pursued nearly as many paths of research or done nearly as much analysis of our results. I don't know what it's like in other fields of engineering, but I know a ton of people who work in signal and image processing that would say the exact same thing.

I once struggled with one particularly tough problem for almost a year- I was developing in C++ at the time, since that was what the final implementation would be in. Again, this wasn't just implementing something that was well-understood...this was also researching new algorithms and doing lots of experimentation with lots of different datasets. After a year I got exasperated and and decided to develop the algorithms in MATLAB first. I reimplemented all the C++ stuff I had already done in the previous year, in about 1 month [!]. In another month I had a working system, much to my amazement. In another month I had tested and analyzed the living crap out of it in ways that I wouldn't have dreamt if I was still trying to do things in C++.

The only real competitor to MATLAB in this field is IDL (not to be confused with Interface Definition Language- you gotta love overloaded acronyms). In the open source world, you've got Octave, SciLab, and SciPy, but I think SciPy has the best shot at weaning people off MATLAB. So far I don't think any of the open source alternatives are quite mature enough for my purposes, but they're getting there. In my (rather specialized) field, Mathcad and Mathematica aren't contenders, for whatever reason.

In conclusion: as a programming language, it can seem a little odd (for instance, all variables are matrices). It's better to think of it as an interactive mathematical prototyping environment designed around a very specialized language. But if you have a need for that kind of tool, it can be a really amazing timesaver.

I'm a huge open source fan, and it really riles me to have to use proprietary software, but it's hard to match what IDL can do. With a little practice you can go from raw data to publication quality plots in a staggeringly short period of time.

So far I've yet to find anything close in the open source world. PerlDL + pgplot is the best contender. The authors seem to be very careful to do things properly and a lot of the functions one needs are already available. But the documentation is a great steaming pile of beetle dung and it's almost impossible for a new person to get started, and they don't (yet?) seem to have some of nifty multi-dimensional array processing features that IDL offers.

If anyone knows of a good alternative, I'd love to hear about it! I'll be watching this thread in the hope that there's a project out there somewhere that's slipped through the cracks.

I used Garble because back in Summer 2000, before the grueling years of my physics PhD program began, I took a 15,000 mile roadtrip around the USA. During most of the driving I had my laptop connected to my GPS, pinging it every minute (through a simple bash script) getting a latitude and longitude. After my trip I compiled all the points, and used IDL to make some nice plots from the data. IDL was cool because I could easily set up map projections w/ my latitude/longitude data, overlap satellite images, and even plot country/state borders (IDL costs $$$$$, but Johns Hopkins University physics department has a large client license on the student terminals, which is nice). Check out my final Mercator Plot and Satellite Perspective Plot. My route is in red (chronologically going clockwise around the country), yellow circles are where I spent the night.

You can read my unfinished online journal here. Yeah, it's been a few years since the trip, and I do really need to finish that journal and clean up the ugly page layouts.

That should have been IDL.

Yep, you're right that Mathematica is not a good comparison -- I stuck that in mainly as a reference to the numerical part of Mathematica, but the symbolic stuff is pretty much unmatched (though Maple fans might disagree).

Much of PDL's development has been motivated by a need for something "like IDL, but more powerful", and I think that's really where PDL shines best: in remote sensing and image processing tasks. It helps a lot that all of CPAN is already present, and that the file I/O and indexing have many fewer "gotchas" than those of IDL. The PGPLOT back-end is great, too, for actual device-independent plotting: how many hours have you spent tweaking your IDL plots to actually print right on the PostScript device?

It's (IMHO) a Good Thing that we have all three of numpy/scipy, Octave, and PDL: each has a different set of strengths. Ultimately, each group really should use the tool that suits them best (and it shouldn't cost more than the workstation it runs on...). The reason I've more-or-less committed to perl development rather than Python or Octave is that it has a nice "natural language", expressive feel to it: it's easy to build pipeline-style, imperative-style, or evaluated-style constructs, whichever is most convenient for the current application.

Of course, the open-source languages have the added benefit that results derived using them are actually reproducible, whereas closed-source languages might conceal irreproducible bugs (in the language or the reduction code) that other groups can't identify.

---I don't understand. They're comparing a bunch of X Servers versus a bunch of Dell PC's?

That's what it seems. Yeah, I know. The compairison sucks.

---What about the guy who's playing MP3's at his desk?

What? Lopster and XMMS arent good enough for him?

---What about the guy who wants to sync to his Palm Pilot?

There's already good sync software for Linux. Just un-endorsed. Hell, They might actually make a "legit" tool if stuff like this happens.

---What about the guy who's using Messenger?

There's buttloads of tools for IM on Linux.

---What about the guy who *NEEDS* a specific piece of software to communicate with his peers?

In limited cases, WIndows is the only answer for now. But as sysad, you could put heavy pressure on a company who does such.

---What about the guy who's burning DVD's of classroom presentations?

Get him a Mac. Most unix dudes could get one working.

---What about the guy who wants to run mid-priced shrink wrapped applications like Mathematica or MATLAB or IDL (all probably less than $10,000 for a single user license, but could get expensive for a big machine).

OK... Your point ?

---What about the guy who runs small simulations -- the kind of thing a reasonable desktop could do in an evening or a weekend? People who run computer centers often complain about 40 hours of computer time on the big boxes.

Help his department build a small cluster for job crunching. COuld even be a "beowulf" cluster if his apps support it. Then he could 'job' out time to other departments. That'll avoid cpu munchers on the main system.

---In short, what about all the flexibility that the Personal Computer gives the user? Why ins't that included in their "TCO" at all?

How about the flexibility of "use the tool that works for the job"? Trust me, you really dont NEED windows anymore.Look at all 3 links at your Math program question.

I don't have any experience with PDL, but the name clearly alludes to IDL. Does that mean that there is any compatibility between both ? Can PDL interpret an IDL macro ?

....actually, Matlab is available for Unix. I've used it at my university. It runs on linux too! It also runs on Sun (i seen it on a sunblade 1000). If you don't have a X11 display (eg, simple terminal), then it'll display the matrices in text format. I haven't tried graphing stuff under text though. If you do have a X11 display, then it's just like it is in Windows.

...if you're into even more hardcore sims, you can use IDL to develop full-fledged apps without having to interface to C++. And the IDL language is pretty similar to matlab, so the learning curve isn't as bad as having to learn C++. IDL is used by quite a bit of EE companies.

I worked for a while at a physics department in a University. They used IDL almost exclusively for number-crunching - especially the atmospheric physics guys. Apparently you could write code rather quickly and easily and it had excellent graphics capabilities. Check out Research Systems Inc.. Mind you it costs a stack. We got it cheaper coz we were a Uni but it still cost heaps.

We had some Matlab around which the theoretical guys used more but IDL was the most prevalent.

Another product to consider is IDL. The only problem with IDL is that it's price tag is similiar to that of Matlab. However, it does provide a complete programming language, a rich set of data analysis and computation routines, it's own development enviroment, and the ability to create complex visualizations and user interfaces.

Most of my colleagues are Windows addicts who battle away getting Excel to handle huge data sets. I do almost all of my work in IDL and find that it works just great.

I switched language when I left uni for the real world (IDL is nice but bloody expensive) and settled for python.
Actually, more like python+numeric+c

I usually experiment in pure python+numeric first and then optimise the parts that need in c. I really love how easy it is to interface c and python (especially compared to IDL). No problems passing numeric arrays to and from python, etc, etc...

Don't forget to check biopython which is always a good place to start. Dunno if there's anything for processing microarrays, though.

Which is a bad thing for businesses, because they cannot keep their training schedule up to date.

Seroiusly, who would remove a feature from their app?

It happens all the time, whenever the cost of maintaing the feature exceeds the revenue.

This should require little bandwidth Obviously for some applications it may, and for some organizations they may already have sufficent bandwidth. However there is no doubt at all that bandwidth is a concern that you must ensure is addressed if you are considering using an ASP.

ASPs do this... it is standard practice.

However with ordinary system, if the vendor decides not to do so, then the customer can choose a different organization to do the customization. This happens all the time. With the ASP model the customer has no possibility of doing this.

Ever heard of SSL? TLS

Only helps with client->server connection, if supported properly by the server. Once the data is on the server it's now up to the server to hold it securely.

Even ``morphological'' studies are no longer done with magnifying glasses and film, but rather with large collections of digital images from spacecraft such as SOHO and TRACE. Image calibration and reduction software is mandatory if one is to do meaningful experimental analysis.

Fortunately, the solar community has by-and-large been good about releasing analysis tools into the public domain -- in fact, there's a homebrew distribution system that grew up, mostly before CVS, to nearly-universal status within the research community. Without the tools that are available via solarsoft, I literally could not do the work that I do without developing similar things myself (in fact, I do develop tools myself, and publish them... but that's another story)

Even within the relatively open solar community, there are software-based barriers to entry. For example, most of the current community develops in a proprietary language called IDL, which was developed in significant part (in its early years) with public funds. The developer, David Stern, started RSI, inc. to capitalize on his language. Currently, IDL licenses start at $1,000 per year, double the current cost of an entry-level workstation.

When workstations cost $10,000 and only large organizations could afford hardware capable of doing image processing, this cost was excusable. But now, in an era of cheap computers, high connectivity, and readily available space-borne solar data, the cost of supporting IDL is the main barrier preventing hobbyists, high school students, and interested amateurs from doing their own research programs. If IDL were open-source and free, RSI might well still exist (under the Cygnus / Red-Hat business model), and solar (and other) research would be much more accessible to the masses.

One may argue that IDL (and its competing product, MatLab) wouldn't have developed into the large, powerful packages that they are without commercialization. But such arguments are spurious: PDL, the Perl Data Language, is entirely open-source and free, and powerful enough that that I am now devloping tools in it instead of in IDL.

I signed the petition, and I encourage you to, too. Publicly funded intellectual property is your property, just as the national forests are your forests. Demand them.

Even ``morphological'' studies are no longer done with magnifying glasses and film, but rather with large collections of digital images from spacecraft such as SOHO and TRACE. Image calibration and reduction software is mandatory if one is to do meaningful experimental analysis.

Fortunately, the solar community has by-and-large been good about releasing analysis tools into the public domain -- in fact, there's a homebrew distribution system that grew up, mostly before CVS, to nearly-universal status within the research community. Without the tools that are available via solarsoft, I literally could not do the work that I do without developing similar things myself (in fact, I do develop tools myself, and publish them... but that's another story)

Even within the relatively open solar community, there are software-based barriers to entry. For example, most of the current community develops in a proprietary language called IDL, which was developed in significant part (in its early years) with public funds. The developer, David Stern, started RSI, inc. to capitalize on his language. Currently, IDL licenses start at $1,000 per year, double the current cost of an entry-level workstation.

When workstations cost $10,000 and only large organizations could afford hardware capable of doing image processing, this cost was excusable. But now, in an era of cheap computers, high connectivity, and readily available space-borne solar data, the cost of supporting IDL is the main barrier preventing hobbyists, high school students, and interested amateurs from doing their own research programs. If IDL were open-source and free, RSI might well still exist (under the Cygnus / Red-Hat business model), and solar (and other) research would be much more accessible to the masses.

One may argue that IDL (and its competing product, MatLab) wouldn't have developed into the large, powerful packages that they are without commercialization. But such arguments are spurious: PDL, the Perl Data Language, is entirely open-source and free, and powerful enough that that I am now devloping tools in it instead of in IDL.

I signed the petition, and I encourage you to, too. Publicly funded intellectual property is your property, just as the national forests are your forests. Demand them.

There is much better technology out there.

The two items we are searching for:

Guns, knives, cased explosives are all metal. A metal detector when used properly should be sufficient for these items.

Non cased explosives, plastiques, etc... these are easily detectable using hyper spectral analysis. This is being done now from sattelites. Information gathered from these devices are so accurate that they can determine what type of crop is growing by examining the properties of vapors being given off. There is absolutely no reason that this cannot be employed on a smaller scale. It could easily be adapted, and I can't understand why it is not. For one quick example of what you can do with this technology look here: click me!

Anyway, I got into this industrial computing course, and we was doing assembly on some 68K boards. I was bored throughout the whole thing which was really pissing off the guy I was working with :-)

Anyway, comes the exam (2 hours) and off I go, programming in assembly as if it was basic and getting a working version of the program in about 1/2 hour! (some weird shit, interfacing the 68K with some IO ports and timers) 15 minutes later and the program was looking a lot less messy than the original version. Around this time comes the teacher to check on me, checks the program, sees that it works, and asks me for my flowchart...

So I tell the teacher that I haven't quite finished my program, that surely there are things to optimize, so I'll draw the freakin flowchart when I'm done with the programming.
Not quite the question order he had in mind (was I supposed to draw the flowchart thingy BEFORE coding??? me??? :), so I got marked down... eventhough I knew my shit like nobody else.

After getting my result and talking to the guy I learned a valuable lesson: I gave him the impression that I could have done the thing my eyes closed and ended-up making some minor mistakes (can't remember what it was, but it was some silly 68K specific shit, or something... took him quite some time to find something to mark me down on, though!) so he hurt me plenty for that (like, that'll teach you being so arrogant :-)

What I'd say is that a mark is just a freakin mark, nothing else... is it going to affect your carrer if you're only the second best in your field? (especially if being the best means so much more work). Is a mark that important that you can't take a joke? Was you going to fill A Complaint because you was marked down? come-on, man, cheer-up! ;-)

---

There is a lot of sucky OOP support out there. IDL is really bad. R on the other hand, has some very simple OOP support, but it is certainly very nice, and makes the job done very nicely. I haven't really gotten the hang of the Perl OOP stuff, but then, my Perl stuff isn't really sophiscated.

Or you could use that for medical imaging.

BUT! It's okay when nothing complicated is involved.. as in just the data m'am. In my case that would be the bladder/prostate/rectum compound, no texture, 1 or 2 lights, no shadow...
If I were to add the patient skin with realistic semi-transparence and wanted to do functional animation as well, things bet much messier, and I (would) do that in pov... The problem then becomes, okay, everybody does it with clusters, but why? is it because 3D accelerated hardware is looked at the wrong way, or is it because this is not really the main focus (a.k.a games! :)

A card designed for med imaging would need to do clever things as marching cubes maybe, and also raytracing from voxels (any other suggestions? :)
A side note: for hardware accelerated opengl and computation, I use IDL on NT but am waiting for a comparable solution in Linux (yep, sure do!)

---

"The Power Mac G4 gives us anywhere from a two- to five-fold boost in performance for computationally intensive tasks," says David Uhlir, director of product marketing for Research Systems. "For example, basic binary operations on arrays in IDL run almost five times faster on Power Mac G4 systems than on otherwise comparable computers."

"Due to the C-interface with the Power Mac G4 chip, coding for the chip has been relatively easy. As an example, modifying all of the basic binary operations on arrays in IDL took only a couple of days. As we identify and target more areas of IDL that can benefit from the G4 chip, support will be added rapidly. This will enable us to have broad support in our upcoming version of IDL."

While I think this is a noble gesture, I think it's also a monumental waste of time. My discussions with the various AXAF folks lead me to believe that you could buy RSI outright for the amount of money being thrown at this brand-new, free equivalent whatever. NIH...

That's from a technical point of view, micros~1 licenses cost probably more than the computer itself, but there you go...

Why I am mentioning DOS?
DOS+DJGPP was the platform I started to do image processing on. A 486DX2-66 with 8Mb RAM... That was in 95. Actually, I used TC too, but extended/expanded memory scheme was just not for me :-)
So how much time do you think it takes to do adaptive equalisation on a 640x640 image, using a DX2? I'd say about 15s...
Okay, you need to do a bit of thinking, be familiar with hand optimisation and the like (loop unrolling and early loop breaking anyone? ;) but think about it! I'm sure if I tried the same using IDL or matlab (sorry, I meant PDL and octave) I'd get worse results on my PII-400...

The key is, it takes much more than a computer degree to do real optimisation by hand, it takes time... and who's willing to take the time?

What about Linux? (this is /. after all)
Well, Linux router allows you to do routing stuff all on one floppy. It works great on a 486, eventhough it may be a little slow a computer to do caching DNS and masquerading at the same time :-) (I'm switching to a Pentium 133 - 16Mb RAM)

So there you go, old computers are great if you can find some use for them... did I mention I love to program Z80 assembly on my Amstrad CPC6128? ;-)

---

Anyway, my point is... with popular 3D games containing as many polygons as Quake, the hardware needed to render scenes at a good frame rate goes down in price because of this popularity, and benefits many other field of which... the medical imaging industry :-)

Way to go folks!!!!

---