Slashdot Mirror
Running a Research Lab on Free Software?
"[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is. Who could blame them? Their Windows DLL is the perfect solution under [most] circumstances.
I'm not the only one around here getting frustrated, but all before me have been defeated. It seems I am to be as well, for today I have started to learn Visual Basic.
Has anyone had any *positive* experiences trying to move a lab from proprietary to free software? Surely the government-funded researchers of the world have a responsibility to ensure that their work is free, as in freedom. However, I have found out the hard way that it's usually just not worth the effort, following such ideals. You just get frustrated by apathetic colleagues, useless product support, and the conventional wisdom that it's OK to ignore your ideals, so long as you get the experiment working. Additionally, my ordeals convince my peers that free software isn't worth the trouble."
I've found it somewhat difficult to use since building their modules is really suggested on a generic rather than stock Redhat kernel (and building with what are claimed to be the RH sources and config files didn't work for me).
As someone who has been a sysadmin for a 20-person femtosecond laser group, may I suggest Labview (www.ni.com)? It runs on Linux & Windows, many hardware cards support it, and it's honestly better than VB.
Your time is your most important resource. Don't waste it recoding.
R.
certain hardware manufacturers utterly refuse to support anything other than Windows
Since you mentioned you did some coding, you may want to check out Linux Device Drivers plus some of their other kernel tweaking/modding books.
simply because certain hardware manufacturers utterly refuse to support anything other than Windows."
Just pick your hardware manufacturers more carefully. There is plenty of analog and digital I/O boards for PCs that have Linux support. Even better, Linux is very popular on embedded systems (like PC104), so you don't even need a whole desktop PC but can use a small, embedded PC running Linux, together with hardware that comes with Linux drivers.
It is also my experience that manufacturers that ship Windows-only hardware are generally substandard. They probably don't support Linux because they are very tight on resources. If they don't give you low-level documentation, it's probably because they don't have it. And you end up between a rock and a hard place with that kind of hardware when VB wants you to upgrade your OS and their proprietary Windows driver won't work anymore.
UNIX itself has a very long tradition for experimental applications, so if there is nothing for Linux, consider getting a cheap Sun workstation with hardware that is supported under Solaris. That will still work a lot better than the Windows stuff, and it will interoperate nicely with Linux machines.
If you absolutely must do something on Windows, use Python, Perl, and/or wxWindows rather than VB. CygWin is also great. That way, your developers will acquire open source and Linux expertise and won't be locked into the Windows upgrade treadmill.
So, while occasionally some cheap peace of Windows hardware may seem alluring, if you just look around a bit more, you'll probably find something at least as good or better for Linux.
I have never known a lab that used VB as its programming environment. Usually it is either c/c++, Java, or one of the math programs (matlab, mathematica, maple, etc...). In general, I would recommend using microcontollers for controlling your experiments. However, you mentioned that you are doing stuff at the quantum level, so these may not be fast enough for you (the ones I use are 20MHZ). However, I must say that the PIC series of microcontollers can be programmed in a variety of languages and has a great deal of flexibility. One of the main problems is that a lot of the software for contolling lab equipment is either homebrew for a specific application (as is the case with some dynamic clamping software in the neurosciences) or made for a wide variety of applications (labview), but is not open source. The best option may be to get a company made environment which can have functions written for it in another programming language and customize it. However, coming from a biology side of things I do not know what your specific needs would be for quantum computing, and thus cannot give any ideas as to that specifically. Good Luck.
I've used comedi with a National Instruments NiDAQ 16 channel acquisition card on a P3 laptop running debian - it worked very well.
However, can I offer the following advice, which may save some people from smashing their head into a bloody pulp against a wall...
* Turn off APM!!! *
You can do this by passing apm=off to the Linux kernel with your bootloader (I think - can't actually check that at the moment) if you don't want to actually remove it from the kernel (APM is useful on a laptop normally).
If you don't do this, you might find your acquisition mysteriously stalling after random intervals. It's to do with APM interrupt handling. Not sure if it's restricted to PCMCIA cards.
Is it really worth your time, your professor's time, or the government grant's time to spend your quantum research dollar in overhead costs as you bang your head away in frustration trying to cludge together some string of 0.2 beta versions of open source data collection programs?
And what about when you leave? Does the next grad student have to spend 3 years learning your absolutely unique software setup instead of learning physics?
In the Big Name(TM) physics lab I work in, grad students cost about $200 a day (to the grant), and postdocs cost about $500 a day. If I need a program that would take me a month to write or costs $2,000 to buy today, it's my job do know to just buy the program.
We used to use LabWindows (call it C++) and VisualBasic, but the last person who know LabWindows left and now looking at the code when things go wrong is a nightmare.
So, anything new is being done in LabView. (Disclaimer - I don't work for National Instruments) Sure, it costs $2K for the good suite, but I guarantee you will make up for it in productivity. Plus, debugging LabView code as a beginner is waaaaay easier than debugging someone's crazy spaghetti C code. With the high turnover rate of a research university, it's very important to retain the chain of knowledge. Otherwise things progress into the realm of Black Boxes.
My opinion is not to waste your valuable research time worrying about software. Especially in quantum computing where you will be left in the dust if you fart around worrying about open source too long.
Best of luck,
Muerte
However you do not have to run VB only. My Windows2k box has opengl, openinventer, vtk, ativePerl, active Python, gVIm, cygwin, gnuc/c++, Devc++, ruby, tk/tlc, apache, php, etc.
Infact the win32 ports for these opensource apps are very well integrated with Windows. FOr example I can use gvim aka VI to replace my editor in VC++, create ole programs in python, and even use Perl to create Excell macro's.
Your Windows based collauges will get use to opensource and be more open later on after they get used to it.
As a scientist I assume you use VB for similiations and or to interface with your devices for experiments.
For similiations try vtk++ and openinventor. Your colleagues probably used them in Irix quite heavily. THe libary comes with a great
If your equipment provider only provides
http://saveie6.com/
Labview is okay, but I much prefer IGOR (www.wavemetrics.com). It's a pretty powerful data acquisition and analysis software package that my mesoscopic physics lab predominantly uses. It runs under Windows and Mac OSes and allows you to automate pretty much anything if you write the code for it. You can buy add-in software that allows you to use the GPIB interface, which many instruments carry.
W.
This way you can use python, and all of the open source goodess that comes with Python, plus you can link to those god forsaken DLLs.
To almost any programming question VB is NOT the answer.
In fact to many programming questions VB is the perfect solution. Why do you think there are so many VB programmers and VB apps out there? Its so easy a complete novice can learn how to build reasonable applications quickly. As applications start getting more complex then by all means use Java, C# or C++, with more experienced developers to do it properly. In a lot of cases VB is perfect for quick simple solutions. Some of the stuff I have seen talented coders produce in VB is not half bad either.
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
It wasn't so much the laptop falling asleep - just having the APM code active in the kernel was doing something screwy to the interrupt handling from the NiDaq card. It would fail acquisition after 15 seconds or so (although fairly randomly).
David Schleef pointed this fix out to me after I asked him very politely about what the $#@$ was going on...
We work on a DARPA project that runs a distrubuted multiagent system which is written in Java across a couple hundred dual-Xeon machines running Linux. The control mechanism is a Ruby application framework that uses Jabber for control and stressing of the multiagent application.
Not only is the project using open source as the infrastructure, but contributing to the open source community with projects like Cougaar and PMD.
We use Linux primarily because of the simplification of administration and maintanence.
Comedi is awsome. The very first driver described leads to The 8255 driver. It's author, Daniel Franklin, recomends "Alessandro Rubini's excellent book Linux Device Drivers (another fine O'Reilly publication)" Ahh, knowledge, what could be finer. Free software, free info. Go get it and become the research tech God you want to be.
Friends don't help friends install M$ junk.
I've worked at Princeton Plasma Physics Laboratory (A US DOE Lab) and we were using Linux for our visualization cluster...despite that it was based off of the Princeton University one which was running Windows.
My current work is at Rutgers University's Dataman Lab working on wireless sensors from Berkeley University and we also run Linux to program for TinyOS.
So all I can say is that there are some places where the researchers are using linux
United Electronics, www.ueidaq.com, make data acquisition cards similar to National Instruments and have linux driver support. Check them out.
If a user is running an app and he/she has just regular user priveldges then even a virus can not do damage. This is because the kernel checks access rights for files, directories, and programs. The important ones are marked "root" and unix will refuse to touch them unless the user is logged in as root. Windows NT, 2k, and XP have different users as well but the internal access rights are different. Any program can access the registry in Windows no matter what the user priveldge is. Thank Windows95 for making this standard. No app will run if you take that priveldge away sadly. This is where all OS and app information is and what causes 80% of the problems with Windows. Also as a developer you need administrator or root privedlges in WIndows to use com and the memory debugger. Another problem with WIndows.
Dll management is the other that causes those GP faults.
Unix is just better designed for this. Windows was never really designed but its a constant evolution from single user dos with a whole bunch of bandaid solutions.
One of these bandaids is the registry. The vast majority of screwups in Windows have do with an app or Windows itself corrupting the registry and fucking up the whole system with it. Only in W2k and XP 5 years after ms introduced the registry with WIndows95 did they began to fix this with registry tools and automated checkups. Its still really bad but registry corruption and dll management has improved. A bad app can still screw this up.
Everything in Unix is a text file so nothing gets corrupt unless its a hardware problem or a user did something dumb as root. Even the hardware is just text files in
Believe me I say that a bad programmer can write a bad app for every os. BSD is no exception but it certianlly is alot cheaper to use and operate. For a lab its essential to have good uptime. Especially if students needs these systems. A problem will usually never require a complete re-install. Just a fix with the app or config file that the particular app uses. Good os design is important in a lab environment.
http://saveie6.com/
It's free, supported, works on most platforms without rewriting the code, and has simple wrappers/interfaces for all kinds of programming languages, so you can still use those DLL's.
Voodoo Girl is the bomb!
I must disagree with the VB bashers here.
Our shop builds test equipment with lots of analog and digital I/O - some of it high speed.
We started off 20 years ago using C but switched to whatever-the-current-version of Microsoft's BASIC was at the time. Each system was unique and you can't beat VB for fast coding/debugging.
In the beginning we had to write assembler code to access the hardware regs on the acq boards for speed but when DLL's began to be supplied with the hardware things got much easier. The huge library of controls and DAQ add-ons for VB just cannot be ignored. For a laboratory the advantages would be similar - fast prototyping and debugging for unique applications with lots of off-the-shelf code and low labor costs. VB has it's faults but for this kind of work it is well suited.
There is really no economic advantage to using free software in this instance.
Labview is better to use to actually acquire the data, but Igor is amazing to display the data.
I work in an IR laser lab actually doing labview development and it is VERY easy to program with.
Either that, or I'm way behind in the quality of Open Source RAD tools
You're way behind in the quality of open source RAD tools.
The python-libglade solution isn't as flashy as VB, but it's an extremely easy way to throw together an easily modifiable GUI app *fast*.
(Btw, I presented on this particular combination to my university's LUG a year or two back after writing a fairly fancy virtual machine/simulator frontend literally overnight in about 100 lines of python code and an XMl file created by dragging and dropping in glade).
I'm currently working for a small research group which is part of a particle physics experiment and we are running entirely on Redhat systems, using many excellent open source tools made available by CERN. In my experience, a Unix like environment works orders of magnitude better than a windows environment, especially when it comes time to automate events. I can't even imagine trying to do what we do in a windows environment. It would be an absolute nightmare trying to run most of the program we write.
There's no sig like SIGSEG
I dont know in what time you live but it must be a long time ago .... .conf files and if people wouldnt just use ADMINISTRATOR on windows they couldnt do as much damage.
> One of these bandaids is the registry. The vast majority of screwups in Windows have do with an app or Windows itself corrupting the registry and fucking up the whole system with it. Only in W2k
> and XP 5 years after ms introduced the registry with WIndows95 did they began to fix this with registry tools and automated checkups. Its still really bad but registry corruption and dll
> management has improved. A bad app can still screw this up.
Apps can only screw up the part where the user has access rights - and even then it never happened to me.
Give your average user a linux box and he will kill himself over stupid
Its the users - a lot of dumb users using windows.
My XP box never ever crashed and it runs winxp 24/7 since you could download the first winxp iso. (Yes I do have a license - bought when it came out)
Same goes for Win2k - never crashed for me and performs blazingly fast as does winxp.
Its the users and you linux zealots should step down from your high horse and learn from windows because its indeed better than linux in many areas.
The only thing linux is still better is firewalling.
Why not use Gambas for the VB stuff in Linux?
I'm working in a lab doing "Quantum Chaos" experiments (manipulating cold atoms to investigate the difference between quantum and chaotic physics). We use the free (as in beer) version of RTLinux to run all our experiments, as timing is important and we didn't want to implement a hard real-time system. I coded most of the gui for the experimental interface using Borland Kylix and everything works quite nicely (apart from some evil memory leaks).
The real problem is the hardware - a real guru set that up for us. He wrote the "drivers" for the I/O cards himself (although that's meant to be a little easier in RTLinux than for normal linux) and also got a scientific grade CCD camera working even though the only linux drivers available were outrageously outdated. Sadly, we will definitely face some issues in the future if we want to upgrade to a new kernel!
Personally, I think the only way to move data files around is with a decent shell. Rename is perfect for all those times I put the wrong parameter in the file names of 160 different data sets. Most of the time our lab works quite smoothly with regard to the OS itself, and it's certainly an improvement on the old Win95/Scientific workplace combo of the past!
I work for one of the larger visual effects companies, and from our direct experiences, and from what I know of our competitors, linux is now basically a given. Over the past 6 months, we have yet to talk to a vendor who doesn't have a package for both windows and linux, or has a concrete "in 3 months" type schedule for their linux product line. We just got a product demo for a new software package written by a startup company, and the 2nd/3rd thing coming out of the rep's mouth was "linux version in 2 weeks". That's an amazing contrast with the laboratory field, I guess - It's been amazing how much the studios have jumped on the bandwagon. To guess, I think a lot of that driving force is the general cost savings - labs that are less "profit"-driven have less incentive to try new ways of doing things, but the visual effect industry is so cutthroat, everyone's always switching to the next "great" thing, and right now, that thing is linux. Will it stay that way? Almost definately, but you never know...
Now, if only someone could beat the SCO guys with a clue stick, we'd be all set. I haven't had any allback yet on that crap (no senior management wandering over asking me for comments), but it's bound to come if there's any serious setbacks in the case. Here's hoping that IBM works 'em over.
The registry has ACLS, just like the filesystem. You can only screw up the keys that you have access to (namely, your own personal profile).
Life in Orange County
control = embedded space (eurotherm or others for temperature control) ;-) but I couldn't care less) ;-)
log = whatever (chose labview...errr my boss chose labview
As an engineer, safety is one of the first things that should come to mind.
And in my experience embedded space provides this with much higher accuracy than any OS on any PC.
Ofcourse on Slashdot you read this extreme data logging things like over 6 months-extreme weather: to me those people did the right thing: linux-labview-comedi drivers.
But for logging-in-the-lab: anything goes.
e.g. log temperature and pressure every minute: we use qbasic-talking about cost control
No, we're not a russian lab...
For controlling motors and controlling through extreme high frequency daq: guess I would drop anything of the above and go with labview and the OS with the decent-driver-at-the-moment
This is going OT but is importand none on the less
Windows NT, 2k, and XP have different users as well but the internal access rights are different. Any program can access the registry in Windows no matter what the user priveldge is. Thank Windows95 for making this standard.
There I was typing away on a post marking you as a troll and then it hit me... You have been using regedit.exe on winnt based systems! Have a look at regedt32.exe, Done? You must have noticed the "security" menu on top. Here you can set full acl`s on every value or "directory" you want. All sorts of fine tuning options there. This beats the unix configuration system by like a hundred times on security. Its more complex (very bad in security land) but also offers great posibilities being able to limit acces to each induvidual value with an ACL with options (dis)allowing induvidual users (or programs run in their context) from querying the value, setting the value, creating subkeys,listing subkeys,"reporting","linking",removing, writing DAC data,writing owner and reading security info. (can`t translate them all from my localized version)Compare this to the crude all or nothing mechanism split by owner, one group and everyone else on a whole config file.Then add the fact that NT kernels have had full filesystem ACL`s (and remote filesystem acls) as well as basic mandatory acces controll (user rights policies) while linux has been strugeling to get even minor posix suport for ACL`s and has been doing nfs by default for a while now... And lets not forget the full acounting features posible on all of the objects with ACL`s on them in winnt.
Now dont get me wrong any OS that comes with a webserver that has had a plain dotdot bug in its filehandling (same bug in win95 remote filesystems) enabled by default is to be laughed at first befor being considered for use on the internet, but blame everything microsoft (office apps, "servers", scripting features, shell, manuals, patching policies, backward compatibility policies, monkey-see-monkey-do training, source distribution policies) for that except the security design of the NT kernel
I have been working with Kmax for as long as I can remember... It is platform independant, Java based data acquisition software that allows you to build graphical toolsheets to interface with many different bus types. It can talk to GPIB, CAMAC, VME, and any other bus that you can write a JNI driver for. You can get a free version of Kmax at Sparrow Corporation, and I think the only thing that is disabled is the Save feature...
From my experience, Kmax has been the most versatile data acquisition software I have ever used. The way it is designed, if you want something more, just write your own KmaxDevice or KmaxDriver (interfaces are documented) and you're all set! It even has options for remote connections over TCP/IP for client-server connections, useful if you want to take a look at your data from home or if you feel like changing some parameters without walking down the hall and mucking with your racks of equipment.
-EowaennorThere's a lot of VME/VXI hardware out there that will also work well with linux kernels (I've done it). Compact PCI (cpci) should work as well, although much of it seems to be driven by x86 / Windows embedded computers...
Any hardware which communicates to the workstation via a standard interface (ethernet, usb, gpib, serial, etc) should work just fine.
The real issue is simply checking the vendors of your hardware BEFORE you make the purchases to see if the hardware is supported. This should be true with any hardware purchase. (It is possible to buy hardware that doesn't work with Windows...)
If you're trying to use legacy instruments (which you already own) whose manufacturer does not support Linux and who refuses to release interface information (because of it's proprietary nature); well then, you're out of luck unless you can kludge an interface with a Windows PC talking to the device(s) (acting as an ad-hoc interface) and a Unix system doing the rest of the work. You'll get better performance this way than trying to have the Windows PC do all the work.
I'd urge you to use Unix or Linux variants for data acquisition and controlls. Windows is not deterministic and DOES NOT do real-time (I don't care what Microsoft says or how fast the machine runs).
that's just my 2cents worth...
gnu scientific library
and
gnu plot
Absolutely killer apps !!!
At PSU Physics , we use a variety of OSS for research. Most notable is our computing cluster which runs particle simulations and such on Linux. I'm unawae of any who have reversed drivers for their instruments to run on OSS in my department, however many researchers use PERL to analyze the results.
- about me