Slashdot Mirror
Running a Research Lab on Free Software?
"[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is. Who could blame them? Their Windows DLL is the perfect solution under [most] circumstances.
I'm not the only one around here getting frustrated, but all before me have been defeated. It seems I am to be as well, for today I have started to learn Visual Basic.
Has anyone had any *positive* experiences trying to move a lab from proprietary to free software? Surely the government-funded researchers of the world have a responsibility to ensure that their work is free, as in freedom. However, I have found out the hard way that it's usually just not worth the effort, following such ideals. You just get frustrated by apathetic colleagues, useless product support, and the conventional wisdom that it's OK to ignore your ideals, so long as you get the experiment working. Additionally, my ordeals convince my peers that free software isn't worth the trouble."
Research companies write their software in VB? No wonder there's still no cure for cancer!
For almost any government project i have seen, Windows is the choice by the government. Getting them to switch over makes no sense to them, because why switch when you have something that works? Cost benefits don't really seem to do anything, but they seem afraid of switching and trying something new because Windows is just the way it has been, and will continue to be for them. Research might be the same way. UNlesss their research IS software like this, they may just want to stick with what has already beenw orking for them.
I've found it somewhat difficult to use since building their modules is really suggested on a generic rather than stock Redhat kernel (and building with what are claimed to be the RH sources and config files didn't work for me).
Funny thing is, the two and a half weeks that the "all BSD" lab lasted, we had no work orders concerning crashes. I learned BSD only because the LAN manager in question made me learn it to move a email server over to it (Novell GroupWise used to be VERY expensive, and this square-headed manager didn't want to pay a bunch of cash to run one secure and isolated mail server on Novell stuff).
Business is business I guess.
As someone who has been a sysadmin for a 20-person femtosecond laser group, may I suggest Labview (www.ni.com)? It runs on Linux & Windows, many hardware cards support it, and it's honestly better than VB.
Your time is your most important resource. Don't waste it recoding.
R.
I work in a Corporate R&D lab and we're pushing open source technologies here, but you have to be patient and strategic.
First off, are you trying to make programmers convert? If so then you're in a losing battle. People will almost always stick with what they're comfortable with. You'll only get them to look at something else if it is 1 or 2 orders of magnitude simpler to use.
Set an example with your work. If you can do your work using OS tools and you can do it quicker, cheaper, and easier then that's how you convince people. Comments like, "But VB is lame. MS is the anti-christ. etc..." do NOT a good case for conversion make.
Finally, if you can't do it in OS then don't. If you're stuck because of specific hardware/OS issues then don't try to fight the beast on those for now. Pick projects and things that you can migrate and move those instead. (Move the intranet site to JSP or PHP or whatnot on Linux/BSD/Apache. Throw out some NT/2000 Domain boxes and use SAMBA instead.) If you can show the advantages of OS tools for certain tasks then you can get people thinking about it. If you can't do it because of the hardware you're using then you're just going to appear as a stubborn zealot to your colleagues.
--- I wish I could hear the soundtrack to my life. That way I'd know when to duck.
Actually, I sometimes feel that this is the number one symptom of the obstacles that OSS/free software face. Multiple times, I have been labeled a nut for wanting to us Linux/OSS. I had a roommate in the dorms once who insisted that I would be "happier" using Windows.
As far as experience moving labs from Windows to OSS, I have never moved an entire "lab" so-to-speak to Linux, but it is my experience that in the past three or so years since I was introduced to Linux, it has made big inroads into becoming easier to install and use. The hardware support is definitely better. I remember when I first tried Linux, support for USB mice was still experimental.
These days, migrations from Windows to Linux have been relatively smoothe. I've had great success moving a fileserver from a Windows NT workstation to a less-used Linux box. Usually, it will eventually go down because of a power outage - so I recently purchased a UPS.
The bottom line is, I think that the problem of colleages who drag their feet on using Linux/OSS will be reduced as major distros become increasingly easy to setup and operate and hardware support improves.
Wh47 d1d j00 541, 31337 15n't t3h r0xor5 ne m0r3???
simply because certain hardware manufacturers utterly refuse to support anything other than Windows."
Just pick your hardware manufacturers more carefully. There is plenty of analog and digital I/O boards for PCs that have Linux support. Even better, Linux is very popular on embedded systems (like PC104), so you don't even need a whole desktop PC but can use a small, embedded PC running Linux, together with hardware that comes with Linux drivers.
It is also my experience that manufacturers that ship Windows-only hardware are generally substandard. They probably don't support Linux because they are very tight on resources. If they don't give you low-level documentation, it's probably because they don't have it. And you end up between a rock and a hard place with that kind of hardware when VB wants you to upgrade your OS and their proprietary Windows driver won't work anymore.
UNIX itself has a very long tradition for experimental applications, so if there is nothing for Linux, consider getting a cheap Sun workstation with hardware that is supported under Solaris. That will still work a lot better than the Windows stuff, and it will interoperate nicely with Linux machines.
If you absolutely must do something on Windows, use Python, Perl, and/or wxWindows rather than VB. CygWin is also great. That way, your developers will acquire open source and Linux expertise and won't be locked into the Windows upgrade treadmill.
So, while occasionally some cheap peace of Windows hardware may seem alluring, if you just look around a bit more, you'll probably find something at least as good or better for Linux.
..you choose whatever platform/software that will do the job the best.
Functionality
Availability
Budget
Useability
Philosophy is far down the list.
Clearly VB is the winner here as it perfectly mimics the unpredictability of quantum mechanics!
Sounds like a Michael Crichton recipe for technology gone bad. I bet the fat guy in your lab gets it first.
"And this is my boy, Sherman. Speak, Sherman." "Hello." "Good boy."
I have never known a lab that used VB as its programming environment. Usually it is either c/c++, Java, or one of the math programs (matlab, mathematica, maple, etc...). In general, I would recommend using microcontollers for controlling your experiments. However, you mentioned that you are doing stuff at the quantum level, so these may not be fast enough for you (the ones I use are 20MHZ). However, I must say that the PIC series of microcontollers can be programmed in a variety of languages and has a great deal of flexibility. One of the main problems is that a lot of the software for contolling lab equipment is either homebrew for a specific application (as is the case with some dynamic clamping software in the neurosciences) or made for a wide variety of applications (labview), but is not open source. The best option may be to get a company made environment which can have functions written for it in another programming language and customize it. However, coming from a biology side of things I do not know what your specific needs would be for quantum computing, and thus cannot give any ideas as to that specifically. Good Luck.
I've used comedi with a National Instruments NiDAQ 16 channel acquisition card on a P3 laptop running debian - it worked very well.
However, can I offer the following advice, which may save some people from smashing their head into a bloody pulp against a wall...
* Turn off APM!!! *
You can do this by passing apm=off to the Linux kernel with your bootloader (I think - can't actually check that at the moment) if you don't want to actually remove it from the kernel (APM is useful on a laptop normally).
If you don't do this, you might find your acquisition mysteriously stalling after random intervals. It's to do with APM interrupt handling. Not sure if it's restricted to PCMCIA cards.
Is it really worth your time, your professor's time, or the government grant's time to spend your quantum research dollar in overhead costs as you bang your head away in frustration trying to cludge together some string of 0.2 beta versions of open source data collection programs?
And what about when you leave? Does the next grad student have to spend 3 years learning your absolutely unique software setup instead of learning physics?
In the Big Name(TM) physics lab I work in, grad students cost about $200 a day (to the grant), and postdocs cost about $500 a day. If I need a program that would take me a month to write or costs $2,000 to buy today, it's my job do know to just buy the program.
We used to use LabWindows (call it C++) and VisualBasic, but the last person who know LabWindows left and now looking at the code when things go wrong is a nightmare.
So, anything new is being done in LabView. (Disclaimer - I don't work for National Instruments) Sure, it costs $2K for the good suite, but I guarantee you will make up for it in productivity. Plus, debugging LabView code as a beginner is waaaaay easier than debugging someone's crazy spaghetti C code. With the high turnover rate of a research university, it's very important to retain the chain of knowledge. Otherwise things progress into the realm of Black Boxes.
My opinion is not to waste your valuable research time worrying about software. Especially in quantum computing where you will be left in the dust if you fart around worrying about open source too long.
Best of luck,
Muerte
[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is.
You have to phrase your request nicely:
"Give us the technical specs, or we will crack your company encryption keys with our quantum computer, access all your specs, and post them on usenet."
However you do not have to run VB only. My Windows2k box has opengl, openinventer, vtk, ativePerl, active Python, gVIm, cygwin, gnuc/c++, Devc++, ruby, tk/tlc, apache, php, etc.
Infact the win32 ports for these opensource apps are very well integrated with Windows. FOr example I can use gvim aka VI to replace my editor in VC++, create ole programs in python, and even use Perl to create Excell macro's.
Your Windows based collauges will get use to opensource and be more open later on after they get used to it.
As a scientist I assume you use VB for similiations and or to interface with your devices for experiments.
For similiations try vtk++ and openinventor. Your colleagues probably used them in Irix quite heavily. THe libary comes with a great
If your equipment provider only provides
http://saveie6.com/
Forget the old stuff, concentrate on new projects, number crunching and data servers. Use the strengths of free software first, then move into areas you can. Old stuff is working, as well as any old M$ junk works, and you will only be frustrated working with equipment no one cares about. "Rebuild" it till it smokes. You are familiar with the superiority of free software available for networking, number crunching and programming. Take old computers no one wants and make them useful. Something as simple as databases, web servers and email servers are helpful. Take the hardware advice you get here and put it into your next proposal. The bottom line will speak for itself and you will have proven the dependability of free software on comodity hardware already.
Friends don't help friends install M$ junk.
You guys are trying to get Windows running on quantum computers? Talk about uncertainty!
To almost any programming question VB is NOT the answer.
In fact to many programming questions VB is the perfect solution. Why do you think there are so many VB programmers and VB apps out there? Its so easy a complete novice can learn how to build reasonable applications quickly. As applications start getting more complex then by all means use Java, C# or C++, with more experienced developers to do it properly. In a lot of cases VB is perfect for quick simple solutions. Some of the stuff I have seen talented coders produce in VB is not half bad either.
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
ActiveState: Python, Perl, Tcl, etc for Windows.
"God fights on the side with the best artillery." - Napoleon, Marshal of France - speaking truth to power
don't fix it. For fuck's sake, did you read your own submission?
I know what I'm missing out on, in the free software world.
Followed immediately by:
I've wasted a *lot* of time and effort trying to implement some very simple stuff with free (and better) alternatives
Yeah you're missing out on the struggle and pain of hacking together ad-hoc solutions to an already-solved problem.
Way to go, buddy.
certain hardware manufacturers utterly refuse to support anything other than Windows
Since you mentioned you did some coding, you may want to check out Linux Device Drivers plus some of their other kernel tweaking/modding books.
Buying a book isn't going to get her the board spec, unfortunately. To get the job done quickly, she'll need the board spec.
Otherwise, she could start by using the generic pci driver to probe the board's parameters in freebsd, or use phob to observe the board's parameters.
This would be an excellent chore for a grad student, as it would provide her with a useful skill while not really interrupting her studies of, e.g. The Effect of Gamma Rays on Man-in-the-Moon Marigolds
Been there, done that. The problem is in obtaining the knowledge. Hey, if they want Linux support all they have to do is make it only mildly difficult and people like us will give it to them. Then they can keep profiting from us. But if they make it so hard that you have to reverse-engineer binary drivers, then they can go find themselves somebody else to sell to.
It's free, supported, works on most platforms without rewriting the code, and has simple wrappers/interfaces for all kinds of programming languages, so you can still use those DLL's.
Voodoo Girl is the bomb!
I must disagree with the VB bashers here.
Our shop builds test equipment with lots of analog and digital I/O - some of it high speed.
We started off 20 years ago using C but switched to whatever-the-current-version of Microsoft's BASIC was at the time. Each system was unique and you can't beat VB for fast coding/debugging.
In the beginning we had to write assembler code to access the hardware regs on the acq boards for speed but when DLL's began to be supplied with the hardware things got much easier. The huge library of controls and DAQ add-ons for VB just cannot be ignored. For a laboratory the advantages would be similar - fast prototyping and debugging for unique applications with lots of off-the-shelf code and low labor costs. VB has it's faults but for this kind of work it is well suited.
There is really no economic advantage to using free software in this instance.
It wasn't true until .NET, but now I have to say, after using about 1/4 of the languages out there (and that's still a lot), VB has truly matured. I'd say that for anything except embedded software, or that which simply must work on *ix, VB is probably the best choice overall. Consider things like ease of use, learning curve, cost per hour per programmer, and it makes some sense.
There's two basic tricks I've discovered over the last couple of years of slowly incresing the use of open source platforms and tools in our research (ok, we do behavioral science stuff, but the politics of IT change are the same).
First, re convincing colleagues that open source / free software has a role in your work: do something they envy. Produce a tool they want to use, or find some existing software that does something useful and cool, or even just do the great unix thing of tying a bunch of small programs that do one or two things well together to do something that no existing monolithic package really offers. Then point out that it either can't be done on the current platform of choice, or, while it can be done, it requires spending $$$ on some proprietary solution. Doing something like this tends to legitimize the use of the toolset you'd like to use, and gives you a good foot in the door for more abitious moves later.
Second, re working with third party suppliers who don't currently produce software or drivers or whatever that work with non-MS platforms. If there's more than one vendor who supplies something that does what you want, pick the smallest one. They're more likely to be interested in finding niche markets, less likely to be bogged down by bureacracy when it comes to doing something new or different. And a three-person company is more likely to have two of the three who've recently been working in your field & remember what it's like trying to do the usual research thing of trying to get an existing tool to do something that no-one's done before - hence more likely to give you access to the kind of more detailed information you might need, even if they can't really expend the effort themselves right now.
Anyway, that's my take on 'what worked' after a couple of years of win-some, lose-some politics around research and IT.
I work for a Navy research lab and we use Linux, Perl, C, Apache, Mozilla, Gnome, Perl/Tk, GnuPlot, etc almost exclusively for our control and monitoring systems. We have had no difficulty controlling Opto22 devices, GPIB, RS422, R485, and one-wire devices. The only windows we use is on the desktop which is unfortunately mandated (see NMCI).
If you need real-time look into QNX.
BTW we operate what is probably the deepest running webserver. We have a vehicle monitoring system that is controlled via a Linux/Apache machine setting at depth reporting topside via fiber connection. This system controls power reports temperatures, pressures, inclination, depth, smoke, leak, etc.
Either that, or I'm way behind in the quality of Open Source RAD tools
You're way behind in the quality of open source RAD tools.
The python-libglade solution isn't as flashy as VB, but it's an extremely easy way to throw together an easily modifiable GUI app *fast*.
(Btw, I presented on this particular combination to my university's LUG a year or two back after writing a fairly fancy virtual machine/simulator frontend literally overnight in about 100 lines of python code and an XMl file created by dragging and dropping in glade).
I've been working with one of the Physics profs at my school (U of Rochester) for the past year, helping him update the software they use for cosmic particle experiments. We use a data aquisition board and particle detectors designed by FermiLab. The software runs on Linux, and accesses the DAQ board through the serial port. My job has mostly been adding a GUI to the program, so that the students running the experiments can concentrate more on getting results than understanding the weird command line interface for the program. For more info on the project, see the FermiLab page for the QuarkNet project, and the PARTICLE project page at the university.
My first choice is Delphi. I don't think I'd ever say Delphi is better at creating quick`n`dirty apps than VB, but I would most certainly say that it is completely on par in that area, with the added benefit of being much more powerful. (My opinions here are based on VB6 and Delphi5, which are the last two I used heavily before being liberated from Windows GUI work.)
The other alternative I can think of is RealBASIC. Their development environment used to only run on Mac OS, even though it could compile apps for either Mac OS or Windows. Nowadays, the environment itself as well as the apps it creates all run on both Mac OS 9/X and Windows, although I've never used the Windows development environment. I've only had limited exposure to RealBASIC, but based just on those few hours, I would highly recommend any fan of VB at least give it a shot--I know if I ever have to go back to Windows GUI work, I certainly will. (It seems it would especially shine for quick`n`dirty apps because it seems to focus more on simplicity and cross-platform rather than feature bloat.)
Perl - $Just @when->$you ${thought} s/yn/tax/ &couldn\'t %get $worse;
I'm working in a lab doing "Quantum Chaos" experiments (manipulating cold atoms to investigate the difference between quantum and chaotic physics). We use the free (as in beer) version of RTLinux to run all our experiments, as timing is important and we didn't want to implement a hard real-time system. I coded most of the gui for the experimental interface using Borland Kylix and everything works quite nicely (apart from some evil memory leaks).
The real problem is the hardware - a real guru set that up for us. He wrote the "drivers" for the I/O cards himself (although that's meant to be a little easier in RTLinux than for normal linux) and also got a scientific grade CCD camera working even though the only linux drivers available were outrageously outdated. Sadly, we will definitely face some issues in the future if we want to upgrade to a new kernel!
Personally, I think the only way to move data files around is with a decent shell. Rename is perfect for all those times I put the wrong parameter in the file names of 160 different data sets. Most of the time our lab works quite smoothly with regard to the OS itself, and it's certainly an improvement on the old Win95/Scientific workplace combo of the past!
[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is.
What exactly did they say when you asked? Have you made sure that they understand what you want to do? (Create a driver that makes the card work on linux, that anyone can get, potentially increasing the sales for the card). The key is to present the request not as "we need this" but as "you will get this if we can get that". They may still not be willing to help and then you explain that whenever you do the purchasing decisions you will prefer a company that provides specifications (or linux drivers). They still might not listen so you may have to wtick with windows, just make sure you remember who foreced you to it whenever you get a budget to buy new equipment.
- We are the slashdot. Resistance is futile. Prepare to be moderated -
Recently this went so far that I had a very candid talk with one of their sales people. I made it clear I would move to a different manufacturer if they would not provide drivers or some means for me to write them. He would ask within their organisation. Several days later I received an email titled "Solution to your GPIB driver problem". To my astonishment, it gave contact details for several other manufacturers of GPIB cards!
So, isn't this weird? They'd rather lose our custom than provide us with a driver, or sufficient details to write our own! And given the amount of stuff we are buying from them that's a pretty big decision.
So now we are talking with National Instruments, and they are very pleased to have us as customers. Moreover, they have Linux drivers for their GPIB cards. I haven't seen these drivers yet, and I sure hope they will be of acceptable quality.
I'm interested in hearing what experiences other people here have had with GPIB cards under Linux - especially those of NI, which we are about to buy. When you mention lousy Linux support, does that include their GPIB cards?
I was in a similar position...I was using Linux to run a laser tweezer setup in our small research group. Linux was such a pain - inconsistent libraries, poor support, flaky drivers unsupported by vendors, etc.
I switched to Windows 2000 Pro for the vastly superior support for data acquisition hardware, framegrabbers, and so on. Along the way I switched from doing GUI stuff in wxWindows/C++ to Matlab. The GUI stuff in Matlab 6.x is pretty good. It's got a bigger learning curve than LabView but it can also be used to implement much of your data analysis as well, with its vast libraries of mathematical functions and decent performance.
With the new hardware coming out, our lab now has a standing policy to only purchase items with USB or FireWire interfaces whenever possible, so the PCs don't even need to be opened up anymore. These drivers are rarely if ever available for Linux.
Use whatever makes your lab the most productive. Standard lab software like LabView, Matlab, and Mathematica are a safe way to implement software since they're so popular and they're more efficient and productive than C/C++ or VB. If you need high performance computing, then go consider F90, C++, etc., but instrumentation should never be controlled like that. You need real time? Buy a real time board.
I am a physicist at Vanderbilt University, in the Free Electron Laser Center, who has been doing various kinds of distributed data acquisition and control for about 25 years. I have run into many of the same problems with closed interfaces the author describes, and am slowly developing a strategy to minimize the impact of this problem.
First, I try to avoid products which depend on a closed library/DLL/whatever to control them. This has resulted in my shifting away from a lot of PCI-card devices to (when possible) external devices which communicate via GPIB/IEEE-488, rs233/422/488 interfaces, USB interfaces, network interfaces and (I hope soon) FireWire/IEEE-1394 interfaces. For such devices, one can often find programming specifications, although not always. Obviously, using the slower interfaces may result in lower performance in high-bandwidth environments, so it won't always be an option. However, the fastest current IEEE-1394 looks very promising, as it can support speeds that only a few years ago would saturate a PCI bus.
I have also discovered another interesting phenomenon: 'hidden' standards. In the past year, I became aware of a little-advertised standard, VXI-11, (www.vxi.org), which is a protocol for communicating with GPIB-like devices over TCP/IP. Although one almost never hears about it, a lot of devices support it. Tektronix and Agilent Infiniium scopes use it, and Ethernet-GPIB converters from Agilent, Tektronix and National Instruments all use this. The protocol is open, sunrpc based, and quite easy to implement. However, each company hides any reference to it deep inside the documentation, and basically provides their own Windows dll for communicating with their devices. The Agilent E5810 Ethernet-GPIB converter (a very elegant box) even calls itself a GPIB-LAN interface for Windows, as its official product name, even though it is almost fully VXI-11 compliant and can be used from any platform. I have no idea why they actively _hide_ its cross-platform compatibility.
<slight advertisement> I am trying to address some of these issues by releasing a lot of the code I am working on to interface various devices. I am a python fan, so I have a sourceforge project PythonLabTools which is a library of open-source code to implement communications with various types of devices which are effectively GPIB devices but run over the net via TCP/IP. I am also adding other classes of interface support to this library. An example is the Verinier Software LabPro, a low-end but quite nice and inexpensive a/d, d/a, and digitial i/o box which communicates via serial and USB. There are also data analysis tools in this package (fitting, et.c), and support for the National Instruments DSTP protocol, which allows LabVIEW to share data over a network, and (in this case) allows a Python program to directly interact with a LabVIEW program. This allows separation of the fancy user interface capabilities of LabVIEW from more intensive data analysis which can be executed in Python. <end advertisement>
I am looking forward to a day when more and more external devices with published interfaces become available. Internal devices (PCI or whatever), of course, take a lot of work to make drivers, but since communications with external devices can be carried out in userland, they are easier to provide cross-platform support for. I am alos watching as bandwidth on external interfaces rises to the point where there may not be any need for internal cards and the hassles they create.
I spent a fair amount of time a couple days ago lobbying our National Instruments representative for opening up the interfaces to more of their devices, especially their FireWire/1394 based products, which are right now only supported on Windows (which _really_ galls me, since Apple and Sony spearheaded this interface!). We are a fairly significant customer of theirs, and I think they actually listen.
There's a lot of VME/VXI hardware out there that will also work well with linux kernels (I've done it). Compact PCI (cpci) should work as well, although much of it seems to be driven by x86 / Windows embedded computers...
Any hardware which communicates to the workstation via a standard interface (ethernet, usb, gpib, serial, etc) should work just fine.
The real issue is simply checking the vendors of your hardware BEFORE you make the purchases to see if the hardware is supported. This should be true with any hardware purchase. (It is possible to buy hardware that doesn't work with Windows...)
If you're trying to use legacy instruments (which you already own) whose manufacturer does not support Linux and who refuses to release interface information (because of it's proprietary nature); well then, you're out of luck unless you can kludge an interface with a Windows PC talking to the device(s) (acting as an ad-hoc interface) and a Unix system doing the rest of the work. You'll get better performance this way than trying to have the Windows PC do all the work.
I'd urge you to use Unix or Linux variants for data acquisition and controlls. Windows is not deterministic and DOES NOT do real-time (I don't care what Microsoft says or how fast the machine runs).
that's just my 2cents worth...
not just a pride thing, unlike many colleagues I know what I'm missing out on, in the free software world
Like what, for example? Under Windows you can run Emacs, Vim, Perl, Python, Ruby, etc. You can get bash or another UNIXy shell if you like, and all the same command line tools. Putting together GUI interfaces for little tools with Visual Basic is a *great* idea. You could use something else, of course, like Tcl/Tk, or a free VB-like system, but VB is very good at what it does.
Reliability isn't an issue, if you're running Windows 2000. I have never had a single crash doing heavy software development under Windows 2000.
To me, it sounds like you just want to avoid Microsoft and Windows at all costs, but you don't have a real reason. In fact, you're even attempting to move away from the OS that most of your peripherals are designed to run under. Very strange.