Running a Research Lab on Free Software?

Neurotensor asks: "My lab is researching quantum computing, and I don't like the fact that Windows / Visual Basic [seems to be] the preferred solution for controlling the experiments. It's not just a pride thing, unlike many colleagues I know what I'm missing out on, in the free software world. I've wasted a *lot* of time and effort trying to implement some very simple stuff with free (and better) alternatives, simply because certain hardware manufacturers utterly refuse to support anything other than Windows." There were older articles that touched on this subject, 2 years ago, but are others still finding themselves in the same situation as the submitter?

"[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is. Who could blame them? Their Windows DLL is the perfect solution under [most] circumstances.

I'm not the only one around here getting frustrated, but all before me have been defeated. It seems I am to be as well, for today I have started to learn Visual Basic.

Has anyone had any *positive* experiences trying to move a lab from proprietary to free software? Surely the government-funded researchers of the world have a responsibility to ensure that their work is free, as in freedom. However, I have found out the hard way that it's usually just not worth the effort, following such ideals. You just get frustrated by apathetic colleagues, useless product support, and the conventional wisdom that it's OK to ignore your ideals, so long as you get the experiment working. Additionally, my ordeals convince my peers that free software isn't worth the trouble."

Comments

Uh...

[Anonymous Coward]

Research companies write their software in VB? No wonder there's still no cure for cancer!

Re:Uh...

[Black Copter Control]

puns aside, I presume that in research many apps are written to test a particular algorithm and thrown away.

When you pay $20K-$200K for a piece of hardware, you find as many ways to use it as possible before you throw it away. Just because a new version of the software comes out a year or two later doesn't mean that the lab is going to be able to afford the new version. If a lab can afford a new version of the $100K toy, chances are that they'll had-me-down the older version to the undergrads.

I can understand that a company doesn't want to support more than one or two OSs, but it gets reall annoying if they don't give you the data so that you can roll your own support.

I take it that there isn't a different company that you can go to to get an equivalent machine??? If there is, then try playing them off against each other. For the more expensive equipment, at least, they should be willing to do some work to get the sale -- if that means releasing the register data, then they'll probably do it.

Re:Uh...

[0x0d0a]

Specifically quick-and-dirty Windows *GUI* apps, kind of like Hypercard was for the Mac.

The question is, if you're going to write a single app and throw it away, why are you wasting time on a GUI at *all*?

Re:Uh...

[fshalor]

The problem is, even in the 20K-200K range of hardware, the software usually sucks. Or the install isn't stable.
One example, (albiet cheaper one) I've been dealing with is a simple scale that sends data over a serial port to a PC. Using VB, you can occasionally get reliable results while taking data into Excel. Even through the software to do this cost as much as the scale.
On the other side of my fence, I've been using Linux for about a year now to do video capture and acquire temperatures from a networked DAQ unit. It's effortless after installed.
I'm looking into similar linux based solutions to running two old units, a DDA-06 and a DAS-16 to record thermocuple data realtime. The linux projects for these are actually really good, far exceding the windows based controls. The problem is the age of the boards (ISA, mid 80's) which are actually still sold today by several vendors including Omega. Unfortunatly, you need to be a god of electrical juices inorder to calibrate the darn things.

Re:Uh...

[flatrock]

For products with simple hardware interfaces, releasiing the register data is often a good solution in order to sell products to customers with very specific needs. At the company I work at we do release the register data for several of our products. It's in the hardware manual you get with the products, and our application engineers often have experience writing drivers for those products and can answer some questions to help customers write their own drivers.

However, we don't mark our products up enough to pay for us to provide the kind of support that some customers need while writing their own dirvers.

I just got a phone call while typing this and just spent the last hour trying to get information for a customer about a hardware feature that we don't use in our own drivers, so none of our software engineers could tell me how it works.

Supporting someone else's driver development is not a simple task when the hardware is not simple task, and the customers usually don't have the proper tools such as a PCI bus analyzer to debug problems when they occur. I've done a lot of ports to a lot of systems, and seen a lot of problems where the driver is written correctly, however the motherboard manufacturer of chipset designer didn't follow or implement the PCI spec properly and problems result. This could result in poor performance, or even data being dropped between the PCI Bus and the memory controller.

Supporting other people's developement can be very expensive.

Supporting Linux drivers also has it's own issues. We provide Linux drivers for our products on both X86 and PPC systems. I've seen drivers that work fine with one kernel revision break with the next minor revision. Linux customers also like to apply kernel patches which sometimes break our drivers. It's impossible to properly retest a complicated driver every time a new kernel revision comes out. We often ship customers a borad and a driver that was tested with a version of Linux other than what the customer is using. We warn the customers of this, however if they run into problems we end up having to load a system with that version of Linux and testing it to discover if there really is a problem, or if the customer has doen something strange with their configuration.

Even though we have Linux drivers there isn't a huge demand for them. This results in us spending a lot and time and money supporting an OS that isn't making us as much money. This is one of the main reasons a lot of hardware manufacturers don't bother with Linux support.

One other reason is that you often use third party ASICs in your products, and how to interface whith their registers is often provided under NDA, so that it's impossible to release to a customer without proper NDAs and licensing agreements.

Yet another reason is that often the hardware provides only a small portion of the functionality of the product. The rest of what makes the product valuable is done in the driver. If you have a product that uses commodity hardware and provides additional functionality though software, you can't simply give away the source code to that software and still pay for your development. People will take your software and buy the hardware from someone else, which can sell the hardware cheaper because they didn't invest in the software development.

Many of these issues depend on the hardware involved. Others depend on the skill and resources of the customer.

In the case of our Company we are willing to work with customers to provide reasonable solutions. However for customers that want to buy one hardware unit, and want us to help them develop a custome driver for their particular OS and platform, those solutions are usually very expensive.

If a driver on Windows with an interface writter in Visual Basic meets the technical needs of your project (including reliability) then it's often a pretty cost effective choice.

Linux in Research

[Linthos]

For almost any government project i have seen, Windows is the choice by the government. Getting them to switch over makes no sense to them, because why switch when you have something that works? Cost benefits don't really seem to do anything, but they seem afraid of switching and trying something new because Windows is just the way it has been, and will continue to be for them. Research might be the same way. UNlesss their research IS software like this, they may just want to stick with what has already beenw orking for them.

Comedi

[vondo]

It's not completely clear to me what exact problem you are trying to solve, but if you are looking for a way to communicate with a variety of data aquistion and control boards, comedi may be part of your solution. It's developed, as I understand it, by LBL (Lawrence Berkley Lab, a.k.a. the US government). They have a list of about 100 boards they support.

I've found it somewhat difficult to use since building their modules is really suggested on a generic rather than stock Redhat kernel (and building with what are claimed to be the RH sources and config files didn't work for me).

Re:Comedi

[Neurotensor]

Well it's so simple you're gonna laugh, I'm controlling a two-axis pulsed-laser spectrometer. The problem is that I've got a National Instruments PCI-7324 stepper motor card to work with and even Comedi (which looks kewl BTW) doesn't support it. I can't blame them, there's just no information out there to start with. NI won't even reply to my emails.

I thought it would be clever to implement a GUI using QT and then simply compiling on Windows with the stubs filled in to use the DLL. How wrong I was. The GUI looks great, but Trolltech's official position is that the Non-Commercial version of QT for Windows isn't supported any more, so you have to use QT 2.3 when 3.1 is out for Linux. I would have even ported my code back to 2.3 but then I would still need MS Visual C++ or Borland C++ Builder 5 (yes, I have version 6 and that's not binary compatible with the QT distribution). In case you're wondering, I did try MinGW but in the time I had available I didn't get anywhere because qmake still looks for a C++ Builder 5 DLL which I don't have. I could download the MS VC++ version of QT 2.3 but I expect to have the same problems all over again. And really if you've paid good money for a development environment like Borland C++ Builder 6, you might as well just use it instead. Oh and I did try to get the NI software running under Wine, I think I might have had some luck that way but I couldn't see how to get the Windows DLL talking to the PCI card in the time I had.

Then came a flurry of ideas for changing the stepper card to something else, say a timer/counter card, and use Comedi. But I quickly came to the conclusion that I've wasted enough time, I have a working stepper card already, and my supervisor really wanted a product using VB anyway because it's the easiest way to get physicists with zero coding experience to use and modify the application to fit their needs. It's logic I can't really beat right now, I wish it weren't so, I *so* wish I could hand him a free software solution with the same benefits to him as a VB solution, but I just can't.

Oh and if you were still wondering, I have to use Win95. That or replace it with the free (as in beer) OS of my choice, which would be Mandrake 9.1. Maybe I'll get an upgrade to Win2K but it bothers me to pay good money for a piece of software I wouldn't need if a) the original Win95 worked without blue-screening randomly, or b) NI would have some Linux support.

There's a guy upstairs who used to have an entire lab running GNU/Linux until he battled Agilent's and NI's lousy Linux support. He finally gave up and converted his lab from Linux to Windows (at great financial cost) so he could keep working on his experiments. Along came the customary driver conflicts, forced expensive updates, etc. etc. It's a sad tale, and it's what I was told when I began asking around in desperation. Now I know he wasn't kidding around :(

Re:Comedi

[Anonymous Coward]

You're REALLY not gonna like this. Not long ago NI came and gave a talk at my school (University of Texas) and basically said "the future is .NET." When asked about Linux, his response was basically "Why would anyone want to use Linux? Windows works." This angered a lot of the crowd (mostly CS students) and then he added that any job applicants better learn .NET or they had no chance. They really don't seem interested in Linux at all.

The basic fact of the matter is that there are industries where free software is not an option. Sometimes you have to abandon your ideals to get things done (as it seems those who came before you did.) Why do you think researchers are always begging for money? They have to pay for expensive, shitty software and there's no other alternative to do the things they want to do. It sucks, but Linux isn't the best solution to every problem (it probably would be in this case, but it's not an option because your hardware doesn't work.) Sometimes that's the way it goes.

Re:Comedi

[malakai]

First off, skip VB6 and anything before it. Go to .Net. You can do true OO programming using VB.Net, C#, C++ and a bunch of other .Net CLI languages. They can all communicate with ANY sort of DLL/COM combo your board manufacutuers give you.

Second, dump Win95 NOW. Your logic on why you don't want too is flawed. WinNT/2K has nothing in common with Win95. They are Two different products. Your not "Upgrading" your buying a different product.

Win95 was designed for the home, and a stepping point from DOS. Not as some sort of platform for stability and IO/Data Gathering, Realtime acquisition..etc. It's a hack and an old one at that. Would you use a build of Linux from 1995? 98? Hell no.

I'm really serious on this last point. Many people fail to realize just how stable Windows 2000 is. I'm amazed Win95 is still floating around in labs like this. Win98 is no better either (same thing). Did it come on the computer? I think i've been running win2k for nearly 5 years now in October. That's about the limit for a development platform imo (win2003 i'll ugprade to at SP1 level).

If you can find an Open Source alternative using Linux and other free stuff great. But honestly, I bet you end up spending more time getting things up and running and working, then programming/inventing.

Whatever you do, dont use VB6. It's not worth 'learning'. Stick to a .Net language, or if you're a good programmer, and you know C/C++ use that.

-Malakai

Re:Comedi

[Lumpy]

Earth to clueless person....

he cant. thereare TONS of real reasearch boards out there that cost more than you will ever spend on a complete computer that is 100% incompatable with Windows NT and it's diraviatives like 2000 and XP.

I have a drawer FULL of $7000.00 a piece video analysis cards that cannot be used in anything higher than a Windows 95 box. and we require them here for several processes and projects.

so get a clue first. Microsoft is a consumer OS not a reasearch OS so they happily ditch compatability all the time with hardware.

VB6 is the last VB that isn't bloated all to hell. I actually reccomend that everyone use VB4 for the quick and dirty here as it's 1/5th the size and much faster than anything that vb6 can produce.

so dont go around offering advice that can completely devastate someone's reasearch when you haven't even the slightest clue about what you are talking about.

When I was a work study

[mao che minh]

When I was a work study I moved a computer lab that I was in charge of over to BSD (exactly 24 workstations). The students didn't seem to mind. The admin didn't mind. The network engineer asked me how I got IE to run in WINE. The LAN manager, however, threw a ton of bullshit my way concerning warranties and support.

Funny thing is, the two and a half weeks that the "all BSD" lab lasted, we had no work orders concerning crashes. I learned BSD only because the LAN manager in question made me learn it to move a email server over to it (Novell GroupWise used to be VERY expensive, and this square-headed manager didn't want to pay a bunch of cash to run one secure and isolated mail server on Novell stuff).

Business is business I guess.

Re:When I was a work study

[imin8r]

The problem I believe is that Microsoft products are so commonly used, that for a relatively small business (as opposed to the whole german goverment) it is detrimental to go against the flow. What about software that is specifically written for windows and not alternative OS's? You CAN make a statement and try to use another OS, but you are only going to hurt yourself from a business perspective

Re:When I was a work study

[cduffy]

And this software *couldn't* crash the operating system if it weren't for bugs in *something* with OS-level privileges allowing the buggy software to bring down the rest of the box. Otherwise just the one buggy program would crash and the rest of the system would get along just fine.

Sure, you have the same problem in Linux or FreeBSD if you have a 3rd-party app twiddling a buggy kernel driver or X server -- but buggy kernel drivers are exceedingly rare these days except in corner cases (say, complicated hardware for which drivers haven't been officially accepted into the kernel yet), and X has gotten dramatically more stable in the last five years or so.

Keep in mind, too -- while a FreeBSD base install is made up of software built by the same team that wrote the kernel, a Linux distribution is almost *all* 3rd-party software. From that perspective, saying that the free OSen will get unstable when there's 3rd-party software available for them is rather untenable.

Re:When I was a work study

[Billly Gates]

I use FreeBSD and the only problems I have which severly fuckup the system are done logged in as root and is caused by myself the user and not an app. A poorly coded Unix app will crash by itself or give a signal 11 error and core dump. It will not corrupt the system and bring other apps down with it. It will just give the error message and close.

If a user is running an app and he/she has just regular user priveldges then even a virus can not do damage. This is because the kernel checks access rights for files, directories, and programs. The important ones are marked "root" and unix will refuse to touch them unless the user is logged in as root. Windows NT, 2k, and XP have different users as well but the internal access rights are different. Any program can access the registry in Windows no matter what the user priveldge is. Thank Windows95 for making this standard. No app will run if you take that priveldge away sadly. This is where all OS and app information is and what causes 80% of the problems with Windows. Also as a developer you need administrator or root privedlges in WIndows to use com and the memory debugger. Another problem with WIndows.

Dll management is the other that causes those GP faults.

Unix is just better designed for this. Windows was never really designed but its a constant evolution from single user dos with a whole bunch of bandaid solutions.

One of these bandaids is the registry. The vast majority of screwups in Windows have do with an app or Windows itself corrupting the registry and fucking up the whole system with it. Only in W2k and XP 5 years after ms introduced the registry with WIndows95 did they began to fix this with registry tools and automated checkups. Its still really bad but registry corruption and dll management has improved. A bad app can still screw this up.

Everything in Unix is a text file so nothing gets corrupt unless its a hardware problem or a user did something dumb as root. Even the hardware is just text files in /dev.

Believe me I say that a bad programmer can write a bad app for every os. BSD is no exception but it certianlly is alot cheaper to use and operate. For a lab its essential to have good uptime. Especially if students needs these systems. A problem will usually never require a complete re-install. Just a fix with the app or config file that the particular app uses. Good os design is important in a lab environment.

Re:When I was a work study

[SN74S181]

Users don't care about the OS crashing. They care about the application crashing. And users of dedicated machines in a research lab care about a dedicated set of applications. So it makes no difference if the application crashes the OS or the application itself just crashes. The user is going to say 'the computer crashed again' and the whole user/root timesharing mindset of the freenixes will be irrelevant.

Likewise, the most critical data on a system actually used for productive work is the user-writable data in the home directory and/or on the shared user-writable network drive. Again, it makes no difference that the OS can't be screwed up by the user. If a script s/he downloads from somewhere smokes his/her home directory, the significant damage is done.

a Linux distribution is almost *all* 3rd-party software

That statement demeans the whole purpose of what's called a 'distribution.' A 'distribution' is and should be a tested and known well-functioning collection of software made into a usable system. That's radically different from the third party software a user drags in from, say freshmeat and inflicts recklessly on a system.

Re:When I was a work study

[Arandir]

A 'distribution' is and should be a tested and known well-functioning collection of software made into a usable system.

If only that were the case. But all too often it is not. Debian and Slackware are exceptions, but the others (even Gentoo) have the motto "ship tomorrow's release today". Debian is utterly stable (if you use stable) but the number one complaint with Debian is that it's so outdated. Slackware is frequently dissed by having a nine month release cycle instead of three or six months. On the other side of the spectrum, there's one very popular distro that I suspect doesn't even have a QA process. Another has the reputation to avoid their dot-oh releases.

I would put up with this shoddy QA if it were done by non-commercial distros, but the strange thing is that the non-commercial distros have the highest quality! Go figure...

Re:When I was a work study

[SpaceCadetTrav]

What about registry access? As long as an app can access the registry it can corrupt. Plain and simple.

The registry has ACLS, just like the filesystem. You can only screw up the keys that you have access to (namely, your own personal profile).

Labview

[Anonymous Coward]

As someone who has been a sysadmin for a 20-person femtosecond laser group, may I suggest Labview (www.ni.com)? It runs on Linux & Windows, many hardware cards support it, and it's honestly better than VB.

Your time is your most important resource. Don't waste it recoding.

R.

Re:Labview

[Anonymous Coward]

I know that where I work LabVIEW is the prime choice, example in mind. We wanted to run a thermometry test on a crymodule in a linac that was misbehaving and we were given specs on wed. by friday we were running the test. If we'd gone the route that the "machine" operators wanted us to (EPICS in our case) it honestly would have been years and way too much hastle.

Re:Labview

[DirtyJ]

Labview is indeed very nice. Once you get used to the graphical programming, you can work very quickly. Interfacing with hardware usually goes quickly and smoothly - especially if you buy National Instruments hardware; of course Labview is set up to interface with NI hardware with minimal pain. And it runs on several OS's, I think. Certainly it runs in Linux.

However, if part of the motivation is moving to free or cheap software alternatives, Labview is not the answer. NI charges one metric shitload for their products. But... for people running and working in a research lab, time is extremely valuable - perhaps moreso than money. You'll probably make up for the money cost of Labview with the time saved easily interfacing software/hardware. Time saved = salary saved, after all...

Re:Labview

[Neurotensor]

I've been told that our lab distrusts LabView because of previous irritations. I don't have any experience with it personally. But as for the Linux version, there still aren't any drivers for my stepper board (NI PCI-7324).

Your time is your most important resource. Don't waste it recoding.

That's some good advice, I agree. But still my ideals die hard.

Do it on a project-by-project basis

[TheConfusedOne]

I work in a Corporate R&D lab and we're pushing open source technologies here, but you have to be patient and strategic.

First off, are you trying to make programmers convert? If so then you're in a losing battle. People will almost always stick with what they're comfortable with. You'll only get them to look at something else if it is 1 or 2 orders of magnitude simpler to use.

Set an example with your work. If you can do your work using OS tools and you can do it quicker, cheaper, and easier then that's how you convince people. Comments like, "But VB is lame. MS is the anti-christ. etc..." do NOT a good case for conversion make.

Finally, if you can't do it in OS then don't. If you're stuck because of specific hardware/OS issues then don't try to fight the beast on those for now. Pick projects and things that you can migrate and move those instead. (Move the intranet site to JSP or PHP or whatnot on Linux/BSD/Apache. Throw out some NT/2000 Domain boxes and use SAMBA instead.) If you can show the advantages of OS tools for certain tasks then you can get people thinking about it. If you can't do it because of the hardware you're using then you're just going to appear as a stubborn zealot to your colleagues.

Re:Do it on a project-by-project basis

[The-Bus]

This honestly is probably one of the most even-handed comments I have ever read here on /. -- expand it and you see why some people won't switch to MS.

Thank you.

the problem

[rock_climbing_guy]

Additionally, my ordeals convince my peers that free software isn't worth the trouble.

Actually, I sometimes feel that this is the number one symptom of the obstacles that OSS/free software face. Multiple times, I have been labeled a nut for wanting to us Linux/OSS. I had a roommate in the dorms once who insisted that I would be "happier" using Windows.

As far as experience moving labs from Windows to OSS, I have never moved an entire "lab" so-to-speak to Linux, but it is my experience that in the past three or so years since I was introduced to Linux, it has made big inroads into becoming easier to install and use. The hardware support is definitely better. I remember when I first tried Linux, support for USB mice was still experimental.

These days, migrations from Windows to Linux have been relatively smoothe. I've had great success moving a fileserver from a Windows NT workstation to a less-used Linux box. Usually, it will eventually go down because of a power outage - so I recently purchased a UPS.

The bottom line is, I think that the problem of colleages who drag their feet on using Linux/OSS will be reduced as major distros become increasingly easy to setup and operate and hardware support improves.

Device Drivers

[The Jonas]

certain hardware manufacturers utterly refuse to support anything other than Windows

Since you mentioned you did some coding, you may want to check out Linux Device Drivers plus some of their other kernel tweaking/modding books.

Re:Device Drivers

[stanwirth]

certain hardware manufacturers utterly refuse to support anything other than Windows

Since you mentioned you did some coding, you may want to check out Linux Device Drivers plus some of their other kernel tweaking/modding books.

Buying a book isn't going to get her the board spec, unfortunately. To get the job done quickly, she'll need the board spec.

Otherwise, she could start by using the generic pci driver to probe the board's parameters in freebsd, or use phob to observe the board's parameters.

This would be an excellent chore for a grad student, as it would provide her with a useful skill while not really interrupting her studies of, e.g. The Effect of Gamma Rays on Man-in-the-Moon Marigolds

Re:Device Drivers

[Neurotensor]

Been there, done that. The problem is in obtaining the knowledge. Hey, if they want Linux support all they have to do is make it only mildly difficult and people like us will give it to them. Then they can keep profiting from us. But if they make it so hard that you have to reverse-engineer binary drivers, then they can go find themselves somebody else to sell to.

pick your hardawre more carefully

[73939133]

simply because certain hardware manufacturers utterly refuse to support anything other than Windows."

Just pick your hardware manufacturers more carefully. There is plenty of analog and digital I/O boards for PCs that have Linux support. Even better, Linux is very popular on embedded systems (like PC104), so you don't even need a whole desktop PC but can use a small, embedded PC running Linux, together with hardware that comes with Linux drivers.

It is also my experience that manufacturers that ship Windows-only hardware are generally substandard. They probably don't support Linux because they are very tight on resources. If they don't give you low-level documentation, it's probably because they don't have it. And you end up between a rock and a hard place with that kind of hardware when VB wants you to upgrade your OS and their proprietary Windows driver won't work anymore.

UNIX itself has a very long tradition for experimental applications, so if there is nothing for Linux, consider getting a cheap Sun workstation with hardware that is supported under Solaris. That will still work a lot better than the Windows stuff, and it will interoperate nicely with Linux machines.

If you absolutely must do something on Windows, use Python, Perl, and/or wxWindows rather than VB. CygWin is also great. That way, your developers will acquire open source and Linux expertise and won't be locked into the Windows upgrade treadmill.

So, while occasionally some cheap peace of Windows hardware may seem alluring, if you just look around a bit more, you'll probably find something at least as good or better for Linux.

Re:pick your hardawre more carefully

[dbrutus]

Eventually your $1200 card will die out and you'll either get a new replacement or a newer hand-me-down. Trailing edge everything always has these problems. It isn't as shiny, functional, easy to use, or as cool as the new stuff that the better funded lab in the next building gets. But every purchase of new stuff today is not only affecting you but all those future frustrated scientists who get to deal with your hand-me-downs as well.

The key isn't to just demand more money (though it can't hurt) but to have a strategy for dealing with the fact that lab equipment can last decades but MS has an official company policy of obsoleting OS versions older than 5 years. That fact alone should make alternative OS support a priority because for a 20 yr lifespan instrument, at least three quarters of its useful life will be run without an underlying supported OS if you choose the MS solution route.

Obviously...

[YrWrstNtmr]

..you choose whatever platform/software that will do the job the best.

Functionality
Availability
Budget
Useability

Philosophy is far down the list.

Re:Obviously...

[MourningBlade]

Yes and no.

A research lab, especially a public one, is first and foremost obligated to do science. To clarify what I mean by this, let me say what I mean:

The experiment must

• Have a clear question in mind, and a method for discerning success from failure (not as easy at it sounds, when your result is not binary).
• Be repeatable by your lab.
• Be repeatable by other labs.

In a physical experiment, it is perfectly acceptable to use a proprietary kit, provided that you can:

• Show that the results are replicable
• Give other labs access to said kit (for a price) and have the ability to produce more
• Show that the results have relevance to what it is that you are trying to achieve

That is true for physical experiments. For manipulation of the data of an experiment, however, the procedure must follow a published or publication-pending method of analysis if you intend to have your research be considered legitimate.

It is coming to pass that algorithms are becoming complex enough and analysists savvy enough, that it is often more practical to produce clear, well-documented source code in addition to your paper than it is to go over and over again the fine points of your method with every interested party.

This leads me to my point: in physical interaction, proprietary and closed methods will most likely remain prevalent for many years to come.

In information manipulation, however, open methods are becoming a dominating trend, if only for the clarity they afford.

In my lab we do bioinformatics research, and we could not do research on the scale we are doing at the pace we are doing it if we were depending upon proprietary software: the proprietary companies cannot compete on customization, new development (after all, we're the ones creating the method), user interface (we're improving all the time, most proprietary packages have ancient user interfaces that are clunky and just plain awful, read: GeneSpring and friends), and cross-lab communication and auditing.

These proprietary solutions are forcing many theoreticians to use software that is "open" in another way: Excel. Yes, I kid you not. Quite a bit of bioinformatic analysis development is done in Excel because the proprietary solutions are just too closed.

The only proprietary companies that are on the right track, in my opinion, are the ones that allow you to use the app as a hub for many other componentized programs.

Best tool for the job

[appleLaserWriter]

Clearly VB is the winner here as it perfectly mimics the unpredictability of quantum mechanics!

Controlling quantum computing using VB?

[DogIsMyCoprocessor]

Sounds like a Michael Crichton recipe for technology gone bad. I bet the fat guy in your lab gets it first.

Programming in VB?

[neuroneck]

I have never known a lab that used VB as its programming environment. Usually it is either c/c++, Java, or one of the math programs (matlab, mathematica, maple, etc...). In general, I would recommend using microcontollers for controlling your experiments. However, you mentioned that you are doing stuff at the quantum level, so these may not be fast enough for you (the ones I use are 20MHZ). However, I must say that the PIC series of microcontollers can be programmed in a variety of languages and has a great deal of flexibility. One of the main problems is that a lot of the software for contolling lab equipment is either homebrew for a specific application (as is the case with some dynamic clamping software in the neurosciences) or made for a wide variety of applications (labview), but is not open source. The best option may be to get a company made environment which can have functions written for it in another programming language and customize it. However, coming from a biology side of things I do not know what your specific needs would be for quantum computing, and thus cannot give any ideas as to that specifically. Good Luck.

NOOOOOOO!!!!!

[frank_adrian314159]

for today I have started to learn Visual Basic...

Don't do it! If you have to learn something, at least go to C#. Visual C++.NET would be OK, too. Both of those are better languages all around. And you can still interface to DLL's with either. To almost any programming question VB is NOT the answer.

Re:NOOOOOOO!!!!!

[pz]

Here's the problem, though (I've faced nearly exactly the same issue as the original poster).

1. You have programming skills, no one else in the lab does (they're all scientists with pure science backgrounds, you're the only one with a mixed engineering/science background), except the staff programmer. He is good, but also thinks there's nothing wrong with VB. Fine. Your code needs to be read, supported, and developed once you're gone.

2. You need a little experimental control program with a GUI that is reasonably easy to debug and modify, and you need it TWO MONTHS AGO.

Deploying something that meets both of these criteria is relatively straightforward if you swallow your pride and write something in VB. Hell, all of the interface card and hardware vendors provide VB example code. Doing it any other way (remember, GUI, easily changed, *yesterday*, etc.) is a heck of a lot harder. Either that, or I'm way behind in the quality of Open Source RAD tools.

Good luck to the original poster. Like many of his predecesors, I gave up and wrote the experimental control in VB. But my data analysis and primary desktop environment is still Linux.

Re:NOOOOOOO!!!!!

[Timesprout]

To almost any programming question VB is NOT the answer.

In fact to many programming questions VB is the perfect solution. Why do you think there are so many VB programmers and VB apps out there? Its so easy a complete novice can learn how to build reasonable applications quickly. As applications start getting more complex then by all means use Java, C# or C++, with more experienced developers to do it properly. In a lot of cases VB is perfect for quick simple solutions. Some of the stuff I have seen talented coders produce in VB is not half bad either.

Re:NOOOOOOO!!!!!

[cduffy]

Either that, or I'm way behind in the quality of Open Source RAD tools

You're way behind in the quality of open source RAD tools.

The python-libglade solution isn't as flashy as VB, but it's an extremely easy way to throw together an easily modifiable GUI app *fast*.

(Btw, I presented on this particular combination to my university's LUG a year or two back after writing a fairly fancy virtual machine/simulator frontend literally overnight in about 100 lines of python code and an XMl file created by dragging and dropping in glade).

Re:Comedi gooooood, APM baaaad.....

[meowsqueak]

I've used comedi with a National Instruments NiDAQ 16 channel acquisition card on a P3 laptop running debian - it worked very well.

However, can I offer the following advice, which may save some people from smashing their head into a bloody pulp against a wall...

* Turn off APM!!! *

You can do this by passing apm=off to the Linux kernel with your bootloader (I think - can't actually check that at the moment) if you don't want to actually remove it from the kernel (APM is useful on a laptop normally).

If you don't do this, you might find your acquisition mysteriously stalling after random intervals. It's to do with APM interrupt handling. Not sure if it's restricted to PCMCIA cards.

Not ready for precision?

[Sheetrock]

There are a number of factors in Free Software that may be a cause for concern.

First, naturally, there is the question of "Who do you sue" when things go wrong (which in itself is a statement of the issues surrounding responsibility in community development.) Commercial software vendors strive to meet a standard of reliability because they've implied with the sale of their software a warranty of fitness.

Also, there is a question of reliability. Commercial software is designed towards a goal; Open Source is designed almost by accident. Integrating commercial products tends to work better than Free Software because of an accumulation of tolerances.

Most importantly, using commercial software makes repeatibility in other labs a more likely possibility. A Windows setup is standard. A Linux (or whatever) setup depends on any number of factors, from the versions of the software to the distributions of the kernels. Irregardless of the general expense involved in setting up commercial software testing, this is perhaps the most important thing: your colleagues won't have a chance at duplicating your results if they don't know what you're running.

Re:Not ready for precision?

[Anonymous Coward]

This post is mindboggling wrong in so many ways it's probably a troll, but here goes...

"who do you sue" isn't relevant; you can't meaningfully sue a software vendor, because the licenses almost always absolve them of all liability for their product's failures beyond perhaps giving you back the purchase price of the software (which is meaningless compared to a failed experiment). if you actually want the software to work, you're often better off with the source code so that you can make it work yourself, instead of having to fight with vendors to prove that it doesn't work, then wait 6 months until the fix fits into their product update cycle. (NB: not all vendors suck, just the big ones)

reliability: for many applications, open source software is better than proprietary because there are _more_ people improving the system. For specialized applications from small companies, there may be only a single engineer, whereas an open source system can benefit from the direct involvement of all of its users. Also, vendors have an agenda to hide problems (you often are required to sign an NDA before receiving bg fixes so that you can't even tell the vendor's other customers about the problems you found), while open source app's status is transparent, so it _has_ to work.

Finally, the "windows" environment is far from standard -- beyond major changes between every different OS MS has shipped (95/98/ME/NT/XP/CE, home/server/embedded, etc.) there are zillions of patch levels and, even worse, random OS patches installed by MS Office, IE, WMA, etc., installers. It's nearly impossible to document, much less duplicate a specific Windows configuration without making a disk image and completely standardizing all hardware. Also, it's literally impossible to purchase most versions of Windows (MS only sells the current versions), so it's impossible to legally duplicate an older environment. With Linux (or BSD) I have access to any revision of the platform and any applications. Not that configuration management is easy, but it's doable. And since there aren't any licensing issues, I can build out any configurations I need without requiring me to keep track of certificates of authenticity, or whether I have to pay a second OS license for a box because I installed a different copy of the OS than the one that was shipped by the OEM.

What's valuable to you?

[Muerte23]

Is it really worth your time, your professor's time, or the government grant's time to spend your quantum research dollar in overhead costs as you bang your head away in frustration trying to cludge together some string of 0.2 beta versions of open source data collection programs?

And what about when you leave? Does the next grad student have to spend 3 years learning your absolutely unique software setup instead of learning physics?

In the Big Name(TM) physics lab I work in, grad students cost about $200 a day (to the grant), and postdocs cost about $500 a day. If I need a program that would take me a month to write or costs $2,000 to buy today, it's my job do know to just buy the program.

We used to use LabWindows (call it C++) and VisualBasic, but the last person who know LabWindows left and now looking at the code when things go wrong is a nightmare.

So, anything new is being done in LabView. (Disclaimer - I don't work for National Instruments) Sure, it costs $2K for the good suite, but I guarantee you will make up for it in productivity. Plus, debugging LabView code as a beginner is waaaaay easier than debugging someone's crazy spaghetti C code. With the high turnover rate of a research university, it's very important to retain the chain of knowledge. Otherwise things progress into the realm of Black Boxes.

My opinion is not to waste your valuable research time worrying about software. Especially in quantum computing where you will be left in the dust if you fart around worrying about open source too long.

Best of luck,
Muerte

You said quantum computing?

[product byproduct]

[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is.

You have to phrase your request nicely:

"Give us the technical specs, or we will crack your company encryption keys with our quantum computer, access all your specs, and post them on usenet."

What about running Free software on Windows

[Billly Gates]

I assume your using special equipment which connect to a pci card on your computers. If that is the case then Windows is the only option.

However you do not have to run VB only. My Windows2k box has opengl, openinventer, vtk, ativePerl, active Python, gVIm, cygwin, gnuc/c++, Devc++, ruby, tk/tlc, apache, php, etc.

Infact the win32 ports for these opensource apps are very well integrated with Windows. FOr example I can use gvim aka VI to replace my editor in VC++, create ole programs in python, and even use Perl to create Excell macro's.

Your Windows based collauges will get use to opensource and be more open later on after they get used to it.

As a scientist I assume you use VB for similiations and or to interface with your devices for experiments.

For similiations try vtk++ and openinventor. Your colleagues probably used them in Irix quite heavily. THe libary comes with a great .chm Windows help file. It is the most complex thing I think I have ever seen behind the openoffice sdk.

If your equipment provider only provides .ocx files aka toolbars I would be in complete dismay. VB is just not designed to do that kind of work. Ask your vendor if they have libraries for other langauges. C/C++ might be your only other option unless they use something proprietary like the Allan Bradely langauge or lisp.

Re:What about running Free software on Windows

[TrekkieGod]

I assume your using special equipment which connect to a pci card on your computers. If that is the case then Windows is the only option.

Not necessarily true. I work on research for the university I study at, and the particular branch I work on uses comedi drivers to interface with our pci data acquisition boards. Take a look, it might support what you're using.

Many things people would ordinarily think linux can't do is already, or will one day be possible, so keep looking. A lot of people tend to think you can't do real-time work on Linux, because it's not a real-time kernel "like windows nt", but we do it with a patched RTAI kernel too. My advice is to research what you need, in any operating system. Check out all your options, and choose what's best for you.

Start with new projects.

[twitter]

Forget the old stuff, concentrate on new projects, number crunching and data servers. Use the strengths of free software first, then move into areas you can. Old stuff is working, as well as any old M$ junk works, and you will only be frustrated working with equipment no one cares about. "Rebuild" it till it smokes. You are familiar with the superiority of free software available for networking, number crunching and programming. Take old computers no one wants and make them useful. Something as simple as databases, web servers and email servers are helpful. Take the hardware advice you get here and put it into your next proposal. The bottom line will speak for itself and you will have proven the dependability of free software on comodity hardware already.

Sometimes it's the other way

[jmv]

I'm doing my Ph.D. at a mobile robotics lab. Most (all?) of the robots we bought, (either from iRobot or ActivMedia) came with Linux pre-installed. I don't think we could even have Windows if we wanted it. I think this is going to be much more frequent in the future. For things that don't have a display anyway (no we don't have 17" monitors on our robots!) what's the point of running Windows?

Yes, it is possible

[bm_luethke]

Depends on the research I guess:

http://www.csm.ornl.gov/
http://www.csm.ornl.go v/torc/

we are mostly linux. We do some windows because we choose too (for varying reasons, most of us really like Open Source but are relativly OS agnostic).

Besides our CS work we run things such as the human genome project, nuclear fallout modelling, vis applications, process data from some of the colliders, basically govt researchy type stuff.

So yes, it is VERY possible to run a research lab under linux, unix, or windows, though there will generally be some mix. Here, compute power is usually linux/unix, developer platforms are generally linux/unix, office work is usually windows - though sometimes windows has some software that is really kick ass.

heisenberg is rolling over in his grave

[stinky wizzleteats]

You guys are trying to get Windows running on quantum computers? Talk about uncertainty!

ActiveState

[N8F8]

ActiveState: Python, Perl, Tcl, etc for Windows.

its' the fault of the professors

[jsse]

who run the labs. In our lab we wouldn't make the purchase when the products could't work on more than one open platforms. It's not like we hate Windows and proprietary products.

A researcher with certain year of experience would have thru the nightmare of getting obsoleted hardware to work - it's not that the hardware itself is getting out-dated, it's about its interface i.e. the device driver. E.g. the VPL optical gloves are still usable today as we've the driver source came with them, even when the company behind were out of contact for years; while we have to kiss those 3d controllers goodbye when their Windows device driver no longer functional in modern Windows system.

Really, we had no problem when we had to code in VB, as long as it works effectively, and gives results for our papers. We just don't want our money wasted on endless upgrade-cycle(if upgradable), we are not running a corporation afterall.

Re:Comedi gooooood, APM baaaad.....

[meowsqueak]

It wasn't so much the laptop falling asleep - just having the APM code active in the kernel was doing something screwy to the interrupt handling from the NiDaq card. It would fail acquisition after 15 seconds or so (although fairly randomly).

David Schleef pointed this fix out to me after I asked him very politely about what the $#@$ was going on...

Open source DARPA projects

[Rich_Kilmer]

We work on a DARPA project that runs a distrubuted multiagent system which is written in Java across a couple hundred dual-Xeon machines running Linux. The control mechanism is a Ruby application framework that uses Jabber for control and stressing of the multiagent application.

Not only is the project using open source as the infrastructure, but contributing to the open source community with projects like Cougaar and PMD.

We use Linux primarily because of the simplification of administration and maintanence.

cool.

[twitter]

Comedi is awsome. The very first driver described leads to The 8255 driver. It's author, Daniel Franklin, recomends "Alessandro Rubini's excellent book Linux Device Drivers (another fine O'Reilly publication)" Ahh, knowledge, what could be finer. Free software, free info. Go get it and become the research tech God you want to be.

Re:cool.

[Neurotensor]

Comedi is awsome.

I second that sentiment!

"Alessandro Rubini's excellent book Linux Device Drivers (another fine O'Reilly publication)" Ahh, knowledge, what could be finer. Free software, free info. Go get it and become the research tech God you want to be.

Well I've read Linux Device Drivers, nearly cover to cover, and I've implemented Linux drivers for the chip I designed during my Engineering Honours project. So I'm not scared of doing it although I know of how much effort it takes. The main thing is you need to have specs for the card. Such things don't seem to exist outside of NI's vaults.

If it ain't broke

[j1mmy]

don't fix it. For fuck's sake, did you read your own submission?

I know what I'm missing out on, in the free software world.

Followed immediately by:

I've wasted a *lot* of time and effort trying to implement some very simple stuff with free (and better) alternatives

Yeah you're missing out on the struggle and pain of hacking together ad-hoc solutions to an already-solved problem.

Way to go, buddy.

We use *nix

[WebMasterP]

I work with a research professor at the University of Washington. I'm hard pressed to find a windows machine on my floor (I think there are two, and they suck). Everything is Solaris. The main server room has a huge Sun Enterprise Server and a 19 computer linux cluster. My group just got its own 2TB storage server, it's linux based... The whole dept. seems to moving from Solaris => Linux if anything... but no where near Windows.

Linux and LabView?

[rco3]

We have a lightning research group (previously mentioned on Slashdot here and here). We use LabView 6.1, on RedHat (7?), to communicate with PIC microcontroller-based instrument control boxes out in the field, to continously monitor the local electric field (which is a good indicator of the favorability of lightning initiation, either natural or triggered), and to arm, calibrate, and disarm all of the oscilloscopes in the various experiments. We leave the system unattended during the winter months, just in case a frontal storm comes through and we get a strike within a half kilometer or so...

We use the Comedi drivers to interface with our National Instruments DAQ card. .VI files are available to access the Comedi drivers from LabVIEW, and our fields monitoring program appears to have been running, unattended, without restart, for over 100 days on our dual Athlon MP 1900+ box.

And if you can't program in LabVIEW, you might oughta practice asking if I want fries with that burger.

Use Python instead of VB

[failrate]

It's free, supported, works on most platforms without rewriting the code, and has simple wrappers/interfaces for all kinds of programming languages, so you can still use those DLL's.

VB and data acquisiton

[efedora]

I must disagree with the VB bashers here.
Our shop builds test equipment with lots of analog and digital I/O - some of it high speed.
We started off 20 years ago using C but switched to whatever-the-current-version of Microsoft's BASIC was at the time. Each system was unique and you can't beat VB for fast coding/debugging.
In the beginning we had to write assembler code to access the hardware regs on the acq boards for speed but when DLL's began to be supplied with the hardware things got much easier. The huge library of controls and DAQ add-ons for VB just cannot be ignored. For a laboratory the advantages would be similar - fast prototyping and debugging for unique applications with lots of off-the-shelf code and low labor costs. VB has it's faults but for this kind of work it is well suited.
There is really no economic advantage to using free software in this instance.

Flamebait, I'm sure... but VB kicks ass now.

[shadowxtc]

It wasn't true until .NET, but now I have to say, after using about 1/4 of the languages out there (and that's still a lot), VB has truly matured. I'd say that for anything except embedded software, or that which simply must work on *ix, VB is probably the best choice overall. Consider things like ease of use, learning curve, cost per hour per programmer, and it makes some sense.

Re:Labview okay, Igor Better

[Nesomir]

Labview is better to use to actually acquire the data, but Igor is amazing to display the data.
I work in an IR laser lab actually doing labview development and it is VERY easy to program with.

the politics of IT change

[spasm]

There's two basic tricks I've discovered over the last couple of years of slowly incresing the use of open source platforms and tools in our research (ok, we do behavioral science stuff, but the politics of IT change are the same).

First, re convincing colleagues that open source / free software has a role in your work: do something they envy. Produce a tool they want to use, or find some existing software that does something useful and cool, or even just do the great unix thing of tying a bunch of small programs that do one or two things well together to do something that no existing monolithic package really offers. Then point out that it either can't be done on the current platform of choice, or, while it can be done, it requires spending $$$ on some proprietary solution. Doing something like this tends to legitimize the use of the toolset you'd like to use, and gives you a good foot in the door for more abitious moves later.

Second, re working with third party suppliers who don't currently produce software or drivers or whatever that work with non-MS platforms. If there's more than one vendor who supplies something that does what you want, pick the smallest one. They're more likely to be interested in finding niche markets, less likely to be bogged down by bureacracy when it comes to doing something new or different. And a three-person company is more likely to have two of the three who've recently been working in your field & remember what it's like trying to do the usual research thing of trying to get an existing tool to do something that no-one's done before - hence more likely to give you access to the kind of more detailed information you might need, even if they can't really expend the effort themselves right now.

Anyway, that's my take on 'what worked' after a couple of years of win-some, lose-some politics around research and IT.

What hardware

[Camel Pilot]

I work for a Navy research lab and we use Linux, Perl, C, Apache, Mozilla, Gnome, Perl/Tk, GnuPlot, etc almost exclusively for our control and monitoring systems. We have had no difficulty controlling Opto22 devices, GPIB, RS422, R485, and one-wire devices. The only windows we use is on the desktop which is unfortunately mandated (see NMCI).

If you need real-time look into QNX.

BTW we operate what is probably the deepest running webserver. We have a vehicle monitoring system that is controlled via a Linux/Apache machine setting at depth reporting topside via fiber connection. This system controls power reports temperatures, pressures, inclination, depth, smoke, leak, etc.

Not quite Quantum Computing, but...

[mtnharo]

I've been working with one of the Physics profs at my school (U of Rochester) for the past year, helping him update the software they use for cosmic particle experiments. We use a data aquisition board and particle detectors designed by FermiLab. The software runs on Linux, and accesses the DAQ board through the serial port. My job has mostly been adding a GUI to the program, so that the students running the experiments can concentrate more on getting results than understanding the weird command line interface for the program. For more info on the project, see the FermiLab page for the QuarkNet project, and the PARTICLE project page at the university.

Our lab is entirely Linux

[bigberk]

We set up a small undergraduate research lab at the University of Manitoba's EE Dept. For the summer we are doing research in networking and telecommunications. All of our workstations are running slackware linux and we find most of what we need in this distro. OK, so this is slightly contrived (we're doing networking after all) but we rely big time upon iptables, tcpdump, iptraf, Sun's java, and lotsa unix utilities. I think people underestimate how many useful tools are in linux.

My Experience

[muon1183]

I'm currently working for a small research group which is part of a particle physics experiment and we are running entirely on Redhat systems, using many excellent open source tools made available by CERN. In my experience, a Unix like environment works orders of magnitude better than a windows environment, especially when it comes time to automate events. I can't even imagine trying to do what we do in a windows environment. It would be an absolute nightmare trying to run most of the program we write.

A couple great alternatives

[ZxCv]

My first choice is Delphi. I don't think I'd ever say Delphi is better at creating quick`n`dirty apps than VB, but I would most certainly say that it is completely on par in that area, with the added benefit of being much more powerful. (My opinions here are based on VB6 and Delphi5, which are the last two I used heavily before being liberated from Windows GUI work.)

The other alternative I can think of is RealBASIC. Their development environment used to only run on Mac OS, even though it could compile apps for either Mac OS or Windows. Nowadays, the environment itself as well as the apps it creates all run on both Mac OS 9/X and Windows, although I've never used the Windows development environment. I've only had limited exposure to RealBASIC, but based just on those few hours, I would highly recommend any fan of VB at least give it a shot--I know if I ever have to go back to Windows GUI work, I certainly will. (It seems it would especially shine for quick`n`dirty apps because it seems to focus more on simplicity and cross-platform rather than feature bloat.)

Re:A couple great alternatives

[Yuioup]

I've been using Delphi for over 2 years now and I can say that Delphi is on par with C++. In Delphi you're not limited to the handful of functions that VB gives you and you can choose to go deeper if you want to, straight down to the assembly level.
Everybody here on /. knocks Delphi down as another "Quick 'n dirty" development environment but it most certainly isn't.
Yuioup

Re:A couple great alternatives

[ZxCv]

My point was that Delphi is the best of both worlds: it was equally powerful whether you needed it for a quick`n`dirty GUI app or an extremely complex project.

Re:windows-only, huh?

[Neurotensor]

You dared me did ya?

Well according to your next post you think I'm working on NMR quantum computing. I'm not. It's an optical scheme involving coherent transients in rare-earth doped inorganic crystals. So we aren't using a bought system, it's a system that's been put together over a decade, out of all sorts of discrete pieces of equipment, many of which were not bought by us for this purpose, all of which must be usable under a given OS for the whole experiment to use that OS. The only software that really can't be replaced is the hardware drivers, and the whole issue is that they can't be replaced without information from the company.

Whilst you are correct that much of the hardware we use, such as DSOs, waveform generators, etc. are OS-independent (via GPIB busses most of the time), we have some things that we simply can't replace and yet they don't work without proprietary drivers. Such as the USB->ISA converter in the pulse sequencer. We do fortunately know everything we need to know about the ISA card, but ARS Technologies manufactured the USB->ISA converter and it uses a magical black-box driver. No help with the specs when I asked them. So we would have to pull it all apart and use the ISA card directly from the PC (which I'm open to do BTW). Also the NI DAQPads are USB devices with equally mysterious USB converters, once again requiring black-box drivers under Windows.

I'm sure you realise that I am one of the people who *wants* to dump Windows in our lab. But I have to start by proving it's both possible and simple, on my own specific project, which for the present month is to retrofit an old optical spectrometer to be controlled from a PC. And before I got here, we had a PC running Win95, and using a broken VB app that talks to an NI PCI-7324 stepper card.

Now that's the thing that consumed going on two weeks of my time now. Instead of implementing a solution using VB, I felt adventurous. I tried GNU/Linux, but found that there's most likely no way without some hardcore reverse-engineering, to get the card going under Linux. Not all of that time was wasted on the card, some of it went into experiments with QT as a cross-platform development environment. Still, I've given up (at least for now) as I still have to use Windows.

I know.

[twitter]

The main thing is you need to have specs for the card. Such things don't seem to exist outside of NI's vaults.

Ah, that's what my knee jerk reaction was all about. Discresion is the better part of valor. Use hardware that's got drivers already for new projects and use the old junk till it blows up. I presume someone will have sucess stories that I was only able to dream of four years ago.

I get ill thinking of VB. Learning C++ and the win95 API was easier for me. One week of happy brainwash VB training tape scarred me for life, "methods" twitch. I saved myself from that hell with a nice little "windows XX API how-to" book with examples and a watcom compiler. It made sense and offered greater control. MFC was required to talk to devices and it was a step in the wrong direction but control, display and communication modules were still seperate Others were lazy or stupid and spageti code VB was used on many other projects with horrid USB interfaces.

Good luck, you suffer a legacy of bad choices and are going to be forced onto VB. So you enter the downward spiral of the M$ maze, chasing mindless changes, befudled by tools that don't work the way they should and mysterious crashes, delays and poor data rates. You shall suffer nights of rebuilding win3.1 machines to take care of those old DA boards that don't have win2k much less XP drivers. If you can even read the poorly commented and ill disciplied spagetti code you have, you will suffer the pain of "porting" VB 4, 5, 6 to whatever is the current version, which might require complete re-writes to save time. Read letters to the editor in VBmagazine if you don't believe me. The more you learn, the worse it looks. Don't gripe too hard, the boss might have written some of that crap.

When it's all said and done, using a seperate machine listen to the device and learn how to talk to it might save you time. Selecting reasonable hardware for future project surely will save you time. Once you get aquisition working once you will be able to replace legacy stuff that breaks. The VB/disposable hardware route can hardly be called a success story.

federal law requires commercial over development

[rpalmeira]

In some cases, esp. where federal monies are concerned, law requires that if a commercial solution exists already that they be used over development of similar/same alternatives. So is the environment is primarily windows (or some versions of linux) then development with free software, where similar commercial products are available, can be illegal.

In my experience...

[mhfs]

I'm working in a lab doing "Quantum Chaos" experiments (manipulating cold atoms to investigate the difference between quantum and chaotic physics). We use the free (as in beer) version of RTLinux to run all our experiments, as timing is important and we didn't want to implement a hard real-time system. I coded most of the gui for the experimental interface using Borland Kylix and everything works quite nicely (apart from some evil memory leaks).

The real problem is the hardware - a real guru set that up for us. He wrote the "drivers" for the I/O cards himself (although that's meant to be a little easier in RTLinux than for normal linux) and also got a scientific grade CCD camera working even though the only linux drivers available were outrageously outdated. Sadly, we will definitely face some issues in the future if we want to upgrade to a new kernel!

Personally, I think the only way to move data files around is with a decent shell. Rename is perfect for all those times I put the wrong parameter in the file names of 160 different data sets. Most of the time our lab works quite smoothly with regard to the OS itself, and it's certainly an improvement on the old Win95/Scientific workplace combo of the past!

Re:When I was a work study [OT]

[Anonymous Coward]

This is going OT but is importand none on the less

Windows NT, 2k, and XP have different users as well but the internal access rights are different. Any program can access the registry in Windows no matter what the user priveldge is. Thank Windows95 for making this standard.

There I was typing away on a post marking you as a troll and then it hit me... You have been using regedit.exe on winnt based systems! Have a look at regedt32.exe, Done? You must have noticed the "security" menu on top. Here you can set full acl`s on every value or "directory" you want. All sorts of fine tuning options there. This beats the unix configuration system by like a hundred times on security. Its more complex (very bad in security land) but also offers great posibilities being able to limit acces to each induvidual value with an ACL with options (dis)allowing induvidual users (or programs run in their context) from querying the value, setting the value, creating subkeys,listing subkeys,"reporting","linking",removing, writing DAC data,writing owner and reading security info. (can`t translate them all from my localized version)Compare this to the crude all or nothing mechanism split by owner, one group and everyone else on a whole config file.

Then add the fact that NT kernels have had full filesystem ACL`s (and remote filesystem acls) as well as basic mandatory acces controll (user rights policies) while linux has been strugeling to get even minor posix suport for ACL`s and has been doing nfs by default for a while now... And lets not forget the full acounting features posible on all of the objects with ACL`s on them in winnt.

Now dont get me wrong any OS that comes with a webserver that has had a plain dotdot bug in its filehandling (same bug in win95 remote filesystems) enabled by default is to be laughed at first befor being considered for use on the internet, but blame everything microsoft (office apps, "servers", scripting features, shell, manuals, patching policies, backward compatibility policies, monkey-see-monkey-do training, source distribution policies) for that except the security design of the NT kernel

Specifications and how to get them

[Hellkitten]

[Hardware Manufacturers] seem to get very upset when somebody asks them what the register-level interface to their card is.

What exactly did they say when you asked? Have you made sure that they understand what you want to do? (Create a driver that makes the card work on linux, that anyone can get, potentially increasing the sales for the card). The key is to present the request not as "we need this" but as "you will get this if we can get that". They may still not be willing to help and then you explain that whenever you do the purchasing decisions you will prefer a company that provides specifications (or linux drivers). They still might not listen so you may have to wtick with windows, just make sure you remember who foreced you to it whenever you get a budget to buy new equipment.

A mixture of Open Source/Commercial Unix/Windows

[grahamlee]

I'm a Physics UG at the University of Oxford, and have done the odd part-time job writing code for some of the groups (I worked for CMP last year and will start in the Teaching Course next month). All of the code that I have written was released as OSS, because other research groups can then modify it and use it in their experiments, without affecting the originality of our work.

The systems I worked on in CMP were almost entirely Red Hat, and some of the code I had to work with was being ported to C/Linux from Delphi/Windows it was stupid to ask people to pay for Delphi licenses just to tweak a few lines of code in a detector controller. Some of the older computers were running Win95 with X11 and SSH clients and were just used as terminals for the beefier Linux boxen.

The computer I'll be using in my next project is a little meatier, a four CPU Sparc box. There are a couple of these in the department, each with a few dozen SunRays attached. The code developed on here will still be OSS, and the box has a fair amount of the GNU userland sitting side-by-side with the Solaris rubbish.

Other than that, my knowledge is a bit sketchy and is inferred through what I hear or see around the department. There are a number of Windows workies around, because software like SPSS or Origin or Minitab only exists on Windows, and there either don't exist Unix or Linux alternatives, or they aren't yet mature enough to want to switch onto. Many of these workstations also have Cygwin or at least DJGPP.

But the main point to be answered is whether or not a research group should be OSS. My opinion is yes as far as in-house code is concerned, because this facilitates collaboration between groups using similar code, hence quickly smoothing bugs. OSS also neatly fits the philosophy of shared information many scientists have.

Should the computers all use Open Source operating systems? I just don't think this should be a requirement in most cases. Yes Linux and the BSDs are stable and mature enough to be used in a research environment, but then so are Slowlaris and Mac OS X, and you can develop and build your code on both of these systems too. The only situation OTTOMH in which having an OSS OS would be directly beneficial is when custom-built embedded devices are required, for instance in HEP detectors or beam controllers. In these circumstances the ability to modify the OS kernel would be useful, but if you're just developing your C code in a Unix environment to be used on bog-standard Unix or Linux machines, then a good set of manpages and a functional cc is really all that's needed.

LabVIEW SUCKS for serious projects.

[nzyank]

Great for prototyping. Horrible for large projects. Try debugging nested loops. What a friggin mess. Anyone who suggests LabVIEW for anything other than a one-day project should be shot. I spent (wasted) 1 1/2 years doing a commercial project with it. Seems nice at first, but slowly you very quickly get diminishing returns from it. I'd rather do stuff in assembler. At least I can see what the fuck's going on. Optimization sucks because who the hell knows what NI does under the hood. Go with VB or C/C++ or ANYTHING before LabVIEW. Please.

GPIB cards

[johannesg]

I have been doing my own little battle with Agilent for several years. I want to have a Linux driver for their GPIB cards, but they are utterly unwilling to provide it.

Recently this went so far that I had a very candid talk with one of their sales people. I made it clear I would move to a different manufacturer if they would not provide drivers or some means for me to write them. He would ask within their organisation. Several days later I received an email titled "Solution to your GPIB driver problem". To my astonishment, it gave contact details for several other manufacturers of GPIB cards!

So, isn't this weird? They'd rather lose our custom than provide us with a driver, or sufficient details to write our own! And given the amount of stuff we are buying from them that's a pretty big decision.

So now we are talking with National Instruments, and they are very pleased to have us as customers. Moreover, they have Linux drivers for their GPIB cards. I haven't seen these drivers yet, and I sure hope they will be of acceptable quality.

I'm interested in hearing what experiences other people here have had with GPIB cards under Linux - especially those of NI, which we are about to buy. When you mention lousy Linux support, does that include their GPIB cards?

Re:GPIB cards

[JGski]

The fact that the rep steered you to NI and other vendors is an indication that Agilent still have the ethics of the HP Way! If they can't provide, they didn't lead you on; they told you how to solve your problem, regardless of the immediate economic impact. In my book, that earns them brownie points for the future even if it isn't GPIB cards. I can't say the same for HP ethics now but that's another posting. That "weird feeling" is the astonishment of seeing ethical behavior!

As a 9yr veteran of Agilent/HP T&M (now ex) I can tell you some of the reasons why they won't support Linux or, at least, offer "NoMAS" information (No Manufacturer Available Support: "accepting this information constitutes a legal contract agreeing to never request support or otherwise use any company resources regarding this information") like the old HP Corvallis used to for "insider information" about ultra popular calculators (HP41, et al).

The number one reason is: support cost or concerns about support cost regardless of such a NoMAS. There is this admirable concept at Agilent (hell, I was heavily involved in creating the first formalized version of it, so it must be good ;-) ) once called "Intrinsic Support" which basically asks "What do we owe a customer contact who simply possesses one of our products regardless of its current support life status or whether this is the original purchaser of the product (assuming not stolen, right)?". Well, it isn't nothing. It's at least "best effort", that is, other product support issues are the priority but if nothing else is on the burner... The "Real" HP always did this anyway. It's sometimes hard to assess those support priorities though so there is a risk of "cost leakage" that can be scary.

Unfortunately one of the side-effects of insisting on observing a more ethical customer relationship has been reduced incentive to innovate or allow customers to innovate in hard times for these kinds of cases.

The other element in this is that the key R&D lead/influencer for these products has been enamoured with the Evil Empire for sometime and has gone completely Cult Koolaid on .NET at the expense of rationality, IMO. You might have "picked up" on this phenomena on their web site! :-p

BTW I've had the same request rejected also, and I had the power of an entire Agilent product division pulling for my cause. Hence, we're going with NI for our product development and an NI GPIB card will be bundled with this Agilent-sold product which will be available on platforms other than Windows! This sounds paradoxical but Agilent, like HP, has product line profit & loss so product line managers are free to "do what it takes" to make their numbers even if that means using a component that competes with another division. It's actually a wise strategy overall.

To the GPIB-hater: if you want to foot the bill for replacing the vast installed base of both HW and SW that is based on GPIB, please, write the check now (easily in the ten of billions of $) and everyone who currently slogs GPIB code will gladly switch to some other standard. Until then, this is what we've got. USB would be nice but no one is going to pay to replace equipment that is perfectly functional, fully depreciated and still generating value despite being purchased decades ago. On top of that, in most cases, it's the physics of the measurement that determine throughput and not the i/f to the computer, so GPIB ends up being perfectly acceptable on all counts but programmer comfort! That darn HP/Agilent and other vendor stuff were made with too much reliability! Some of the newer stuff has various newer i/f but for products that have a use life of 20-30 years, even USB is new and untested in the grand scheme of things - it could become a has-been i/f in a blink like Mac ADB is now.

JGSki

Running a Research Lab on Free Software

[fireweaver]

All I want to know is why the fuck are the hardware makers so damn reluctant to tell you how to use thier products (as in write drivers for them)? Does Microsoft have that much of a stranglehold on them or is there some other consideration (such as possible appropriation of trade secrets or some other such nonsense.)

Matlab!

[erf]

I was in a similar position...I was using Linux to run a laser tweezer setup in our small research group. Linux was such a pain - inconsistent libraries, poor support, flaky drivers unsupported by vendors, etc.

I switched to Windows 2000 Pro for the vastly superior support for data acquisition hardware, framegrabbers, and so on. Along the way I switched from doing GUI stuff in wxWindows/C++ to Matlab. The GUI stuff in Matlab 6.x is pretty good. It's got a bigger learning curve than LabView but it can also be used to implement much of your data analysis as well, with its vast libraries of mathematical functions and decent performance.

With the new hardware coming out, our lab now has a standing policy to only purchase items with USB or FireWire interfaces whenever possible, so the PCs don't even need to be opened up anymore. These drivers are rarely if ever available for Linux.

Use whatever makes your lab the most productive. Standard lab software like LabView, Matlab, and Mathematica are a safe way to implement software since they're so popular and they're more efficient and productive than C/C++ or VB. If you need high performance computing, then go consider F90, C++, etc., but instrumentation should never be controlled like that. You need real time? Buy a real time board.

LabVIEW also runs on OS X

[daveschroeder]

...as does IGOR, which someone else mentioned:

LabVIEW

IGOR

Some straegic thoughts

[Mendenhall]

I am a physicist at Vanderbilt University, in the Free Electron Laser Center, who has been doing various kinds of distributed data acquisition and control for about 25 years. I have run into many of the same problems with closed interfaces the author describes, and am slowly developing a strategy to minimize the impact of this problem.

First, I try to avoid products which depend on a closed library/DLL/whatever to control them. This has resulted in my shifting away from a lot of PCI-card devices to (when possible) external devices which communicate via GPIB/IEEE-488, rs233/422/488 interfaces, USB interfaces, network interfaces and (I hope soon) FireWire/IEEE-1394 interfaces. For such devices, one can often find programming specifications, although not always. Obviously, using the slower interfaces may result in lower performance in high-bandwidth environments, so it won't always be an option. However, the fastest current IEEE-1394 looks very promising, as it can support speeds that only a few years ago would saturate a PCI bus.

I have also discovered another interesting phenomenon: 'hidden' standards. In the past year, I became aware of a little-advertised standard, VXI-11, (www.vxi.org), which is a protocol for communicating with GPIB-like devices over TCP/IP. Although one almost never hears about it, a lot of devices support it. Tektronix and Agilent Infiniium scopes use it, and Ethernet-GPIB converters from Agilent, Tektronix and National Instruments all use this. The protocol is open, sunrpc based, and quite easy to implement. However, each company hides any reference to it deep inside the documentation, and basically provides their own Windows dll for communicating with their devices. The Agilent E5810 Ethernet-GPIB converter (a very elegant box) even calls itself a GPIB-LAN interface for Windows, as its official product name, even though it is almost fully VXI-11 compliant and can be used from any platform. I have no idea why they actively _hide_ its cross-platform compatibility.

<slight advertisement> I am trying to address some of these issues by releasing a lot of the code I am working on to interface various devices. I am a python fan, so I have a sourceforge project PythonLabTools which is a library of open-source code to implement communications with various types of devices which are effectively GPIB devices but run over the net via TCP/IP. I am also adding other classes of interface support to this library. An example is the Verinier Software LabPro, a low-end but quite nice and inexpensive a/d, d/a, and digitial i/o box which communicates via serial and USB. There are also data analysis tools in this package (fitting, et.c), and support for the National Instruments DSTP protocol, which allows LabVIEW to share data over a network, and (in this case) allows a Python program to directly interact with a LabVIEW program. This allows separation of the fancy user interface capabilities of LabVIEW from more intensive data analysis which can be executed in Python. <end advertisement>

I am looking forward to a day when more and more external devices with published interfaces become available. Internal devices (PCI or whatever), of course, take a lot of work to make drivers, but since communications with external devices can be carried out in userland, they are easier to provide cross-platform support for. I am alos watching as bandwidth on external interfaces rises to the point where there may not be any need for internal cards and the hassles they create.

I spent a fair amount of time a couple days ago lobbying our National Instruments representative for opening up the interfaces to more of their devices, especially their FireWire/1394 based products, which are right now only supported on Windows (which _really_ galls me, since Apple and Sony spearheaded this interface!). We are a fairly significant customer of theirs, and I think they actually listen.

Re:Comedi gooooood, APM baaaad.....

[Pxtl]

Speaking of NI, some labs I've worked with use LabVIEW. Yes, its prohibitively expensive (especially compared to VB) but its spectacular for scientific RAD. Plus, the programming is wholly graphical, which should be refreshing to those scientists that have no experience with text-based programming.

An opensource group would do well to attempt some sort of "workalike" to the language - the ease-of-use is stunning.

That being said, part of the reason it is popular is that instrument companies are pretty good for providing drivers for LabVIEW. An open-source project would lack that unless they also implemented their driver system, which would probably get that project in big trouble.

Kmax

[Eowaennor]

I have been working with Kmax for as long as I can remember... It is platform independant, Java based data acquisition software that allows you to build graphical toolsheets to interface with many different bus types. It can talk to GPIB, CAMAC, VME, and any other bus that you can write a JNI driver for. You can get a free version of Kmax at Sparrow Corporation, and I think the only thing that is disabled is the Save feature...

From my experience, Kmax has been the most versatile data acquisition software I have ever used. The way it is designed, if you want something more, just write your own KmaxDevice or KmaxDriver (interfaces are documented) and you're all set! It even has options for remote connections over TCP/IP for client-server connections, useful if you want to take a look at your data from home or if you feel like changing some parameters without walking down the hall and mucking with your racks of equipment.

-Eowaennor

GPIB?

[Andy Dodd]

Not sure what protocols your company uses, but at my company, nearly everything is GPIB controlled. Most stuff in the past was either done using LabVIEW or manually - In my lab it's almost all Perl these days thanks to me.

http://www.mock.com/gpib/ has an excellent library for GPIB in Perl. It only supports older equipment, but at least in the GPIB world, nearly everything has a published command set, and the library makes adding support for other instruments extremely simple. (I have yet to find an instrument that didn't have detailed programming info. It takes me an average of an hour to implement most of the funcionality of a new piece of equipment. Sadly, the work I've done on that lib will most likely stay in-house.)

Now if only Octave had the external interfacing capabilities that Matlab did. (Even with some of the "Matlab compatibility" libraries like octave-forge, Octave is still missing a ton of signal processing toolkit functions like psd() - Octave will also not interface with any of our digital I/O cards). Even Matlab under Linux won't help us here.

In short: We've basically replaced LabVIEW with Perl here (to much rejoicing - People at this facility are more familiar with text-based programming than GUI programming.), but I don't forsee us replacing Matlab any time soon.

Linux DAQ hardware

[confused one]

National Instruments, as a company (no, I don't work for them) has been very good about supporting Linux. All of their software is available in Linux versions, All of their hardware has Linux drivers. And they release I/O port maps and hardware specs for those who want to develop their own drivers.

There's a lot of VME/VXI hardware out there that will also work well with linux kernels (I've done it). Compact PCI (cpci) should work as well, although much of it seems to be driven by x86 / Windows embedded computers...

Any hardware which communicates to the workstation via a standard interface (ethernet, usb, gpib, serial, etc) should work just fine.

The real issue is simply checking the vendors of your hardware BEFORE you make the purchases to see if the hardware is supported. This should be true with any hardware purchase. (It is possible to buy hardware that doesn't work with Windows...)

If you're trying to use legacy instruments (which you already own) whose manufacturer does not support Linux and who refuses to release interface information (because of it's proprietary nature); well then, you're out of luck unless you can kludge an interface with a Windows PC talking to the device(s) (acting as an ad-hoc interface) and a Unix system doing the rest of the work. You'll get better performance this way than trying to have the Windows PC do all the work.

I'd urge you to use Unix or Linux variants for data acquisition and controlls. Windows is not deterministic and DOES NOT do real-time (I don't care what Microsoft says or how fast the machine runs).

that's just my 2cents worth...

Try GNUPLOT and GNU SCIENTIFIC LIBRARY

[Anonymous Coward]

gnu scientific library

and

gnu plot

Absolutely killer apps !!!

What exactly are you missing out on?

[Junks Jerzey]

not just a pride thing, unlike many colleagues I know what I'm missing out on, in the free software world

Like what, for example? Under Windows you can run Emacs, Vim, Perl, Python, Ruby, etc. You can get bash or another UNIXy shell if you like, and all the same command line tools. Putting together GUI interfaces for little tools with Visual Basic is a *great* idea. You could use something else, of course, like Tcl/Tk, or a free VB-like system, but VB is very good at what it does.

Reliability isn't an issue, if you're running Windows 2000. I have never had a single crash doing heavy software development under Windows 2000.

To me, it sounds like you just want to avoid Microsoft and Windows at all costs, but you don't have a real reason. In fact, you're even attempting to move away from the OS that most of your peripherals are designed to run under. Very strange.

PSU Physics / OSS

[XtAt]

At PSU Physics , we use a variety of OSS for research. Most notable is our computing cluster which runs particle simulations and such on Linux. I'm unawae of any who have reversed drivers for their instruments to run on OSS in my department, however many researchers use PERL to analyze the results.

Use OSS, think OSS

[TheSwirlingMaelstrom]

I'm an astrophysicist in a non-profit research institute. We have very few 'special' hardware requirements like the poster, and so are not as limited by what OS hardware vendors are willing to support, but I've come across similar situations.

When it comes to proprietary hardware with proprietary drivers on a limited number of (non-free) OSes, you're stuck. Everytime you talk to the vendor, or the support staff, you need to ask them about Linux (or BSD, or whatever) support. Be the bee in their bonnet that gets them thinking about supporting other operating systems. They don't necessarily have to GPL their software and drivers (I know, sacrilege!), but the ability to use your hardware on Linux means your one step closer to moving the lab over to Linux. Also, even if the data-acquisition is on, for example, Windows, that doesn't mean the data-analysis has to be on the same OS.... (Unless the data format is proprietary =8-( )

However, being a researcher, you should be used to the concept of peer-reviewed publications: nothing is published in established journals without having being scrutinized by other researchers in the same field. The same concept applies to OSS: open source software, at sometime in its public life, is viewed by enough people that bugs, cheats, etc., will probably be caught (things slip through, as they do in the scientific peer-review process, but the idea is sound). If you're doing research in the uncharted regions of physical science, you can't expect that someone would have written all of the software you need to get there and understand what you discover. This means that you, the researcher, are obligated to write the software. This software should be open sourced and peer-reviewed, saving your collegues and other researchers the same headache. What is done in scientific research is often governed by the 'publish or perish' doctrine. But keep in mind that what you publish might not have to be scientific papers: if everyone in quantum computing labs around the world knows of or makes use of your software, you will have more exposure than publishing a few obscure papers.

Also, despite the fact that you might be analysing unique data, I would hazard to guess that large amounts of the mathematics and statistics you would be using are not unique. Save yourself time when writing your software and don't re-invent the wheel: use publicly available mathematical and statistical packages. There are enough out there that I'm not going to bother giving URLs...

I look forward to seeing your project (not in VB or C#) on sourceforge =;-)

(Oh, and when your Windows-bound collegues ask about using your software on their OS, you can say "Sorry, it's only supported on Linux." That'll make you feel better, trust me =8-)

You gotta be nuts.

[twitter]

VB and LabView are the defacto standards.

Au Contrere, my silly AC frere. If you got stuck with some fancy device that only works in windblows, the least you can do for youreself is learn to use it's C interface. Almost anything that has a dumb VB interface also comes with one that works with C. While VB works for M$ and only M$, C is everywhere and a much better thing to learn. All VB can add for you is a dinky interface and loads of heartache and rewrites.

Lab view might be useful, but real labs don't hamper themselves with vendor lock in nighmares like VB.