Managing Parallel Development in Two Languages?

Abhaga asks: "I work for a technology startup and our research work is mostly done in Matlab. The technology has matured, and now we are looking to build prototypes and products in C++. However, the dilemma is about the further research work/enhancements to the system. Should they be done in Matlab and continuously ported to C++, or is it better to move to C++ once and for all, at this point of time? Anyone having experience with similar things, what should we keep in mind while deciding on one or other."

Comments

Do one thing at a time.

[Anonymous Coward]

Every development project should have a proof of concept phase. You need to know that the underlying idea will work. Get something working however you can. Once you have done that you always have a fallback position that you know works. That's the stage where you use Matlab.

Trying to write C++ code and develop the math at the same time means that you have four times the trouble debugging. If you have a problem you won't be sure whether it's in the math or the code. If you get the math right first, you know that any problems you have are in the code.

Start High, Then Low

[Webz]

With neither experience in parallel development or MATLAB, here's something I've read before (regarding Ruby and C++)...

Start in whatever language happens to be easiest/most high level. Easiest in that whatever helps you express your final product the fastest. Then, when this prototype is up and running, go ahead and reprogram it in C++ for speed.

Think of using the first language as a roadmap, where you can concentrate on organizing your thoughts and getting user requirements out of the way. Done purely in C++, you may be subject to premature optimization or just wasting time re-inventing constructs and concepts that are trivial in the other language.

Why do this at all?

[kognate]

First, ask yourself why you want to port existing code to a new language? Presumably, the people who are writing
the Matlab code have a facility with Matlab and are subject matter experts that are doing the heavy lifting (algorithmically speaking). Are the C++ coders the same people? If they're not, can you afford to spend the time/staff to do the porting? Should the
original code even be in Matlab in the first place?

You can call matlab libraries from C++ code, which would seem to be the best of both worlds. Then you wouldn't have to port anything.

Lastly, this is not the kind of question that will get answered well on Slashdot. People who have never used matlab will make assumptions and not understand that it is very unlikely that C++ will have the kind of simulation and and capabilities that Matlab does. Besides, a lot of the time Matlab people (scientists, engineers, quants, etc) may be comfortable working Matlab but not C++, so you do what you can to make it possible for them to work. Also, the suggestion that Mathworks will raise pricing and hold your work hostage is laughable: They already do that, their pricing is crazy.

Re:Why do this at all?

[Anonymous Coward]

I'd say it strongly depends on what it is and who the people are that will have to do the porting. (btw. I'm working with matlab as well as c++ in a research environment that is largely matlab-dominated.)

You have to realize that the speed at which the developement team works will break in significantly once they start to migrate to c++ (and it will stay below the former speed !!). Matlab is a very powerful tool when it comes to numerical tasks or data evaluation. It is also very forgiving regarding "quick and dirty" programming styles. You just don't have to wory about what is habbening behind the curtain, much different than when programming in c++.

If some guys in your team claim they already worked in c++ and they think migration won't be any problem, be sure to check wether they really had programming experiences with a PROJECT OF THIS SCALE. It is much more difficult to build a complex application in c++ than in matlab without producing unreadable unreliable and malfunctioning spaghetti-code. This is something especially non-IT researchers (such as me) tend to overlook.

And if it's a piece of software that will require user interaction, be sure to either have the GUI and the program framework up and running before the migration of the "worker" code starts or else have an IT PROFESSIONAL evaluate how much work it will be to create the GUI for your application. Because creating a GUI to your application is MUCH more difficult than creating one in matlab (calculate the time in minutes you needed to do it in matlab times thousand).

Daniel.

Re:The easy way

[EsbenMoseHansen]

octave 2.9 is pretty awsome. We use it (for solving a lot of Lp problems, with some branch-and-bound), and it works beautiful.

As for the question... I would question the wisdom in abandoning octave (or matlab) at all, but if you do need to do it, do it in small steps. At least, that is the best way in my experience.

Been there, Done That...

[occamboy]

I assume that the algorithms and math are in Matlab. Matlab is much better than C++ for developing and unit testing math "stuff". However, shipping Matlab libraries with your application means a more complicated setup, licensing issues - and it will look pretty "bush league" (try it you'll see what I mean). Also, I'm guessing that your "domain experts" are more comfortable with Matlab than with C++ - which is why you're asking this question.

I would continue to develop algorithms in Matlab, and use the Matlab compiler to move the algorithms to C++ for integration with the C++ "presentation layer" code. Then compile and ship an all-C++ product.

Re:Do it in C++ from the start

[mangu]

if you're implementing an algorithm to solve a 2nd order differential equation using the finite element method or using the shooting method it is hard

Hard? Only if you cannot or don't want to use existing libraries for C++. Now try to find a pre-packaged solution for "they want a button for downloading the data in the same dialog that lets them open an Excel spreadsheet" or any of the infinite other changes one always gets to do in any non-trivial GUI.

Similar Situation

[robbyjo]

I've been in similar situation, except that we chose Java as opposed to C++ for the "lower" level language. For the higher level langauge, we have both Matlab and R. We're also dealing with research situation. Here's my experience...

• Prototypes are best done in higher level language, in this case, Matlab. Hands down. You want to test a new research methodologies fast. You don't want to get bogged down by unneccessary programming constructs. Moreover, it's the scientist's job who do the prototyping (since he invents the algorithm, right?). Scientists are more familiar with Matlab than C++ or Java.
• Be aware that prototypes are ALWAYS poorly structured. More often than not, they're more like spaghetti code and/or copy and paste. Prototypes are just prototypes. They're just proof of concepts that a particular method works.
• Consequently, you may want a cadre of C++/Java developers to do the structuring. It's more like 4 low level developers per 1 scientist who does the prototyping. Often times the scientists don't know low level languages well. So, it's the C++/Java developers' job to figure out the scientist's program. Of course, the scientist would have to explain how the algorithm / source code works. On the contrary, reading MatLab / R is NOT as hard as what people says here (they must be smoking cracks. Don't listen to them). The only trouble is to familiarize yourself with Matlab / R API, which can be cryptic for some.
• With respect to libraries, Matlab / R have a throng of ready made scientific libraries. Of course, for C++ you can use GSL, LAPACK, ATLAS, etc. But the problem is that sometimes the library call does NOT correspond one-to-one. So, you'll need to tinker around to find out how each library call behaves. Moreover, for some operations, like Matrix / Vector operations, are very simple in Matlab / R, since matrices / vectors can be treated as if they were scalars. Be careful in porting those.
• In addition, you'll need to profile the library call. Make sure you actually GAIN speed with such library calls (or else you'll need to use something else). In addition, speed is NOT the only concern. Accuracy is VERY important. You don't want to use a speedy library with expense of accuracy. In scientific programs, this tradeoff is OFTEN NOT desirable. Make SURE the libraries have the same accuracy level. This is often the grounds to dismiss some unknown library who only claims that they're fast. Losing one degree of accuracy is often a BIG issue in scientific library. For example: If a library is at least 10^-16 accurate may not be acceptable as opposed to a library with an accuracy of at least 10^-17. Think of simulations, which may have millions upon millions of iterations. One degree difference in accuracy often makes a HUGE difference in the final result. Therefore, it is OFTEN more desirable to obtain an open source library where you can inspect the algorithm and point out places where a library call may lose accuracy.
• Familiarize yourselves with many scientific algorithms that improves accuracy. In Matlab or R, they autodetect pathological situations where accuracy can lose. You'll need to do that manually in C++. For example: If you have a nearly singular matrix, you'll need SVD for better accuracy. In general, you can use QR decomposition. If the accuracy is not really an issue, LU decomposition might be enough. In Matlab / R, it can detect the matrix automatically when you try to invert the matrix and use the appropriate algorithm. Pay attention to that. Make sure your C++ program also behaves similarly.
• You MUST make A LOT of regression tests. If it is possible, make the prototype run with the same file format as in the final product. If it isn't possible, convert the file format first and then confirm with the scientist that both of them are exactly the same. Make sure that for all tests, both returns the same numbers

Re:Similar Situation

[mangu]

If you have a nearly singular matrix, you'll need SVD for better accuracy. In general, you can use QR decomposition. If the accuracy is not really an issue, LU decomposition might be enough. In Matlab / R, it can detect the matrix automatically when you try to invert the matrix and use the appropriate algorithm.

Then Matlab must have improved a lot since I last used it (Version 4). A problem I had was that matrices were well behaved with small test cases, but became ill conditioned when we used actual working data. Rewriting everything to use SVD in those cases was a real PITA, since many functions used QR internally and one had to modify the Matlab toolboxes library code. I ended with many adapted functions, like invfreqs_svd instead of invfreqs, for instance.

My conclusion was that, although it did textbook examples perfectly, Matlab wasn't robust enough to handle real-life problems.

Re:Proprietry lock-in

[try_anything]

Matlab also allows the expensive guys with math PhDs to work quickly in a pleasant, familiar, supportive environment. Those guys are smart enough to learn C++ and deal with memory management and templates (often helpful for supremely efficient math code), but it's not a good use of their time if it can be avoided. If you need C++, let the cheap programmers transcribe the Matlab work into C++ and do the tedious job of debugging in C++, while the math guys stay happy and productive in Matlab.

Re:C++

[mangu]

For example: every time you write a function, should it take its arguments by value, by reference, or by pointer?

Good question. I'll answer that if you answer this: every time I get to a street intersection, should I turn right, turn left, or go straight ahead? The answer to both is: it depends. Where do you want to go? If the argument is a basic type that will not be changed, use a value, if it needs to be changed, use a reference, if it's a large structure or array, use a pointer.

A language that ignores such details will not be necessarily safer or easier to code. For instance, should strings be mutable or not? C++ lets you choose, use the "const" modifier to make a string immutable. In languages where this property is fixed you cannot just ignore it, you may have to work around it, at the cost of possible bugs or inefficiency. When I was learning Python, I spent a lot of time in a program because a string wasn't changing when I tried to modify it. After I found out that strings in Python are immutable by design, I had to redesign my program.

C++ hard to learn, but there is a consolation, you only have to learn it once. The biggest problem in learning C++ isn't the complexity of the language itself, but the learning curve. In other languages you can start small and learn as you go, but to be effective in C++ you have to learn many details before you start using it efficiently. OTOH, the syntax is rather well-behaved, you don't have the anomalies you find in Perl, for instance, where variable types depend on the first character of the name, or in Ruby where a block of code is delimited by an "end" in some circumstances and by braces in other cases.

After you have become experienced, coding in C++ is easier than in more "helpful" languages, because you always have the choice of the best method to do everything. Knowing C++ is like riding a cross-country motorcycle. You can go to places you can't reach with either a bus (easier to ride), or a Ferrari (faster in well paved roads with light traffic).