The Power of Multi-Language Applications

wbav queries: "I've been programming for a number of years, and someone always asks, 'What language do you use, Java or C++?'. Now personally, I find that question a little biased, mainly because, of how I program. Rather than making one massive program, adding in all the support I need to make up for weaknesses in languages, I prefer to make several different apps that call each other, each using the strengths of that particular language. I tend to use C++ as my controlling program, and then execute Perl, PHP, or Java depending on what will give me the best performance for and cause me the least amount of pain to accomplish the task at hand. Do you guys use this kind of method, or do you try to do everything in one program? What advantages or disadvantages do you see in creating one program compared to many programs?"

Comments

well

[djweis]

I can see some merit in that, but if you pass on maintenance to someone else, you could have just increased the number of people required to support the project, or require the new developers to learn more new languages before they can be useful. It's easier to find someone skilled in foo than to find someone that knows foo, bar, baz, etc.

Re:well

[Nelson]

I agree. I hope he described his system poorly and really meant something else.

The fact is, lot's of large projects are multi-language based right now. Not always in ways you might expect but for example, a large project I worked on for a very large and successful computer company had a large C++ code base. It has a set of no less than 35 support "scripts" written in Korn shell, it has an elaborate build process written in "make" which has portions that break out into Korn or Bourne shell to read environment variables and do various things. If you want to be pedantic you could easily point to pure C in the code base as well... And that doesn't include the java based client that was written later. Honestly I don't know of an easier or better way to run it, not for something like that. It was a very large "system" and not really an "app" or a "program" there were dozens of programs in the system that were sold as a package.

With web based apps I don't think it's terribly uncommon to have C or C++ doing some kind or service oriented task and then have something like perl or php interface to it and format some HTML. I could easily see a C++ server with some support scripts in Bourne shell (perhaps entries in /etc to start it up) feeding XML type data to a PHP or Perl script running on apache or something like that. As long as things are kept sane I could see it being a very easy way to develop an app and probably a sane one. I don't see someone writing C or C++ code to do what PHP does any better than PHP does it or producing the product very quickly. As a consultant, most of my clients would gladly pay for a few lines of php if it means I get done in 2 months instead of 4 months.

Optimization across processes can be tricky...

[gatkinso]

...just the cost of the context switch can be enought to deter one from this.

Cross platform development just got that much more complex.

Add in the cost (not always $) of additional development tools, maintaining such a beast, and obtaining the required knowledge/skills can make your approach daunting - esp on large scale multideveloper projects.

Re:Optimization across processes can be tricky...

[lkaos]

I hear these arguments alot, but they are just not right.

1. multi-processor scaling, generally a no brainer with seperate processes, if they are broken up intelligently

This is what threads are for. Now, there has to be communications amoung the processes and the overhead to accomplish this is greatly reduced if done in a single application. Now, you can bitch about programming threads but if you can break the program into entirely seperate processes, you shouldn't have to worry about race conditions since you code is already so seperate.

2. reduced memory foot print - well broken up processes mean you don't have to worry about loading up a ton of libraries all at once, you can let the os deal with memory as processes are kicked off and finished

Well, one would hope that if multiple processes need to be using a common library that it be made into a shared library. There is only ever one instance of a shared library in memory. The little bit about avoiding memory leakage by having processes restart frequently requires a great deal of latency when actions occur. If your application can handle this latency, and you want to be that lazy about programming, then just have your application restart every n minutes.

3. security - different processes can be run as untrusting groups, a great boon especially if one processes needs to run as root!

There are very few circumstances when a process should ever have to run as root. Why do a bunch of apps have to run in a group whereas a single app can't? If an application absolutely requires some bit of access to resources only given to root, then having that application just exist presents a huge problem since you are effectively making a resource that many smart people think shouldn't be given to arbitrary applications and giving it to them.

qmail is written in C, and this is definitely the C way to do things, but in C++, objects present a much cleaner interface to all of this. That is the beauty of OOD. qmail really should be written in C++ if there is such to encapsulate things. (although OOD is accomplishable in C, albeit very, very ugly).

use 'em all

[gerardrj]

I personally like the 'use whatever gets the job done' technique.
Writing differnt parts of apps in difernt languages is the easiest way for me to accomplish a task, and to support the software later. For instance, writing some things in AWK or PERL will dramatically reduce the complexity of a C++ program. I Write some things in C or Assembly (I gave up machine language with my 6809) for speed or source code control.
There is something to be said for developing a library of code all in one language, but for my purposes the multi-language approach works best.

oh god yes

[jabbo]

I'd go insane if I tried to use a compiled language for frequently-changing applications (eg. web interfaces to purchasing systems, database large object manipulation & indexing, etc.) but likewise I'd grow old waiting for things to happen if the cores weren't in C.

Personally I dislike C++ because I find it harder to port than ANSI C. But I wrote plenty of it when I was learning how to program (before I learned Scheme and Perl and Java and got some latitude in my paradigms). Either C or C++ is great for speed and hooking into the OS or an Apache module or whatever. It's also less tempting for admins or rookies to mess with C code because bad code may not compile.

On the other hand, object-oriented (or at least modular) PHP and Perl code, and decently-written Java code, is much easier to adapt to changing demands. I stick with PHP and Perl, myself, and I use Perl as kind of a glue-core layer between C applications and PHP interfaces. If I had more time I'd probably write the hooks as PHP extensions, but Perl is just so damn powerful when you use it right.

I'm not sure what people are thinking when they specify that everything on a project "needs" to be done in Java or whatever. It's not particularly hard to use my code, since it's all just calls to libraries that automagically do the hard stuff. More importantly, I tend to use POD or Javadoc-style comments in everything, and don't put it anywhere else. That way I am forced to keep it all up to date, because that's where *I* remind myself what arguments to feed what methods!

Re:You scare me.

[MWright]

Actually, I think you could argue that it would make debugging easier. The program is proken down into parts, each doing a certain task. If one part has a bug, you only need to debug that specific part, rather than wading through masses of code that aren't related.

Boy, I hope I never run into one of your programs.

[Wakko Warner]

Imagine trying to install it on a machine without one of the 80 billion interpreters or libraries or virtual machines your code needs in order to run? Is it multiplatform? How do you maintain this stuff? Do you get paid to write it, or is this simply stuff you write for yourself and use by yourself? I guess, if the former, there's quite a bit of job security there. But I'd shoot code that looked like that, just by the sound of it.

Re:Quite the Contrary

[plover]

I, too, am working on a large programming project. It's primarily written in C++, but I use other languages to write quick-n-dirty tests for the objects.

A curious side benefit I've found is that when I cross language boundaries, I tend to expose more flaws than if I just had a small main() testing the objects. Crossing language boundaries forces me to code the interface to each object very properly and distinctly.

John

Think about it

[Ars-Fartsica]

I'm not sure what people are thinking when they specify that everything on a project "needs" to be done in Java or whatever.

Building software is a function of economics. If you've dumped considerable resources into training, code licenses, an existing software base, or other real-world issues, you will be very inclined to insist that your "chosen" language be used in future projects unless there is an overwhelming issue that prohibits it.

IMHO your approach is fine

[wrinkledshirt]

I don't think there's anything wrong with your method. Certainly if you can coordinate everything, your approach has definite advantages.

Personally, I've found that a good knowledge of C can pretty much be used to fix any limitations of any other language (and pretty much every language has better string-handling routines than C ;). It's also a great glue language, so that if you've got one language that doesn't query a database, but it can interact with C, then you can write the C stuff to query the database and have the other portion of the program call that. Pretty much everything out there has a C-based API.

In my case, I coded a messageboard website in PHP, which does everything really well, except that when it came time to work in a search engine for the site, I figured I wouldn't want to go through some sequential parsing of all the files through PHP each time because it would probably be superslow and interpreted each time.

Now, you can redirect from a php page to a c-based cgi page and vice-versa (just have to know your HTTP headers), and you can code sockets in c in linux pretty easily. So what I have is a daemon process running in the background in C, with all the information indexed in a binary search tree (from when it was loaded) and ready to receive connections. The php search page hands off to the cgi page, which queries the daemon with some data, receives the resulting data, and redirects the output as part of the header to a php page, which displays the results. The search daemon in the background is all c-based, with queries to postgresql, and the way it's set up now, could receive CRON commands to rebalance the tree, repopulate the tree, add new messages to be indexed, dump all the contents to an XML file, etc.

Now, PHP has bindings to pretty much every database worth its salt, does great string manipulation, can interact with XML files, can interact with LDAP (I think), has some socket capabilities, and they're working on making it work with GTK+ and also be CORBA compatible. PHP combined with a web-browser can pretty much create the perfect client, and of course, you do whatever the hell you want with the server...

The main problem is having a good protocol to communicate between everything. That's nothing to scoff at. But I still think your approach is sound.

Re:Release and maintenance problems.

[KyleCordes]

[Not only does my employer need to find someone who can code in C, C++, Java, Perl, Python, shell script, and assembler]

You are certainly right in the extreme case, but this is often overdone. If your employer is not hiring people who can learn a bit of Python when they need it, perhaps they should hire different people. If you can save a lot of time and money why adding a second language to a project, it may well be worthwhile. Adding the 5th language very likely is never worth it.

Framework

[mugnyte]

For a long time, many of the readers of /. have laboured to put together a private, then public "Framework"

While not discussing the merits of such an endeavor, they almost always have a productivity gain result, with the added propeller-head feature of making the dev team "kings of the castle" of How It Should Be Done.

Every OS is merely an minimal acceptable level of a framework.

By writing applications in a single language, platform or environment, you leverage your knowledge in the language. BFD. BUT! When you organize it on a grand scale as a Framework, you can make extraordinary differences...and you avoid the hodge podge of applications that may or may not live as long as your users need them.

In my world, enterprise applications written in a massive and ever-more-portable / feature-filled / wrappable C++ framework are the norm. They last years and become somewhat of a family hierloom that we take pride in.

We have budding frameworks (both built and purchased) for VB and Delphi - but they slip and slide through versions way too often. Ever 3 years the language is updated to take advantage of the Next Big Thing. In a lower-level framework, you can add in these features because you're closer to the understanding of basic toolset.

Combined with a Component take on development, one can merely add a object to the mix and suddenly the security model, status reporting, version control and documentation stay compatible.

If you are trying to use different frameworks to achieve solid goals merely from your single opinion of "best for the job" - you are missing a lot of management skills.

mug

Welcome to the World of Web Development

[thomis]

On any given page in my current project you are seeing the results of VBscript or JavaScript running on the server (ASP), usually JS running on the browser(for DHTML effects and validation), XML/XSLT (formatting large result sets and general neato-ness), and FoxPro (the database. Don't laugh, I'm trapped in an M$ shop, and FoxPro is the most subversive dev tool we use (no licensing for server, just whip up a COM, and it kicks M$QL's ass))

This seems pretty common in the Web world, and yes, it explains alot... but I definitely try to pick the right tool for the job (dates are easier to handle in VBS, but forget about regular expressions, that's a job for JS).
I think I'd get bored quick if I had to develop in on language. Then again, it might lead to a deep understanding of the language.

Of course, I could be wrong.

Perl + PHP

[justinstreufert]

I worked at eGrail, whose flagship product is coded in a mixture of both PHP and Perl. Web user interface is in PHP with a fast internal parsing system written in Perl.

There are definite advantages to this design; PHP is probably the easiest language in which to write a web GUI, and Perl greatly simplified the process of building the incredibly complex parser.

The disadvantages: eGrail has (or at least had) a ridiculously long list of dependencies; one needed both a working web server with PHP4 and many extensions, not to mention Perl 5 with a host of 3rd party modules. TWO seperate database interface libraries are required.

I think it was a good thing for eGrail, but it's a carefully balanced tradeoff.

Justin

Re:This is not human languages!

[scrytch]

Imperative languages are pretty much isomorphic. Learn one that's sufficiently rich and you pretty much know 'em all.

The story gets different when you start with other paradigms. Try teaching yourself Haskell or prolog in a morning. And it's languages like that that are often a reason to pick one over the other, because the problem space demands it. I could write a web form processor with db access in any language ... but trying to write a fraud detection system in C is like trying to build a house with a hammer (actually the fraud detection system I'm thinking of was written in APL ... yowza)

Weird languages

[YoJ]

A lot of people seem to be saying that it is dangerous to use "alternative" languages for major projects because people that must maintain the code may not be familiar with the language. This is undoubtedly a valid concern. One neat idea to get around this is to use a high-level language as a tool for creating a project written in a lower level language. For example, if the target application is some sort of GUI app, then someone may decide to use Java. But Java is not a particularly high-level language, so you decide to use Lisp. You create a Lisp "environment" that facilitates creating Java source code. Lisp functions create abstract data types representing Java programs with certain attributes, e.g. widgets and widget behaviors. Then you write a Lisp function that translates the data structure into valid Java source code. Developing the program then becomes less of a mundane Java programming task, and more high-level and fun.

This method has a loss of productivity from the added complexity of the task, but this can easily be made up for in increased productivity at the meta-level. I.e. if your application has many buttons with only slightly different functionality, a single Lisp function call creates each button. Changing behaviors and adding or deleting parts of the program becomes simple. Best of all, the end result is a valid Java program that anyone who knows Java can understand. If the person who comes after you does not understand Lisp or the idea of program generation, it doesn't matter. They have a Java program to hack on. If they do happen to understand Lisp, even better, they can pick up where you left off with a big productivity gain. Lisp is particularly suited to this type of thing, but any good high-level language would work.