The Challenge of Cross-Language Interoperability
CowboyRobot writes "David Chisnall of the University of Cambridge describes how interfacing between languages is increasingly important. You can no longer expect a nontrivial application to be written in a single language. High-level languages typically call code written in lower-level languages as part of their standard libraries (for example, GUI rendering), but adding calls can be difficult. In particular, interfaces between two languages that are not C are often difficult to construct. Even relatively simple examples, such as bridging between C++ and Java, are not typically handled automatically and require a C interface. The problem of interfacing between languages is going to become increasingly important to compiler writers over the coming years."
I don't even like .Net, but they won this round years ago.
every language but c and go suck
thats a fact
It took me until the third sentence to work out that it's referring to programming languages.
That said.... C rules!
What do you need multiple languages for anyway? Java does everything you could want. Java is a powerful, object-orientated, cross-platform language, with fully developed GUIs, such as Swing.
To demonstrate the superiority of Java, I can point to such leaders in their field as Eclipse, Minecraft, and this awesome applet I saw on someone's homepage once.
Anyone still using ancient and difficult to use languages such as C++ (let alone C!) are obviously crazy, and probably should be committed for their own good before they go on spree of shooting (not just themselves, but) other people in the foot. Java makes it almost impossible to shoot yourself, let alone others, in the foot.
Moreover, because Java is licensed under the GNU GPL, you can leverage the wisdom of crowds, and the powerful "many eyes make bugs shallow" concept to be confident that Java is the best.
And with just-in-time, Java is as fast as any other language, so you don't have to worry about the speed of execution. Instead, you can focus on developer time, and Java's just faster in that regard.
With built-in, from the ground up, support for Unicode, Java is there for the future, and is ready to be used across the multiverse (as soon as those aliens get their scripts into Unicode). Beat that C, with your lack of a string type.
And if you aren't convinced, tell me why do so many top enterprises use this language? You don't see ads from Fortune 500 companies looking for Ruby "developers" do you?
HELP MY ACCOUNT HAS BEEN HACKED BY AN ILLIBERAL ART STUDENT SET TO DESTROY THE INTERWEBZ!
I hope JNI burns in hell
The fault here lies specifically with Java and C++. Java's JNI is poorly designed and makes no serious attempt to make interoperability easy because it is against Java's philosophy. C++'s runtime is deliberately completely implementation dependent because C++'s designers thought that might let implementors squeeze out a little bit of performance.
Nevertheless, tools like Swig make even Java/C++ interoperation fairly easy, and many other languages try to accommodate C++ well (cf. Boost::Python).
Theoretically in a large organization with a huge and demanding application allowing developers to be able to interface with some sort of API with whatever language they choose may seem like a very flexible solution. I would be very worried that after a few years your code base would not only include a broad spectrum of languages but even potentially a broad spectrum of versions of those languages.
I suspect that you would find yourself having to learn Haskell for one emergency and Erlang for the next.
Where a multi language compatible API would be great though is when you start to migrate your system from one language to another. If you could do it piece by piece deploying each small well tested piece before moving on to the next I would suspect that many a disaster would be avoided.
We needed to interface OCaml and C++ in our project. SWIG was already being used to interface C and Java in other parts of the codebase, so we decided to give SWIG a try...with a few hiccups we managed to get it to work. Now that the infrastructure is in place adding new data structures to share is pretty easy.
I used to be a hardcore C programmer, but now I love OCaml and think it is wonderful for certain applications.
There is no getting away from having to mix multiple languages for a fair sized project I think.
Simple. Instead of having to suffer through problems with the stack do very simple network calls to servers set aside to do just this, sort of like DNS today. Throw a buzzword on it like SDI Net, or something else cool sounding, and have ISPs maintain these SDI Net servers. Simple.
In particular, interfaces between two languages that are not C are often difficult to construct
(Yes, I know what they mean. I think they meant to write "In particular, interfaces between two languages (when neither is C) are often difficult to construct". :)
Still, it amuses me to think about making an interface from C... to C
I'm sitting right now and coding an interface between Java and Erlang using JInterface, and the actual JInterface part is pretty good. It makes Java act as an Erlang node. But Java is just a horrible, horrible language.
Everything on the Java side gets so bloated it is just silly. What would take one line on the Erlang side can easily take 20 lines on the Java side.
I know you are just trolling, but...
Swing is the most horrible toolkit I've ever tried to use. Even monkeys can make a better one.
Eclipse is never needed for a real language, if you need an IDE, there is something wrong with your language.
Java does the opposite of trying to help you shoot yourself in the foot, by making the code bloated and spread out. More lines = more bugs, that is the simple fact.
Just-in-time does not make java "just as fast" automatically. I have 8 cores on my machine, and I can easily and safely make Erlang programs that max out all of them, and without any mutexes. The mutex model is just broken and as we move to more and more cores, you need languages that can easily and safely scale.
"Java unicode from the ground up", MY ASS! Why would you even make .toString() the default when converting from one type to the next. You always want .getBytes()!!!!
And stop smoking your crack for at least 5 seconds before lying to yourself about Fortune 500 companies not hiring anything but Java programmers. What do you think they coded Facebook chat in, for just one example? Not Java!
That should be enough fodder for the trolls.
Have you used Netflix, VUDU, Amazon, etc to watch movies (or, ok, World of Warcraft to play a game for this audience)? Then you have probably already seen this. Actually, even the WoW reference is unnecessary for anyone who understands game engines - almost all of them now use native cores + interpreted scripting engines.
Though I have to laugh at the article's choice of examples - holy crap, is CS academia really that far behind everyone else?
"You can no longer expect a nontrivial application to be written in a single language." ... that hugely depends on your definition of "nontrivial", but generally: yes, I can expect that and see no counter-indication for it.
The only exception is to have a kind of "core framework" which can be scripted and has an embedded interpreter (eg. python or similar), but that scripting language is then not a first-class-citizen on equal footing with the rest of the program, but a sort of "slave system" (does that make any sense?).
Use sockets. In majority of cases the performance is more than good enough, especially if designed properly, and you get network transparency "for free".
Sure there are cases where sockets are not appropriate, but IMHO they are far too seldom used.
One facet of this is that increasingly, any application of any great interest runs over the network, often using HTTP, so its UI is coded in JavaScript or somesuch and the back end can be any motley arrangement you like that happens to work.
Even traditionally single-host applications like word processors nowadays involve mechanisms for installing software updates and online dictionaries or cloud-based functionality. As for games, you can't even launch most single-player games that should have no need of a network connection without having to log in.
The popularity of the mobile app model is such that this trend is only going to increase. As such, the issue is not about binary runtime language compatibility frameworks like SWIG, but more about network protocols.
A programming language is just a syntax to express some concepts. The real question is: Why do we need a hundred different syntaxes to express the same concepts?
On this aspect, Python does handle interoperability pretty well (at least with C and C++). It might just have a little bit too many options: .h-like file
* ctypes: connect to any C library directly (you just have to not do any mistake in parameters, as there is not check)
* Python C extention: write a wrapper in C.
* SWIG: "automatically" generates the wrapper, based on some
* cython: write C code using python syntax
Personally, I just use ctypes or cython, and it's quite easy to interpolate with any software library I need.
WoW uses Lua, which is actually known for having a very simple and easy to use interface with C code. I even found an example on the Wikipedia page: http://en.wikipedia.org/wiki/Lua_(programming_language)#C_API
I really don't see the need to use any language other than C. It is simple, elegant, straightforward, portable and versatile, it allows you to do everything you need without too much cargo cult and there is an enormous number of tools available.
This is what Enterprise Busses were invented for (DDS, 0MQ etc).
Seriously,
This is the plight of windows ecosystems. Linux/Unix has had abilities in this regard for many many years. It's stable it's rock solid and it performs.
Java is a bit of a nightmare for performance. But a ton of Enterprise is written in Java. Depending on your role you would argue the same thing for python, perl ruby etc. The later languages tend to perform better when calling native libs and vice versa. .NET is an abomination of performance an security disasters. These issues are backed into it's architecture and require far to much skill and background to avoid.
Now compilers are NOT the issue. It's runtime binding that is the issue. Compilers do a great job. Runtime binding is where things really fall down. Again .NET. The more you can resolve binding issues prior to the runtime event the better. For example I'm sure a ton of people on /. can rant about the issue with RMI and the destruction it inflicts at runtime. I call RMI SUPER LATE BINDING. .NET is founded on the similar principles and it SUCKS.
Apps like SWIG allow you to create compile time bindings between libraries. Which though not perfect create far more robust interfaces between high level languages. Something you can actually test. Since interfaces from libs also tend to be stable. The are also robust.
Yes of course it is possible to build robust integrations which bind at run time. But the level of effort is ridiculous and is rarely justified in budgets of medium to small Enterprise.
If we mutate this thread to discuss hardware design languages, Verilog won the popularity contest. I really don't want to get involved with any project that dares mix VHDL and Verilog ... So mixing languages and HDL should be off the table :(
A quine chain in ~50 languages: https://github.com/mame/quine-relay
Now that's what I call cross-language interoperability.
For bonus points, they're even in alphabetical order.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
It it surprising to me that the article makes no mention of language workbenches, when they are a good solution to the problem the article describes. See our preprint https://peerj.com/preprints/112v2/ for an example of language interoperability, and references therein for descriptions of language workbenches.
Oh no...
xxxx is better than yyyy because..
Langugages (and really the platform / libraries are more important) ech have their own strength.
What you don't want to do is to solve a problem, add a prgramming language, because then suddenly tou have 3 problems instead of one:
-being to able to write in 2 langugages. (the subset of emplyess able to do that is much much smaller than those having good skills in one languges
-interface between them (rarely done, so few people understand the details.
-maintaining for a long time mulple languages.
If you run a JAVA enterpise. Fine. If you run perl/php, Fine. If you run C? ueh....cobol? Keep it!
Shlaer-Mellor / Executable UML have been offering this type of language independence for over 20 years. The method works from business to embedded software. All that's required is model compiler support for the target language, which can be bought off the shelf or made in-house. Currently model compilers exist for C, C++, Java, Ada, System C, and I'm sure there's more that I haven't encountered.
Verilog won the popularity contest
Does that make it a better language?
Also VHDL is still quite popular some places. I've worked for several companies in the last 10-15 years that use it. I suspect VHDL and Verilog will continue to co-exist. As far as tool chains are concerned, they're both essentially front ends. It's not hard to support both, and most do.
I get annoyed when the premise is so flawed, but stated as fact.
Major projects have been cross-language for decades. In the 70s, Fortran + C + Assembly were in most big languages. Or large systems using COBOL having to interface to non-COBOL systems. By the 80s, many had bits of Pascal (Borland was huge, remember?) and BASIC, with important routines hand-optimized in assembly. Or C. By the 90s, we had SQL and native code, DLLs written in random languages, and often VB for the UI. Most of what you use on a daily basis probably is browser-hosted but includes active controls in C# or C++, back end code in PHP or Java, database code in SQL and browser code in JavaScript or ActionScript. Many of my Android apps are mostly Java with some kernel-level support in C.
Yes, each language has it's own way... COM, exports, dllimport, etc. Until it has one, it's not a very functional language. But this isn't a new development.
Sheesh, even modern Fortran compilers can be interfaced directly with C code (use ISO_C_BINDING) — I do it all the time with gfortran. If even Fortran can get it right, surely the Java folks can agree on a standard way to get this done!
The success of Android (Java) and iOS (Objective-C) showed us that allowing several languages on a platform just adds unnecessary complexity. One language is sufficient to develop great applications.
IBM solved this problem 20 years ago with their Integrated Language Environment (ILE), not that anyone cares.
(Damn noobs think every problem they encounter is new and unique).
Proverbs 21:19
Instead of getting my programmers from Glasgow, I am sending my current programmers -- broke, penniless, desperate -- to live on the streets of Glasgow for a year.
Those that survive the ordeal will return to my Xanadu-like pleasure palace, where they will serve out their remaining years in matchless luxury.
-kgj
Paul Graham liked to blog about how awesome Lisp is. Apparently, he did some web back-ends in Lisp and was able to stay ahead of the competition. Now, Lisp has some awesome features. The two that stick out to me are (a) Lisp macros, which are arbitrary Lisp code run at compile time that emits arbitrary Lisp code that gets compiled, allowing some seriouly powerful constructs to be created very concisely and (b) massive libraries. That being said, I suspect that Paul Graham and his cohorts were more successful at using Lisp compared to another back-end language because they were both very skilled at the language and also super smart. The language they chose is actually MUCH less important. If they’d chosen another language, they would have done probably just about as well.
If you’re not a super smart programmer, I recommend Java (among other languages). Pass-by-reference and garbage collection obviate a lot of coding mistakes. And with Jit compilation, you get pretty darn fast code. Under 99% of circumstances Java performance is way better than adequate and makes MUCH better use of programmer engineering time.
But if what you want is super fast performance, a super smart C++ programmer will beat a super smart Java programmer. It’s a lot HARDER to get better results from C++, but the ceiling is higher. Some reasons for this:
- In memory-constrained environments, garbage collection imposes some overhead. I’ve worked on huge programs near the VM size limit whose performance was limited by GC performance. Most of the time, incremental GC in another thread is a win, but it can be a huge liability in memory constrainted environments. Instead, manual memory management in C++ allows you to make tighter use of the memory space and performs better in a memory-constranied environment.
- In a CPU-constrained environment, background GC steals cycles away from computation.
- Just like how macros are a huge win in Lisp, C++ templates generate customized code at compile time that can have huge perfrormance benefits. This is why C++ sort is faster than the C library qsort: The C++ sort is a template that inlines the comparison function for the type you’re sorting, rather than making a method call. In Java, you MIGHT get some of this from a fabulous JIT compiler.
- Compiling to bytecode is a huge information loss. If you used gcj to compile Java to native code, there’s the potential to have less information loss and therefore optimize better based on the programmer intent. But normally, Java uses bytecode as an intermediate. With more programmer intent knowledge, the C++ compiler can make smarter optimizations.
I could add more things, but I have other real work to do. Before I go, I’ll cap this off with two practical thoughts. You CAN get better performance from C++ than Java. Do you WANT to?
- Java has massive libraries too, where the critical parts are written in optimized native code, so if you make heavy use of Java libraries, you’ll see almost no difference in performance with regard to CPU throughput (GC being a separate issue). With no perceptible impact on peformance, less code to write, fewer common programming errors, and better use of engineering time, Java is quite often just an all-around practical win over C++. I say this as a programmer who prefers C++.
- If you’re REALLY REALLY concerned about performance, use Fortran. The language is more restrictive, providing the compiler with even more freedom to optimize. (For instance, no aliasing, no recursion.)
OMG tried to tackle the language interoperability with CORBA ( Common Object Request Broker Arch ).
They created another language: IDL
wait.
- these are not the droids you are looking for -
I like to use message queuing technologies like AMQP or ZeroMQ to achieve this kind of integration. Much simpler than link editing everything into one monstrous binary artifact.
Yeah, there are a bunch of games/engines that use Lua for their scripting now (and others that use Javascript, and some that implemented their own language, eg. QuakeC).
Not sure why my original post was labeled "flamebait". I guess someone out there in "CS academia" had mod points :) (it was a joke, people!)
On the web we have tens of languages on the server side communicating to browsers & crawlers. Crawlers are written in nearly every language.
How did we do it? A standard protocol.
So, stand up 2 programs, have a standard for message passing, and continue working. Forget library-like integration: you don't want some giant code in your process space anyway.
- Testing is simpler.
- Fractional deployment is equal to having DLLs, but simpler resource file management.
- Crashes are easier to debug, and far less insulation is necessary for a good overall experience.
- Scalability becomes simpler.
- The initiator of an internal issue can be hard to determine, but many solutions exist & it's often unnecessary to know.
Summary: Do one thing, do it well
Science & open-source build trust from peer review. Learn systems you can trust.
Verilog won the popularity contest
Does that make it a better language?
Also VHDL is still quite popular some places. I've worked for several companies in the last 10-15 years that use it. I suspect VHDL and Verilog will continue to co-exist. As far as tool chains are concerned, they're both essentially front ends. It's not hard to support both, and most do.
As far as I can tell, SystemVerilog won the popularity contest. This imports a number of VHDL ideas into verilog that make life easier. What it fails to do is import the operator overloading and resolution functions that works so effectively in VHDL. In VHDL if you want to define a new logic value, like say 'P' for values driven from unpowered logic, you can simply overload the std_logic operators, or replace std_logic, and the simulator will work for you. No so in Verilog. The logic values (1, 0, H, L, X etc) are wired into the language. You can't add a 'P'. Consequently, SystemVerilog makes it really difficult to verify low power fine-grained power gated designs.
Of course nobody making tool purchasing decisions considers these things.
I should use this sig to advertise my book ISBN-13 : 978-1501515132.
In particular, interfaces between two languages that are not C are often difficult to construct
I'm under the impression that this one of the primary use cases of service-oriented architecture.