Generic VMs Key To Future of Coding

snydeq writes "Fatal Exception's Neil McAllister calls for generic VMs divorced from the syntactic details of specific languages in order to provide developers with some much-needed flexibility in the years ahead: 'Imagine being able to program in the language of your choice and then choose from any of several different underlying engines to execute your code, depending upon the needs of your application.' This 'next major stage in the evolution of programming' is already under way, he writes, citing Jim Hugunin's work with Python on the CLR, Microsoft's forthcoming Dynamic Language Runtime, Jython, Sun's Da Vinci Machine, and the long-delayed Perl/Python Parrot. And with modern JITs capable of outputting machine code almost as efficient as hand-coded C, the idea of running code through a truly generic VM may be yet another key factor that will shape the future of scripting."

Comments

One standard, several implementations

[Anonymous Coward]

One standard, several implementations? Sounds nice in theory, just like the numerous standards that Sun has outputted where each vendor delivers its own implementation (JPA, JDBC, J2EE among others). However, in practise you pick *one* vendor and *one* implementation and run with it. Only a fool would dare switching implementation mid-development, making the choice really just academic, because there are always minor differences that "shouldn't" matter, but does.

Re:One standard, several implementations

[MyDixieWrecked]

You also run into problems where someone creating an implementation of your language's VM may create one that's less complete or robust than another.

This will also get interesting when you see an implementation of a language for one vendor's VM significantly outperform one on another, or also implementation-specific security issues; where a certain framework is secure on one vendor's platform, but not so much in another.

Also, security will be very platform/vendor-specific. Imagine a product where there's a security issue that spans many languages.

This whole idea sounds very good in theory, and may eventually work very well in practice, but I foresee this working well with a couple of very popular languages for one or two vendors' products and not so hot on many "me too" products.

Re:One standard, several implementations

[jlarocco]

One standard, several implementations? Sounds nice in theory, just like the numerous standards that Sun has outputted where each vendor delivers its own implementation (JPA, JDBC, J2EE among others). However, in practise you pick *one* vendor and *one* implementation and run with it. Only a fool would dare switching implementation mid-development, making the choice really just academic, because there are always minor differences that "shouldn't" matter, but does.

That's true, but there are still several reasons why it keeps the vendors on their toes more than a technology with a single vendor, like Microsoft's crap.

It may be a pain to switch implementations, but it's still easier than a complete rewrite using something else. Switching C++ compilers is much easier than switching to a different language. Swapping J2EE implementations is a pain, but still easier than rewriting using Ruby on Rails. So the vendor may be able to make your life difficult, but not too difficult because they know you can switch.

Secondly, multiple vendors supplying several different, but largely similar products makes word of mouth and customer relations much more important. If Vendor A and Vendor B both have products implementing standards C and D, their reputations become much more important in deciding which product to use. There's more incentive to keep the customer happy. That's a good thing.

So no, one standard with multiple implementations won't solve every problem, but it makes the general situation better.

Sort of like generic database access layers?

[sphealey]

Reminds me of architects and developers who create generic database access engines so their product can be "platform independent" and then wonder why its performance is so bad no matter which of the six major databases is used.

sPh

And...

[Colin Smith]

Software development recursively disappears up it's own arse.

We already have different, generic, virtual machines. They are called operating systems. They run on bits of silicon and steel.

You can't fix the problems you have writing software by running away from them

Re:And...

[TheRaven64]

I totally agree. The summary explained exactly how my code works already. I write C, Smalltalk, Objective-C and C++ (if I really can't avoid it) code. I then use a magical tool called a 'compiler' which turns it in to code for a language-agnostic virtual machine called 'the {x86,SPARC,PowerPC,ARM} instruction set' which then runs it. The important part is not the VM, it's the libraries. With my Smalltalk compiler I can add methods to objects written in Objective-C, subclass classes written in either language with the other. I can write high-level application logic in Smalltalk, mid-level code in Objective-C, and really performance-critical stuff in inline assembly in some C functions called from Objective-C methods. I can access a wealth of libraries written in C, C++, or Objective-C.

Actually, I do use a virtual machine, since my Smalltalk compiler is built on top of LLVM, but this VM is similar to an idealised form of a real CPU, and fairly language agnostic. Currently, I only use it for optimisation and statically emitting native code, but I could use it for run-time profiling and dynamic optimisations too.

Oh, and real men write their own compilers.

Re:And...

[Dolda2000]

I could not agree more, and none to my surprise, TFA was full of inflated fluff and very little substance. It was hard enough to wade through it even to find anything substantial at all, but let me highlight some of the things that can be found:

In fact, many developers would rather be freed from the hassles imposed by traditional systems programming languages. VM-based languages offer such features as automatic garbage collection, runtime bytecode verification, and security sandboxes -- all of which translate into peace of mind.

Of course garbage collection has been a feature of LISP since its inception, which has been compilable to machine code since... the 60s? Not to mention the garbage collection libraries available for C and other languages. I'd care to call that point bogus.

Likewise, runtime bytecode verification isn't necessary with a hardware CPU. It's just made to ensure that a JVM doesn't encounter any illegal instructions or jump to code outside the current protection domain. Hardware CPUs can do illegal instruction checking in parellel with execution without penalties, and virtual memory makes the jump checks pointless as well. Not to mention that it is less restricted, so that one can implement such things as tail-call optimization or continuations without reimplementing the CPU.

Oh, and of course, operating systems have had security sandboxes called "processes" since... the 60s? Of course, one could well argue that it would be swell to be able to further control a process' privileges to a degree not available on, say, Linux or NT, but that isn't exactly something that requires a VM.

Dynamic languages, on the other hand, mean efficient coding; their high-level syntax makes it easy to conceptualize applications and build prototypes rapidly.

Yeah. But as Lisp, Psycho and countless others have demonstrated, they don't need a VM to run efficiently.

The great advantage of a generic VM, as opposed to a language-specific one, is flexibility.

Of course, exactly what a "generic" VM entails does not seem to be entirely clear to the author. Or at least, I can't find anything about it in TFA.

Re:Wait, this sounds familliar!

[Psychotria]

Well I have to agree (mostly). What on Earth is "hand-coded" C? And why is it better than... wait... what other kind of C is there?

The point?

[orclevegam]

Am I the only one that sees this as completely ass backwards? I mean, part of the lure of scripting languages is that we skip that whole compile phase of things, and so achieve a certain degree of platform independence. So long as the system being targeted has a implementation of the scripting languages interpreter, you just run the script inside of it, and you can distribute the same script (more or less) for any system with an interpreter. Now they're talking about essentially compiling a scripting language to one of several different byte codes to target one of several different VMs, which then of course need implementations on whatever systems you're targeting. How is this an improvement over the previous way of doing things?

What exactly are we getting out of this? The language developers don't have to worry about the details of the underlying machine, but as a trade off they now need to write implementations for whatever VM is out there, which is turn will require them to worry about the details of the underlying machine, so we've just pushed that pain point down one level of abstraction, but not eliminated it. The only up side I can see to the entire thing is language interoperability which is nice and all, but how does that fit in with the multiple-VM approach being touted here? Each language is most likely going to require some minor changes in order to support interoperability at the VM level, and of course there will be quirks and gotchas on each VM as well. Unless all the VM developers get together and agree on the exact changes that will be required to each language we could end up with a situation in which each language will come in multiple slightly different syntaxes depending on exactly which VM it targets.

VM?! Why are you drinking that kool-aid?

[the_skywise]

Microsoft promised this with .NET. (Just buy our tools and you build to .NET and run on all Windows platforms, XP SP1, XP SP2 AND Vista! It's sooo much better than that... Java thing.)

Microsoft promised us this with Windows CE. (Just buy our tools and with a simple compiler switch, voila, you're targetting CE... it couldn't be easier.)

Microsoft couldn't even do it with DirectX where OpenGL could (Oh hey, that XBox directX.. it works a little differently than Windows DirectX)

For that matter, the Windows Printer driver APIs aren't consistent (Yeah, we know it's called GetMarginSpaceFromEdge but driver A measures the edge from half an inch in and driver b measures the edge from the print head detects the edge of the page which is sometimes an inch greater than the page itself...)

Y'know what the greatest VM is right now? i386! And has been for nigh-on 10 years!

I LIKE Microsoft product, don't get me wrong... but I'm not going to buy Visual Studio 2011 which has no other changes than a GUI enhancement and the ability to target my development towards the hot new sweetness.DNET API's so 3 years later, Microsoft can abandon .DNET for DCOM# because, hey, thats what our research said people wanted and it'll be supported on Windows 7.1.1 along with Blackbird 2.0

The problem is type conversion between languages

[MarkWatson]

There is a JSR to address this on the JVM but I am not convinced that interop between languages on a single VM will be transparent. I mix Java libraries with JRuby and I often end up writing thin facade classes to make interop better.

It's not the syntax, stupid

[jonaskoelker]

generic VMs divorced from the syntactic details of specific languages

The syntax of programming languages is something understood by the front-end of a compiler. It then translates the code into code that does the same thing in the back-end language (such as JVM/PyVM/x86/LLVM bytecode). Neither back-end knows about the syntax of the front-end language.

The real challenge is to adopt conventions on the back-end VM that allows different languages to talk together. It'd be straightforward to implement an x86 emulator on top of the JVM and run the ${language} VM on that x86. Wow, you now have ${language} running on the JVM. So? You can't talk to the Java library that way.

If you want languages to talk together, they need to agree on data representation formats and calling conventions. Try getting object.field if you don't know where field is relative to the base address of object. Try calling object.method() if you don't know the format (or location) of object.__vtbl.

Also, the semantics of some operations have to be considered if a language has to deal with a foreign object model. Let's say we target the Java VM. How do you implement multiple inheritance? What does .super do on a class with multiple parents? How do you implement "Object *p = malloc(...); *p = my_object;"? How do you implement C++'s delete? How do you implement python's generators?

To support a set of languages, the VM must support the union of features. To make the languages talk together smoothly, the VM must support each feature in a reasonably straightforward way. The two demands pull the VM in opposite directions.

I don't want to just poo-poo this idea, but my experience with dealing with the Java VM (I've written a java-important-subset compiler in my compiler course) is that it's tightly coupled to the Java way of doing things. My experience with different languages (C, C++, Java, python, perl, ruby, haskell, scheme) says that things are different enough that you can transfer most of what you know from one language to another [at least for the oo/procedural], but that the devil is in the details, and the VM has to handle all values of $details.

like the UCSD P-system?

[YesIAmAScript]

The future is the 70s?

Didn't UCSD do this in the 70s w/ pcode?

[WillAdams]

If memory serves, all of their compilers compiled to a genericized ``pcode'' for which multiple engines existed (one per processor architecture I believe it was) --- all that was missing was multiple implementations per architecture.

William