Generic VMs Key To Future of Coding

snydeq writes "Fatal Exception's Neil McAllister calls for generic VMs divorced from the syntactic details of specific languages in order to provide developers with some much-needed flexibility in the years ahead: 'Imagine being able to program in the language of your choice and then choose from any of several different underlying engines to execute your code, depending upon the needs of your application.' This 'next major stage in the evolution of programming' is already under way, he writes, citing Jim Hugunin's work with Python on the CLR, Microsoft's forthcoming Dynamic Language Runtime, Jython, Sun's Da Vinci Machine, and the long-delayed Perl/Python Parrot. And with modern JITs capable of outputting machine code almost as efficient as hand-coded C, the idea of running code through a truly generic VM may be yet another key factor that will shape the future of scripting."

Comments

Will we ever see Parrot?

[CRCulver]

I remember some years ago the elation people felt when Parrot was announced. At last, we could leverage the strengths of either Python or Perl--or whatever other interpreted languages--but work with a common interpreter. But then the hype started to die down, and the last edition of O'Reilly's book on the subject appeared over four years ago. Within the Python community, interest in Parrot seems completely dead. Are the Perl folks going it alone, and when might we see the project reach a successful deployment?

Re:Will we ever see Parrot?

[chromatic]

Patrick Michaud wrote a bare-bones Python implementation in eight hours. It doesn't support all of Python, but it supports a large amount -- and, to my knowledge, he'd never implemented a Python compiler or interpreter before. That project, Pynie, has languished for a while, as he's spending more time working on Rakudo (the Perl 6 implementation on Parrot), but it's a viable port just waiting for someone to work on it. Lua is functionally complete as of 5.1 (I believe), and Tcl, PHP, and Ruby are in progress.

You can play with the latest versions of all of these languages on Tuesday, 21 October, when we make our next monthly stable release (though partcl just moved to a separate repository, so you can check out the current version there on a different schedule).

Re:Will we ever see Parrot?

[chromatic]

Note that x is read-only, and y is read-write. I assume that if you don't put rw after an attribute, it's read-only. Otherwise, there's not much point having rw.

That's true, but note that the rw attribute only applies to the accessor method. A12 says:

The traits of the generated method correspond directly to the traits on the variable.

Further, it says:

In any event, even without "is rw" the attribute variable is always writable within the class itself (unless you apply the trait is constant to it).

The idea's a little tricky at first, but those types of declarations (and the public/private attribute distinction you get with the use of twigils or has versus my govern the visibility and type of access only outside the class.)

One standard, several implementations

[Anonymous Coward]

One standard, several implementations? Sounds nice in theory, just like the numerous standards that Sun has outputted where each vendor delivers its own implementation (JPA, JDBC, J2EE among others). However, in practise you pick *one* vendor and *one* implementation and run with it. Only a fool would dare switching implementation mid-development, making the choice really just academic, because there are always minor differences that "shouldn't" matter, but does.

Sort of like generic database access layers?

[sphealey]

Reminds me of architects and developers who create generic database access engines so their product can be "platform independent" and then wonder why its performance is so bad no matter which of the six major databases is used.

sPh

Re:Sort of like generic database access layers?

[isny]

It's VMs all the way down! Or is it turtles...

Re:Sort of like generic database access layers?

[gazbo]

Gah! Database independent, yet almost invariably used on exactly one RDBMS. And you just know some of the more obscure query syntax makes the application on top of it database dependent anyway.

Yeah, it annoys the tits off me also.

And...

[Colin Smith]

Software development recursively disappears up it's own arse.

We already have different, generic, virtual machines. They are called operating systems. They run on bits of silicon and steel.

You can't fix the problems you have writing software by running away from them

Re:And...

[TheRaven64]

I totally agree. The summary explained exactly how my code works already. I write C, Smalltalk, Objective-C and C++ (if I really can't avoid it) code. I then use a magical tool called a 'compiler' which turns it in to code for a language-agnostic virtual machine called 'the {x86,SPARC,PowerPC,ARM} instruction set' which then runs it. The important part is not the VM, it's the libraries. With my Smalltalk compiler I can add methods to objects written in Objective-C, subclass classes written in either language with the other. I can write high-level application logic in Smalltalk, mid-level code in Objective-C, and really performance-critical stuff in inline assembly in some C functions called from Objective-C methods. I can access a wealth of libraries written in C, C++, or Objective-C.

Actually, I do use a virtual machine, since my Smalltalk compiler is built on top of LLVM, but this VM is similar to an idealised form of a real CPU, and fairly language agnostic. Currently, I only use it for optimisation and statically emitting native code, but I could use it for run-time profiling and dynamic optimisations too.

Oh, and real men write their own compilers.

Re:And...

[Dolda2000]

I could not agree more, and none to my surprise, TFA was full of inflated fluff and very little substance. It was hard enough to wade through it even to find anything substantial at all, but let me highlight some of the things that can be found:

In fact, many developers would rather be freed from the hassles imposed by traditional systems programming languages. VM-based languages offer such features as automatic garbage collection, runtime bytecode verification, and security sandboxes -- all of which translate into peace of mind.

Of course garbage collection has been a feature of LISP since its inception, which has been compilable to machine code since... the 60s? Not to mention the garbage collection libraries available for C and other languages. I'd care to call that point bogus.

Likewise, runtime bytecode verification isn't necessary with a hardware CPU. It's just made to ensure that a JVM doesn't encounter any illegal instructions or jump to code outside the current protection domain. Hardware CPUs can do illegal instruction checking in parellel with execution without penalties, and virtual memory makes the jump checks pointless as well. Not to mention that it is less restricted, so that one can implement such things as tail-call optimization or continuations without reimplementing the CPU.

Oh, and of course, operating systems have had security sandboxes called "processes" since... the 60s? Of course, one could well argue that it would be swell to be able to further control a process' privileges to a degree not available on, say, Linux or NT, but that isn't exactly something that requires a VM.

Dynamic languages, on the other hand, mean efficient coding; their high-level syntax makes it easy to conceptualize applications and build prototypes rapidly.

Yeah. But as Lisp, Psycho and countless others have demonstrated, they don't need a VM to run efficiently.

The great advantage of a generic VM, as opposed to a language-specific one, is flexibility.

Of course, exactly what a "generic" VM entails does not seem to be entirely clear to the author. Or at least, I can't find anything about it in TFA.

LLVM plug

[Anonymous Coward]

article didn't include it, but this open source project seems to have similar goals

http://llvm.org/

Wait, this sounds familliar!

[neokushan]

Sure this sounds quite a bit like something Microsoft, of all people, tried to create? That's right, I'm talking about .Net! Microsoft loved touting how you could develop .Net applications in C#, C++ or even good ol' VB and it should all work the same and even interoperate.
But it's .Net and I'm sure anyone with any experience knows that despite the supposed advantages, it has quite a few disadvantages as well. But at least it made VB somewhat useful again.

None the less, I wouldn't hold my breath on this one, sounds like a pipe dream to me and I'm sure some would argue - what's the point in running your code through a VM if you can just run it natively?

On a side note: As efficient as hand-coded C? In my experience, 90% of the time someone tries to write "efficient" C, they end up causing more problems than it's worth (early-optimisation and all that). Perhaps it should be reworded to say something like Hand-crafted C from a C Master".

Re:Wait, this sounds familliar!

[_jameshales]

There are some programming language implementations that "compile to C".

The future of Python is PyPy

[YA_Python_dev]

Within the Python community, interest in Parrot seems completely dead.

Generic VMs are so 2005, the future of Python runtime is PyPy. From a single implementation of Python (written in Python), they can compile Python code to C, JVM, automatically create a customizable JITed VM, etc...

Check them out: they are doing some seriously cool stuff and they can use a bit of help.

The point?

[orclevegam]

Am I the only one that sees this as completely ass backwards? I mean, part of the lure of scripting languages is that we skip that whole compile phase of things, and so achieve a certain degree of platform independence. So long as the system being targeted has a implementation of the scripting languages interpreter, you just run the script inside of it, and you can distribute the same script (more or less) for any system with an interpreter. Now they're talking about essentially compiling a scripting language to one of several different byte codes to target one of several different VMs, which then of course need implementations on whatever systems you're targeting. How is this an improvement over the previous way of doing things?

What exactly are we getting out of this? The language developers don't have to worry about the details of the underlying machine, but as a trade off they now need to write implementations for whatever VM is out there, which is turn will require them to worry about the details of the underlying machine, so we've just pushed that pain point down one level of abstraction, but not eliminated it. The only up side I can see to the entire thing is language interoperability which is nice and all, but how does that fit in with the multiple-VM approach being touted here? Each language is most likely going to require some minor changes in order to support interoperability at the VM level, and of course there will be quirks and gotchas on each VM as well. Unless all the VM developers get together and agree on the exact changes that will be required to each language we could end up with a situation in which each language will come in multiple slightly different syntaxes depending on exactly which VM it targets.

Colas: Coke, Pepsi and Jolt point a way forward

[itsybitsy]

Ian Piumarta and the VPRI [http://vpri.org] are doing some amazing work related to this story.

COLAs: Combined Object Lambda Architectures - A Complete System in 20,000 Lines of Code.
http://piumarta.com/software/cola
The system is slowly evolving towards version 1.0 which
        * is completely self-describing (from the metal, or even FPGA gates, up) exposing all aspects of its implementation for inspection and incremental modification;
        * treats state and behaviour as orthogonal but mutually-completing descriptions of computation;
        * treats static and dynamic compilation as two extremes of a continuum;
        * treats static and dynamic typing as two extremes of a continuum; and
        * late-binds absolutely everything: programming (parsing through codegen to runtime and ABI), applications (libraries, communications facilities), interaction (graphics frameworks, rendering algorithms), and so on.

http://piumarta.com/papers/colas-whitepaper.pdf
http://piumarta.com/papers/EE380-2007-slides.pdf
http://piumarta.com/pepsi/objmodel.pdf
http://www.vpri.org/html/work/NSFproposal.pdf

Allen Wirfs-Brock and Dan Ingalls are currently working on bringing notions like Colas to the browser so that we can use any programming language WE choose to for our browser based applications. Check out their interview here.
http://channel9.msdn.com/posts/Charles/Dan-Ingalls-and-Allen-Wirfs-Brock-On-Smalltalk-Lively-Kernel-Javascript-and-Programming-the-Inter/

VM?! Why are you drinking that kool-aid?

[the_skywise]

Microsoft promised this with .NET. (Just buy our tools and you build to .NET and run on all Windows platforms, XP SP1, XP SP2 AND Vista! It's sooo much better than that... Java thing.)

Microsoft promised us this with Windows CE. (Just buy our tools and with a simple compiler switch, voila, you're targetting CE... it couldn't be easier.)

Microsoft couldn't even do it with DirectX where OpenGL could (Oh hey, that XBox directX.. it works a little differently than Windows DirectX)

For that matter, the Windows Printer driver APIs aren't consistent (Yeah, we know it's called GetMarginSpaceFromEdge but driver A measures the edge from half an inch in and driver b measures the edge from the print head detects the edge of the page which is sometimes an inch greater than the page itself...)

Y'know what the greatest VM is right now? i386! And has been for nigh-on 10 years!

I LIKE Microsoft product, don't get me wrong... but I'm not going to buy Visual Studio 2011 which has no other changes than a GUI enhancement and the ability to target my development towards the hot new sweetness.DNET API's so 3 years later, Microsoft can abandon .DNET for DCOM# because, hey, thats what our research said people wanted and it'll be supported on Windows 7.1.1 along with Blackbird 2.0

Plus ca change....

[bfwebster]

My first thought on reading this was an old software engineering maxim, usually (and probably correctly) attributed to Don Knuth:

There is no complexity problem in programming that cannot be eased by adding a layer of indirection. And there is no performance problem in programming that cannot be eased by removing a layer of indirection.

Universal VMs are old as the hills (anyone [else] here old enough to have programmed on the UCSD p-System?). We shift towards VMs to gain independence and portability, and then we shift back to direct, spot or JIT compilation to improve performance. It's an old, old dance, and one that will likely go on for years to come. ..bruce..

The problem is type conversion between languages

[MarkWatson]

There is a JSR to address this on the JVM but I am not convinced that interop between languages on a single VM will be transparent. I mix Java libraries with JRuby and I often end up writing thin facade classes to make interop better.

Bollocks

[Colin Smith]

Oh, and real men write their own compilers.

Real men code in P".

 

It's not the syntax, stupid

[jonaskoelker]

generic VMs divorced from the syntactic details of specific languages

The syntax of programming languages is something understood by the front-end of a compiler. It then translates the code into code that does the same thing in the back-end language (such as JVM/PyVM/x86/LLVM bytecode). Neither back-end knows about the syntax of the front-end language.

The real challenge is to adopt conventions on the back-end VM that allows different languages to talk together. It'd be straightforward to implement an x86 emulator on top of the JVM and run the ${language} VM on that x86. Wow, you now have ${language} running on the JVM. So? You can't talk to the Java library that way.

If you want languages to talk together, they need to agree on data representation formats and calling conventions. Try getting object.field if you don't know where field is relative to the base address of object. Try calling object.method() if you don't know the format (or location) of object.__vtbl.

Also, the semantics of some operations have to be considered if a language has to deal with a foreign object model. Let's say we target the Java VM. How do you implement multiple inheritance? What does .super do on a class with multiple parents? How do you implement "Object *p = malloc(...); *p = my_object;"? How do you implement C++'s delete? How do you implement python's generators?

To support a set of languages, the VM must support the union of features. To make the languages talk together smoothly, the VM must support each feature in a reasonably straightforward way. The two demands pull the VM in opposite directions.

I don't want to just poo-poo this idea, but my experience with dealing with the Java VM (I've written a java-important-subset compiler in my compiler course) is that it's tightly coupled to the Java way of doing things. My experience with different languages (C, C++, Java, python, perl, ruby, haskell, scheme) says that things are different enough that you can transfer most of what you know from one language to another [at least for the oo/procedural], but that the devil is in the details, and the VM has to handle all values of $details.

Javascript - as a VM intermediate language(!)

[lkcl]

no don't laugh, it works very well! there are a number of very good reasons for this.

1) javascript is actually an incredibly powerful language, in particular due to the concept of "prototype"ing.

2) javascript, thanks to web browsers, has an unbelievably large amount of attention spent on it, to optimise the stuffing out of it. as a result, the latest incarnation to hit the streets - the V8 engine - actually compiles to i386 or ARM assembler.

3) the number of "-to-javascript" compilers is really quite staggering. see the comments from pyv8 article for an incomplete list.

GWT has a java-to-javascript compiler; Pyjamas has a python-to-javascript compiler. There's a ruby-to-javascript compiler - the list just goes on.

then there's the pypy compiler collection, which has javascript as a back-end. (and, for completeness, it's worth mentioning that it also has a CLR backend, LLVM.org backend, and a java backend).

like the UCSD P-system?

[YesIAmAScript]

The future is the 70s?