JIT vs AOT Compilation

jg21 writes "This article on "Penguin-Driven" JVMs takes a look the performance of Java GUI applications based on the JFC/Swing API, and contends that the JIT-powered JVMs can't match a JVM with an ahead-of-time compiler ported to the Linux/x86 platform. With AOT compilation, says the CTO who has written this piece, real-world Swing applications performed perceivably faster. One is left wondering, will we now see the 'microbenchmark war' carried into the Linux camp?"

Comments

What about dynamic compliation?

[Muda69]

IMO, this is the way to go. Dynamic compilation is a mix of interpretation and translation. It defers compilation for a body of code, until it believes that it will have an impact upon the runtime of the program. Dynamic compilation has the same benefits as JIT compilation, but mitigates the compilation times and pauses by limiting the amount of code it actually compiles. Additionally, dynamic compilation can take advantage of runtime characteristics of the program itself, allowing it to perform optimizations like monomorphic inlining. (Although it wouldn't be fair to elide the fact that AOT compilers could potentially make use of feedback/runtime profiling to achieve some of the same characteristics).

Re:What about dynamic compliation?

[mmusson]

One of the points in the article is that the JIT compilers did not do well with Swing code because it did not have hot spots. Any type of after the fact compile is not going to perform well if it can't identify the parts of the code that need to be compiled.

ibm's daisy

[ophix]

doesnt ibm have a project named daisy that is a JIT vm running on top of the very machine it emulates?

i seem to remember reading that they were able to achieve up to 25% performance increase by doing so (by taking advantage of run time profiling of the executable)

i might be mistaken though

Introspective behaviour

[12357bd]

Why not spend our ever increasing computing power to create 'introspective' programs? I mean programs that dinamically examines and changes program behaviour, looking for 'better' execution modes.

VM's are a perfect environment for that kind of programming, ie, why not feature a JVM that dinamically adjust/perfect bytecode execution methods on a program by program basis?.
The article spots something old, optimizing is not a one size fits all matter, what is good for a given case is bad for another.

As our procressing power increases, we can achieve that 'programming about programming' indirection level.

Re:Introspective behaviour

[12357bd]

We need to go further, examine local execution instances, ie. I dayly run a given program (say Jedit), but I am only using/executing 50% of the code, and usually on a strongly patterned way. What about a JVM that detects that kind of behaviours and adapt itself ?.

That's why I am talking about 'introspective' programming, programs that continously examine how other/selfs programs are run and extract/apply usefull information.

Re:Introspective behaviour

[p3d0]

Think about usage patterns, maybe I am wrong but no actual VM execution model adapts program execution at running/history characteristics.

Yep, I think you're wrong. Look up "value profiling" for instance. It means finding values that are effectively constant on a particular run of a program, and recompiling that program on the assumption that the value will indeed be constant.

For instance, imagine the UNIX "wc" (word count) program were written in Java. An advanced JIT compiler could tell, for instance, that you have specified the "-l" option, and therefore you only need to count lines. The portions of the program that count letters and words would get removed as dead code, which probably speeds up the inner loop a great deal.

Moden JIT compilers produce self-profiling code, and then re-compile with the profiling information to produce a specialized version of the code that will run faster under the current conditions.

If that's not the "introspection" you're talking about, then perhaps I have misunderstood. Can you give an example?

Re:Introspective behaviour

[arkanes]

I think what he's saying is caching some of that information across program invocations, so that the next time you invoke "wc -l" the JVM looks at its JIT cache and pulls up (depending on whats available or practical) the profiling information it has from the last run or the actual compiled code.

This seems obvious and I wonder why it's not widespread. Maybe it is and I just don't know it? Maybe the work of figureing out whether or not your cache is stale outweighs the benefits? Taken to it's logical conclusion, this ends up turning into AOT compilation, of course, so there's some point where the disadvantages outweigh the advantages...

Re:Introspective behaviour

[p3d0]

As you say, you end up with a slippery slope that leads to AOT compilation, which would be fine, except that it makes you ask the question, why are we jitting in the first place? At that point, whatever answers you come up with often work against keeping persistent data between runs of the same application.

However, it's certainly not impossible in principle, and I'm sure there are profitable points on the JIT-versus-AOT spectrum that have yet to be explored.

Why Not Both?

[RAMMS EIN]

Why compare JIT against AOT? Why not have both?

AOT compilation makes for fast start up time and fast run time. JIT advocates claim that it can lead to better performance, as more optimizations can be performed with run-time information. So why not combine the two? Compile it before the first run, and further optimize it at run-time where appropriate. That way, you get the best of both worlds.

Re:Why Not Both?

[breadbot]

That's an interesting suggestion. I think it would be tough to pull off because, in order to detect hot spots, you need performance metrics. Compiled code that generates metrics on itself (for example, running with embedded profiling tools) is usually very slow. To take advantage of pre-compiling, you'd basically need a profiler with as little speed penalty as possible.

Also, it may turn out to be easier to recompile based on the original bytecode rather than the compiler machine code. Which would mean you'd have to ship both versions with every binary, unless the machine code was JIT'd at runtime.

On the other hand, if you had a way to optimize the already-compiled code, then heck, you'd have a general-purpose application accelerator. I think I remember reading, a few years ago, about research group at HP that started out working on aggressive processor emulation techniques and ended up with an optimizing recompiler. They were surprised to found that, when they targeted the same processor that the original binary came from, their application sped up 10-20% thanks to optimizations they could apply that the compiler couldn't have known about (cross-library inlining, for example).

Re:Why Not Both?

[OmniVector]

this is what i have been clamoring for for years. why don't we just cache JIT compilation information somewhere, and upon launching an application, check a hash of the system resources (memory, cpu, other information that aids to JIT compilation performance) and if the hash hasn't changed, just use the pre-JIT compiled binary information? and even if it DID change and the platform is still the same, why not keep the pre-JIT compiled information and redo the JIT compilation in the background?

something must be incredibly wrong with this idea, because it's such a stupid, simple, solution to the problems with JIT performance that clearly there's a reason Sun hasn't done it yet. but what do i know.. i'm just a 4th year CS undergrad.

Re:Why Not Both?

[Anonymous Coward]

You might want to look at LLVM. One of the ideas behind the project is to utilize information from the entire lifetime of the program to perform optimizations.

1998 called

[p3d0]

They want their idea back.

Let me clarify

[p3d0]

If you can show me a commercial JIT compiler that doesn't already do this, I'll eat my hat. What you call "dynamic compilation" is so routine and mundane that when people talk about "JIT compilers" these days, that is exactly what they are talking about. Nobody blindly compiles on the first invocation of a method any more.

What you have mentioned is not the crux of the problem. I can't say much more because I do confidential work on IBM's JIT compiler.

Re:Let me clarify

[p3d0]

Yes, I have actually done this myself for x86 with my pet language's interpreter/compiler, so I'm not just talking out my arse.

Oh, ok. For a moment there, I thought I was a professional JIT developer, but apparently I was mistaken.

Re:What's the point in Java bytecode anyway?

[p3d0]

It is parse trees. It's stack-based, which is pretty much just a post-order traversal of expression trees. Think of bytecode as a file format for describing expression trees.

Augh! Are they TRYING to drive me insane?

[Sentry21]

When will the TLA madness end? Oh, the humanity! At least provide definitions, for those god-fearing folk who may be interested but not up-to-date.

JVM: Java Virtual Machine, the virtual environment that every Java bytecode program runs within, abstracting real hardware for the program in question.

GUI: Graphical User Interface

JFC: Java Foundation Classes - the basic classes that are provided to developers upon which, or rather, with which, to build their programs.

API: Application Programming Interface, a defined way for software to interface with other software (i.e. to make library calls)

JIT: Just-In-Time compilation, compiles the program when it is being launched, for the machine it is being launched on, in order to prevent poor performance by compiling every instruction whenever it needs to be done

AOT: Mentioned in the article text, it means Ahead of Time. For details, read the linked story.

CTO: Chief Technology Officer. Name given to an executive in charge of new or current technology.

Now that you know what is going on, RTFA.

--Dan