JIT vs AOT Compilation

jg21 writes "This article on "Penguin-Driven" JVMs takes a look the performance of Java GUI applications based on the JFC/Swing API, and contends that the JIT-powered JVMs can't match a JVM with an ahead-of-time compiler ported to the Linux/x86 platform. With AOT compilation, says the CTO who has written this piece, real-world Swing applications performed perceivably faster. One is left wondering, will we now see the 'microbenchmark war' carried into the Linux camp?"

Comments

What about dynamic compliation?

[Muda69]

IMO, this is the way to go. Dynamic compilation is a mix of interpretation and translation. It defers compilation for a body of code, until it believes that it will have an impact upon the runtime of the program. Dynamic compilation has the same benefits as JIT compilation, but mitigates the compilation times and pauses by limiting the amount of code it actually compiles. Additionally, dynamic compilation can take advantage of runtime characteristics of the program itself, allowing it to perform optimizations like monomorphic inlining. (Although it wouldn't be fair to elide the fact that AOT compilers could potentially make use of feedback/runtime profiling to achieve some of the same characteristics).

Re:What about dynamic compliation?

[mmusson]

One of the points in the article is that the JIT compilers did not do well with Swing code because it did not have hot spots. Any type of after the fact compile is not going to perform well if it can't identify the parts of the code that need to be compiled.

Re:What about dynamic compliation?

[Anonymous Coward]

So far, all dynamic compilation advocates have managed to demonstrate is programs that - given a sufficiently long runtime, and we're talking weeks not minutes, are able through the use of dynamic optimisation to match the speed of traditionally compiled software.

That's quite impressive, considering that programs executing in virtual machines are generally at least an order of magnitude slower than machine code. But it doesn't make it a superior option, I'm afraid.

Not yet, anyway...

Re:What about dynamic compliation?

[Anarioch]

This is exactly what Suns client VM does - it'll only compile a block of code if it is executed multiple times; if you use the server VM it'll precompile all code though.

Re:What about dynamic compliation?

[jovlinger]

I'm suprised that JREs don't ship with class libraries precompiled into native libraries.

I'm curious how much performance could be gained from a highly optimized precompiled class library.

Re:What about dynamic compliation?

[mmusson]

The Java Advanced Imaging API which is part of the class library ships with a Java and some native implementations. This is done purely for performance reasons because the pure Java version does not perform well.

In fairness to Java, I think this is because that library is not written well. My company made its own imaging routines that cover part of its functionailty and they are substantially faster.

ibm's daisy

[ophix]

doesnt ibm have a project named daisy that is a JIT vm running on top of the very machine it emulates?

i seem to remember reading that they were able to achieve up to 25% performance increase by doing so (by taking advantage of run time profiling of the executable)

i might be mistaken though

Re:ibm's daisy

[RAMMS EIN]

``doesnt ibm have a project named daisy that is a JIT vm running on top of the very machine it emulates?''

No. Daisy is a project that emulates other architectures (x86, PowerPC, and also JVM) on top of a VLIW processor.

Re:ibm's daisy

[jovlinger]

HP's dynamo.

Introspective behaviour

[12357bd]

Why not spend our ever increasing computing power to create 'introspective' programs? I mean programs that dinamically examines and changes program behaviour, looking for 'better' execution modes.

VM's are a perfect environment for that kind of programming, ie, why not feature a JVM that dinamically adjust/perfect bytecode execution methods on a program by program basis?.
The article spots something old, optimizing is not a one size fits all matter, what is good for a given case is bad for another.

As our procressing power increases, we can achieve that 'programming about programming' indirection level.

Re:Introspective behaviour

[RAMMS EIN]

``why not feature a JVM that dinamically adjust/perfect bytecode execution methods''

The HotSpot JVM actually does this.

Re:Introspective behaviour

[12357bd]

We need to go further, examine local execution instances, ie. I dayly run a given program (say Jedit), but I am only using/executing 50% of the code, and usually on a strongly patterned way. What about a JVM that detects that kind of behaviours and adapt itself ?.

That's why I am talking about 'introspective' programming, programs that continously examine how other/selfs programs are run and extract/apply usefull information.

Re:Introspective behaviour

[p3d0]

Why not spend our ever increasing computing power to create 'introspective' programs?

We do. That's what modern JVMs do.

Re:Introspective behaviour

[12357bd]

We do. That's what modern JVMs do.

I don't think so.
The actual introspection level is low, very low, in fact systems and not only VMs should be designed with introspection in mind to have a satisfactory environment for adaptative behaviour.
Think about usage patterns, maybe I am wrong but no actual VM execution model adapts program execution at running/history characteristics. We are still simply 'running' programs, the system/vm is not designed to learn/find the best way to run those programs at each execution instance, that's why i am talking about the need/possibility of an aditional indirection level.

Re:Introspective behaviour

[arkanes]

We've got this for normal compilers already in the form of profile based compilation. JIT does it per instance. I don't know for sure that any current JIT has a way of saving that sort of information/state across invocations but seems like a really obvious extension.

Re:Introspective behaviour

[p3d0]

Think about usage patterns, maybe I am wrong but no actual VM execution model adapts program execution at running/history characteristics.

Yep, I think you're wrong. Look up "value profiling" for instance. It means finding values that are effectively constant on a particular run of a program, and recompiling that program on the assumption that the value will indeed be constant.

For instance, imagine the UNIX "wc" (word count) program were written in Java. An advanced JIT compiler could tell, for instance, that you have specified the "-l" option, and therefore you only need to count lines. The portions of the program that count letters and words would get removed as dead code, which probably speeds up the inner loop a great deal.

Moden JIT compilers produce self-profiling code, and then re-compile with the profiling information to produce a specialized version of the code that will run faster under the current conditions.

If that's not the "introspection" you're talking about, then perhaps I have misunderstood. Can you give an example?

Re:Introspective behaviour

[arkanes]

I think what he's saying is caching some of that information across program invocations, so that the next time you invoke "wc -l" the JVM looks at its JIT cache and pulls up (depending on whats available or practical) the profiling information it has from the last run or the actual compiled code.

This seems obvious and I wonder why it's not widespread. Maybe it is and I just don't know it? Maybe the work of figureing out whether or not your cache is stale outweighs the benefits? Taken to it's logical conclusion, this ends up turning into AOT compilation, of course, so there's some point where the disadvantages outweigh the advantages...

Re:Introspective behaviour

[p3d0]

As you say, you end up with a slippery slope that leads to AOT compilation, which would be fine, except that it makes you ask the question, why are we jitting in the first place? At that point, whatever answers you come up with often work against keeping persistent data between runs of the same application.

However, it's certainly not impossible in principle, and I'm sure there are profitable points on the JIT-versus-AOT spectrum that have yet to be explored.

Re:Introspective behaviour

[12357bd]

Well, the main idea is that currently a running program instance is a sum of factors, ie let's suppouse a java program; we have the bytecode, the JVM, the program parameters,and the system state.

Introspection adds three facts, a) the ability to inspect/analize/record actual or past runnings instances, b) the ability to try/apply different execution models and c) predictive behaviour to to try/use/evaluate the results.

Current optizing techniques are still 'static', we learn that is good or bad to use heuristic 'a' on situation 'b' (ie inlinning on a given cpu cache size, or your -l param example). What we are still not doing is dinamically learning/adjusting execution models (ie, if a given program is usually called to parse a big file, adjust allocation/prefetching to optimize that process).

Is not about JIT vs AOT, is 'globally static' versus 'locally dinamic' optimization. In simple words: if i am using always the same secuence of program actions, why the system is not 'smart' enough to detect-it and adjust for an optimal response? You need introspection to do that kind of behaviour.

Re:Introspective behaviour

[Lally Singh]

Yeah, it's called 'LISP'.

Re:Introspective behaviour

[Anonymous Coward]

dinamically examines and changes
dinamically adjust/perfect bytecode execution

"dynamically", "dynamically adjusts/perfects".

Re:Introspective behaviour

[Anonymous Coward]

I dayly run a given program

"daily".

usually on a strongly patterned way

"in a".

Re:Introspective behaviour

[p3d0]

if a given program is usually called to parse a big file, adjust allocation/prefetching to optimize that process

I think the OS usually does a good job with that. Do you have any reason to think that there's an opportunity here?

Re:Introspective behaviour

[cakoose]

As you say, you end up with a slippery slope that leads to AOT compilation, which would be fine, except that it makes you ask the question, why are we jitting in the first place?

Although "Just In Time" does imply that compilation is only done immediately prior to execution, that is only accurate of the early implementations. Though newer VMs still say JIT, I think JIT has come to mean "dealing with a canonical representation that is not directly executable." Caching is a transparent optimization in that the semantics of what's going on doesn't change. The VM's "CompileMethod()" function is still called and still has the same function signature. The only difference is that it might run faster (when a cached copy is available).

JIT is a generalization of the AOT workflow. In a JIT framework, you can still compile everything ahead of time and only once but you have the option of trying other things.

forget i said anything

[ophix]

next time i will RTFA before posting (when resnet here decides i can open the article's website)

after finally being able to look at the article i was mistaken as to what they meant by AOT compile (i just thought they meant native compile from source, not what they actually mean)

Why Not Both?

[RAMMS EIN]

Why compare JIT against AOT? Why not have both?

AOT compilation makes for fast start up time and fast run time. JIT advocates claim that it can lead to better performance, as more optimizations can be performed with run-time information. So why not combine the two? Compile it before the first run, and further optimize it at run-time where appropriate. That way, you get the best of both worlds.

Re:Why Not Both?

[breadbot]

That's an interesting suggestion. I think it would be tough to pull off because, in order to detect hot spots, you need performance metrics. Compiled code that generates metrics on itself (for example, running with embedded profiling tools) is usually very slow. To take advantage of pre-compiling, you'd basically need a profiler with as little speed penalty as possible.

Also, it may turn out to be easier to recompile based on the original bytecode rather than the compiler machine code. Which would mean you'd have to ship both versions with every binary, unless the machine code was JIT'd at runtime.

On the other hand, if you had a way to optimize the already-compiled code, then heck, you'd have a general-purpose application accelerator. I think I remember reading, a few years ago, about research group at HP that started out working on aggressive processor emulation techniques and ended up with an optimizing recompiler. They were surprised to found that, when they targeted the same processor that the original binary came from, their application sped up 10-20% thanks to optimizations they could apply that the compiler couldn't have known about (cross-library inlining, for example).

Re:Why Not Both?

[RAMMS EIN]

It's not as difficult as it may seem. You just ship the bytecode as for Java. Then, the receiver compiles the code before running it. You can then run a run-time optimizer similar to the one in the Java HotSpot VM alongside the program. You could even save the run-time optimized version of the code. Or leave out the run-time optimizer, if you prefer.

Native machine code may not be a lot harder to optimize than JVM bytecode (which also consists of low-level instructions). And if it is, you can always keep a copy of the JVM bytecode around.

Re:Why Not Both?

[OmniVector]

this is what i have been clamoring for for years. why don't we just cache JIT compilation information somewhere, and upon launching an application, check a hash of the system resources (memory, cpu, other information that aids to JIT compilation performance) and if the hash hasn't changed, just use the pre-JIT compiled binary information? and even if it DID change and the platform is still the same, why not keep the pre-JIT compiled information and redo the JIT compilation in the background?

something must be incredibly wrong with this idea, because it's such a stupid, simple, solution to the problems with JIT performance that clearly there's a reason Sun hasn't done it yet. but what do i know.. i'm just a 4th year CS undergrad.

Re:Why Not Both?

[Anonymous Coward]

You might want to look at LLVM. One of the ideas behind the project is to utilize information from the entire lifetime of the program to perform optimizations.

Re:Why Not Both?

[j3110]

Makes sense, and I've been asking the same question...

Another thing I would like to see is AOT compiled classes libraries that can be inlined by modifying the actual machine code. Viruses have been playing with relative addresses for some time, so why not Java?

Honestly though, I think the reason they haven't done this is because of the disk space. That's a lot of space to be using.

On a side note, I think IBM's JVM does have the class library precompiled. Too bad I can't really use it very much because I ran into threading issues on Linux with it. In all honesty, it was probably my fault for not marking an appropriate function synchronized.

Re:Why Not Both?

[Anonymous Coward]

Nah, it's really simple. You just open up the scull cavity, install a rocket engine, then sew her back up. Piece of cake.

Re:Why Not Both?

[the quick brown fox]

Microsoft provides this for .NET, in the form of the "ngen" tool. Although it isn't commonly used by developers (nor does MS really encourage them to), when you install the .NET Framework, the standard libraries are "ngen-ed".

(FWIW, my company tried ngen to speed up our app's startup time, and it made surprisingly little impact.)

1998 called

[p3d0]

They want their idea back.

Let me clarify

[p3d0]

If you can show me a commercial JIT compiler that doesn't already do this, I'll eat my hat. What you call "dynamic compilation" is so routine and mundane that when people talk about "JIT compilers" these days, that is exactly what they are talking about. Nobody blindly compiles on the first invocation of a method any more.

What you have mentioned is not the crux of the problem. I can't say much more because I do confidential work on IBM's JIT compiler.

Re:Let me clarify

[Anonymous Coward]

Nobody blindly compiles on the first invocation of a method any more.

Uh huh. Keep in mind that 'blindly' compiling from bytecode to machine code gives a nice speedup (typically *at least* 2x, even if you only call functions instead of inlining adds, etc). The extra fraction of a millisecond per function required for compilation is typically dwarfed by the time it takes to read the bytecode or program text from disk. And if your interpreter bothers to verify the bytecode before executing it (most do these days), then it might as well also compile (verification and compilation can be done in a single quick pass).

Yes, I have actually done this myself for x86 with my pet language's interpreter/compiler, so I'm not just talking out my arse.

Re:Let me clarify

[p3d0]

Yes, I have actually done this myself for x86 with my pet language's interpreter/compiler, so I'm not just talking out my arse.

Oh, ok. For a moment there, I thought I was a professional JIT developer, but apparently I was mistaken.

What's the point in Java bytecode anyway?

[RAMMS EIN]

Regardless of whether you use AOT or JIT compilation, if you're compiling anyway, what's the point in Java bytecode? I mean not why it exists, but why is it like machine code? Wouldn't it be more compact, quicker to interpret, and easier to optimize if it were higher level (say, like parse trees detailing the method invocations)?

Re:What's the point in Java bytecode anyway?

[p3d0]

It is parse trees. It's stack-based, which is pretty much just a post-order traversal of expression trees. Think of bytecode as a file format for describing expression trees.

Re:NMA

[FooAtWFU]

It does. Can someone tell me what AOT is so that I don't need to RTFA?

Quick article summary!

[Anonymous Coward]

"Please buy our product."

Re:NMA

[Gogo Dodo]

AOT is an Ahead-of-Time compiler. It compiles Java to native code, instead of Java bytecode.

Augh! Are they TRYING to drive me insane?

[Sentry21]

When will the TLA madness end? Oh, the humanity! At least provide definitions, for those god-fearing folk who may be interested but not up-to-date.

JVM: Java Virtual Machine, the virtual environment that every Java bytecode program runs within, abstracting real hardware for the program in question.

GUI: Graphical User Interface

JFC: Java Foundation Classes - the basic classes that are provided to developers upon which, or rather, with which, to build their programs.

API: Application Programming Interface, a defined way for software to interface with other software (i.e. to make library calls)

JIT: Just-In-Time compilation, compiles the program when it is being launched, for the machine it is being launched on, in order to prevent poor performance by compiling every instruction whenever it needs to be done

AOT: Mentioned in the article text, it means Ahead of Time. For details, read the linked story.

CTO: Chief Technology Officer. Name given to an executive in charge of new or current technology.

Now that you know what is going on, RTFA.

--Dan

Re:Augh! Are they TRYING to drive me insane?

[Eric Giguere]

You think that's bad, try keeping everything straight with J2ME (Java 2 Micro Edition) programming. I put the most common ones up on my J2ME section:

J2ME acronym list

Eric
BlackBerry programming stuff

Re:Augh! Are they TRYING to drive me insane?

[SpookyFish]

Ok, it isn't a *T*LA, but my favorite is still:
PCMCIA-
People Can't Memorize Computer Industry Acronyms

Re:Augh! Are they TRYING to drive me insane?

[Anonymous Coward]

Not even in a high-scored reply, and already slashdoted...

Poor server.

Re: Welcome to unemployment!

[Anonymous Coward]

I'm sorry you got fired for posting on slashdot. :(

Look on the bright side: now you've got time to write a design your own JIT from the ground up instead of blindly following someone else's design. :)

Re:NMA

[Anonymous Coward]

You don't need to RTFA; just read the blurb again: "an ahead-of-time compiler".

The article is slightly mood

[angel'o'sphere]

I don't like the article.

For several reasons I think they cheated and they coin old terms into new terms for no reason (except publicity). E.g. they mix up AOT with static compilation.

AOT is an extension of JIT compilation, and not a compilation done ahead of the first execution.

A VM based on a JIT compiler loads always only byte code and compiles the byte-code to native code on the fly depending on several algorithms/heuristics.

A AOT compiler does JUST THE SAME on the first invocation of the byte code program, but it saves the generated native code to be available for later invocations of the same program. So for the later invocations, the compilation was ahead of time, not for the first one.

The Excelsior JET is no AOT compiler but just a static compiler. However they seem to jump on the bandwagon of a new buzzword, because the term AOT is only used in academic circles currently (besides that Java 5.0 tries to cash compile informations and even native code over invocations).

The next flaw is this: they measure a GUI application over only 600 "loops". As the exact form of the loop is not published, I have the impression they made the loop in a way that a "standard" HotSpot JIT does predictable not even attempt to compile much of the covered code.

To understand that you have to know how a HotSpot JIT works. Two "big" strategies are used: fiddling with loops, this is a short code fragment, inside of one method, consider it as a for loop or while loop. You can do different things with it ... not interesting here.

The second big thing is call stack sampling. Suppose you have very often the method calls A, B, C on the call stack, the VM decides to check if anything can be inlined or if any of those methods is jitted.

In the "benchmark" of the article, you neither have a loop (that one is in the external driver program hidden) nor stack samples which occur often enough to cause a JIT compilation.

Excellent literature about JIT compilation and strategies can be found here: http://www.research.ibm.com/jalapeno/publication.h tml and here: http://www-124.ibm.com/developerworks/oss/jikesrvm /info/papers.shtml

IMHO: the Excelsior people tricked the HotSpot/JIT VMs in not doing their job, and thus you see the big performance gain. There are other tricks thinkable, e.g. not using the standard SWING library but a tweaked one, just remove some "synchronized" keywords if you are sure the benchmark does not need them, and voila ...

angel'o'sphere

(+1, FP)

[Anonymous Coward]

Foundation Profile, n000bz!

MOD AC UP

[12357bd]

Mod AC post up, it's informative.

marketing versus knowledge

[happycorp]

This discussion is interesting in that we've got some actual knowledgeable people responding to what is essentially a marketing article disguised as a technical one.

Without this, I would have partially believed the article (though even I noted that they did not report the results for hotspot -server).

Two observations.

[glen_upstate_ny]

JET seems reasonably priced at least. It has been pointed out elsewhere that while the GNU JRE is not finished the compiler is. GNU GCJ is a replacement for javac that can generate either native code or bytecodes. And since SWT is open source you should be able to write completely open source Java apps now or at some point in the future. Also native compilation solves the problem of decompilation of bytecodes. The legal status of all this is way beyond the scope of this post. O'reilly has a good book about legal issues that covers both GNU and Sun's licenses. Glen Pepicelli http://www.glenp.net