Help crack the Java 1.6 Classfile Verifier

pdoubleya writes "As part of the development of Mustang (Java 1.6), Sun is developing a new, smaller and faster classfile verifier which they want your help in trying to break. As Sun VP Graham Hamilton puts it in his blog entry, "As part of Mustang we will be delivering a whole new classfile verifier implementation based on an entirely new verification approach. The classfile verifier is the very heart of the whole Java sandbox model, so replacing both the implementation and the basic verification model is a Really Big Deal.... The new verifier is faster and smaller than the classic verifier, but at the same time it doesn't have the ten years of reassuring shakedown history that we have with the classic verifier." You can read about the new verifier on Gilad Bracha's blog, and join the new Crack the Verifier initiative to if you can break it. Read all about the Crack the Verifier - Challenge."

Comments

Take Java seriously

[rexguo]

Before those who go on to dismiss Java for various reasons (no matter how ignorant they are), take a look at the presentation given by Google at this year's JavaZone conference on how Google is using Java internally at extreme scales. Among them are AdWords and GMail.

Re:Take Java seriously

[Anonymous Coward]

Java may not be "slow" any more, but it's an INSANE memory hog. Part of this is because the heap NEVER shrinks - as you allocate more memory, the heap just grows and grows until it reaches the heap limit (which can be user-set). The other part is because you need to load in large numbers of "support" classes, along with the virtual machine itself, just to get to something as simple as "Hello World".

Now, I need to explain that "heap NEVER shrinks" bit, because people are going to hop in and start talking about the garbage collector. Well, yes. The garbage collector does indeed free memory. HOWEVER, it only returns the memory to the Java VM heap, NEVER to the operating system.

So, simple real-world example. Java VM starts with 8MB heap. Your application allocates 40MB worth of objects, increasing the total heap size to 64MB. Java now has 24MB "free" memory within the VM, and 40MB used. You can determine this in code using Runtime.freeMemory() to get the memory free in the heap, and Runtime.maxMemory() to get the maximum memory the heap has available.

Your program then "releases" 38MB worth of that data as unneeded. Eventually the GC collects that. Now you have only 2MB used, but the maximum heap size (as determined by Runtime.maxMemory()) REMAINS at 64MB. So now Java has 62MB "free" within the VM. Even if your program never uses more than 8MB after that, Java will ALWAYS keep the 64MB heap. It will NEVER shrink it.

This is fine in server setups, but it absolutely blows for client apps, where it's not unusual for a certain process to require a lot of memory, and then be ready to release it back to the OS.

Ultimately, on client apps, this winds up causing swapping, as chunks of unused VM heap get swapped out in favor of applications that actually need the memory they allocate. Swapping = slow.

You could call this behavior a "memory leak" but it's almost certainly by design.

Java may not be slow, but it's definitely bloated.

Re:Take Java seriously

[justsomebody]

While I agree with you on all accounts I can't help but comment you.

Both Java and .Net have the same problem. Sloppy memory. As long as you don't use a lot of atomic memory blocks with higher load on machine where it runs, everything is ok and just as you said it. I tested both on the same tests and always fallen in the same problem. No direct memory control, GC waited until it was too late, and everything started to crawl. GC somehow avoids doing work if software is taking most of the CPU, it is also the same reason why Java beats C or C++ on speed tests (Every speed test where they try to proove that Java or .Net is faster than C or C++) uses allocation and freeing of memory in some loop. And while both C and C++ actualy do free memory, Java and .Net just mark that as garbage and wait for GC to clean up the mess which doesn't happen if load is too high. Just take any test where Java was faster and test loop to make 2^32 instead of 1000 or 10000 calls. This way you will actualy use more memory than you actualy have with allocations.

p.s. Not bashing, just saying results of my testing. If you can suggest some approach, do that, but so far not even one person suggested something I haven't tried yet. Both Java or .Net would be a real gift for me if only I could use them for my needs.

Re:Take Java seriously

[AKAImBatman]

The reasons to use Java on the server are quite simple. The combination of factors that attracted developers to Java in the first place make them want to use it on the server. Those factors are:

1. Cross-platform capability - Many companies still prefer to deploy applications on large Sun, IBM, or Linux (name your brand) servers. However, these companies would also like to give their developers Windows desktops so they can interact with the rest of the company. (Who most likely uses MS Office/Outlook.) As long as you avoid explicit path names, it is quite easy (and common!) to develop on a Windows machines but deploy on a Unix or Unix-like machine.

2. Automatic Memory Management - So your server is running along, and suddenly someone generates an unexpected error. In Java you can sleep soundly because even the worst programmer would have a hard time doing anything to completely take down the application. If you use a language that allows direct memory management, you have a good chance of that new guy coding a General Protection Fault/Segfault. The result is that your entire system coredumps when you least expect it.

3. Security - While Java is able to control the Security of the ENTIRE JVM through its security framework, most companies are happy with the lack of buffer overruns, code injection techniques, and other common attacks. That's not to say that a poor programmer can't put a security hole in the application wide enough to drive a Mack truck through, but at least you can rely on the underlying system not to betray you.

4. Flexibility - The Java server side frameworks are exceedingly flexible in their designs. For example, the servlet framework allows you to plug in your own custom server page technology. I have seen many a programmer (including myself) implement something like Reports by simply linking the ".rpt" extension to a custom servlet. The servlet then loads the requested configuration file and executes it. Very nice.

Another example is servlet filters. Need a security framework added in a hurry? Just add a filter servlet! It will execute before the rest of the code, allowing you to check the variables and security permissions to ensure that the client isn't trying any funny business.

5. API - When Java was first introduced, it absolutely creamed all the competing languages in the richness of its bundled API. As time has worn on, this has changed. However, Java still enjoys a sizable lead over even C/C++ with features such as Type IV (tested cross-platform, pure Java) JDBC database drivers. Unlike ODBC, many of these drivers have been tuned for excellent performance. Similarly, there are free APIs for handling Office Documents, PDF Creation/Editing, SOAP/XML-RPC communications, Object-Relational mapping, Image Management/Creation/Editing, CORBA, XML Databases, XSL-T, etc. While these APIs are all available for C/C++, there are significant cross-platform issues with many of them, as well as a lack of common "pluggable" APIs that allow for one API to many implementations.

Other languages have a hit/miss score with these sorts of features, often not providing these features, providing only a small subset, or only being available in an expensive commercial package.

6. Dynamic Loading - While C/C++ can manage dynamic loading of shared objects, it's a very difficult thing to implement. Java does it out of the box, with a full reflection API and interface support, thus allowing such wonderful code as Beans, Servlets, Pluggable Drivers, self-organizing code, and a host of other features that other systems can't compete with.

(If you don't believe me, try adding support for a feature in PHP sometime. "It's so simple! Just install the SO and recompile PHP!" Meh.)

7. Performance - This may sound like an odd thing to say, but the performance of Java is a key selling feature. Java server applications may execute more slowly than one written in C/C++ (just as C/C++ may execute more slowly than

Re:Take Java seriously

[TheRaven64]

I would refer you to some research done around a decade ago, which involved running a MIPS emulator on a MIPS machine. The emulator was doing dynamic optimisation, and got around a 10% speed increase over the same code running directly on the hardware.

A Java VM does some things that are simply not possible with C. To inline a C function, you need the source for both - this leads to some really ugly things like putting simple functions in headers, which should be reserved for interface definitions, not implementation details. The Java VM will inline functions on the fly. This can potentially give a huge performance boost - I got almost a 50% speed increase on some C code I was recently writing by shuffling things around to allow the compiler to inline some common functions.

The other advantage of higher-level languages is that they provide more semantic information to the optimiser. Consider the trivial example of autovectorisation. In C, if you want to do an operation on a vector, you will usually iterate over every element and perform the operation. The compiler then needs to check that there are no dependencies between loop iterations, which can be non-trivial. In a language like FORTRAN or Smalltalk, you can simply perform an operation on a vector type. The compiler then just needs to check if the operation you are trying to perform corresponds to one or more vector unit instructions, and substitute these in to your code. This is much easier to do.

C is a fairly easy language to write optimised code in for any CPU up to and including a 386. For anything more modern, you will find yourself fighting a language which is simply not designed to deal with parallelism - and compiler writers find themselves fighting even harder.

Re:Don't believe the hype!

[craigmarshall]

> C is portable, fast, very complex and since 35+ years the leading standard for professional OS and APP development.

I agree that C is portable and fast, however I don't it can be called very complex.

The smallest programming language manual I have ever owned (and I've owned quite a number), has to be "The C Programming Language", often hailed as the One True Reference to the language. How can it be that complex if the manual is less than half the size of most of my other manuals? I think languages (in general) have got more complex since then. The size of the .Net Framework is huge, there's no way that's simpler than the C standard library. Then you've got to think about reflection, inheritance, dozens of things that C just doesn't have.

If what you mean is that C programs end up looking more complex, that's probably because C is used for systems programming. If you mean that you have to write more code to do it in C, then you may have a point, but I think C is actually one of the simpler languages. The closer to assembly you get, the simpler the language has to be.

Craig

Re:Why not prove it?

[fuzzy12345]

Here's a flame to all the respondents to the parent post. They say (if I may paraphrase) "Code verification is hard. I want my MOOMMMMMY!" Well, it's certainly more difficult, in the short run, than the "throw it against the wall and see if it sticks" approach. But it has been done, it isn't as hard as the naysayers are making out, and it's one of those things that you don't improve at unless you try. Google VLISP for an example of a provably correct compiler.

One thing's for sure: Improvements in software quality will be harder to come by if everybody's attitude continues to be "Bugs are inevitable. Formal proofs are beyond us. Let's keep doing it the old way."

you have no clue

[RelliK]

You obviously have never worked on large server-side applications. Other posters have already listed some reasons for java's popularity. I'll add some more:

* java is nearly as fast as C++ according to all the benchmarks I've seen. Yes, really. The perception of java as being "slow" is simply the legacy of the old awt apps. Yes, the awt gui was (and is) slow. Server-side java applications are not. The "much better performance" is simply not there, particularly for typical enterprise apps.

* *All* the enterprise apps (which is the area where java is particularly successful) store stuff in a database and/or talk to remote apps. Newsflash: a database query or a remote procedure call is *orders of magnitude slower* than an in-process procedure call. Once you include DB/RPC into the equation, whatever little speed advantage C++ has is wiped out completely.

* This is CS 101: performance of a program is largely determined by the algorithm used. You can write a linear search in assembly, and it will be very fast for small lists. But for large lists, a binary search written in shell script will beat it.

* In an enterprise application scalability is much more important than raw speed. So what if I can write a C++ app that's 20% faster than an equivalent java app? Java has frameworks that make it easy to write an app that you can scale horizontally (i.e. by adding more boxes). Easy being the keyword.

* Developer's time is much more expensive than runtime. It is *much faster* to write an app in java than in C++. And for all but the smallest/simplest apps it is faster to write the app in java than in PHP/perl/whatever.

If it's a safety/security issue then again you could build the same thing in a native compiled language, sandbox and all.

Uhhhm, yes. Safety and security are *big* issues in enterprise apps. Show me *one* native language and platform that does it. You are saying it like one can just wave a magic wand and have it built in no time. "You could build the same thing" is not "it's already built".

I mean really, is it just because Java provides a lot of easy to use API's?

Yes. among all the other things I've mentioned.

These are just a few reasons why java is so popular in enterprise apps. Sure, I wouldn't write a game in java, but for enterprise apps, it's perfect. Why java and not PHP/perl/? Simply because java is better. It has all the advantages of compiled laguages (type safety, variable declaration checking, syntax checking, etc.) without some of the disadvantages (manual memory management). Think of java compiler as a sanity checker for your code. It will catch common mistakes like typos, missing return statements, invalid function parameters, etc. A scripting language will not complain about that, but force you to spend hours tracking down the bug. That's why java is faster to develop in than any scripting language for large apps.