Slashdot Mirror

← Back to Stories (view on slashdot.org)

On the Horizon: an Apache-License Version of Java

Posted by timothy on from the better-definition-of-trusted-computing dept.

mparaz writes "Geir Magnusson of the Apache Software Foundation announced a J2SE 5 implementation project called 'Harmony.' It covers the virtual machine and the class libraries, and aims to pass the Sun specification. A FAQ is available."

29 of 268 comments (clear)

  1. great news by Sv-Manowar · · Score: 4, Insightful

    Could this be an essential aid to Tomcat and the increasing number of projects the apache foundation are managing within the Java space, such as ANT. This can only be a good thing

    --
    Business Voyeur
    1. Re:great news by timeOday · · Score: 5, Informative
      If it ever happens, yes... but so far it's just a message on a bulletin board. Implementing the JVM itself is no trivial task, and would take years to reach the performance and stability of Sun's JVM even with huge resources. They have chosen their own unique architecture so I don't think code reuse is in their plan.

      Then there are the class libraries, which have sprawled to a massive scale, and in comparison make implementing the JVM look easy. Look at Wine, which still isn't an alternative for Win32 (only selected applications are supported), after years and years of work. Or Mono, which cannot and probably never will run arbitrary .Net apps.

    2. Re:great news by afabbro · · Score: 3, Insightful
      Flash back to 1993...

      Will anyone ever reimplement the Unix kernel? So far it's just a message on USENET. Implementing the Unix kernel itself is no trivial task and it would take years years to reach the performance and stability of Sun's kernel even with huge resources. They have chosen their own unique architecture so I don't think code reuse is in their plan.

      Then there is /usr/lib, which has sprawled to a massive scale...

      --
      Advice: on VPS providers
    3. Re:great news by wfmcwalter · · Score: 5, Informative
      Well, an elementary JVM is trivial, but a half-decent one has some harder stuff. Assuming they'll want to get Harmony up to the standard of Apache-httpd and Apache-tomcat, the JVM component will be a big job too.

      The bytecode loop and elementary classloader are indeed straightforward (which is why there are so many of them hanging around), and doesn't really get harder between a barely-working JVM and a decent one. Lots of other stuff, however, does:

      • A dumb, blocking, non-generational mark-and-sweep garbage collector is fairly straightforward (handling blocking and native methods wrt GC is a complication, if a manageable one). But for a box serving lots of connections on a busy website, you don't really want half second long pauses while the GC sweeps the whole memory. You really need a generational collector, and you really want one that's either non-blocking (yes, that's hard) or resumable. The nice thing is that the GC is fairly self contained (not entirely, as it interworks with synchronisation and the native method interface) and there are lots of university research groups who have done lots of research on GCs (for java and other languages) so it should be possible either to pick up some research collector or farm the work out to some eager masters students.
      • Efficient management of native synchronisation resources has an important effect on scalability. Mature JVMs go to great lengths to marshall the number of native synchronisation primitives the JVM instance uses (e.g. with some kind of mutex pool, assigning an OS mutex to a java one only when it is in scope). They can work without this for a while, but it needs done eventually. I see Doug Lea is onboard, and this kind of stuff is Doug's meat-and-drink.
      • The verifier is hard to get right. Sun's one is the product of careful design and then of several analyses by third parties. For example, one univerity wrote a verifier from the JVM spec in a formal language (Z or something) and then threw millions of randomly generated program fragments at their one and the Sun one. Where the two differed, the group analysed the program in depth. From this they found dozens of cases where the two verifiers differed materially; most were due to different interpretations of the JVM spec (which, one hopes, resulted in the language of the spec being tightened) but about ten were nontrivial holes in the Sun verifier. Last time I was involved with this (a few years ago) Sun insisted that all Java licencees (even those who had written their own JVM etc. entirely from scratch) run the Sun verifier. Luckily, the verifier is like the GC - it's a subject of ongoing academic research, so there are universities who might be persuaded to do the heavy lifting. And for a trusted enterprise setup you can do without the verifier anyway (it's really only needed for untrusted mobile code like applets or JINI things).
      • But the really big task is dynamic compilation. A bytecode-interpreted system isn't credible for any real application. A JIT will do, at the expense of sluggish performance and drastic memory-munching. A real hotspot-like smart, self-monitoring dynamic compiler is really necessary for a quality JVM. I guess the Harmony folks will spend a lot of effort here, as it's a lot of work and its too tightly bound to your JVM internals to either farm out or to allow you to easily take something off the shelf.
      Still, I hope I've not sounded too negative. It's all doable (python and mono do most of this stuff between them, neither with a vast team) and the lack of a free or open JVM has been an uncomfortable gap between LAMP and tomcat for too long.

      Heck, maybe it's just a strategy to get Sun to open Tiger sooner rather than later.

      --
      ## W.Finlay McWalter ## http://www.mcwalter.org ##
    4. Re:great news by AstroByte · · Score: 3, Interesting
      Most of your points are right, however, I take issue with the statement that the bytecode loop is trivial. This, I'm afraid, is a widely held misconception :)

      While a simple switch-based interpreter is trivial to implement, it will perform abysmally on modern processors because of the overhead of the computed jumps. Writing an efficient interpreter has been an active research area in itself (see http://www.complang.tuwien.ac.at/projects/interpre ters.html for some good papers). In fact, the difference between a poor interpreter and a good interpreter can be greater than that between an intepreter and a JIT.

      Several improvements over a switched interpreter are possible. Firstly, indirect-threading can be used to minimise dispatch overhead. This can further be improved by moving to direct-dispatching, but this requires rewriting the original bytecode stream. Splitting instruction fetch and address computation via prefetching can also lead to substantial gains, as can the use of "super-instructions".

      Additionally, an interpreter can attempt to do stack-caching (either simple top-of-stack caching or more advanced 2 or more levels). This can be used to overcome the inefficiency of executing stack-based bytecodes on a register-based CPU.

      Many of these techniques move into the grey area between interpreters and JIT compilers. But no commercial VM uses a simple bytecode interpreter. Even with a JIT, modern VMs still initially execute code using the interpreter, only compiling the "hot spots" to native code. With full JIT compilation, the start-up time of the VM becomes prohibitive.

      Many of these techniques are also used in some of the open-source VMs. For example, the interpreter in JamVM, depending on architecture, makes use of direct-dispatching, super-instructions, prefetching, and true, 2 level stack-operand caching. It is many times faster than a simple bytecode interpreter, and it has not been trivial to implement!

  2. Better for the Linux User by MoogMan · · Score: 5, Insightful

    Cool! This will be useful for the majority of Linux desktops, because it means it could be installed as part of a default install, rather than having to download it and install it afterwards (==hell for lots of users).

    1. Re:Better for the Linux User by Chris+Kamel · · Score: 3, Interesting

      Don't you think a reimplementation of the VM is too much of a price to pay for such a small convenience?

      --
      The following statement is true
      The preceding statement is false
    2. Re:Better for the Linux User by IamTheRealMike · · Score: 3, Insightful

      The real question that's on most peoples lips and conspicuously not answered in the FAQ is what is wrong with the GNU implementation. They mention that Classpath and GCJ already exist but fail to mention why these are not open source enough. Red Hat is putting a lot of effort into Free Java - why does Apache feel the need to compete with this?

    3. Re:Better for the Linux User by IamTheRealMike · · Score: 5, Informative

      Right, but so what? It can "run" .class bytecode files by compiling them to native code on the fly, this is how gcjwebplugin works AFAIK. The difference between a VM and compiler is mostly one of semantics, there's no compelling enough reason to reimplement the VM.

    4. Re:Better for the Linux User by DrXym · · Score: 4, Interesting
      The same could be said for Kaffe, gcj, classpath et al. And in fact it probably has been, each in their time being heralded as a way to break from Sun.


      Sadly the reality is that no Java is even remotely as reliable or complete as Sun's implementation for the desktop let alone anywhere else. Major work had to be done to gcj just to make Open Office 2.0 run, which hardly speaks for its maturity. And other impls such as Kaffe are missing critical security functionality such as byte code verification. And enterprise level functionality? Forget it.


      Personally I'd love to see a free and open source Java, but its taken years to get this far and its still not there yet.

    5. Re:Better for the Linux User by beforewisdom · · Score: 4, Insightful

      "sneaky" ?

      I really hate to defend Sun,...really, I do, but they are the ones who spent a ton of money and work developing Java.

      They make it available for free of charge.

      what jerks

    6. Re:Better for the Linux User by FireBreathingDog · · Score: 3, Insightful
      Garbage collection can't guarantee code that's completely free of memory leaks. You can still leak memory in Java is you have a long-lived data structure (Set or Map, for example) that holds references longer than it needs to.

      Let's say you've got an improperly written session management routine where users are added to a Set when they log in, but there's a bug that forgets to remove them when they log our or their session expires. Over time, you'll start having memory issues.

      Of course, it's still true that there's much less likelihood of leaks in a GC environment.

      --
      Shame on Google.
  3. Kaffe, Classpath... by otisg · · Score: 3, Informative

    For those too lazy to click through to that blog entry, Kaffe, Classpath and other solutions already exist, and this is not the first.... although coming from Apache carries some weight.

    --
    Simpy
    1. Re:Kaffe, Classpath... by Fefe · · Score: 4, Informative

      Had you bothered to read the blog entry yourself before commenting, you would have noticed that Kaffe and Classpath members are part of this project.

      This does appear to be a consolidation project. We have several contenders for Open Source JVMs under Linux, but most of them lack in some way or the other compared to the Sun and IBM JVMs. So having one up-to-date one instead of five not-quite-there-yet ones is a step forward.

    2. Re:Kaffe, Classpath... by Anonymous Coward · · Score: 3, Informative

      Some Classpath members may be part of this project - but the Classpath code ain't . This is due to the incompatible license. Writing all that java class library code took years. It will be interesting to see whether the Classpath code will simply be lifted (wink, wink) or rewritten from scratch. It's a huge task. The VM, by comparison, is a dime a dozen. Any moderated motivately hobbyist can knock off one in a few months. It may not run quickly, but it will run. The java class libraries' quality and completeness is 99% of the problem.

  4. basic architectural blueprint by anandpur · · Score: 4, Insightful


    http://people.apache.org/~geirm/harmony.jpg

  5. Parent uninformed or troll by Chris_Jefferson · · Score: 3, Funny
    The link you post to is the FSF's problem with Java's current licence. Their actual opinion on the Apache License v2.0 is below. It's incompatable due to patent related issues that the GPL doesn't (and probably should) deal with. It's a fine free software license:

    Apache Software License, version 2.0

    This is a free software license but it is incompatible with the GPL. The Apache Software License is incompatible with the GPL because it has a specific requirement that is not in the GPL: it has certain patent termination cases that the GPL does not require. (We don't think those patent termination cases are inherently a bad idea, but nonetheless they are incompatible with the GNU GPL.)

    --
    Combination - fun iPhone puzzling
  6. Re:reminds me of QT by eurleif · · Score: 3, Informative

    There are issues with it for non-commercial software which isn't GPLed. There aren't issues with it for commercial software which is GPLed. GPL != non-commercial.

  7. Re:Not GPL compatible by squiggleslash · · Score: 4, Informative
    In case you're serious, the Apache Sofware License 2.0 is considered a Free license, just not a GPL compatable one. The FSF actually quite likes it:
    This is a free software license but it is incompatible with the GPL. The Apache Software License is incompatible with the GPL because it has a specific requirement that is not in the GPL: it has certain patent termination cases that the GPL does not require. (We don't think those patent termination cases are inherently a bad idea, but nonetheless they are incompatible with the GNU GPL.)
    It's quite probable that once version 3 of the GPL is released, there'll be a strong effort from both sides to get some compatability between the two as incompatable licenses hurt everyone, whatever your ideological differences. Version 3's important because it's the one the FSF has suggested it'll deal with the issue of patents in.

    In the mean time, the Apache group's choice of license for their Java project makes perfect sense given a major, if not the major, use for Java these days is for back-end work of web-fronted applications. Apache's Tomcat sometimes seems to be more popular than Apache itself. (I said seems people, seems); I can't think of any other reason why the Apache people would be organizing this, though it surprises me they're going for J2SE and not J2EE compatability.

    So, no. There's no "Java trap" inherent in developing code for Apache Harmony.

    --
    You are not alone. This is not normal. None of this is normal.
  8. Excellent! by multipartmixed · · Score: 3, Funny

    And after it passes the Sun spec, we can fix it to be useful (since we have the source) with a simple header change:

    #define sleep(a) while(0) ..that should eliminate half of the code, decreasing binary size and actually performing. ;)

    --

    Do daemons dream of electric sleep()?
  9. Re:Divide and Conquer by multipartmixed · · Score: 4, Insightful

    C *is* cross-platform.

    The system libraries, on the other hand.. well, that has nothing to do with the language. If you want cross-platform code, use cross-platform libraries.

    If you can stick to using only functions in K&R and the POSIX Programmer's Reference Guide, you will find that your code (if written properly) will run damn near anywhere.

    If you want a little more functionality (as much as you need, really) without GUI, adding the Apache Runtime Library will get you there -- portably. Especially under unices and workalikes.

    C++ -- I'm not qualified to comment on that.

    --

    Do daemons dream of electric sleep()?
  10. Re:Not GPL compatible by hexene · · Score: 3, Informative

    I can't think of any other reason why the Apache people would be organizing this, though it surprises me they're going for J2SE and not J2EE compatability.

    But they are. J2EE is a superset of J2SE, and by adding Apache Geronimo you'd get the complete stack. Admittedly Geronimo is aiming for J2EE 1.4 rather than J2EE 5.0 at the moment, but J2EE 5.0 doesn't really exist yet. ;o)

  11. JamVM by Hugo+Graffiti · · Score: 5, Interesting
    I hope they choose JamVM for the VM. It's a fairly new VM but impressively lean and mean (100k executable that still supports the full spec). From the JamVM web site, here is a list of the main features:

    • Uses native threading (posix threads). Full thread implementation including Thread.interrupt()
    • Object references are direct pointers (i.e. no handles)
    • Supports class loaders
    • Efficient thin locks for fast locking in uncontended cases (the majority of locking) without using spin-locking
    • Two word object header to minimise heap overhead (lock word and class pointer)
    • Execution engine supports basic switched interpreter and threaded interpreter, to minimise dispatch overhead (requires gcc value labels)
    • Stop-the-world mark and sweep garbage collector
    • Thread suspension uses signals to reduce suspend latency and improve performance (no suspension checks during normal execution)
    • Full object finalisation support within the garbage collector (with finaliser thread)
    • Garbage collector can run synchronously or asynchronously within its own thread
    • String constants within class files are stored in hash table to minimise class data overhead (string constants shared between all classes)
    • Supports JNI and dynamic loading for use with standard libraries
    • Uses its own lightweight native interface for internal native methods without overhead of JNI
    • JamVM is written in C, with a small amount of platform dependent assembler, and is easily portable to other architectures.
    1. Re:JamVM by shutdown+-p+now · · Score: 4, Informative

      No JIT, though. Which makes it nearly useless on the server, where Java is mostly used.

  12. binary compatibility? by palinurus · · Score: 3, Interesting
    I bet they can pull off a really nice VM. The existence of multiple VM implementations has yet to produce the kind of community fragmentation that a lot of people have prophesized, is a credit to the strength of sun's spec writing, and has been good for the platform overall (my java apps, both client and server, run without modification on os x...) I wonder about the class libraries, though.

    It seems like maintaining binary compatibility between serialized classes (esp. for collections and java.lang classes) is essential, at least if you want to do J2EE stuff in the long run. It will be at least a nuisance to, say, reimplement java.util.HashMap in a binary-compatible way without illegally appropriating Sun's IP (something the project seems pretty conscious of in their charter/FAQ).

    It's not impossible, but I think the IP challenge there is the real issue (not to mention the fact that your implementation is going to be constrained to being nearly identical to Sun's, at least in terms of overall strategy, if not line-by-line). If you read Sun's code in one window, and then write the same member variables in the same order in another window, is that copying code or not? And even if you do write something completely different (say, going with the HashMap example), you have to come up with some kind of transformation from your choice of state variables to sun's serialized state variables, which could look pretty nasty.

    I also pity the poor bastard that has to write those AWT libraries...

  13. Re:Divide and Conquer by ssj_195 · · Score: 3, Insightful
    What is "fragmented" about it? If you have a computer with more than 256MB of RAM, then you can happily run either KDE or GNOME - the difference is merely one of personal preferences. An application written for GNOME works damn-near perfectly under KDE, and vice-versa, due in large part to the efforts of freedesktop.org. If you are talking about distros, then the only two that a "Joe Sixpack"-type of home user will need to know about are Ubuntu and SUSE (I'd go with Ubunutu, personally). Granted, packages are not interoperable between the two, but since both have good, up-to-date versions of the same software a mouse-click away, who cares?

    I'm sorry, but I'm just really not seeing this supposed "fragmentation" as a barrier to Linux on the desktop.

  14. Harmony could use Parrot by mattr · · Score: 5, Interesting
    I'm just someone not involved in language development, and so I'm sorry if I'm out of line. Most developers in language X just sit back in admiration at the olypmian efforts of language, compiler, vm or kernel designers. But I would like to humbly suggest that Harmony people talk with the parrot people. Parrot already has a java bytecode converter proof of concept, initial code, will run on tons of platforms, and has perl and python people too. It is GPL compatible and licensable under the Perl Artistic License.

    The reason I suggest this is that it would appear that the main purpose of the Harmony project is to create a vibrant, inclusive community. In that case, the open source world, Harmony, and Parrot, plus users of java, perl, python, ruby and tcl (for starters) can all benefit by combining two disparate groups of all-star programmers working in potentially complementary areas.

    If any parts of the Harmony project can use parts being developed for Parrot, much time would be saved and the quality of both projects could increase. In addition, it would likely be easier for the Harmony project to meet its stated goals of collaboration and sharing of runtime components, etc. to do so with parrot. The Parrot FAQ also talks a bit about VM development, including working with a JVM, it sure sounds like there is some overlap with Harmony.

    Perhaps the Parrot people don't need any help (I doubt they would say so though) and maybe the Harmony VM people can't stand the idea of not building from ground zero, or using only the Apache license and nothing else. If any of these three maybes are true then it is a sad story.

    Also, I may be out of line but it sounds like parrot will enable sharing of code from different languages at runtime. If so that will just magnify what Harmony is trying to do in terms of bringing people together.

    So humbly I would like to say that the ideas of creating a specification and reference implementation, and promoting collaboration and sharing of modular code sounds wonderful, and focusing on these and not wasting time reinventing the wheel could be a great move for Harmony, and contribute to refocusing the brainpower of the free software world, in the spirit of the Harmony and Parrot projects.

    My guess is that Harmony has some really smart people and they are also well aware of the Parrot effort. Maybe some are already involved for all I know. Any comments one way or the other?

  15. Re:Divide and Conquer by Master+of+Transhuman · · Score: 4, Insightful


    I shouldn't have posted above while I had mod points, since this troll crap is modded "Insightful" by the Windows trolls moderators and other idiots.

    Look, stupid, this is not just a "licensing fetish" (although as has been discussed, there is a perfectly good reason for Apache to not use the GPL or like Sun's license.)

    The point of this project is to provide a compatible free Java that Apache can use to underpin its numerous Java-based projects.

    It's an excellent idea - unless Sun ever comes out with a truly OSS license. And if they do, it will probably be because such a project is gaining traction.

    --
    Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
  16. Interesting project name.. by salimma · · Score: 3, Interesting

    .. does anyone else get a déja vù from the KDE-sponsored attempt to clone Qt back in the non-QPL 1.x days?

    --
    Michel
    Fedora Project Contribut