Slashdot Mirror
Using Java In Low Latency Environments
twofishy writes "Something I've noticed amongst financial service companies in London is a growing use of Java in preference to C/C++ for exchange systems, High Frequency Trading and over low-latency work. InfoQ has a good written panel discussion with Peter Lawrey, Martin Thompson, Todd L. Montgomery and Andy Piper. From the article: 'Often the faster an algorithm can be put into the market, the more advantage it has. Many algorithms have a shelf life and quicker time to market is key in taking advantage of that. With the community around Java and the options available, it can definitely be a competitive advantage, as opposed to C or C++ where the options may not be as broad for the use case. Sometimes, though, pure low latency can rule out other concerns. I think currently, the difference in performance between Java and C++ is so close that it's not a black and white decision based solely on speed. Improvements in GC techniques, JIT optimizations, and managed runtimes have made traditional Java weaknesses with respect to performance into some very compelling strengths that are not easy to ignore.'"
This Slashvertisement brought to you by: Dice.com
I find java helps me with my latency :)
Programs written in some languages run more quickly than those written in other languages. Some languages allow you to write programs more quickly than other languages. Therefore, the choice of language is always a tradeoff. There is no perfect programming language, and no, not all programming languages are alike as you seem to believe. If I want to write a program quickly, I write it in Python even though I know the program will run quite slowly.
What a fool believes, he sees, no wise man has the power to reason away.
It's a good thing that these days computers have plenty of memory. It's not quite yet true on smaller devices such as smartphones, but it will be soon. With each year, memory use is becoming less of a problem and how to take advantage of parallel processing is becoming more of a problem.
What a fool believes, he sees, no wise man has the power to reason away.
well I am a master of PHP, so let's find out...
I'm god, but it's a bit of a drag really...
Java can be compiled to native code.
C and C++ can be interpreted.
You do know that Java is compiled to native, don't you?
The only reason Java is slower than C, is because C can have unchecked arrays, or low level access to CPU registers or vector SIMD.
Given the same code written in both C and Java, and including C range bounds checking (to make it as safe as Java), the speed will be the same. Or, quite possibly Java would be quicker once the JVM starts stripping away the checks once it realises there is no possibilities of bounds been breached.
Java is far, far safer than C++. C++ does not enforce type safety at all. For example, in Java you cannot possibly have a buffer overrun or access freed memory as you can in C++. I think most of the security notices are about C and C++ programs, not Java programs. I think you're referring to the Java runtime, which is written in, you guessed it, C.
What a fool believes, he sees, no wise man has the power to reason away.
Traders are typically working with monster servers outfitted with over a hundred gigabytes of RAM, not tiny desktop workstations that need to swap just to move the mouse. I won't say that memory usage is no object, but there is almost no reason not to throw extra memory at a process if it wants it.
Your trading engine runs in Java and leaks four gigabytes an hour? No problem. Just give it 64G of stack and do half an hour of garbage collection after the market closes. Is that not enough? Okay, give it more. Don't have that much available? Get more. Can't afford it? Now you're just pulling my leg. Buying extra memory is cheaper than debugging a live system where any slip-up could cost you thousands of dollars in missed trades or penalties.
It's a weird world, but somehow it works that way.
Challenge Accepted!
The chest beating about Java vs C is kinda sad. Look, I've spent the past 20 years hating java with the fire of a 1000 suns, but having been kinda forced to use it lately I've realised its actually not a bad language, in fact its quite neat and well thought out (But god help me, somehow its date handling is even more broken than javascript)
The problem is all the verbose cruft that goes with it. The giant overly complicated frameworks that require configuring 50 different XML files fed through a labrynith undocumented build process that allows you to write terse and insane pattern-madness code ....or alternatively just fire up JDBC and write a bloody SQL query instead.
I think JAVA could shine if people just threw out about 15 years of insane and overly complicated frameworks and took some tips from the python and perl people and replace them with some simple but effective libraries that do one thing and do it well.
But hey, at least its not bloody javascript!
Excuse the Unicode crap in my posts. That's an apostrophe, and slashdot is busted.
Java is slow because it is Garbage-Collected, not because it runs in a Virtual Machine.
Memory usage is more important than the virtual machine for performance for anything more complex than calculating Fibonacci numbers, as it affects hard drive swapping and cache misses. That's what is making Java programs **feel** slow. The hard disk is the bottleneck, not the CPU.
Wrong. This hasn't been true for more than 15 years 1996. It's a JIT-compiled language, which means it has a slow startup time, but once the VM is warm, it can actually outperform C on numerical code. Really.
The only reason Java is slower than C, is because C can have unchecked arrays, or low level access to CPU registers or vector SIMD.
That and C and C++ have faster allocation and deallocation of temporaries. And they have no runtime specification of what's going on, so the optimizer is allowed to do more.
Specifically, C/C++ (one of the few times where such a phrase is meaningful) require the programmer to specify stack allocation or heap allocation. Stack allocated objects are completely free. At the point of function call and return, it simply uses a different size for the increment/decrement.
Java has an excellent heap allocator (very fast) and can do good escape analysis, but very fast is aloways slower than free and escape analysis is never as good as by-hand specification.
In terms of the optimizer, java specifies all sorts of things about how stack traces from exceptions must be accessed and so on. C/C++ don't specify anything about those. As a result, the optimizer is free to remove more stuff than in C/C++ than Java. If you compile with -O3, it's not uncommon to find the debugger thinks you're in a completely different place than seems to make sense.
Or, quite possibly Java would be quicker once the JVM starts stripping away the checks
The thing is, java trades some safety and tighter specification for some performance loss. C and C++ allow the programmer to specify more. While the JVM is very good at removing a lot of stuff, it's always fighting an uphill battle to remove stuff that simply isn't there in C and C++.
SJW n. One who posts facts.
Many JVM implementations can make use of SIMD at run time because they know what they're running on. Additionally, in many algorithms, the pointer indirection cost in C is higher than the bounds checking cost in Java (because pointers make life hard on optimizing compilers, and HotSpot doesn't have that disadvantage). Cache coherency also plays a big role. See the link in my reply to parent comment.
Java runs in a virtual machine, and is compiled "just in time".
For "real time" applications, that need guarenteed latency, this is a no no.
Java's garbage collection, running is also a no no.
Real time applications need hard deadlines, that are deterministic. A function must always return X mSec, regardless of any other things like GC, etc.
Just imagine if say a pacemaker ran Java.
"I detected an unusual event, but I have to run GC first, then process the event..oh, what the patient is dead?"
Why don't we have for ultimate speed, a java interpreter written in java, running on a java interpreter written in java running on... ad nauseaum?
I don't want a 15 min boot time for my embedded system. there is a reason most Android tablets have huuuge lag at unexpected times. Give me access to the bare metal any day. I'll figure out memory management myself, thank you very much!
Then again, I come from the embedded days, when memory was and sometimes still is, measued in Kb, not Gb.
Java is great for student programmers and others of limited skill that need to produce *something* ,at limited cost. They can't really harm the underlying system, and don't need to lean any hard concepts like memory management and real-time deadlines.
For simple applications that don't deal with a lot of dynamic memory, Java is almost as fast as C/C++. Unfortunately, trading systems are not such - I know because I worked in that domain for some time, in the 1980's and again in the 2000's. Java's biggest issue is garbage collection - it is decidedly non-deterministic. IE, it can have SERIOUS impacts upon performance, and at times that cannot be pre-determined. In my experience (30+ years), if you need consistency and speed, then you do not select Java for your environment - and I do a LOT of Java development. I just don't use it when I need the software to have a small footprint, be fast, and stay out of the way of other system processes.
Manual memory management is a whole class of bugs that Java doesn't have to deal with. A good C/C++ programmer shouldn't have *that* much difficulty, but it does add to debugging costs and detracts from mental focus on the algorithmic aspects of the problem.
"Wouldn't a C++ programmer generate an applicable program effectively as quickly as a Java programmer?"
No, some languages simply are slower to develop with and debug. The problem is also made worse depending on the frameworks and IDEs available. As an example, you're going to get your work done way quicker writing an application manipulating dates and times using C# and Visual Studio than you are Java and Eclipse because until Java 8 Java's date time functionality is shit and Eclipse is a dog slow IDE. With Java 8 and say NetBeans or JDeveloper though things will be pretty similar.
At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java. C/C++ give you more scope for optimisation and more control over memory management as a benefit of that though. It's about trade-offs and figuring out what matters.
But it's possible to be great at both C++ and Java without having to descend into petty arguments as to which is better and know when to use each in response to a specific task, and that's the sort of great programmer these institutions will be looking for and this is really what they're talking about - both have their place but in some cases getting a trading application to market a day earlier than the competition even at slight latency trade-off may be enough to net your company a few million dollars advantage. In other cases, the latency improvements of a highly optimised C++ application may instead be the key to scooping up those extra millions.
FYI the author of that quote is the chief engineer of a C system on which enterprise, low latency trading systems are built. So when he says it, I would tend to give that more than the passing thought.
Those items quoted have indeed been recently shown to bring performance close to similarly developed C++ systems. Though you are right that the layer of indirection will always mean overhead, if you are working low level enough and real time is not your goal, that indirection is what allows your code to be run time optimized and compensate for the cost of the overhead.
For applets in a web-browser. Running a Java server is a different beast with different concerns.
Ram is cheap, and you have control over the max heap size on your java call, so I don't know why you think disk swapping is a problem. Not only that, but garbage collection can actually speed you up in some cases: if you can defer memory management from a time when it would slow you down to a time when you're sitting on free time anyway (waiting for io, etc), you are actually sometimes better off.
Low latency is not measured in the millisecond, but in the microsecond. I'm so tired of hearing about how special the HFT think they are.
Some people also work on multi-million line projects which compile in a minute or less. It's a trade-off. Most compiler issues are related to obscene (and generally unnecessary) amounts of header dependencies. 1000 lines takes an almost immeasurably small period of time to compile, but a 2 line file can take a minute if it includes the wrong headers. The ISO C/C++ library or boost is pure evil in these regards.
A well formed program can compile extremely rapidly. A poorly formed program can often compile extremely quickly with enough CPUs working in parallel across a fast enough network. However a decent program will take time for the initial compile but take almost no time for each consecutive compile when only a file here or there changes.
Also, to make things politically incorrect on Slashdot, good tools like Visual C++ 2012 can even take poor code and make it compile quickly if the projects files are designed well. Do your coding there and compile it later on GCC.
Really.
Meh.
http://benchmarksgame.alioth.debian.org/u64/java.php
Or not. Benchmarks are a game and everyone likes to play, except the users of brainfuck for whom creating a program which completes is essentially winning.
The thing is people have been trumpeting "blah is as fast as C/C++/Fortran" since nearly the inception of C and C++ and probably shortly after the inception of FORTRAN.
I think there is a kernel of truth in that the reason C, C++ and FORTRAN/Fortran are always the targets is because they are almost always either the fastest or with a few percent of the fastest.
When people start comparing execution speeds of Haskell, Python, C#, C++, $NEWLANGUAGE etc consistently against Java then I'll believe that Java is indeed king of the hill.
SJW n. One who posts facts.
These are HFT folks, they have essentially unlimited money. Take a TB of RAM, use half for a tmpfs and you no longer worry about IO. A safer route and more expensive would be to just use PCIe SSDs.
Unlimited money solves a lot of problems.
Uh since around 1.3 the JIT optimization for java has led to blindly fast code containing optimizations which are not even available to C++ . Dynamic compiling allows for branch prediction to be more accurately, unlike malloc, the GC knows where to look for free memory and returns it from the last bit of memory you just requested, if you know where pointers are pointing at compile time, you can put them in registers. C++ and other statically compile languages don't have this information, so it stores them in cache, but the JIT can acquire this information and store it in a register. It's the difference between a register to register test and reading from disk.
There are tons of other stuff like this. I don't have it committed to memory, and compiler technology is not my thing but if you look around you'll see that actually GC and JIT are theoretical advantages in terms of speed and those advantages are being realized. It can even figure out what chip it's being run on at runtime and optimize the code for that chip.
The Java is Slow Meme is left over from 1995 before there was even HotSpot.
Not bashing any other language here. C# could also avail itself of these advantages.
Leaving aside my political beliefs that high speed trading needs to be banned, let me say this.
Switching to Java should yield similar results to C++. What matters is whether the programmer understands the memory architecture of the run-time environment well enough to not have issues. Generally, you'll find that even the best programmers in either language will overlook things like garbage collectors and memory fragmentation issues. It's a time-to-market thing. When working with large dynamic data sets, it doesn't matter if you're using Java or C++, the developer needs to be able to adapt their code to perform well on the system.
Code written without considering the processing time of memory management will probably work much better in C++ than Java. That said for huge data sets, Java could perform better since the memory itself is location independent and it is highly probable that you're gain a great deal of performance from being able to defragment memory. Consider however that the garbage collector and the defragmenter will have unpredictable times which can cause multi-millisecond hiccoughs during processing.
I recommend if you take this route, you hire a compiler geek to work on staff optimizing the memory operations.
These are HFT folks, they have essentially unlimited money.
Unlimited money solves a lot of problems.
But not, apparently, the problem of an ongoing obsession with making even more money.
systemd is Roko's Basilisk.
I'm not a Java user, so I've never directly tuned for things like GC, nor do I interact with it directly. Warm up is a different story.
I interact with quite a few exchanges (over all kinds of protocols). Most are, unsurprisingly, written in Java. Almost all of them perform terribly at the beginning of the week. The issue is a standard one: the JVM hasn't JITted important code paths, and it won't until several thousand requests come in. For a standard throughput-oriented program, this doesn't matter -- the total time wasted running interpreted code is small. For a low-latency network service, it's a different story entirely: all of this wasted time happens exactly when it matters.
The standard fix seems to be to write apps to exercise their own critical paths at startup. This is *hard* when dealing with front-end code on the edge of the system you control. Even when it's easy, it's still something you have to do in Java that is entirely irrelevant in compiled languages.
If JVMs some day allow you to export a profile of what code is hot and re-import it at startup, this problem may be solved. Until then, low-latency programmers should weigh the faster development time of Java with the time spent trying to solve annoying problems like warm-up.
This. I've actually personally witnessed a case where a java test application outperformed a (roughly) equivalent C++ application by a startling amount. The two applications were more of a benchmark program than a real app, since it was only testing the timing characteristics of allocating many thousands of objects on the heap, discarding some of them, allocating more, then discarding more, and so on.... The C++ version used the default new and delete operators, while the java version just used the normal new keyword and left the discarding of memory of old objects to the GC. The Java version outperformed the C++ version by more than a factor of 10. Altering the C++ version so that it used custom new and delete operators for the objects, pooling unused objects instead of always just returning them to the heap as the default delete operator did, the C++ version of the application sped up considerably, outperforming the Java version only slightly over a period of about one billion allocations. In consideration of this experience, I think that one is left with answering the question for themselves of whether spending the extra time to specifically optimize a C++ program is truly worth the marginally improved performance. Sometimes it might be... but sometimes it won't.
File under 'M' for 'Manic ranting'
I built a real-time interface to some hardware in Smalltalk 20-years ago! (Yikes!) Since Java and Smalltalk have the same GC theory (at least, simplistically), some of my experiences would translate. I also spent 5-years running high-performance, low(ish) latency stuff in Java too. The simple rules of low-latency Java (or Smalltalk) are: - Create less garbage. Don't tell me that the GC is an issue when you are creating huge garbage. If your application is running in a tight loop (small amount of code being run a LOT of times), spend time on understanding the GC generational model... create only small temporary objects, and really focus on understanding where every object goes. If you can avoid the stop-the-world gc from kicking in (and you can), then most of the issues go away. - Force a GC before you go into low-latency land. This was something I could do in Smalltalk 20-years ago... I would force a GC right before I went into time-sensitive stuff. - Define what you mean by low-latency. I've worked on major systems (100s of millions of financial transactions per day with strict latency requirements) where there was a team dedicated to it, and frankly, Java wasn't nor never would be an option. (In fact, we worked with mainframe operating systems people can't even name anymore!) However, most people don't mean that when they say low-latency. - Finally, if what you really mean is low-latency measured in 100s of milliseconds, that isn't really hard and Java application servers do that in their sleep now. (I've build business transactional applications that do 100,000 transaction with 700,000 SQL calls per second on some standard software and some beefy hardware) Just work inside the best practices space of these products. It is very easy to say "Java doesn't perform" when your code is crap.
Use RAII consistently, and use containers (from stl or otherwise) which have asserts() on bounds-checking. Bonus points for a tiny unit-test (which can therefore run at the end of every compilation). You'll be amazed at how stable, maintainable, easy to debug and performant your code will be.
Do the hardcore pointer handling only where the profiler tells you that it matters and there's no way java even gets close in performance
Honestly, when your task is, "Grab data from mysql, dump it to screen." There really is nothing faster than coding it in PHP. Its quick, its easy, and it gets the job done as fast as a shell script.
Garbage collection creates a different class of problems, namely that the performance characteristics of your program become non-deterministic. This is a Big Deal for certain classes of applications such as video games and in particular HFT. Would you like to be the person explaining why the GC caused your program to stall at an inopportune time for 50ms and lost somebody a few million bucks?
In the real world so what if you know how to write better software. If you have a job to maintain some old large software, you're mostly stuck with its compile times.
Rewriting it "better" would take many years - you'd be sacked long before you succeeded.
And you'll notice even on your linked counterexample there are 5/11 examples where Java is within the margin of error on CPU time.
Yeah and in the rest C++ is quite definitively faster. In other words as I claimed, C++ is either the fastest or within a small percentage, across the board. There are precious few examples where C or C++ is substantially outperformed by another language (and in almost all of those precious few examples, it's Fortran) and plenty of examples against just about any other languages where it is a clear winner.
Take the time to go back and read the link I posted. "Java vs C" goes back and forth depending on the algorithm in question and the cache characteristics of the target platform.
I did though the one I posted has much more recent results and no dead links. All the benchmarks are similar. In some cases C++ is slightly otuperformed by Java, in other cases C++ substantially outperforms Java.
Mostly it doesn't matter.
But when people routinely benchmark against Java rather than C++, then I'll accept Java as the faster language.
SJW n. One who posts facts.
It's hardly a weird world.
Water leaks into basements. Many times, the solution is to use a sumppump to pump the water out rather than a very costly rebuilding of the house and surrounding terrain.
I've bought many products in my life (just recently a car tire pump) that tells you not to run it for more than 15 minutes. It might over heat.
Many people have cars that start leaking oil as they age. They don't spend the money to fix it as long as they can just add enough oil to keep it going.
And who hasn't used duct-tape to seal leaks?
It's hardly a weird world out there.
Any JVM that would run this code will be tunable, including the ability to tune the GC so it becomes more deterministic. The fact that your desktop app runs in 'use all memory then GC' does not mean that is the only way the JVM can work.
Use a deterministic garbage collector.
C/C++ is much more prone to memory leaks than Java. Yes, you could chase them all down but this takes time. For a long running process this is definitely a factor.
There is no reason Java can not run just as fast and predictably as C/C++, given people who know what they are doing.
Yes, yes there is. Well, more than one actually. Here we go again:
(1) Unbounded stop the world of the JVM causing thread stalls (GC being the canonical example)
(2) Other stop the world's in the JVM due to code profiling causing a newly compiled version to be swapped in (maybe they've switched to atomics here, I'm not sure)
(3) Java's internal profiling and housekeeping absorbs CPU cycles
(4) Everything is allocated on the heap. If you have an array of objects - it's an array of pointers to things in the heap. There is no packed array of structs
These are the ones I've run up against when attempting to reduce scheduling latency in Java.
You can reduce the impact of (1), (2) and (3) using a realtime Java VM - but then you're not really writing Java - you can't use standard containers or standard libraries as they allocate memory all over the place. Also, the ahead of time compilation means you've lost the benefit of that lovely JIT re-profiling and optimisation.
(4) Can be mitigated a little by using NIO buffers with long offsets - but it's a mess and adds unnecessary extra computations to something that should just be pointer + offset.
To get down to it - with the standard Java VM I get scheduling latencies occasionally larger than 20ms - with C++ my thread scheduling latencies are measure in nanoseconds (i.e. less than a microsecond).
This is on a real time kernel with all appropriate thread priorities and such.
Cheap RAM means you can run bigger and faster C++ apps too you know, not just Java. C++/C will beat java in almost every test that counts because skilled C/C++ developers can write code that handles memory better than most GCs.
All those moments will be lost in time, like tears in rain.
The article suggest that one solution is to simply not do GC until the end of the trading day. I have to admit that's a good pragmatic solution, certainly so for HF prop traders.
Not enough memory on your servers? Add more. Still not enough? Add even more. The cost would be a pittance for any prop trading company worth its salt.
In the course of every project, it will become necessary to shoot the scientists and begin production.
That's a claim you need to prove as the article is saying the opposite.
That's a claim the article needs to prove. It was just a bunch of assertions, many by people who have a vested interest in the use of Java.
You are obviously a free thinker shill. Go back to your free thinking and let us get back to the regularly scheduled bashing of everything related to corporate, government, or anything else a 13 year old boy doesn't fully understand yet.
The article you just linked to is describing...wait for it...flaws in the security sandbox which is used for browser plugins. A standard JRE app is not sandboxed, unless you're using a custom ClassLoader for plugins or weird stuff like that. Can you give an example of a datacenter Java application where this would be relevant and how it would be exploited? Why a typical enterprise server app would be running user supplied .class files is beyond me.
Garbage Collection with a bad implementation this is true. But garbage collectors do not need to cause non-determinism; they can be used even in real time systems. The problems is that people re-invent the wheel and don't learn from all the research in the past that sped GC up. Or more likely, the system was originally designed to be a simple quick-and-dirty scripting language but its scope has grown so that now the hastily written GC is unsuitable for the new requirements.
To be fair, good GC is hard, and it is extremely difficult to do it portably. Some good GC techniques need hooks into the operating system (ie, you want to mark pages as clean or dirty). Doing this portably sometimes means you see implementations that sit GC up on top of malloc, so that GC is merely a way to eliminate manual allocating and freeing, and you end up with performance headaches with memory fragmentation.
Yes it's true, if you use a malloc and free directly and no GC, you can still end up with non-deterministic allocation performance! Especially with a naive malloc I see used a lot on small systems, where you just have a linked list of available regions so you have a linear search to find regions big enough. And you end up with fragmentation on any system that does not have relocatable memory (ie, most traditional non-interpreted systems), so many embedded systems prefer to use memory pools to fix those problems.
I got this one, boys.
Let me translate what you said to a car analogy, then you can see if your statement still makes sense.
"Your car is unsafe because it was recalled due to a seat-belt issue. You should ride a motorbike like me. It's never been recalled for seat belt issues."
.there is enough of everything for everyone.
"...by experience with Java apps is that they are incredibly slow, rather unresponsive and crash often."
Then you have no experience with Java application development. Or you're trolling, whichever.
Or possibly you're talking about browser applets, but that's not what this story is about.
If you don't have knowledge on the topic to contribute, why post at all?
At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java.
You're funny.
I think we have about a dozen calls to new and free in a few hundred thousand lines of code in our server. The vast majority of memory allocation in C++ is hidden in libraries like STL, which we presume have been debugged.
In Java, on the other hand, you have to be very careful with memory management or you'll either end up pausing for long periods for garbage collection or crashing with out of memory errors. Instead of worrying about whether people remember to call free after allocating something, you end up putting in caches so you can reuse objects rather than reallocate them. We've used some Java libraries designed for high-performance financial uses and they try very hard not to allocate any objects that they'll later need to clean up.
Programmers believing that 'you don't have to worry about memory management in Java' is the reason so many Java programs are a slug-like mass of memory bloat.
What hinders you to use Joda Time if you think that Java's data and time are insufficient?
Any non-trivial application have dozens or more dependencies, one more or less really is not important.
I find the strength of Java is the enterprise grade open source libraries, one for every possible scenario.
http://www.mueller-public.de - My site http://www.anr-institute.com/ - Advanced Natural Research Institute
But that's the point isn't it? Sometimes those hits from the garbage collector simply don't matter and if they don't matter because performance in this respect isn't the greatest concern then you don't have to bother yourself with it, whilst in C++ you have to bother yourself with it whether performance is a concern or not - in other words C++ always has an inherently slower time to market because of this inherent underlying trait and that's the point of TFA.
If optimisation is more important than time to market then you're absolutely right, switch to C++ because then Java's time to market advantage might be lost.
It's about trade-offs and as I said, sometimes C++ is the right choice, sometimes it's Java, but your anecdote is merely a single case where C++ is the right choice not Java, a single anecdote doesn't disprove Java's worth for every case though.