Slashdot Mirror
Using Java In Low Latency Environments
twofishy writes "Something I've noticed amongst financial service companies in London is a growing use of Java in preference to C/C++ for exchange systems, High Frequency Trading and over low-latency work. InfoQ has a good written panel discussion with Peter Lawrey, Martin Thompson, Todd L. Montgomery and Andy Piper. From the article: 'Often the faster an algorithm can be put into the market, the more advantage it has. Many algorithms have a shelf life and quicker time to market is key in taking advantage of that. With the community around Java and the options available, it can definitely be a competitive advantage, as opposed to C or C++ where the options may not be as broad for the use case. Sometimes, though, pure low latency can rule out other concerns. I think currently, the difference in performance between Java and C++ is so close that it's not a black and white decision based solely on speed. Improvements in GC techniques, JIT optimizations, and managed runtimes have made traditional Java weaknesses with respect to performance into some very compelling strengths that are not easy to ignore.'"
I find java helps me with my latency :)
Programs written in some languages run more quickly than those written in other languages. Some languages allow you to write programs more quickly than other languages. Therefore, the choice of language is always a tradeoff. There is no perfect programming language, and no, not all programming languages are alike as you seem to believe. If I want to write a program quickly, I write it in Python even though I know the program will run quite slowly.
What a fool believes, he sees, no wise man has the power to reason away.
well I am a master of PHP, so let's find out...
I'm god, but it's a bit of a drag really...
Java can be compiled to native code.
C and C++ can be interpreted.
You do know that Java is compiled to native, don't you?
The only reason Java is slower than C, is because C can have unchecked arrays, or low level access to CPU registers or vector SIMD.
Given the same code written in both C and Java, and including C range bounds checking (to make it as safe as Java), the speed will be the same. Or, quite possibly Java would be quicker once the JVM starts stripping away the checks once it realises there is no possibilities of bounds been breached.
Java is far, far safer than C++. C++ does not enforce type safety at all. For example, in Java you cannot possibly have a buffer overrun or access freed memory as you can in C++. I think most of the security notices are about C and C++ programs, not Java programs. I think you're referring to the Java runtime, which is written in, you guessed it, C.
What a fool believes, he sees, no wise man has the power to reason away.
Traders are typically working with monster servers outfitted with over a hundred gigabytes of RAM, not tiny desktop workstations that need to swap just to move the mouse. I won't say that memory usage is no object, but there is almost no reason not to throw extra memory at a process if it wants it.
Your trading engine runs in Java and leaks four gigabytes an hour? No problem. Just give it 64G of stack and do half an hour of garbage collection after the market closes. Is that not enough? Okay, give it more. Don't have that much available? Get more. Can't afford it? Now you're just pulling my leg. Buying extra memory is cheaper than debugging a live system where any slip-up could cost you thousands of dollars in missed trades or penalties.
It's a weird world, but somehow it works that way.
Challenge Accepted!
The chest beating about Java vs C is kinda sad. Look, I've spent the past 20 years hating java with the fire of a 1000 suns, but having been kinda forced to use it lately I've realised its actually not a bad language, in fact its quite neat and well thought out (But god help me, somehow its date handling is even more broken than javascript)
The problem is all the verbose cruft that goes with it. The giant overly complicated frameworks that require configuring 50 different XML files fed through a labrynith undocumented build process that allows you to write terse and insane pattern-madness code ....or alternatively just fire up JDBC and write a bloody SQL query instead.
I think JAVA could shine if people just threw out about 15 years of insane and overly complicated frameworks and took some tips from the python and perl people and replace them with some simple but effective libraries that do one thing and do it well.
But hey, at least its not bloody javascript!
Excuse the Unicode crap in my posts. That's an apostrophe, and slashdot is busted.
Java is slow because it is Garbage-Collected, not because it runs in a Virtual Machine.
Memory usage is more important than the virtual machine for performance for anything more complex than calculating Fibonacci numbers, as it affects hard drive swapping and cache misses. That's what is making Java programs **feel** slow. The hard disk is the bottleneck, not the CPU.
The only reason Java is slower than C, is because C can have unchecked arrays, or low level access to CPU registers or vector SIMD.
That and C and C++ have faster allocation and deallocation of temporaries. And they have no runtime specification of what's going on, so the optimizer is allowed to do more.
Specifically, C/C++ (one of the few times where such a phrase is meaningful) require the programmer to specify stack allocation or heap allocation. Stack allocated objects are completely free. At the point of function call and return, it simply uses a different size for the increment/decrement.
Java has an excellent heap allocator (very fast) and can do good escape analysis, but very fast is aloways slower than free and escape analysis is never as good as by-hand specification.
In terms of the optimizer, java specifies all sorts of things about how stack traces from exceptions must be accessed and so on. C/C++ don't specify anything about those. As a result, the optimizer is free to remove more stuff than in C/C++ than Java. If you compile with -O3, it's not uncommon to find the debugger thinks you're in a completely different place than seems to make sense.
Or, quite possibly Java would be quicker once the JVM starts stripping away the checks
The thing is, java trades some safety and tighter specification for some performance loss. C and C++ allow the programmer to specify more. While the JVM is very good at removing a lot of stuff, it's always fighting an uphill battle to remove stuff that simply isn't there in C and C++.
SJW n. One who posts facts.
For simple applications that don't deal with a lot of dynamic memory, Java is almost as fast as C/C++. Unfortunately, trading systems are not such - I know because I worked in that domain for some time, in the 1980's and again in the 2000's. Java's biggest issue is garbage collection - it is decidedly non-deterministic. IE, it can have SERIOUS impacts upon performance, and at times that cannot be pre-determined. In my experience (30+ years), if you need consistency and speed, then you do not select Java for your environment - and I do a LOT of Java development. I just don't use it when I need the software to have a small footprint, be fast, and stay out of the way of other system processes.
Manual memory management is a whole class of bugs that Java doesn't have to deal with. A good C/C++ programmer shouldn't have *that* much difficulty, but it does add to debugging costs and detracts from mental focus on the algorithmic aspects of the problem.
"Wouldn't a C++ programmer generate an applicable program effectively as quickly as a Java programmer?"
No, some languages simply are slower to develop with and debug. The problem is also made worse depending on the frameworks and IDEs available. As an example, you're going to get your work done way quicker writing an application manipulating dates and times using C# and Visual Studio than you are Java and Eclipse because until Java 8 Java's date time functionality is shit and Eclipse is a dog slow IDE. With Java 8 and say NetBeans or JDeveloper though things will be pretty similar.
At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java. C/C++ give you more scope for optimisation and more control over memory management as a benefit of that though. It's about trade-offs and figuring out what matters.
But it's possible to be great at both C++ and Java without having to descend into petty arguments as to which is better and know when to use each in response to a specific task, and that's the sort of great programmer these institutions will be looking for and this is really what they're talking about - both have their place but in some cases getting a trading application to market a day earlier than the competition even at slight latency trade-off may be enough to net your company a few million dollars advantage. In other cases, the latency improvements of a highly optimised C++ application may instead be the key to scooping up those extra millions.
Ram is cheap, and you have control over the max heap size on your java call, so I don't know why you think disk swapping is a problem. Not only that, but garbage collection can actually speed you up in some cases: if you can defer memory management from a time when it would slow you down to a time when you're sitting on free time anyway (waiting for io, etc), you are actually sometimes better off.
Some people also work on multi-million line projects which compile in a minute or less. It's a trade-off. Most compiler issues are related to obscene (and generally unnecessary) amounts of header dependencies. 1000 lines takes an almost immeasurably small period of time to compile, but a 2 line file can take a minute if it includes the wrong headers. The ISO C/C++ library or boost is pure evil in these regards.
A well formed program can compile extremely rapidly. A poorly formed program can often compile extremely quickly with enough CPUs working in parallel across a fast enough network. However a decent program will take time for the initial compile but take almost no time for each consecutive compile when only a file here or there changes.
Also, to make things politically incorrect on Slashdot, good tools like Visual C++ 2012 can even take poor code and make it compile quickly if the projects files are designed well. Do your coding there and compile it later on GCC.
Really.
Meh.
http://benchmarksgame.alioth.debian.org/u64/java.php
Or not. Benchmarks are a game and everyone likes to play, except the users of brainfuck for whom creating a program which completes is essentially winning.
The thing is people have been trumpeting "blah is as fast as C/C++/Fortran" since nearly the inception of C and C++ and probably shortly after the inception of FORTRAN.
I think there is a kernel of truth in that the reason C, C++ and FORTRAN/Fortran are always the targets is because they are almost always either the fastest or with a few percent of the fastest.
When people start comparing execution speeds of Haskell, Python, C#, C++, $NEWLANGUAGE etc consistently against Java then I'll believe that Java is indeed king of the hill.
SJW n. One who posts facts.
Uh since around 1.3 the JIT optimization for java has led to blindly fast code containing optimizations which are not even available to C++ . Dynamic compiling allows for branch prediction to be more accurately, unlike malloc, the GC knows where to look for free memory and returns it from the last bit of memory you just requested, if you know where pointers are pointing at compile time, you can put them in registers. C++ and other statically compile languages don't have this information, so it stores them in cache, but the JIT can acquire this information and store it in a register. It's the difference between a register to register test and reading from disk.
There are tons of other stuff like this. I don't have it committed to memory, and compiler technology is not my thing but if you look around you'll see that actually GC and JIT are theoretical advantages in terms of speed and those advantages are being realized. It can even figure out what chip it's being run on at runtime and optimize the code for that chip.
The Java is Slow Meme is left over from 1995 before there was even HotSpot.
Not bashing any other language here. C# could also avail itself of these advantages.
Leaving aside my political beliefs that high speed trading needs to be banned, let me say this.
Switching to Java should yield similar results to C++. What matters is whether the programmer understands the memory architecture of the run-time environment well enough to not have issues. Generally, you'll find that even the best programmers in either language will overlook things like garbage collectors and memory fragmentation issues. It's a time-to-market thing. When working with large dynamic data sets, it doesn't matter if you're using Java or C++, the developer needs to be able to adapt their code to perform well on the system.
Code written without considering the processing time of memory management will probably work much better in C++ than Java. That said for huge data sets, Java could perform better since the memory itself is location independent and it is highly probable that you're gain a great deal of performance from being able to defragment memory. Consider however that the garbage collector and the defragmenter will have unpredictable times which can cause multi-millisecond hiccoughs during processing.
I recommend if you take this route, you hire a compiler geek to work on staff optimizing the memory operations.
These are HFT folks, they have essentially unlimited money.
Unlimited money solves a lot of problems.
But not, apparently, the problem of an ongoing obsession with making even more money.
systemd is Roko's Basilisk.
I'm not a Java user, so I've never directly tuned for things like GC, nor do I interact with it directly. Warm up is a different story.
I interact with quite a few exchanges (over all kinds of protocols). Most are, unsurprisingly, written in Java. Almost all of them perform terribly at the beginning of the week. The issue is a standard one: the JVM hasn't JITted important code paths, and it won't until several thousand requests come in. For a standard throughput-oriented program, this doesn't matter -- the total time wasted running interpreted code is small. For a low-latency network service, it's a different story entirely: all of this wasted time happens exactly when it matters.
The standard fix seems to be to write apps to exercise their own critical paths at startup. This is *hard* when dealing with front-end code on the edge of the system you control. Even when it's easy, it's still something you have to do in Java that is entirely irrelevant in compiled languages.
If JVMs some day allow you to export a profile of what code is hot and re-import it at startup, this problem may be solved. Until then, low-latency programmers should weigh the faster development time of Java with the time spent trying to solve annoying problems like warm-up.
This. I've actually personally witnessed a case where a java test application outperformed a (roughly) equivalent C++ application by a startling amount. The two applications were more of a benchmark program than a real app, since it was only testing the timing characteristics of allocating many thousands of objects on the heap, discarding some of them, allocating more, then discarding more, and so on.... The C++ version used the default new and delete operators, while the java version just used the normal new keyword and left the discarding of memory of old objects to the GC. The Java version outperformed the C++ version by more than a factor of 10. Altering the C++ version so that it used custom new and delete operators for the objects, pooling unused objects instead of always just returning them to the heap as the default delete operator did, the C++ version of the application sped up considerably, outperforming the Java version only slightly over a period of about one billion allocations. In consideration of this experience, I think that one is left with answering the question for themselves of whether spending the extra time to specifically optimize a C++ program is truly worth the marginally improved performance. Sometimes it might be... but sometimes it won't.
File under 'M' for 'Manic ranting'
Use RAII consistently, and use containers (from stl or otherwise) which have asserts() on bounds-checking. Bonus points for a tiny unit-test (which can therefore run at the end of every compilation). You'll be amazed at how stable, maintainable, easy to debug and performant your code will be.
Do the hardcore pointer handling only where the profiler tells you that it matters and there's no way java even gets close in performance
Honestly, when your task is, "Grab data from mysql, dump it to screen." There really is nothing faster than coding it in PHP. Its quick, its easy, and it gets the job done as fast as a shell script.
Garbage collection creates a different class of problems, namely that the performance characteristics of your program become non-deterministic. This is a Big Deal for certain classes of applications such as video games and in particular HFT. Would you like to be the person explaining why the GC caused your program to stall at an inopportune time for 50ms and lost somebody a few million bucks?
And you'll notice even on your linked counterexample there are 5/11 examples where Java is within the margin of error on CPU time.
Yeah and in the rest C++ is quite definitively faster. In other words as I claimed, C++ is either the fastest or within a small percentage, across the board. There are precious few examples where C or C++ is substantially outperformed by another language (and in almost all of those precious few examples, it's Fortran) and plenty of examples against just about any other languages where it is a clear winner.
Take the time to go back and read the link I posted. "Java vs C" goes back and forth depending on the algorithm in question and the cache characteristics of the target platform.
I did though the one I posted has much more recent results and no dead links. All the benchmarks are similar. In some cases C++ is slightly otuperformed by Java, in other cases C++ substantially outperforms Java.
Mostly it doesn't matter.
But when people routinely benchmark against Java rather than C++, then I'll accept Java as the faster language.
SJW n. One who posts facts.
It's hardly a weird world.
Water leaks into basements. Many times, the solution is to use a sumppump to pump the water out rather than a very costly rebuilding of the house and surrounding terrain.
I've bought many products in my life (just recently a car tire pump) that tells you not to run it for more than 15 minutes. It might over heat.
Many people have cars that start leaking oil as they age. They don't spend the money to fix it as long as they can just add enough oil to keep it going.
And who hasn't used duct-tape to seal leaks?
It's hardly a weird world out there.
Any JVM that would run this code will be tunable, including the ability to tune the GC so it becomes more deterministic. The fact that your desktop app runs in 'use all memory then GC' does not mean that is the only way the JVM can work.
Use a deterministic garbage collector.
The article suggest that one solution is to simply not do GC until the end of the trading day. I have to admit that's a good pragmatic solution, certainly so for HF prop traders.
Not enough memory on your servers? Add more. Still not enough? Add even more. The cost would be a pittance for any prop trading company worth its salt.
In the course of every project, it will become necessary to shoot the scientists and begin production.
This struck me as well. In the original article, they put a bound of 100 milliseconds as counting as "low latency" in their book, but Wikipedia's article on algorithmic stock trading puts it in the microseconds instead:
"Low-latency traders depend on ultra-low latency networks. They profit by providing information, such as competing bids and offers, to their algorithms microseconds faster than their competitors."
https://en.wikipedia.org/wiki/Algorithmic_trading#Low-latency_trading
100 milliseconds is about 10,000 times too slow, even if we are talking in the neighborhood of "tens of microseconds faster", rather than just "microseconds faster".
Here's an article from 2011 where they justified spending $300M ($0.3B) on a new transatlantic cable to eke out an extra 6 milliseconds: http://www.telegraph.co.uk/technology/news/8753784/The-300m-cable-that-will-save-traders-milliseconds.html That's just 6% of the 100 milliseconds that these Java guys are willing to waste running Java instead of C code.
Garbage Collection with a bad implementation this is true. But garbage collectors do not need to cause non-determinism; they can be used even in real time systems. The problems is that people re-invent the wheel and don't learn from all the research in the past that sped GC up. Or more likely, the system was originally designed to be a simple quick-and-dirty scripting language but its scope has grown so that now the hastily written GC is unsuitable for the new requirements.
To be fair, good GC is hard, and it is extremely difficult to do it portably. Some good GC techniques need hooks into the operating system (ie, you want to mark pages as clean or dirty). Doing this portably sometimes means you see implementations that sit GC up on top of malloc, so that GC is merely a way to eliminate manual allocating and freeing, and you end up with performance headaches with memory fragmentation.
Yes it's true, if you use a malloc and free directly and no GC, you can still end up with non-deterministic allocation performance! Especially with a naive malloc I see used a lot on small systems, where you just have a linked list of available regions so you have a linear search to find regions big enough. And you end up with fragmentation on any system that does not have relocatable memory (ie, most traditional non-interpreted systems), so many embedded systems prefer to use memory pools to fix those problems.
I got this one, boys.
Let me translate what you said to a car analogy, then you can see if your statement still makes sense.
"Your car is unsafe because it was recalled due to a seat-belt issue. You should ride a motorbike like me. It's never been recalled for seat belt issues."
.there is enough of everything for everyone.
At the end of the day with C/C++ you have to deal with memory management and that's just one additional piece of work that you don't have to be so concerned with with Java.
You're funny.
I think we have about a dozen calls to new and free in a few hundred thousand lines of code in our server. The vast majority of memory allocation in C++ is hidden in libraries like STL, which we presume have been debugged.
In Java, on the other hand, you have to be very careful with memory management or you'll either end up pausing for long periods for garbage collection or crashing with out of memory errors. Instead of worrying about whether people remember to call free after allocating something, you end up putting in caches so you can reuse objects rather than reallocate them. We've used some Java libraries designed for high-performance financial uses and they try very hard not to allocate any objects that they'll later need to clean up.
Programmers believing that 'you don't have to worry about memory management in Java' is the reason so many Java programs are a slug-like mass of memory bloat.