Slashdot Mirror

Yeah, that sucks. This looks like a promising alternative for the time being. http://www.azulsystems.com/pro...

Our memory usage scales with load. Our load scales with usage. Predictions about growth in popularity of our product are all very well, but no excuse for not monitoring for impending doom

Of course. But testing will tell you something like "a single instance with a 32GB heap will support 9000 tx/sec with acceptable 99.9% latency". So you can monitor traffic levels and scale out as appropriate well before something monitoring GCs starts seeing problems. Where I work we deal with request rates in the 100k/s range and so if things go wrong they do so very fast - the trick is to know the limits and stay well away from them!

(especially since we have some legacy code that doesn't scale horizontally and so we have to keep throwing more memory at the problem for those services until we can fix that).
Oh fun :) Be wary of getting too big. I'm a JVM fan but if you start going above 100GB you need to be careful - GC pauses can start getting extremely significant and tuning new/eden becomes very important. Over 200GB and you're bleeding edge. If you have the budget look at Azul - their stuff is amazing.

It isn't that insane. Instead of remapping one page at a time the queue up a number of changes and commit them all at once. This is because on stock Linux every page table re-map creates a lot of cache coherency traffic to make sure all the processors know of the new mapping. By committing them all at once they only need one round of that.

http://www.azulsystems.com/sites/default/files/images/c4_paper_acm.pdf

Section 5.1 is where it starts going into some details on that.

There is a very efficient, hardware-assisted Java runtime available from Azul, but that pretty much just proves your point. You need dedicated hardware to make Java scream.

Modern C++, if you're not dumb about how you use it, lets you avoid all of the C's unsafety, automagically, and it can enforce many safety constraints for you at compile time, too. I don't really understand why anyone writing big, scalable server applications would want to use Java when running the same stuff on C++ will cost you less in datacenter power & cooling.

The execution of C/C++ is faster than Java almost always. The development time of Java is faster than C/C++ almost always.

HFT is now evolving. It's no longer about microseconds but it's more about smarter algorithms. Instead of making 1 penny on a hundred million trades per hour, it's about making $1.00 per million trades per hour. Coding an algorithm sitting next to the Math Ph.D. who came up with it is more important to get it out to market in hours rather than days. You only have a small time period until your algorithm is discovered and opposing HFT machines can re-code their systems to guard against it. Sure, Java will use tons more memory but buying 1TB of ECC RAM is pocket change for these HFT shops. The code will execute slightly slower, but these shops can now deploy their own custom algorithms, going from concept to code to production in a matter of hours. It's much harder to do that with C/C++ because the code is so verbose and it simply takes longer to write equivalent code. In addition, all of the good low level C/C++ devs are paid mid 6-digits or better ($400-600k). It's cheaper and easier to get an expert level Java guy from the internal IT dept for $150k and sit him between an algorithms guy and a trader on the desk than trying to lure a good C++ guy from another shop and dealing with the slimy recruiters and the fraud candidates. You also have the JVM benefits (memory management, type safety,etc.)

Lastly, everybody here is missing the point talking about Java performance. This isn't your stereotypical enterprise data center running a line-of-business app to process invoices. These guys lay out hundreds of millions of dollars just for their hardware/infrastructure budget. Most of these HFT shops have their own custom JVM implementations, made for pauseless GC in low-latency situations along with their own customized Linux kernels or hardware-assisted JVMs. The better shops have their own custom networking stack and protocols. The best of the best run on their own hardware that was designed with customized CPUs for their servers. The future is using these custom fabs to produce RISC-based chips with the customized JVM integrated onto a chip in the form of an ASIC. Throw in gigantic L1 and L2 CPU caches so the instruction code itself is always executed on the chip itself. It's not something commercially viable and has failed in the past, but for a very specific market (HFT for example) you can make Java code scream. Yes, with enough money you can make an ultra-optimized Java implementation run faster than stock C/C++. Source: my career in finance

Azul Zing and their Pauseless and C4 collector algorithms are leading the pack of pause-free JVM GC options at the moment..

http://paperhub.s3.amazonaws.com/d14661878f7811e5ee9c43de88414e86.pdf

http://www.azulsystems.com/

``This is simply an exercise in massively parallel CPU design, not an effort to make a faster CPU. That's why they are shipping them to researchers, so they can study and learn how to develop uses for such massively parallel systems.''

Perhaps it would be interesting to mention Azul Systems at this point. They sell systems with 108 to 864 cores, so they may know a thing or two about "massively parallel".

``This is simply an exercise in massively parallel CPU design, not an effort to make a faster CPU. That's why they are shipping them to researchers, so they can study and learn how to develop uses for such massively parallel systems.''

Perhaps it would be interesting to mention Azul Systems at this point. They sell systems with 108 to 864 cores, so they may know a thing or two about "massively parallel".

Here is the URL in case people are interested in checking this out - http://www.caucho.com/resin-4.0/doc/quercus.xtp .
In summary:
It is OpenSource, 100% Java and it brings all the advantages of using a JVM to PHP - performance (JIT), Safety, Scalability (clustering/load balancing), quality tools (Development, Profilers). One can use most of the Java technologies in PHP to ease development even further - XA Transactions, JNDI, Connection pooling, object caching for example.

Besides, improving performance of this pure Java PHP implementation ought to be easier than improving the PHP runtime. (Java6 onwards the available tools to debug and optimize Java applications have made significant progress. jmap/jhat , easy heap dumps on OutOfMemory, Object Query Language etc. already come bundled with the JVM and then there are Eclipse and NetBeans GUI profilers.)

Also worth checking out Dr. Cliff Click's extensive Java vs. C performance blog post - http://blogs.azulsystems.com/cliff/2009/09/java-vs-c-performance-again.html .

Ah thanks! that seems to work!! And here is PDF link to the slides, if any body is still following this story - http://www.azulsystems.com/events/javaone_2009/session/2009_J1_HardwareCrashCourse.pdf

The slides can also be found here here on Azul Systems' website (i.e. the company Cliff works at).

Twitter could've been designed to permit horizontal scaling. Regrettably the article didn't say much about this approach. They are improving the vertical scalability of the application by switching to first-class threads (via the JVM), but are they not eventually going to hit the limits for vertical scaling?!

Being on the JVM means they can use tools like azul compute servers depending on their architecture they might be able to use different JVM scaling tricks. Once you are on the JVM you get to cheat with tools like that.

Azul Systems have already thought about this: See the Vega 3 7300 Series for their 864 core SMP Java based server.

Azul Systems already ships a 54-core chip with systems up to 864-way SMP. Not quite thousands, but getting close. Today.

Their page

Up to 864 processor cores and 768 GB of memory designed for highly scalable Java applications:
Vega 3 Series Compute Appliances

Granted, it doesn't run Linux (or if it does, it's kept hidden from the user.)

Granted, it doesn't run Linux (or if it does, it's kept hidden from the user.) But with these awesome specifications, I have to wonder why they don't just sell general purpose computers -- people would port Linux to them, and they'd clean up! Is there something special about their processors that they're good at doing java or what?

Simple, take one of Azul's Java CPUs.

• Corporate America, all military branches and the U.S. Gov. in general for supporting the worst programming OS and platform in the world: MS .NET and the related hodgepodge of glued together tools ever devised under the name of: Software Engineering.
• The U.S. Congress and Senate for allowing the abuse of the 1950 H1B Visa Act. And, for allowing large finanical institutions to outsource and offshore their huge Java based (web-based) business applications to China, Pakistan and India et. al.
• http://www.comptia.org/ what's-up-with-that? A few weeks in some center or school and you have an A+ certification or MCSE! Let's see one of these vocation school Software Engineers design and install a wire-wrap card into an AT bus (EISA if you are up to it)! Let's see just one of these Geek Squad types write a single line of Assembly language and explain in detail what is happening on the registers and memory while the program is executing?
• I was an ANSI C programmer for 7 years in a heavy duty Research Facility. And, Java saved my programming career. I would have quit by now with all the frustrations that are allowed in C++ and ANSI C. C is a great tool for Systems programming and smaller projects but the ever bigger Enterprise level applications are still written in Java and its FrameWorks spawned from Sun Microsystems Java JDK. Java VM and the new JDK concurrent classes and folks like: http://www.azulsystems.com/ are putting a serious implementation into what is possible using Java. I don't see similar inroads in any other language
• Quoting Peter Lin of Jakarta Apache JMeter project:
  I find .NET fine for simple applications that are easily outsourced, but for large enterprise applications it's lacking in most areas.

What we really need is for software to catch up. Luckily some programs like Premiere, Photoshop have supported multiple CPU's for a while now. But games, etc can really benefit from this. Just stick AI on 1 core, terrain on another, etc etc.

Have any onf you people visted Azul's website? This is not an Intel compatible machine.
It is going to only run a Java Virtual Machine so anything written in Java will run on it.
Windows will not run on it. I took some operating system courses in college and the intel
architecture is a huge mess of hideousness of backwards compatibility that luckily only operating system implementers have to deal with. By only running Java these guys get to sidestep the whole mess and focus
on massively optimizing the hardware architecture for running java code.

http://www.azulsystems.com/products/nap.html