Apache Hello World Benchmarks

Joshua Chamas writes "I have been running the Hello World benchmarks for years, and I have finally published the Apache Hello World Benchmarks site based on this data. Most people have a love-hate relationship with benchmark data, but I think its critical information to have whenever choosing what's right for your project. The beauty of these benchmarks is that they are open source, so one can run them easily on their Apache/UNIX system and pry them apart to see what makes them tick!"

Comments

Mason: Very suprising

[LunaticLeo]

I am very suprised about the "slowness" of Mason and it's memory consumption. Mason's advantage is the cacheing of pseudo-compiled components. I wonder if this Benchmark suite is reasonably testing Mason's scalability (does it scale linearly or logrithmicly, or what).

PHP Speed Improvements

[x-empt]

PHP scripts are compiled in run-time. You can speed up PHP significantly by using a "Cache" module that stores precompiled php scripts (compiled ONLY the first time they are requested) in memory that is shared among requests. APC is a great one and is available at: http://apc.communityconnect.com/

Zend (http://zend.com) Also has a number of PHP goodies! Expect some significant speed improvements when using a cache! I highly recommend them!

x

Re:PHP Speed Improvements

[ayafm]

I had installed Zend Cache recently while benchmarking but it seemed to offer no speedup, but I am not sure I actually had it installed correctly because of the lack of apparent difference. Because the amount of code is so small on these tests, I would not be surprised if the caching didn't help, as it seems that it is more geared to larger code bases that would really benefit from this. It may be that a better benchmark in the future for this would create some 10K lines of code and then run that for its output.

I will however give the zend cache another chance in the future, or might wait for the zend engine to become part of the standard PHP release, as they are in alpha for that now.

Results.

[cant_get_a_good_nick]

Wow, we use some tomcat at work, and I'm surprised as hell by those. I always assumed mod_perl was a memory hog.

I wonder why they didn't include JBoss or WebLogic? WebLogic, I can understand - expen$ive... but JBoss is free, it's on sourceforge.

BTW: This comment is echoing in a very empty room...

Re:Results.

[The Mayor]

JBoss typically uses Tomcat (v3 or v4 Catalina) or Resin for serving dynamic web pages. Both Tomcat Catalina (v4) and Resin are included in the benchmark.

Re:Results.

[ayafm]

mod_perl can be not too bad on memory, but it depends what you are doing with it. If you look at the environments that run on mod_perl like Embperl, Apache::ASP, Template Toolkit, HTML::Mason, AxKit, etc, you will see more memory usage than using raw mod_perl itself, because the amount of actual code running is much greater, but then people use these environments because of the greater application services they provide than raw mod_perl handlers, so its a trade off. I have known web sites with 20K to 50K lines of perl/mod_perl code and it scales fine as long as one is proficient with tuning mod_perl applications.

As far as benchmarking other java application environments, I will do so as long as they are easy to set up, and benchmarking is allowed in their evaluation license. For example, I did not benchmark Chilisoft ASP because they have a clause in their license that excludes benchmarking, whereas Resin/Caucho did not. I'll check out JBoss and see if I can get it working.