Some recent attention in the PHP framework community has been focused on the recent publication of Symfony 2 Preview benchmarks showing that Symfony 2 outperforms Zend Framework by a factor of 3.5. It also outperforms every other benchmarked framework. [...] My fellow Zend Frameworkers, we cannot allow this to stand. [...] I have created the benchmark of benchmarks. Well, to be honest, I only really edited another benchmark. But still, it will prove Zend Framework is faster than everything else out there.

His benchmarks are a little different from the rest with both the usual comparisons between the framework request numbers and a bit of explanation on how the Zend Framework came out on top - he cheated. What it really boiled down to was how the code that was tested was written. His code was optimized in a few different ways (All-in vs the Optimized/More-Optimized/What-The-Fuck-Optimized) and he includes custom benchmarks to show how they differ. What it really boils down to from his perspective is simply put:

Benchmarks. Useless. Final words? Know your framework! All this benchmarking nonsense does little good unless it's plastered with disclaimers.

But benchmarks, for all their decision-making aid, fail under the best of circumstances for one simple reason: they're not real life. Never more is this true than in Joe Stagner's blog post on whether Windows or Linux, and PHP or ASP was faster. [...] Benchmarks come loaded with all sorts of problems. It doesn't matter if it's Microsoft doing them or Apple doing them; they don't mimic real-world conditions, and any number of factors affect how the benchmarks are rendered.

Keeping these things in mind, Brandon looks at some of the issues he found with how the benchamrks were run, the largest of which was comparing the ASP.NET results (with byte code caching) to a PHP install without APC enabled.

mysqlnd saves memory. It consumes half as much memory as libmysql. This is what we have been convinced of. This is what we taught you. Then I tried to test it and made Andrey get nervous for a few hours'¦ Meanwhile he is fine again and we can announce: mysqlnd saves memory, not only in theory, we tested it - we can proof it, can we?

The proof comes in the form of a few "tricks" and some of the settings that the software can use to optimize buffer sizes. Benchmarks (and the code to run them) are also included to show what the differences are between mysqlnd and libmysql.

The Resin high-performance, open source application server features load balancing for increased reliability. Caucho's Quercus(TM) is a fast, 100% Java implementation of the PHP language allowing developers to use PHP flexibility for the web interface and Java for stability.

He also talks about some key points surrounding this server as effects PHP developers:

• Performance
• Development
• Capability
• Security
• Scalability

As such, I have prepared a new set of benchmarks for Cake, Solar, Symfony, and Zend Framework using an almost identical methodology as last time.

The last time is a set of previous benchmarks he ran to test the response times for each of the four frameworks - CakePHP, Solar, Symfony, and the Zend Framework.

The stats have changed a little from the previous results, but overall it's pretty similar - the latest version of each of them shows Solar coming out on top and Symfony as the slowest in its "limitation of responsiveness in a dynamic environment". You can download the log files and the complete report of the tests here.

UPDATE: Paul has also made the code for the benchmark project available to the public on a Google project - http://code.google.com/p/web-framework-benchmarks/.

Included in the list of tests were:

• Reading an integer array
• Parsing variable names
• Splitting out text
• working with a random number generator.

For each of the tests, they have a few different methods of doing the same thing and the results (in milliseconds) plotted out in bar graphs. Some of the results are surprising, but most are expected (if you work with optimizing PHP much at all). It's a good amount of coverage of functionality, though - most aspects of everyday programming are here.