Rasmus mentioned on IRC how he ran the [command line] tests: simply renaming the ".phpt" test files as ".php", and invoking them through the CLI webserver. The SKIPIF sections get executed, but that doesn't affect the desired outcome of testing multiple HTTP requests with different input scripts. [Here] are some quick shell scripts I put together to automate testing the OCI8 extension with valgrind.

He uses the OCI8 extension as an example, showing how to set up these scripts to enable the execution of the tests, fire up the web server and execute Valgrind to help monitor the memory of the execution.

A question I am asked on a regular basis is, how you can test a file upload with PHP. In this blog post, I take a precise look at this topic and show you how to test your file uploads from within your standard testing environment, using widely unknown testing framework for PHP.

He shows how to use a custom $_FILES superglobal to mimic the upload process noting, however, that this won't work due to possible file handling on the backend. His alternative is to use a phpt test to push a raw posted file to the application and then check the results. He then shows how to take these functioning tests and drop them back into PHPUnit via it's "PhpTestCase" handling. You can find full code examples here.

The beauty about PHPT is that you need to know very little other than writing PHP code. A little knowledge into the inner workings of PHP will of course help you in finding areas of code that need testing, and how best to test them, but just knowing PHP is enough to start.

He leads you through a five step process that'll have you up and writing tests in no time - setting up your environment, looking for something to test, writing and executing a test and submitting the results back to the PHP project. He sprinkles in a few code bits and screenshots to help point you in the right direction.

In preparation I decided it was high time to sit down and figure out what testing PHP is all about. People kept telling me it was easy but I had no clear picture of how all the pieces went together - there are different ways of doing the same thing and although I have been keen to get involved with testing for some time, I haven't been able to get started until now.

She looks at the automated tests as a part of the build ("make test") and some of the screens from the lcov testing results. She also recommends reading up on the phpt documentation to help you get going.

Last year's TestFest was a huge success. The worldwide initiatives by usergroups and individuals gave a nice addition to the code coverage for PHP itself. This year, the TestFest period has been extended to 3 months, starting the beginning of this month and ending end of june. But a nice bunch of European usergroups including the Dutch usergroup are combining TestFest on may 9th!

TestFest events are happening all over the world - you can see if there's one near you on this page of the PHP.net wiki. For those attending php|tek this year, there'll also be a TestFest going on during the Hackathon event (read our interview about the event here).

One of the problems I found when designing the new code for PEAR 1.4.0 (back in the day) was that it was very difficult to determine whether changes would break things. The main problem revolves around the colossal size of the test suite. [...] This is a real problem when trying to develop with any kind of flow. If, after every change, one needs to sit through 35 minutes of tests, one will never develop anything of substance.

What he wanted was an application that could detect only the files modified and tests those with the results put into the code coverage report. To fill the need, he created test-modified.php to run just the phpt tests needed.

Roy Ganor explains in his blog:

Since Michael has just finished implementing the second phase for PHP 5.3 support in PDT, we can now expose unit tests and ask users to add more cases to the code assist tests repository.

His post includes the basic format for the tests (as written in pdtt, a clone of the phpt format) and a clip from the PDT wiki page about why they need them. There's no automatic way to submit them but if you want more information on the project and testing, you can always send them an email to find out more.

In any case, now that work on ext/phar has shifted primarily to maintenance mode, and namespaces are finally ancient history, I've shifted all of my coding energy to getting Pyrus, PEAR's next-generation installer, ready to ship.

Pyrus is the PEAR installer as rewritten for the next major PHP release (5.3) and uses a lot of the new features it offers (including full archive support, SQLite 3, combined configuration files and several new developer-centric additions). He also includes a sample bit of code that he worked up to run code coverage reports against the PEAR packages. He includes links to three different examples of the report's output.

The test system therefore produces a bunch of files, a file containing the expected output, one containing the actual output and a diff between these as relevant files. The problem there is that the diff, for being portable, is using a quite simple mechanism which only shows the lines which differ without any context.

[...] Lately I've changed my way of working and use vim less, I still use it, but I use NetBeans as an IDE more and more. So I thought a bit about that test issue and searched my brain for my Java skills and started playing around to see whether I manage to write a NetBeans plugin which can run the tests and report the results in a usable way.

Hes created a project for the plugin (where you can download the latest version - 0.6.0) and install it to your local IDE copy. It adds a toolbar icon, asks for the location of the binaries to test and runs the diff quickly and easily. You can see a screenshot of the tool in action here.

It's absolutely amazing how much you can mess up unit tests just by changing their order! (Trevi_* comes after Tilllate_*) Everyone knows that tests are supposed to be independent. But we all know how it is.

He asks for recommendations on what to do to help the situation. Comments on the post (including ones from Lukas Smith and Sebastian Bergmann) mention using PHPT, a new version of PHPUnit that will do just what he wants and whether or not to use Singletons.