What is the first word that pops into your head when you think about unit-testing? I’m guessing “Fun”, amiright? Ok maybe not fun. Unless repeatedly slamming your head against a brick wall is your idea of fun. Trying to build comprehensive tests for existing code is an exercise in patience. Like the Olympics of patience. Sometimes when I come up for air and wipe the blood from my head (and wall), I realize that while I was toiling away in the unit-test dungeon, I stumbled on something useful. Maybe it’s a feature I had not explored before, or a solution to a tricky situation. Maybe it’s something that will help inject a little fun into your PHPUnit experience.

Before I start throwing out random ideas, let me say that I really like PHPUnit: lots of knobs, good documentation, active development. PHP is the Yoga instructor of programming languages, but that flexibility means it’s easy to write poor quality code. When code and unit-tests don’t get along, the right answer is to send the code to bed without dinner, and if it’s really bad, ground it for the rest of the week. We can blame the code all we want, but sometimes there is no choice but to tweak the system to find a way to make it all work.

One of the biggest obstacles in testing is state. Things like global variables, static values or class instances can easily create a situation in which a test passes when run alone, but fails when run as a part of a suite. There are other types of states to consider aside from just the disastrous PHP global namespace. Sessions for example. Or the state of data in a test db or on disk. Tests that fail intermittently are hard to debug, and usually it’s due to an overlooked state issue.

Tests should be as insulated and independent as possible, and PHPUnit has a feature called “runInSeperateProcess” that forces each test to run in its own PHP process. Using this feature is not as simple as it sounds. If you have a bootstrap file defined in your phpunit.xml file, and it includes any code that defines a constant, you will get an error about redefining said constant in your tests. What gives? I thought each test runs in its own process? It does, but from what I can determine (using the throw-it-against-the-wall method), only the test code itself is run per-process. Assuming that poorly substantiated statement to be true, here is a pattern that does work with process separation.

Process Separation

First the phpunit.xml file, WITHOUT a bootstrap

<phpunit strict="true" colors="true">
    <testsuite name="my_awesome_tests">
<file>./my_awesome_tests.php</file>

Then the my_awesome_tests.php file, with the bootstrap included in the setUp() method and the @runInSeperateProcess and @preserveGlobalState annotations.

<?php

class My_Awesome_Tests extends PHPUnit_Framework_TestCase {

    public function setUp() {
        require 'bootstrap.php';
    }

    /**
     * @preserveGlobalState disabled
     * @runInSeparateProcess
     */
    public function my_test_one() {
    }
    /**
     * @preserveGlobalState disabled
     * @runInSeparateProcess
     */
    public function my_test_two() {
    }
}
?>

Finally, the bootstrap looks something like this

<?php

/* all the things */
error_reporting(E_ALL | E_STRICT);

/* determine current absolute path used for require statements */
define('APP_PATH', dirname(dirname(__FILE__)).'/');

/* get mock objects */
require APP_PATH.'tests/mocks.php';

/* get the code we want to test */
require APP_PATH.'lib/framework.php';

/* get the stubs */
require APP_PATH.'tests/stubs.php';

?>

mocks.php contains stand-in objects used as arguments to methods we want to test. The stubs.php file contains wrappers around abstract classes and traits so we can test them more easily. One advantage of this pattern is it makes it possible to pre-define constants in the setUp() method before the code being tested is loaded, so a test can exercise a code path that triggers on a non-default constant value (Assuming the code being tested checks for an already defined constant). Since mocks are loaded before the code being tested, we can also leverage this to override things unfriendly to testing.

Overriding Stuff

It’s good practice to limit mocking and overriding to a minimum. The more code that is mocked out, the less actual code that is being tested. There are however some built-in PHP functions that simply don’t play nice, like setcookie or header or session_start or error_log or die – you get the idea. Using the pattern as described above in “Process Separation”, we can easily add some override behavior to deal with these problems (sadly this does require changes to the code being tested).

In our mocks.php file we create a class of all static methods for built-in functions that don’t play well with others.

class BuiltIns {
    public static function php_header($header_str) { return true; }
    public static function php_die($msg) { return true; }
    public static function php_error_log($str) { return true; }
}

In the code to be tested, we setup a mirror image of this class that runs the actual built in functions, but only loads if the class is not yet defined.

if (!class_exists('BuiltIns')) {
    class BuiltIns {
        public static function php_header($header_str) {
            return header($header_str);
        }
        public static function php_die($msg=false) {
            return die($msg);
        }
        public static function php_error_log($str) {
            return error_log($str);
        }
    }
}

Then we replace occurrences of these built-in functions in the code to be tested. So this:

if ($error_str) {
   error_log($error_str);
}

Becomes this:

if ($error_str) {
   Builtins::php_error_log($error_str);
}

WOOT! Now we don’t have to worry about an errand error_log spoiling our unit-test party. We can even do something useful in the mocked out versions, maybe a fake session_start() call can populate $_SESSION or another constant in setUp can toggle success or failure from the mocked out function. The sky is the limit people!

Coverage

I have only recently started to look at PHPUnit’s coverage options. When I first tried it out, it bailed with a cryptic message and I was sad. Some head scratching and a few apt-get installs later, I was blown away. The HTML coverage report is incredibly useful. By default it sucks up other included files, so if you are dealing with a big code-base it can be handy to limit coverage to just code you are actively testing. I like to define these limits and enable the coverage report in my phpunit.xml file with something like this:

    <filter>
        <whitelist addUncoveredFilesFromWhitelist="false" processUncoveredFilesFromWhitelist="false">
            <directory suffix=".php">../lib</directory>
            <exclude>
            <file>../lib/not_tested.php</file>
            </exclude>
        </whitelist>
    </filter>
    <logging>
        <log type="coverage-html" target="./coverage.html" charset="UTF-8" highlight="false" lowUpperBound="35" highLowerBound="70"/>
    </logging>

The report is comprehensive. It has summary charts for coverage, complexity, risk analysis, and even line by line detail that makes it brain-dead easy to see what your tests are hitting, and more importantly, what they are missing. Coverage alone does not make a good unit test, but it’s a great tool to help improve your tests.

Extending PHPUnit_Framework_TestCase

This post is really dragging on so I will leave you with one additional trick I came across building unit-tests for a billing system. We wanted a test suite that we could run across a variety of products, but the tests code would be nearly identical. Duplicating the tests for each product was a maintenance nightmare. We needed a way to run virtually the same test code across multiple products. Here is what I came up with:

Start by extending the PHPUnit_Framework_TestCase class with an abstract class. This will be where the actual test code lives.

<?php

abstract class Base_Tests extends PHPUnit_Framework_TestCase {
    public function test_something() {
    /* your normal test code goes here */
    }
}
?>

Next extend that class for each product you want to test, and define the product as a member variable in the setUp() method so it can be accessed from the test method scope:

<?php
class Product_A_Tests extends Base_Tests {
    public function setUp() {
        $this->product = 'product a';
    }
}
?>

PHPUnit will not try to run the tests in the abstract class, but it will find and run the tests in the classes that extend it. For each product you want to test, just extend the base class and define the product details in setUp().

I’m hardly a PHPUnit expert, and there are surely improvements to these ideas or even completely better ways to accomplish the same thing. All of these examples minus the last one were taken from a unit-test suite for some Open Source software I’m working on. You can see the still-in-progress test code at Github, and an example of the coverage report from PHPUnit at jasonmunro.net.