How to do unit testing

Overview

Unit tests are one of the greatest advances in programming since object orientation. They allow for a safe development process, refactoring without fear, and can sometimes replace documentation since they illustrate quite clearly what an application is supposed to do. Unit testing can also be used to avoid regression. Refactoring a method can create new bugs that didn't use to appear before. That's why it is also a good practice to run all unit tests before deploying a new realease of an application in production - this is called regression testing. Symfony supports and recommends unit testing, and provides tools for that.

There is no perfect solution for unit testing PHP applications built with symfony. This chapter decribes three solutions, each answering the need only partially. If you have an extensive approach to unit testing, you will probably need to use all the three.

Simple test

There are many unit test frameworks in the PHP world, mostly based on Junit. Symfony integrates the most mature of them all, Simple Test. It is stable, well documented, and offers tons of features that are of considerable value for all PHP projects, including symfony ones. If you don't know it already, you are strongly advised to browse their documentation, which is very clear and progressive.

Simple Test is not bundled with symfony, but very simple to install. First, download the Simple Test PEAR installable archive at SourceForge. Install it via pear by calling:

$ pear install simpletest_1.0.0.tgz

If you want to write a batch script that uses the Simple Test library, all you have to do is insert these few lines of code on top of the script:

<?php
require_once('simpletest/unit_tester.php');
require_once('simpletest/reporter.php');
 
?>

Symfony does it for you if you use the test command line; we will talk about it shortly.

Note: Due to non backward-compatible changes in PHP 5.0.5, Simple Test is currently not working if you have a PHP version higher than 5.0.4. This should change shortly (an alpha version addressing this problem is available), but unfortunately the rest of this tutorial will probably not work if you have a later version.

Unit tests in a symfony project

Default unit tests

Each symfony project has a test/ directory, divided into application subdirectories. If you generated a scaffolding, you might already find a few tests in files labeled like myproject/test/myapp/mymoduleActionsTest.php:

<?php
 
class mymoduleActionsWebBrowserTest extends UnitTestCase
{
  private
    $browser = null;
 
  public function setUp ()
  {
    // create a new test browser
    $this->browser = new sfTestBrowser();
    $this->browser->initialize('hostname');
  }
 
  public function tearDown ()
  {
    $this->browser->shutdown();
  }
 
  public function test_simple()
  {
    $url = '/mymodule/index';
    $html = $this->browser->get($url);
    $this->assertWantedPattern('/mymodule/', $html);
  }
}
 
?>

The UnitTestCase class is the core class of the Simple Test unit tests. The setUp() method is run just before each test method, and tearDown() is run just after each test method. The actual test methods start with the word 'test'. To check if a piece of code is behaving as you expect, you use an assertion, which is a method call that verifies that something is true. In Simple Test, assertions start by assert. In this example, one unit test is implemented, and it looks for the word 'user' in the default page of the module. This autogenerated file is a stub for you to start.

As a matter of fact, every time you call a symfony init-module, symfony creates a skeleton like this one in the test/[appname]/ directory to store the unit tests related to the created module. The trouble is that as soon as you modify the default template, the stub tests don't pass anymore (they check the default title of the page, which is 'module $modulename'). So for now, we will erase these files and work on our own test cases.

Add a unit test

Let's imagine that you need to create a Util class that normalizes a string by removing all special characters. Before even writing the class, write the unit test. For that, add a couple of test cases that illustrate extensively what you expect of the class in a UtilTest.php file (all the test case files must end with Test for Simple Test to find them):

<?php
 
require_once('Util.class.php');
 
class TagTest extends UnitTestCase
{
  public function test_normalize()
  {
    $tests = array(
      'FOO'       => 'foo',
      '   foo'    => 'foo',
      'foo  '     => 'foo',
      ' foo '     => 'foo',
      'foo-bar'   => 'foobar',
    );
 
    foreach ($tests as $string => $normalized_string)
    {
      $this->assertEqual($normalized_string, Util::normalize($string));
    }
  }
}
 
?>

Note: As a good practice, we recommend that you name the test files using the class they are supposed to test, and the test cases using the methods they are supposed to test. Your test/ directory will soon contain a lot of files, and finding a test might prove difficult in the long run if you don't.

Unit tests are supposed to test one case at a time, so we decompose the expected result of the text method into elementary cases. We want the Util::normalize() method to return a lower-case version of its argument, without any spaces - either before or after the argument - and without any special characters. The five test cases defined in the $test array are enough to test that.

For each of the elementary test cases, we then compare the normalized version of the input with the expected result, with a call to the ->assertEqual() method. This is the heart of a unit test. If it fails, the name of the test case will be output when the test suite is run. If it passes, it will simply add to the number of passed tests.

We could add a last test with the word ' FOo-bar ', but it mixes elementary cases. If this test fails, you won't have a clear idea of the precise cause of the problem, and you will need to investigate further. Keeping to elementary cases gives you the insurance that the error will be located easily.

Note: The extensive list of the assert methods can be found in the Simple Test documentation.

Unit tests accessing the database

If you want to include unit tests which need a connection to the database, initialize it in the setUp() method as follows:

public function setUp ()
{
  // initialize the database manager
  $databaseManager = new sfDatabaseManager();
  $databaseManager->initialize();
}

You can then use the database connections and Propel objects just like in actions.

Running unit tests

The symfony command line allows you to run all the tests at once with a single command (remember to call it from your project root directory):

$ symfony test myapp

Calling this command executes all the tests of the test/myapp/ directory, and for now it is only those in our new UtilTest.php set. These tests will not pass and the command line will show:

Warning: main(Util.class.php): failed to open stream: No such file or directory in WEB_DIR/sf_sandbox/test/frontend/UtilTest.php on line 3

This is normal: the Util class doesn't exist yet. Create the Util.class.php now in the lib/ directory:

class Util
{
  public static function normalize($string)
  {
    $n_string = strtolower($string);
 
    // remove all unwanted chars
    $n_string = preg_replace('/[^a-zA-Z0-9]/', '', $n_string);
 
    return trim($n_string);
  }
}

Launch the tests again. This time they will pass, and the command line will show:

$ symfony test myapp
Test suite in (test/myapp)
OK
Test cases run: 1/1, Passes: 5, Failures: 0, Exceptions: 0

Note: Tests launched by the symfony command line don't need to include the Simple Test library (unit_tester.php and reporter.php are included automatically).

Test driven development

The greatest benefit of unit tests is experienced when doing test-driven development. In this methodology, the tests are written before the function.

With the example above, you would write an empty Util::normalize() method, then write the first test case ('Foo'/'foo'), then run the test suite. The test would fail. You would then add the necessary code to transform the argument into lowercase and return it in the Util::normalize() method, then run the test again. The test would pass this time.

So you would add the tests for blanks, run them, see that they fail, add the code to remove the blanks, run the tests again, see that they pass. Then do the same for the special characters.

Writing tests first helps you to focus on the things that a function should do before actually developing it. It's a good practice that others methodologies, like eXtreme Programming, recommend as well. Plus it takes into account the undeniable fact that if you don't write unit tests first, you never write them.

One last recommendation: keep your unit tests as simple as the ones described here. An application built with a test driven methodology ends up with roughly as much test code as actual code, so you don't want to spend time debugging your tests cases...

Simulating a web browsing session

Web applications are not all about objects that behave more or less like functions. The complex mechanisms of page request, HTML result and browser interactions require more than what's been exposed before to build a complete set of unit tests for a symfony web app.

We will examine three different ways to implement a simple web app test. The test has to do a request to a mymodule/index page, and assume that some "mytext" text. We will put this test into a mymoduleTest.php file, located in the myproject/test/myapp/ directory.

The sfTestBrowser object

Symfony provides an object called sfTestBrowser, which allows your test to simulate browsing without a browser and, more important, without a web server. Being inside the framework allows this object to bypass completely the http transport layer. This means that the browsing simulated by the sfTestBrowser is fast, and independent of the server configuration, since it does not use it.

Let's see how to do a request for a page with this object:

$browser = new sfTestBrowser();
$browser->initialize();
$html = $browser->get('uri');
 
// do some test on $html
 
$browser->shutdown();

The get() request takes a routed URI as a parameter (not an internal URI), and returns a raw HTML page (a string). You can then proceed to all kinds of tests on this page, using the assert*() methods of the UnitTestCase object.

You can pass parameters to your call as you would in the URL bar of your browser:

$html = $browser->get('/myapp_test.php/mymodule/index');

The reason why we use a specific front controller (myapp_test.php) will be explained in the next section.

The sfTestBrowser simulates a cookie. This means that with a single sfTestBrowser object, you can request several pages one after the other, and they will be considered as part of a single session by the framework. In addition, the fact that sfTestBrowser uses routed URIs instead of internal URIs allows you to test the routing engine.

To implement our web test, the test_Index() method must be built as follows:

class mymoduleTest extends UnitTestCase
{
  public function test_Index()
  {
    $browser = new sfTestBrowser();
    $browser->initialize();
    $html = $browser->get('/myapp_test.php/mymodule/index');
    $this->assertWantedPattern('/mytext/', $html);
    $browser->shutdown();
  }
}

Since almost all the web unit tests will need a new sfTestBrowser to be initialized and closed after the test, you'd better move part of the code to the ->setUp() and ->tearDown() methods:

class mymoduleTest extends UnitTestCase
{
  private $browser = null;
 
  public function setUp()
  {
    $this->browser = new sfTestBrowser();
    $this->browser->initialize();
  }
 
  public function tearDown()
  {
    $this->browser->shutdown();
  }
 
  public function test_Index()
  {
    $html = $this->browser->get('/myapp_test.php/mymodule/index');
    $this->assertWantedPattern('/mytext/', $html);
  }
}

Now, every new test method that you add will have a clean sfTestBrowser object to start with. You may recognize here the auto-generated test cases mentioned at the beginning of this tutorial.

The WebTestCase object

Simple Test ships with a WebTestCase class, which includes facilities for navigation, content and cookie checks, and form handling. Tests extending this class allow you to simulate a browsing session with a http transport layer. Once again, the Simple Test documentation explains in detail how to use this class.

The tests built with WebTestCase are slower than the ones built with sfTestBrowser, since the web server is in the middle of every request. They also require that you have a working web server configuration. However, the WebTestCase object comes with numerous navigation methods on top of the assert*() ones. Using these methods, you can simulate a complex browsing session. Here is a subset of the WebTestCase navigation methods:

We could easily do the same test case as previously with a WebTestCase. Beware that you now need to enter full URIs, since they will be requested from the web server:

class mymoduleTest extends WebTestCase
{ 
  public function test_Index()
  {
    $this->get('http://myapp.example.com/myapp_test.php/mymodule/index');
    $this->assertWantedPattern('/mytext/');
  }
}

The additional methods of this object could help us test how a submitted form is handled, for instance to unit test a login process:

public function test_Login()
  {
    $this->get('http://myapp.example.com/myapp_test.php/');
    $this->assertLink('sign in/register');
    $this->clickLink('sign in/register');
    $this->assertWantedPattern('/nickname:/');
    $this->setField('nickname', 'testuser');
    $this->setField('password', 'testpwd');
    $this->clickSubmit('sign in');
    $this->assertWantedPattern('/test user logged in/');      
  }

It is very handy to be able to set a value for fields and submit the form as you would do by hand. If you had to simulate that by doing a POST request (and this is possible by a call to ->post($uri, $parameters)), you would have to write in the test function the target of the action and all the hidden fields, thus depending too much on the implementation. For more information about form tests with Simple Test, read the related chapter of the Simple Test documentation.

Selenium

The main drawback of both the sfTestBrowser and the WebTestCase tests is that they cannot simulate JavaScript. For very complex interactions, like with AJAX interactions for instance, you need to be able to reproduce exactly the mouse and keyboard inputs that a user would do. Usually, these tests are reproduced by hand, but they are very time consuming and prone to error.

The solution, this time, comes from the JavaScript world. It is called Selenium and is better when employed with the Selenium Recorder extension for Firefox. Selenium executes a set of action on a page just like a regular user would, using the current browser window.

Selenium is not bundled with symfony by default. To install it, you need to create a new selenium/ directory in your web/ directory, and unpack there the content of the Selenium archive. This is because Selenium relies on JavaScript, and the security settings standard in most browsers wouldn't allow it to run unless it is available on the same host and port as your application.

Note: Beware not to transfer the selenium/ directory to your production host, since it would be accessible from the outside.

Selenium tests are written in HTML and stored in the selenium/tests/ directory. For instance, to do the simple unit test mentioned above, create the following file called testIndex.html:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html; charset=UTF-8" http-equiv="content-type">
  <title>Index tests</title>
</head>
<body>
<table cellspacing="0">
<tbody>
  <tr><td colspan="3">First step</td></tr>
 
  <tr>
    <td>open</td>
    <td>/myapp_test.php/</td>
    <td>&nbsp;</td>
  </tr>
 
  <tr>
    <td>clickAndWait</td>
    <td>link=go to mymodule</td>
    <td>&nbsp;</td>
  </tr>
 
  <tr>
    <td>assertTextPresent</td>
    <td>mytext</td>
    <td>&nbsp;</td>
  </tr>
 
</tbody>
</table>
</body>
</html>

A test-case is represented by an HTML document, containing a table with 3 columns: command, target, value. Not all commands take a value, however. In this case either leave the column blank or use a   to make the table look better.

You also need to add this test to the global test suite by inserting a new line in the table of the TestSuite.html file, located in the same directory:

...
<tr><td><a href='./testIndex.html'>My First Test</a></td></tr>
...

To run the test, simply browse to

http://myapp.example.com/selenium/index.html

Select 'Main Test Suite', than click on the button to run all tests, and watch your browser as it reproduces the steps that you have told him to do.

Note: As Selenium tests run in a real browser, they also allow you to test browser inconsistencies. Build your test with one browser, and test them on all the others on which your site is supposed to work with a single request.

The fact that Selenium tests are written in HTML could make the writing of Selenium tests a hassle. But thanks to the Firefox Selenium extension, all it takes to create a test is to execute the test once in a recorded session. While navigating in a recording session, you can add assert-type tests by right clicking in the browser window and selecting the appropriate check under the Append Selenium Command in the pop-up menu.

You can save the test to a HTML file to build a Test Suite for your application. The Firefox extension even allows you to run the Selenium tests that you have recorded with it.

Note: Don't forget to reinitialize the test data before launching the Selenium test.

A few words about environments

Web tests have to use a front controller, and as such can use a specific environment (i.e. configuration). Symfony provides a test environment to every application by default, specifically for unit tests. You can define a custom set of settings for it in your application config/ directory. The default configuration parameters are:

test:
  .settings:
    # E_ALL | E_STRICT & ~E_NOTICE = 2047
    error_reporting:        2047
    cache:                  off
    stats:                  off
    web_debug:              off

The cache, the stats and the web_debug toolbar are set to off. However, the code execution still leaves traces in a log file (myproject/log/myapp_test.log). You can have specific database connection settings, for instance to use another database with test data in it.

This is why all the external URIs mentioned above show a myapp_test.php: the test front controller has to be specified - otherwise, the default index.php production controller will be used in place, and you won't be able to use a different database or to have separate logs for your unit tests.

Note: Web tests are not supposed to be launched in production. They are a developer tool, and as such, they should be run in the developer's computer, not in the host server.