=================
 Basic API Usage
=================

.. currentmodule:: zope.testrunner

The test runner consists of an importable module. The test runner is
used by providing scripts that import and invoke the `run` method from
the module. The `testrunner` module is controlled via command-line
options. Test scripts supply base and default options by supplying a
list of default command-line options that are processed before the
user-supplied command-line options are provided.

Typically, a test script does 2 things:

- Adds the directory containing the zope package to the Python
  path.

- Calls the test runner with default arguments and arguments supplied
  to the script.

  Normally, it just passes default/setup arguments.  The test runner
  uses `sys.argv` to get the user's input.

This testrunner_ex subdirectory contains a number of sample packages
with tests. Let's run the tests found here. First though, we'll set up
our default options:

    >>> import os.path
    >>> directory_with_tests = os.path.join(this_directory, 'testrunner-ex')
    >>> defaults = [
    ...     '--path', directory_with_tests,
    ...     '--tests-pattern', '^sampletestsf?$',
    ...     ]

The default options are used by a script to customize the test runner
for a particular application. In this case, we use two options:

path
  Set the path where the test runner should look for tests.  This path
  is also added to the Python path.

tests-pattern
  Tell the test runner how to recognize modules or packages containing
  tests.

Now, if we run the tests, without any other options:

    >>> from zope import testrunner
    >>> import sys
    >>> sys.argv = ['test']
    >>> testrunner.run_internal(defaults)
    Running zope.testrunner.layer.UnitTests tests:
      Set up zope.testrunner.layer.UnitTests in N.NNN seconds.
      Ran 156 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer1 tests:
      Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
      Set up samplelayers.Layer1 in N.NNN seconds.
      Ran 9 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer11 tests:
      Set up samplelayers.Layer11 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer111 tests:
      Set up samplelayers.Layerx in N.NNN seconds.
      Set up samplelayers.Layer111 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer112 tests:
      Tear down samplelayers.Layer111 in N.NNN seconds.
      Set up samplelayers.Layer112 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer12 tests:
      Tear down samplelayers.Layer112 in N.NNN seconds.
      Tear down samplelayers.Layerx in N.NNN seconds.
      Tear down samplelayers.Layer11 in N.NNN seconds.
      Set up samplelayers.Layer12 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer121 tests:
      Set up samplelayers.Layer121 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Running samplelayers.Layer122 tests:
      Tear down samplelayers.Layer121 in N.NNN seconds.
      Set up samplelayers.Layer122 in N.NNN seconds.
      Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    Tearing down left over layers:
      Tear down samplelayers.Layer122 in N.NNN seconds.
      Tear down samplelayers.Layer12 in N.NNN seconds.
      Tear down samplelayers.Layer1 in N.NNN seconds.
    Total: 321 tests, 0 failures, 0 errors and 0 skipped in N.NNN seconds.
    False

we see the normal testrunner output, which summarizes the tests run
for each layer. For each layer, we see what layers had to be torn down
or set up to run the layer and we see the number of tests run, with
results.

The test runner returns a boolean indicating whether there were
errors. In this example, there were no errors, so it returned False.

(Of course, the times shown in these examples are just examples. Times
will vary depending on system speed.)
