26.4.
unittest — Unit testing framework¶
Source code: Lib/unittest/__init__.py
(If you are already familiar with the basic concepts of testing, you might want to skip to the list of assert methods.).
To achieve this,
unittest supports some important concepts in an
object-oriented way:
- test fixture
- A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions. This may involve, for example, creating temporary or proxy databases, directories, or starting a server process.
- test case
- A test case is the individual.
See also
- Module
doctest
- Another test-support module with a very different flavor.
-.
The script
Tools/unittestgui/unittestgui.py in the Python source distribution is
a GUI tool for test discovery and execution. This is intended largely for ease of use
for those new to unit testing. For production environments it is
recommended that tests be driven by a continuous integration system such as
Buildbot, Jenkins
or Hudson.
26
Passing the
-v option to your test script will instruct
unittest.main()
to enable a higher level of verbosity, and produce.
26.
Test modules can be specified by file path as well:
python -m unittest tests/test_something.py
This allows you to use the shell filename completion to specify the test module. The file specified must still be importable as a module. The path is converted to a module name by removing the ‘.py’ and converting path separators into ‘.’. If you want to execute a test file that isn’t importable as a module you should execute the file directly instead.
You can run tests with more detail (higher verbosity) by passing in the -v flag:
python -m unittest -v test_module
When executed without arguments Test Discovery is started:
python -m unittest
For a list of all the command-line options:
python -m unittest -h
Changed in version 3.2: In earlier versions it was only possible to run individual test methods and not modules or classes.
26 3.2: The command-line options
-b,
-c and
-f were added.
New in version 3.5: The command-line option
--locals.
The command line can also be used for test discovery, for running all of the tests in a project or just a subset.
26.4.3. Test Discovery¶
New in version 3.2.
Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be modules or packages (including namespace
Note
As a shortcut,
python -m unittest is the equivalent of
python -m unittest discover. If you want to pass arguments to test
discovery the
discover sub-command must be used explicitly..
Changed in version 3.4: Test discovery supports namespace packages.
26.4.4. Organizing test code¶
The basic building blocks of unit testing are test cases — single
scenarios that must be set up and checked for correctness. In
unittest,
test cases are represented by
unittest.TestCase instances.
To make your own test cases you must write subclasses of
TestCase or use
FunctionTestCase.
The testing code of a
TestCase instance should be entirely self
contained, such that it can be run either in isolation or in arbitrary
combination with any number of other test cases.
The simplest
TestCase subclass will simply implement a test method
(i.e. a method whose name starts with
test) in order to perform specific
testing code:
import unittest class DefaultWidgetSizeTestCase(unittest.TestCase): def test_default_widget_size(self): widget = Widget('The widget') self.assertEqual(widget.size(), (50, 50)).
Tests can be numerous, and their set-up can be repetitive. Luckily, we
can factor out set-up code by implementing a method called
setUp(), which the testing framework will automatically
call for every single test we run:
import unittest class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') def test_default_widget_size(self): self.assertEqual(self.widget.size(), (50,50), 'incorrect default size') def test_widget_resize(self): self.widget.resize(100,150) self.assertEqual(self.widget.size(), (100,150), 'wrong size after resize')
Note
The order in which the various tests will be run is determined by sorting the test method names with respect to the built-in ordering for strings.
If the
setUp() method raises an exception while the test is
running, the framework will consider the test to have suffered an error, and
the test method will not be executed.
Similarly, we can provide a
tearDown() method that tidies up
after the test method has been run:
import unittest class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') def tearDown(self): self.widget.dispose()
If
setUp() succeeded,
tearDown() will be
run whether the test method succeeded or not.
Such a working environment for the testing code is called a fixture.
Test case instances are grouped together according to the features they test.
unittest provides a mechanism for this: the test suite,
represented by
unittest‘s
TestSuite class. In most cases,
calling
unittest.main() will do the right thing and collect all the
module’s test cases for you, and then execute them.
However, should you want to customize the building of your test suite, you can do it yourself:
def suite(): suite = unittest.TestSuite() suite.addTest(WidgetTestCase('test_default_size')) suite.addTest(WidgetTestCase('test_resize')) return suite.
26, with optional set-up and tear-down methods:
testcase = unittest.FunctionTestCase(testSomething, setUp=makeSomethingDB, tearDown=deleteSomethingDB).
26.4.6. Skipping tests and expected failures¶
New in version 3.1.¶.
Skipped modules will not have
setUpModule() or
tearDownModule() run.
26.4.7. Distinguishing test iterations using subtests¶
New in version 3.4.
When some of your tests differ only by a some very small differences, for
instance some parameters, unittest allows you to distinguish them inside
the body of a test method using the
subTest() context manager.
For example, the following test:
class NumbersTest(unittest.TestCase): def test_even(self): """ Test that numbers between 0 and 5 are all even. """ for i in range(0, 6): with self.subTest(i=i): self.assertEqual(i % 2, 0)
will produce the following output:
====================================================================== FAIL: test_even (__main__.NumbersTest) (i=1) ---------------------------------------------------------------------- Traceback (most recent call last): File "subtests.py", line 32, in test_even self.assertEqual(i % 2, 0) AssertionError: 1 != 0 ====================================================================== FAIL: test_even (__main__.NumbersTest) (i=3) ---------------------------------------------------------------------- Traceback (most recent call last): File "subtests.py", line 32, in test_even self.assertEqual(i % 2, 0) AssertionError: 1 != 0 ====================================================================== FAIL: test_even (__main__.NumbersTest) (i=5) ---------------------------------------------------------------------- Traceback (most recent call last): File "subtests.py", line 32, in test_even self.assertEqual(i % 2, 0) AssertionError: 1 != 0
Without using a subtest, execution would stop after the first failure,
and the error would be less easy to diagnose because the value of
i
wouldn’t be displayed:
====================================================================== FAIL: test_even (__main__.NumbersTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "subtests.py", line 32, in test_even self.assertEqual(i % 2, 0) AssertionError: 1 != 0
26.4.8. Classes and functions¶
This section describes in depth the API of
unittest.
26.4.8.1. Test cases¶
- class
unittest.
TestCase(methodName='runTest')¶
Instances of the
TestCaseclass represent the logical test units in the
unittestuniverse. This class is intended to be used as a base class, with specific tests being implemented by concrete subclasses. This class implements the interface needed by the test runner to allow it to drive the tests, and methods that the test code can use to check for and report various kinds of failure.
Each instance of
TestCasewill run a single base method: the method named methodName. In most uses of
TestCase, you will neither change the methodName nor reimplement the default
runTest()method.
Changed in version 3.2:
TestCasecan be instantiated successfully without providing a methodName. This makes it easier to experiment with
TestCasefrom the interactive interpreter. 3.2. 3.2.
run(result=None)¶
Run the test, collecting the result into the
TestResultobject passed as result. If result is omitted or
None, a temporary result object is created (by calling the
defaultTestResult()method) and used. The result object is returned to
run()‘s caller.
The same effect may be had by simply calling the
TestCaseinstance.
Changed in version 3.3: Previous versions of
rundid not return the result. Neither did calling an instance.
skipTest(reason)¶
Calling this during a test method or
setUp()skips the current test. See Skipping tests and expected failures for more information.
New in version 3.1.
subTest(msg=None, **params)¶
Return a context manager which executes the enclosed code block as a subtest. msg and params are optional, arbitrary values which are displayed whenever a subtest fails, allowing you to identify them clearly.
A test case can contain any number of subtest declarations, and they can be arbitrarily nested.
See Distinguishing test iterations using subtests for more information.
New in version 3 accept a msg argument that, if specified, is used as the error message on failure (see also
longMessage). Note that the msg keyword argument can be passed to
assertRaises(),
assertRaisesRegex(),
assertWarns(),
assertWarnsRegex()only when they are used as a context manager.
assertEqual(first, second, msg=None)¶
Test that first and second are equal. If the values do not compare equal, the test will fail.
In addition, if first and second are the exact same type and one of list, tuple, dict, set, frozenset or str or any type that a subclass registers with
addTypeEqualityFunc()the type-specific equality function will be called in order to generate a more useful default error message (see also the list of type-specific methods).
Changed in version 3.1: Added the automatic calling of type-specific equality function.
Changed in version 3.2:
assertMultiLineEqual()added as the default type equality function for comparing strings. 3.1.
assertIsNone(expr, msg=None)¶
assertIsNotNone(expr, msg=None)¶
Test that expr is (or is not)
None.
New in version 3.1.
assertIn(first, second, msg=None)¶
assertNotIn(first, second, msg=None)¶
Test that first is (or is not) in second.
New in version 3.1. 3.2.
It is also possible to check the production of exceptions, warnings, and log messages using the following methods:
assertRaises(exception, callable, *args, **kwds)¶
assertRaises(exception, msg=None) and possibly the msg arguments are given, return a context manager so that the code under test can be written inline rather than as a function:
with self.assertRaises(SomeException): do_something()
When used as a context manager,
assertRaises()accepts the additional keyword argument msg. 3.1: Added the ability to use
assertRaises()as a context manager.
Changed in version 3.2: Added the
exceptionattribute.
Changed in version 3.3: Added the msg keyword argument when used as a context manager.
assertRaisesRegex(exception, regex, callable, *args, **kwds)¶
assertRaisesRegex(exception, regex, msg=None)
Like
assertRaises()but also tests that regex matches on the string representation of the raised exception. regex may be a regular expression object or a string containing a regular expression suitable for use by
re.search(). Examples:
self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ')
or:
with self.assertRaisesRegex(ValueError, 'literal'): int('XYZ')
New in version 3.1: under the name
assertRaisesRegexp.
Changed in version 3.2: Renamed to
assertRaisesRegex().
Changed in version 3.3: Added the msg keyword argument when used as a context manager.
assertWarns(warning, callable, *args, **kwds)¶
assertWarns(warning, msg=None)
Test that a warning is triggered when callable is called with any positional or keyword arguments that are also passed to
assertWarns(). The test passes if warning is triggered and fails if it isn’t. Any exception is an error. To catch any of a group of warnings, a tuple containing the warning classes may be passed as warnings.
If only the warning and possibly the msg arguments are given, return a context manager so that the code under test can be written inline rather than as a function:
with self.assertWarns(SomeWarning): do_something()
When used as a context manager,
assertWarns()accepts the additional keyword argument msg.
The context manager will store the caught warning object in its
warningattribute, and the source line which triggered the warnings in the
filenameand
linenoattributes. This can be useful if the intention is to perform additional checks on the warning caught:
with self.assertWarns(SomeWarning) as cm: do_something() self.assertIn('myfile.py', cm.filename) self.assertEqual(320, cm.lineno)
This method works regardless of the warning filters in place when it is called.
New in version 3.2.
Changed in version 3.3: Added the msg keyword argument when used as a context manager.
assertWarnsRegex(warning, regex, callable, *args, **kwds)¶
assertWarnsRegex(warning, regex, msg=None)
Like
assertWarns()but also tests that regex matches on the message of the triggered warning. regex may be a regular expression object or a string containing a regular expression suitable for use by
re.search(). Example:
self.assertWarnsRegex(DeprecationWarning, r'legacy_function\(\) is deprecated', legacy_function, 'XYZ')
or:
with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): frobnicate('/etc/passwd')
New in version 3.2.
Changed in version 3.3: Added the msg keyword argument when used as a context manager.
assertLogs(logger=None, level=None)¶
A context manager to test that at least one message is logged on the logger or one of its children, with at least the given level.
If given, logger should be a
logging.Loggerobject or a
strgiving the name of a logger. The default is the root logger, which will catch all messages.
If given, level should be either a numeric logging level or its string equivalent (for example either
"ERROR"or
logging.ERROR). The default is
logging.INFO.
The test passes if at least one message emitted inside the
withblock matches the logger and level conditions, otherwise it fails.
The object returned by the context manager is a recording helper which keeps tracks of the matching log messages. It has two attributes:
records¶
A list of
logging.LogRecordobjects of the matching log messages.
Example:
with self.assertLogs('foo', level='INFO') as cm: logging.getLogger('foo').info('first message') logging.getLogger('foo.bar').error('second message') self.assertEqual(cm.output, ['INFO:foo:first message', 'ERROR:foo.bar:second message'])
New in version 3 3.2: 3.1.
assertRegex(text, regex, msg=None)¶
assertNotRegex(text, regex, msg=None)¶
Test that a regex search matches (or does not match) text. In case of failure, the error message will include the pattern and the text (or the pattern and the part of text that unexpectedly matched). regex may be a regular expression object or a string containing a regular expression suitable for use by
re.search().
New in version 3.1: under the name
assertRegexpMatches.
Changed in version 3.2: The method
assertRegexpMatches()has been renamed to
assertRegex().
New in version 3.2:
assertNotRegex().
New in version 3.5: The name
assertNotRegexpMatchesis a deprecated alias for
assertNotRegex().
assertCountEqual(first, second, msg=None)¶
Test that sequence first contains the same elements as second, regardless of their order. When they don’t, an error message listing the differences between the sequences will be generated.
Duplicate elements are not ignored when comparing first and second. It verifies whether each element has the same count in both sequences. Equivalent to:
assertEqual(Counter(list(first)), Counter(list(second)))but works with sequences of unhashable objects as well.
New in 3.1. 3.1.
assertSequenceEqual(first, second, msg=None, seq_type=None)¶
Tests that two sequences are equal. If a seq_type is supplied, both first and second 3.1.
assertListEqual(first, second, msg=None)¶
assertTupleEqual(first, second, 3.1.
assertSetEqual(first, second, msg=None)¶
Tests that two sets are equal. If not, an error message is constructed that lists the differences between the sets. This method is used by default when comparing sets or frozensets with
assertEqual().
Fails if either of first or second does not have a
set.difference()method.
New in version 3.1.
assertDictEqual(first, second, msg=None)¶
Test that two dictionaries are equal. If not, an error message is constructed that shows the differences in the dictionaries. This method will be used by default to compare dictionaries in calls to
assertEqual().
New in version 3.1.
This class attribute determines what happens when a custom failure message is passed as the msg argument to an assertXYY call that fails.
Trueis the default value. In this case, the custom message is appended to the end of the standard failure message. When set to
False, the custom message replaces the standard message.
The class setting can be overridden in individual test methods by assigning an instance attribute, self.longMessage, to
Trueor
Falsebefore calling the assert methods.
The class setting gets reset before each test call.
New in version 3.1. 3.2..
Changed in version 3.1: In 3.1 this was changed to add the test name to the short description even in the presence of a docstring. This caused compatibility issues with unittest extensions and adding the test name was moved to the
TextTestResultin Python 3.2. 3.1. 3.1.
-.
26.4.8.1.1. Deprecated aliases¶
For historical reasons, some of the
TestCase methods had one or more
aliases that are now deprecated. The following table lists the correct names
along with their deprecated aliases:
Deprecated since version 3.1: the fail* aliases listed in the second column.
Deprecated since version 3.2: the assert* aliases listed in the third column.
Deprecated since version 3.2:
assertRegexpMatchesand
assertRaisesRegexphave been renamed to
assertRegex()and
assertRaisesRegex().
Deprecated since version 3.5: the
assertNotRegexpMatchesname in favor of
assertNotRegex().
26.4.8 may be called several times on a single suite (for example when counting tests or comparing for equality) so the tests returned by repeated iterations before
TestSuite.run()must be the same for each call iteration. After
TestSuite.run(), callers should not rely on the tests returned by this method unless the caller uses a subclass that overrides
TestSuite._removeTestAtIndex()to preserve test references.
Changed in version 3.2: In earlier versions the
TestSuiteaccessed tests directly rather than through iteration, so overriding
__iter__()wasn’t sufficient for providing tests.
Changed in version 3.4: In earlier versions the
TestSuiteheld references to each
TestCaseafter
TestSuite.run(). Subclasses can restore that behavior by overriding
TestSuite._removeTestAtIndex().
In the typical usage of a
TestSuiteobject, the
run()method is invoked by a
TestRunnerrather than by the end-user test harness.
26.4.8 attributes:
errors¶
A list of the non-fatal errors encountered while loading tests. Not reset by the loader at any point. Fatal errors are signalled by the relevant a method raising an exception to the caller. Non-fatal errors are also indicated by a synthetic test that will raise the original error when run.
New in version 3.5.
TestLoaderobjects have the following methods:
loadTestsFromTestCase(testCaseClass)¶
Return a suite of all test cases contained in the
TestCase-derived
testCaseClass.
A test case instance is created for each method named by
getTestCaseNames(). By default these are the method names beginning with
test. If
getTestCaseNames()returns no methods, but the
runTest()method is implemented, a single test case is created for that method instead.
loadTestsFromModule(module, pattern=None. The pattern argument is passed as the third argument to
load_tests.
Changed in version 3.2: Support for
load_testsadded.
Changed in version 3.5: The undocumented and unofficial use_load_tests default argument is deprecated and ignored, although it is still accepted for backward compatibility. The method also now accepts a keyword-only argument pattern which is passed to
load_testsas the third argument..
Changed in version 3.5: If an
ImportErroror
AttributeErroroccurs while traversing name then a synthetic test that raises that error when run will be returned. These errors are included in the errors accumulated by self.errors. the import failure is due to
SkipTestbeing raised, it will be recorded as a skip instead of an error.
If a package (a directory containing a file named
__init__.py) is found, the package will be checked for a
load_testsfunction. If this exists then it will be called
package.load_tests(loader, tests, pattern). Test discovery takes care to ensure that a package is only checked for tests once during an invocation, even if the load_tests function itself calls
loader.discover.
If
load_testsexists 3.2.
Changed in version 3.4: Modules that raise
SkipTeston import are recorded as skips, not errors. Discovery works for namespace packages. Paths are sorted before being imported so that execution order is the same even if the underlying file system’s ordering is not dependent on file name.
Changed in version 3.5: Found packages are now checked for
load_testsregardless of whether their path matches pattern, because it is impossible for a package name to match the default pattern..
failures¶
A list containing 2-tuples of
TestCaseinstances and strings holding formatted tracebacks. Each tuple represents a test where a failure was explicitly signalled using the
TestCase.assert*()methods.
skipped¶
A list containing 2-tuples of
TestCaseinstances and strings holding the reason for skipping the test.
New in version 3.1. 3.2.
failfast¶
If set to true
stop()will be called on the first failure or error, halting the test run.
New in version 3.2.
wasSuccessful()¶
Return
Trueif all tests run so far have passed, otherwise returns
False.
Changed in version 3.4: Returns
Falseif there were any
unexpectedSuccessesfrom tests marked with the
expectedFailure()decorator..
addSubTest(test, subtest, outcome)¶
Called when a subtest finishes. test is the test case corresponding to the test method. subtest is a custom
TestCaseinstance describing the subtest.
If outcome is
None, the subtest succeeded. Otherwise, it failed with an exception where outcome is a tuple of the form returned by
sys.exc_info():
(type, value, traceback).
The default implementation does nothing when the outcome is a success, and records subtest failures as normal failures.
New in version 3.4.
- class
unittest.
TextTestResult(stream, descriptions, verbosity)¶
A concrete implementation of
TestResultused by the
TextTestRunner.
New in version 3.2:=None, descriptions=True, verbosity=1, failfast=False, buffer=False, resultclass=None, warnings=None, *, tb_locals=False)¶
A basic test runner implementation that outputs results to a stream. If stream is
None, the default,
sys.stderris used as the output stream. This class has a few configurable parameters, but is essentially very simple. Graphical applications which run test suites should provide alternate implementations. Such implementations should accept
**kwargsas the interface to construct runners changes when features are added to unittest.
By default this runner shows
DeprecationWarning,
PendingDeprecationWarning,
ResourceWarningand
ImportWarningeven if they are ignored by default. Deprecation warnings caused by deprecated unittest methods are also special-cased and, when the warning filters are
'default'or
'always', they will appear only once per-module, in order to avoid too many warning messages. This behavior can be overridden using Python’s
-Wdor
-Waoptions (see Warning control) and leaving warnings to
None.
Changed in version 3.2: Added the
warningsargument.
Changed in version 3.2: The default stream is set to
sys.stderrat instantiation time rather than import time.
Changed in version 3.5: Added the tb_locals parameter.

run(test)¶
This method is the main public interface to the TextTestRunner. This method takes a
TestSuiteor
TestCaseinstance. A
TestResultis created by calling
_makeResult()and the test(s) are run and the results printed to stdout.
unittest.
main(module='__main__', defaultTest=None, argv=None, testRunner=None, testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, failfast=None, catchbreak=None, buffer=None, warnings=None either the name of a single test or an iterable of test names.
The warnings argument specifies the warning filter that should be used while running the tests. If it’s not specified, it will remain
Noneif a
-Woption is passed to python (see Warning control), otherwise it will be set to
'default'.
Calling
mainactually returns an instance of the
TestProgramclass. This stores the result of the tests run as the
resultattribute.
Changed in version 3.1: The exit parameter was added.
Changed in version 3.2: The verbosity, failfast, catchbreak, buffer and warnings parameters were added.
Changed in version 3.4: The defaultTest parameter was changed to also accept an iterable of test names.
26.4.8.3.1. load_tests Protocol¶
New in version 3.2., pattern)
where pattern is passed straight through from
loadTestsFromModule. It
defaults to in a directory containing a package, either from the
command line or by calling
TestLoader.discover(), then the package
__init__.py will be checked for
load_tests. If that function does
not exist, discovery will recurse into the package as though it were just
another directory. Otherwise, discovery of the package’s tests will be left up
to
load_tests which
Changed in version 3.5: Discovery no longer checks package names for matching pattern due to the impossibility of package names matching the default pattern.
26.4.
26.4.
26.4.
26.4.10. Signal Handling¶
New in version 3.2..
unittest.
removeResult(result)¶
Remove a registered result. Once a result has been removed then
stop()will no longer be called on that result object in response to a control-c.): ...