systest - System test framework.¶
Installation¶
pip install systest
Description¶
Execute a sequence of test cases in serial and/or parallel.
Test cases in a list are executed in serial and test cases in a tuple are executed in parallel, in separate Python threads.
This framework is created with integration/system test in mind. The
framework is not intended as a replacement for unittest
, but
rather to be a complement to it.
Documentation: http://systest.readthedocs.org/en/latest
Example usage¶
See the test suite: https://github.com/eerimoq/systest/blob/master/tests/test_systest.py
For example, the sequence below starts with test case
MyTestCase("1")
. When MyTestCase("1")
has been executed,
MyTestCase("2")
and the list of MyTestCase("3")
and
MyTestCase("4")
are executed in parallel. When both
MyTestCase("2")
and the list of MyTestCase("3")
and
MyTestCase("4")
has been executed, MyTestCase("5")
is
executed. Then the sequence ends.
import logging
import systest
LOGGER = logging.getLogger(__name__)
# Define a testcase.
class MyTestCase(systest.TestCase):
"""Test case description.
"""
def __init__(self, name):
super(MyTestCase, self).__init__()
self.name = "my_testcase_" + name
def run(self):
LOGGER.info("Hello!")
self.assert_equal(1, 1)
self.assert_true(1 == 1)
self.assert_in(1 in [1, 2])
self.assert_none(None)
with self.assert_raises(RuntimeError) as cm:
raise RuntimeError('foo')
self.assert_equal(str(cm.exception), 'foo')
main("my_sequence",
MyTestCase("1"),
(
MyTestCase("2"),
[
MyTestCase("3"),
MyTestCase("4")
]
),
MyTestCase("5"))
The output is:
Name: my_sequence
Date: 2016-02-02 18:42:40.446213
Node: erik-VirtualBox
User: erik
---------------------------------------------------------------
Name: my_testcase_1
Description:
Test case description.
Hello!
my_testcase_1: PASSED in 0m 0s
---------------------------------------------------------------
Name: my_testcase_2
Description:
Test case description.
Hello!
my_testcase_2: PASSED in 0m 0s
---------------------------------------------------------------
Name: my_testcase_3
Description:
Test case description.
Hello!
my_testcase_3: PASSED in 0m 0s
---------------------------------------------------------------
Name: my_testcase_4
Description:
Test case description.
Hello!
my_testcase_4: PASSED in 0m 0s
---------------------------------------------------------------
Name: my_testcase_5
Description:
Test case description.
Hello!
my_testcase_5: PASSED in 0m 0s
---------------------- Test summary begin ----------------------
[
[
my_testcase_1: PASSED,
(
my_testcase_2: PASSED,
[
my_testcase_3: PASSED,
my_testcase_4: PASSED
]
),
my_testcase_5: PASSED
]
]
Execution time: 0m 0s
Result: PASSED (passed: 5, failed: 0, skipped: 0, xpassed: 0, xfailed: 0)
----------------------- Test summary end -----------------------
Classes¶
-
class
systest.
TestCase
(name=None)[source]¶ Base class of a test case executed by the sequencer.
-
run
()[source]¶ The test case logic. This function is called be the sequencer to execute the test case.
A test case can either pass or fail. It fails if any exception is raised, otherwise it passes.
-
dry_run
()[source]¶ Called by the sequencer for a dry run execution. Should return the estimated execution time of the test case.
-
assert_text_equal
(first, second)[source]¶ Raises an exception if
first
andsecond
are not equal.This is equivalent to
assert_equal
except it requires the arguments to be multi-line strings. The description of the failure is presented in the exception as a diff. This is an easier way to determine what has gone wrong in multi-line text.
-
assert_in
(member, container)[source]¶ Raise an exception if given member member is not found in given container container.
-
assert_not_in
(member, container)[source]¶ Raise an exception if given member member is found in given container container.
-
assert_greater_equal
(first, second)[source]¶ Raise an exception if
first
is not greater than or equal tosecond
.
-
assert_less_equal
(first, second)[source]¶ Raise an exception if
first
is not less than or equal tosecond
.
-
-
class
systest.
Sequencer
(name, testcase_pattern=None, dry_run=False, force_serial_execution=False)[source]¶ -
dot_digraph
()[source]¶ Create a graphviz dot digraph of given test sequence.
The slowest execution path has bold edges.
Use the program
dot
to create an image from the output of this function.dot -Tpng -Gdpi=200 -o mysequence.png mysequence.dot
-
run
(*tests, **kwargs)[source]¶ Run given testcase(s).
Test cases may be grouped into lists and tuples. All test cases in a list are executed in serial and all test cases in a tuple are executed in parallel. Lists and/or tuples may be used in multiple levels in the sequence.
For example, the sequence below starts with test case Test1. When Test1 has been executed, Test2 and the list of Test3 and Test4 are executed in parallel. When both Test2 and the list of Test3 and Test4 has been executed, Test5 is executed. Then the sequence ends.
[ Test1(), ( Test2(), [ Test3(), Test4() ] ), Test5() ]
-
-
systest.
main
(name, *tests, **kwargs)[source]¶ Run given tests tests from the command line.
Use console_log_level to set the console log level.
Use file_log_level to set the file log level.
-
systest.
configure_logging
(filename=None, console_log_level=None, file_log_level=None)[source]¶ Configure the logging module to write output to the console and a file. The file name is filename-<date>.log if filename is not None, otherwise the file name is
systest-<date>.log
.Use console_log_level to set the console log level. It is
INFO
by default.Use file_log_level to set the file log level. It is
DEBUG
by default.