Unittest versus pytest

Some differences between them:
1) unittest is standard, it is a built-in module while pytest is a third party module.
2) The syntax of the 2 api is distinct while pytest is able to work with tests using the most of unittest api(It has some limitations especially about the subtest feature)
3) Pytest is more flexible by default.It means that with unittest for some requirements we may need to specify some parameters or configuration and sometimes even add some third-party module and write boilerplate code.
for example some important differences:
– the default test file pattern covers both « test » as suffix and « test » as prefix while by default unittest covers only « test » as prefix
To overcome that with unittest, we need to specify some parameters.
– Generate a report is simpler to do with pytest.
To overcome that with unittest, we need to use a third-party module.
And if we need both : specifying file test pattern and generating a report, it requires still more effort.

My opinion:
pytest provides a very simple API to perform assertions but as drawback it is also more limited while unittest provides a more complete api in terms of functions and features.
For example for sequences assertions is more complete, we can indeed assert the order or not by using either assertEqual() or assertCountEqual() .
The unittest api is more verbose and maybe really painful to configure but personally I find it more python way, but actually in larger projects it is not really the to way use because its integration with ci tools is really bad.
I hope in the future the api will be augmented in terms of easiness of use and features.

unittest

Execute specific test classes

If we mix in the same directory the source code and the test code, it is very simple, but it is also a very bad practice because we don’t want to ship the source code with the test code and also we don’t want to modify by mistake one or the other one.So putting the source code in a specific directory and putting the test code in another directory is advised.
In this not advised way, to execute our tests, we just need to position our working directory in the base directory and to run unittest with as parameter the test file.

If the source directory and the test directory are located inside the same parent directory

Let’s we have this layout for our project containing the source code and the unit tests:


The foo package contains a subpackage bar and each one of these have a python file(module): respectively computer.py and hello.py.
To test these modules,we have test classes these are located in the tests directory and the test filenames are computer_test.py andhello_test.py.
Both the tests directory and the foo directory are located in the same directory: foo_project_example
To execute the tests:
– Go to the common parent directory(foo_project_example).
– Execute unittest like that:
To execute a single test class:
python -m unittest tests.foo.computer_test
python -m unittest tests.foo.bar.hello_test
To execute a list of test classes:
python -m unittest tests.foo.computer_test tests.foo.bar.hello_test

Execute all tests or tests of a package

To be honest, the syntax is not really obvious because we have multiple ways to do that.
In any case the idea is to use the discover subcommand of unittest.
Not that by default we don’t need to explicitly specify this subcommand if we don’t provide any other parameters:
python -m unittest
If we need to specify some other parameters such as the pattern or the starting directory, we need to explicitly specify the discover subcommand.
A very simple way to execute all tests without carrying about where the test directory is located is:
python -m unittest discover -p '*test.py' .

To specify where tests are located, the syntax is more elaborated.
For example if you want to discover tests in the tests directory and with the test file pattern *test.py :
python -m unittest discover -s tests -t . -p '*test.py'
A variant is to choose a specific package to execute by adjusting the starting directory parameter:
python -m unittest discover -s tests.foo.bar -t . -p '*test.py'
python -m unittest discover -s tests.foo -t . -p '*test.py'

Note:
In many cases we may need to specify the top directory (-t) if we specify the start directory (-s) (https://docs.python.org/3/library/unittest.html).

    Test discovery supports namespace packages for the start directory. Note that you need to specify the top level directory too (e.g. <code>python -m unittest
    discover -s root/namespace -t root</code>).

Produce a tests execution report

unittest doesn’t provide any built-in feature to generate a test report.
To achieve that, we have to use a third-party module.
BEWARE: even with that, the ci tools may not be compatible with the report generated(For example with gitlab, it is not compatible),
SO YOU SHOULD REALLY CONSIDER IF UNITTEST IS THE WAY WITH LARGER PROJECTS.
We can use the xmlrunner module(xmlrunner official website).
If our tests follows the unittest default convention for file test pattern, we can just do:
python -m xmlrunner discover
By default, the report is generated in the current working directory and each test file produces a report file. To specify the output directory for the report we can provide the output parameter such as:
python -m xmlrunner -o report discover
If we need to specify the test file pattern, things are more complicated because this third-party module does not provide a parameter to specify a test file pattern as the unittest module does.
To achieve it, we need to define a simple program that will execute all tests by using the xmlrunner configured with unittest as we wish.
For example, we create the tests_not_aggregated_report.py file inside the tests directory :

import unittest
 
if __name__ == '__main__':
    import xmlrunner
 
    testsuite = unittest.TestLoader().discover(start_dir='tests',
                                               top_level_dir='.', pattern="*_test.py", )
    test_results = xmlrunner.XMLTestRunner(output='reports/') \
        .run(testsuite)

And we execute this program by using vanilla python:
python tests/tests_not_aggregated_report.py

If we want to specify the pattern and also to aggregate all test results inside a single file
Here also we define a simple program to execute all tests but besides we specify as output a file with an extension. The module will guess we want to we use the same file for all test reports.
the tests_aggregated_report.py file:

import unittest
 
if __name__ == '__main__':
    import xmlrunner
 
    with open('results.xml', 'wb') as output:
        testsuite = unittest.TestLoader().discover(start_dir='tests',
                                                   top_level_dir='.', pattern="*_test.py", )
        test_results = xmlrunner.XMLTestRunner(output=output) \
            .run(testsuite)

And we still execute this program by using vanilla python:
python tests/tests_aggregated_report.py

pytest

It is really simpler to use.
If we reuse the previous project layout example, we go to the root directory (foo_project_example), then:

To execute specific test located in the tests directory:
python -m pytest tests/foo/computer_test.py tests/foo/test_computer.py
We can also provide a path with a wildcard:
python -m pytest tests/foo/computer_test.py tests/foo/*

To execute all tests located in the tests directory, whatever the ‘test’ file pattern:
python -m pytest tests

To generate a test execution report:
python -m pytest tests --junitxml=reports.xml