Pytest Marks: Mark Entire Directory / Package

I am testing several versions of a component using Pytest. Some tests can run on all versions, some are version specific. For example tests | |-- version1_tests | |-- test_featur

Solution 1:

You can register markers to collected tests at runtime using item.add_marker() method. Here's an example of registering markers in pytest_collection_modifyitems:

import pathlib
import pytest


def pytest_collection_modifyitems(config, items):
    # python 3.4/3.5 compat: rootdir = pathlib.Path(str(config.rootdir))
    rootdir = pathlib.Path(config.rootdir)
    for item in items:
        rel_path = pathlib.Path(item.fspath).relative_to(rootdir)
        mark_name = next((part for part in rel_path.parts if part.endswith('_tests')), '').removesuffix('_tests')
        if mark_name:
            mark = getattr(pytest.mark, mark_name)
            item.add_marker(mark)

Write the code to conftest.py in the project root dir and try it out:

$ pytest -m "common or version2" --collect-only -q
tests/common_tests/test_feature_common_1.py::test_spam
tests/common_tests/test_feature_common_1.py::test_eggs
tests/common_tests/test_feature_common_2.py::test_spam
tests/common_tests/test_feature_common_2.py::test_eggs
tests/common_tests/test_feature_common_n.py::test_spam
tests/common_tests/test_feature_common_n.py::test_eggs
tests/version2_tests/test_feature_2_1.py::test_spam
tests/version2_tests/test_feature_2_1.py::test_eggs
tests/version2_tests/test_feature_2_2.py::test_spam
tests/version2_tests/test_feature_2_2.py::test_eggs
tests/version2_tests/test_feature_2_n.py::test_spam
tests/version2_tests/test_feature_2_n.py::test_eggs

Only tests under common_tests and version2_tests were selected.

Explanation

For each collected test item, we extract the path relative to project root dir (rel_path), first part of rel_path that ends with _tests will be used as source for the mark name extraction. E.g. collect_tests is the source for the mark name collect etc. Once we have the mark name, we create the mark (using getattr since we can't use the property access) and append the mark via item.add_marker(mark). You can write your own, less abstract version of it, e.g.

for item in items:
    if `common_tests` in str(item.fspath):
        item.add_marker(pytest.mark.common)
    elif `version1_tests` in str(item.fspath):
        item.add_marker(pytest.mark.version1)
    elif `version2_tests` in str(item.fspath):
        item.add_marker(pytest.mark.version2)

Registering markers

With a recent version of pytest, you should receive a PytestUnknownMarkWarning since the dynamically generated markers were not registered. Check out the section Registering markers for a solution - you can either add the mark names in pytest.ini:

[pytest]
markers =
    common
    version1
    version2

or add them dynamically via pytest_configure hook, e.g.

def pytest_configure(config):
    rootdir = pathlib.Path(config.rootdir)
    for dir_ in rootdir.rglob('*_tests'):
        mark_name = dir_.stem.removesuffix('_tests')
        config.addinivalue_line('markers', mark_name)

Solution 2:

The simplest way is to put __init__.py files in each directory

pytestmark = ['version1']