monoprop

Testing

Running the tests for the Python and C++ API, with or without MPI.

The monoprop test suite requires installing the package from source. If you follow the installation guide in Building from source, you should have all the tools needed to run the tests.

Running the tests

Python tests

The Python tests run with pytest. Without MPI:

just test              # build, then the Python and C++ suites
just test-py           # the Python suite against whatever is installed

This collects two suites: monoprop's own (tests/) and the one belonging to the monoprop-bench-tools workspace member (packages/monoprop-bench-tools/tests/). The member has to be installed for the second to import, which is what the workspace-test dependency group is for:

uv sync --all-groups --all-extras   # or: uv sync --group workspace-test --all-extras

The plain test group covers monoprop's suite alone; it is deliberately free of workspace members so it can be installed against a built wheel in CI.

The MPI tests need an MPI-enabled build, which the default source build does not produce. The just recipes build an MPI-enabled extension and launch the suite under mpiexec:

just test-mpi                  # build with MPI, then every MPI leg CI runs
just test-py-mpi               # the Python MPI legs only, against the installed build

just test-py-mpi runs the tests that use a world communicator (-m mpi) five times: on 1, 2 and 4 ranks; on 3 ranks with monoprop_ROUTING=splitmix, since linear routing refuses a rank count that is not a power of two; and on 2 ranks with monoprop_PARTITIONS=2. Each test times out after 300 seconds, so a hung collective fails instead of stalling the run. To run one file by hand:

mpiexec --map-by :OVERSUBSCRIBE -n 4 uv run --no-sync pytest tests/test_monoprop_mpi.py --with-mpi -m mpi

The suite's tests/conftest.py provides --with-mpi; no MPI pytest plugin is needed. Tests marked with @pytest.mark.mpi are skipped without that option. Use @pytest.mark.mpi(min_size=2) when a test needs at least two ranks. Under --with-mpi, only rank zero reports pytest output; the other ranks print their failures to stderr, tagged [rank <r>].

Under more than one rank, tests/conftest.py gives each rank its own pytest --basetemp, pytest-of-<user>/mpi-rank-<n> under $PYTEST_DEBUG_TEMPROOT when set, else the system temporary directory, so concurrent ranks never prune each other's temporary directories. Each run clears its rank's directory first, and every run by the same user reuses mpi-rank-<n>: run one MPI test run at a time per user, or pass distinct --basetemp values. An explicit --basetemp overrides this and is shared by every rank.

To do it by hand, build with MPI on, then run under mpiexec with --no-sync so each rank reuses that build. uv rebuilds on its own when the MPI setting changes, since the build options are part of its cache key:

monoprop_ENABLE_MPI=ON uv sync --all-extras
mpiexec --allow-run-as-root -n 2 uv run --no-sync python -m pytest tests --with-mpi

monoprop_ENABLE_MPI (or the monoprop.mpi config-setting) also provisions mpi4py as a build input; see Building from source for why. Installing the mpi extra alone only provides mpi4py and does not enable C++ MPI support. Tests marked mpi are skipped whenever monoprop.MPI_ABIS is empty, and the comm fixture then has only its None case. Accordingly, the CI matrix has explicit cache-free MPI build lanes on Linux x86-64, Linux ARM64, and macOS; only those lanes run the suite under mpiexec.

Testing release wheels

Release wheels carry one MPI backend per ABI family, so CI tests them against real MPI libraries rather than the editable build. To do the same locally, point just test-wheel at a built wheel (or a directory of them); it installs the wheel and mpi4py into a fresh venv, checks that the MPI backend loads, and runs the MPI-marked tests under mpiexec and then tests/test_mpi_backend.py:

just test-wheel dist/ wheel            # mpi4py binary wheel against the MPI on this machine
just test-wheel dist/ source           # mpi4py built against the local mpicc
just test-wheel dist/ wheel impi-rt    # extra requirements, e.g. Intel MPI from PyPI

just inspect-mpi <include dir> <abi> runs the configure-time MPI inspection on an installed MPI and checks that it reports the expected ABI family (mpich, openmpi or mpiabi) and meets the version floor.

The CI setup action (.github/actions/setup) takes two inputs for these jobs: build-deps: "false" skips the build dependencies when a job only tests a built wheel, and mpi: mpich installs MPICH instead of Open MPI (mpi: "on").

C++ unit tests

The C++ tests run through CTest. The build tree is produced by uv sync (via scikit-build-core); do not invoke cmake --preset to configure directly, as the project requires scikit-build-core's build environment to configure correctly. Once the tree exists, run the tests with:

uv sync --all-extras
ctest --test-dir build/editable/Release --output-on-failure

With MPI, reuse the MPI-enabled uv sync from the Python MPI section above, then run CTest against its tree. MPI builds use their own tree, Release-mpi, so switching between the variants rebuilds incrementally:

monoprop_ENABLE_MPI=ON uv sync --all-extras
ctest --test-dir build/editable/Release-mpi --output-on-failure

Or simply just test-mpi.

The full MPI rank matrix (several rank counts) is driven by a helper script:

ctest -S tools/ctest-mpi-matrix.cmake -VV

CI gives every compiler and MPI combination a distinct setup-uv cache suffix. uv's cache key covers the build options, but not the compiler selection (CXX).

A matrix job reports one check per combination, so its check names change whenever a lane is added or removed. Each matrix workflow therefore ends with a small aggregation job — All tests in test.yml, All flake checks in nix.yml — that runs with if: always(), depends on the matrix, and fails unless the matrix succeeded or was skipped. Branch protection requires those stable names, so the matrix dimensions can change without touching the ruleset.

CTest runs each Boost case as its own process, so an MPI build's MPI_Init probes every fabric device per case, whether or not the test sends anything. monoprop_TEST_EXCLUDE_MPI_FABRIC=ON (default) skips that probe for the single-process serial variants only; multi-rank variants keep the full component set, since they exchange real messages. Turn it off by adding SKBUILD_CMAKE_DEFINE="monoprop_TEST_EXCLUDE_MPI_FABRIC=OFF" to the MPI build above.

CTest registers every Boost case individually as a serial variant. A case that needs several ranks declares it with a precondition (cpp/tests/RankPreconditions.h); in a serial entry Boost skips it and CTest reports it as skipped. When the build has MPI enabled and a launcher is found, CTest also registers the whole suite once per rank count: linear routing on the counts in monoprop_MPI_TEST_PROCS (default 2;4, labels mpi, mpi-<n>, mpi-linear) and monoprop_ROUTING=splitmix on monoprop_MPI_SPLITMIX_TEST_PROCS (default 2;3;4, labels mpi, mpi-<n>, mpi-splitmix). Each runs one mpiexec because the ranks have to reach the same collectives, allows oversubscription, and times out after 600 seconds. Only monoprop_MPI_TEST_PROCS has a config-setting and an environment variable; monoprop_MPI_SPLITMIX_TEST_PROCS is a CMake cache variable. Set a single rank count with SKBUILD_CMAKE_DEFINE="monoprop_MPI_SPLITMIX_TEST_PROCS=3" monoprop_ENABLE_MPI=ON uv sync --all-extras. SKBUILD_CMAKE_DEFINE splits on ;, so set a list on the configured tree instead: cmake -D monoprop_MPI_SPLITMIX_TEST_PROCS="2;3" build/editable/Release-mpi. Rebuild afterwards, since CTest registers the variants when the test binary is built, not at configure time, and check the result:

cmake --build build/editable/Release-mpi
ctest --test-dir build/editable/Release-mpi -N -L mpi-splitmix

CI runs the non-MPI and MPI labels separately with --no-tests=error, so an MPI lane fails if its MPI variant was not registered. Standalone MPI-enabled C++ test executables initialize MPI before constructing propagators and destroy the propagators before finalizing MPI. Select the groups locally with just test-cpp and just test-cpp-mpi, which are the same commands CI runs. just test-cpp picks the build tree from monoprop_ENABLE_MPI, so run the serial group of an MPI build with monoprop_ENABLE_MPI=ON just test-cpp, as CI's MPI jobs do. just test-mpi runs both C++ groups and the Python MPI legs.

Run just code-coverage to collect coverage in separate serial and MPI-enabled Coverage builds and render monoprop-coverage/index.html. The MPI collection runs the serial and MPI CTest labels plus the MPI-marked Python tests with two ranks. Python ranks write parallel coverage shards, while C++ coverage uses atomic gcov counters. The code-coverage-collect and code-coverage-aggregate recipes also drive the QA workflow, which publishes the combined reports to Codecov and SonarQube. MPI collection requires covered lines in multiple detail/mpi sources so a silently serial build cannot pass as MPI coverage.

Golden baselines

A refactor that is meant to change no result is checked against a captured baseline rather than by reading the diff. tools/capture-baseline.py propagates every tests/data/*.msgpack fixture at a spread of cutoffs and cutoff types, plus a few hand-written qubit problems, and dumps each run's term count, its full (monomial indices, coefficient) set and its expectation value:

just capture-baseline golden   # seed, once, on an unmodified tree
just diff-baseline             # rebuild, re-capture, diff against the golden

Terms are dumped in the engine's own returned order, not sorted, and diff-baseline is a byte-wise diff. That is deliberate: the order runs from SplitmixHash through probe order, MPI owner routing and insertion order into floating-point accumulation order, so a silent reorder is a regression, not an implementation detail.

For a change that reorders terms on purpose, --compare REF CAND --tol holds the term set and the term count exactly and compares coefficients and energies to a relative tolerance instead. Reach for it only when the reordering is the point; it is not a way to wave through a diff that was supposed to be stable.

The capture is rank-local, so under mpiexec each rank writes its own manifest-rank<r>.json. Output lands in .baseline-capture/, which is gitignored.

Adding tests

Python tests

Place new test files under tests/, named test_<module>.py. We suggest following pytest guidelines.

Simple unit test

Use pytest.mark.parametrize for straightforward parametric tests:

import pytest
from monoprop.pauli import Pauli

class TestPauli:
    def test_default_qubits_are_range(self):
        p = Pauli("XYZ")
        assert p.string == "XYZ"
        assert p.qubits == (0, 1, 2)

    @pytest.mark.parametrize(
        ("string", "expected"),
        [
            ("IZ", Pauli("Z", 1)),
            ("ZI", Pauli("Z", 0)),
        ],
    )
    def test_identity_letters_dropped(self, string, expected):
        assert Pauli(string, (0, 1)) == expected

Data-driven integration test

The problem fixture in tests/conftest.py is parametrized over the three general reference problems: LiH, RX/RZ/RY/RZ rotations, and a random circuit. Each invocation loads a fresh problem from tests/data/ using load_problem() in tests/reference_data.py. Its explicit case IDs make failures easy to identify. conftest.py also provides comm: None (serial) and, under --with-mpi with an MPI build, a duplicate of COMM_WORLD. Use it for results every rank agrees on, such as energies and gradients; test rank-local state such as evolved_operator serially, with comm=None.

from monoprop import MajoranaPropagator


def test_energy_matches_reference(problem, comm):
    mp = MajoranaPropagator(
        problem.operator,
        problem.monomial_circuit.initial_state,
        cutoff=2 * problem.n_modes,
        comm=comm,
    )
    mp.propagate(problem.monomial_circuit.to_circuit())
    energy = mp.expectation_value()
    assert abs(energy - problem.exact_expval) < 1e-8

For tests that specifically require the LiH molecule and its commutator data, request lih_problem. For orbital-rotation checks using the S0 molecule, request orbital_rotation_problem.

def test_lih_reference(lih_problem): ...

Use @pytest.mark.parametrize with pytest.param(..., id="...") for small in-memory tables. When each scenario needs substantial or optional-library setup, separate test functions keep the setup next to its assertions and construct objects only when the test runs.

Adding a new msgpack fixture

Add a .msgpack file to tests/data/ following the schema in tests/data/README.md. To exercise it in every test requesting problem, add a pytest.param entry to that fixture's params list in tests/conftest.py:

pytest.param("my_new_problem", id="my_new_problem"),

If the problem is only appropriate for specific tests, define a named fixture instead:

import pytest

from tests.reference_data import load_problem


@pytest.fixture
def my_new_problem(lazy_shared_datadir):
    return load_problem(lazy_shared_datadir / "my_new_problem.msgpack")

C++ tests

C++ tests live in cpp/tests/ and use Boost.Test. They are registered automatically via CMake: new *.cpp files in cpp/tests/ are picked up on the next configure, so no source-list edit is needed.

Simple unit test

#include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_CASE(my_basic_check) {
    int result = 2 + 2;
    BOOST_TEST(result == 4);
}

Data-driven test using reference fixtures

Use the ExampleDataFix fixture class from TestUtilities.h to load the same msgpack data as Python tests, and BOOST_DATA_TEST_CASE_F to parametrize over it:

#include <boost/test/data/monomorphic.hpp>
#include <boost/test/data/test_case.hpp>
#include <boost/test/unit_test.hpp>

#include "TestUtilities.h"

using namespace test_utils;
namespace utf = boost::unit_test;
namespace bdata = utf::data;

BOOST_DATA_TEST_CASE_F(ExampleDataFix,
                       my_new_test,
                       bdata::make(ds_pare_values) ^ bdata::make(ds_schrodinger_enabled),
                       pare,
                       sch_enabled) {
    const auto schrodinger_cutoff = make_schrodinger_cutoff(sch_enabled, cutoff);
    SimulatorConfig cfg{
        .schrodinger_cutoff = schrodinger_cutoff
                                  ? std::optional<unsigned int>(*schrodinger_cutoff)
                                  : std::nullopt,
        .cutoff_type = cutoff_type,
        .basis_change = basis_change,
    };
    // use data.actual_expval as the reference value
    test_evolve_build_graph<n_modes>(data, cfg, pare, data.actual_expval);
}

See also

On this page

Running the testsPython testsTesting release wheelsC++ unit testsGolden baselinesAdding testsPython testsSimple unit testData-driven integration testAdding a new msgpack fixtureC++ testsSimple unit testData-driven test using reference fixturesSee also