# Testing Tests are driven through `make` from the repository root. Python tests are the default and live in `tests/` as `test_*.py`. C++ tests live in `gtests/` as `test_nopython_*.cpp` and are reserved for behaviour that cannot or should not be reached from Python. - `make pytest`: run the full Python test suite. - `make pytest PYTEST_OPTS="tests/test_buffer.py::SimpleArrayBasicTC"`: forward options verbatim to pytest to run a subset. - `make run_pilot_pytest`: Python tests that need the pilot GUI. - `make gtest`: build and run the full C++ test suite. - `make pyprof`: run the profiling benchmarks (see {doc}`/compute/profiling`). After `make gtest` has built the binary, a single C++ test can be run directly: ```sh ./build/rel/gtests/run_gtest --gtest_filter=Suite.Test ``` where `` is the active Python major and minor version, e.g. `314`. ## Automatic Testing on GitHub Actions Continuous integration runs on GitHub Actions. The workflows live in `.github/workflows/`. Most checks run on every change, while the heavier and slowly-changing ones run on a nightly schedule. Each job drives the same `make` targets described above, so a failure can usually be reproduced locally. The fast set runs on every pull request and `master` push: - `check_skip`: decides whether to skip the heavy jobs (see below). - `lint`: `make cformat`, `cinclude`, `checkascii`, `checktws`, `flake8`, and clang-tidy on the diff, on ubuntu and macOS. - `standalone_buffer`: the standalone buffer build on ubuntu. - `build`: `make gtest` plus `make pytest` with Qt off and on, and the pilot, on ubuntu and macOS (Release). - `build_windows` (Release): the Windows build and tests. The heavy set runs on the nightly schedule only: - `build_windows` (Debug): the second Windows configuration. - `nouse_install`: the `setup.py install` packaging path. - the ASAN/UBSAN sanitizer build (`-DUSE_SANITIZER=ON` over the gtest suite). - `profiling`: the benchmark suite. - the Windows portable artifact. The nightly run is the superset: the fast set plus these extras. Compiler caches (`ccache` on Linux and macOS, `sccache` for MSVC on Windows) are saved by pushes to `master` and by nightly runs, and restored by pull requests. A pull request on a non-default base branch cannot read them and runs cold. ### Skipping in a pull request - A pull request skips the heavy jobs when it carries the `skip-ci` label or a repository member writes `[skip-ci]` alone on a line of its description or a comment. A documentation-only pull request (only `doc/**`, `*.md`, `*.rst`, or `contrib/prompt/**`) skips them automatically. - A pull request that touches no C++ or build file skips the Windows build but still runs the Python build and lint. ### Repository variables These variables tune the workflows. Set them as repository variables (under Settings, then Secrets and variables, then Actions, then the Variables tab). Each is read with a default, so an unset variable keeps the default behavior. - `MMGH_NIGHTLY`: set to `enable` to let the nightly schedule run its jobs. Unset, every scheduled run is skipped. - `MMGH_PUSH_RUN_BRANCH`: which branches run the fast set on a `push`. Use `*` for all branches, or a branch name (matched as a substring). Unset, a push runs nothing. - `MMGH_FORCE_PROFILE`, `MMGH_FORCE_NOUSE_INSTALL`, `MMGH_FORCE_SANITIZER`: set any to `enable` to force that nightly-only job to run on any event, so it can be exercised from a pull request. - `MMGH_TIMEOUT_BUILD` (45), `MMGH_TIMEOUT_LINT` (45), `MMGH_TIMEOUT_STANDALONE_BUFFER` (10), `MMGH_TIMEOUT_NOUSE_INSTALL` (30), `MMGH_TIMEOUT_PROFILE` (30): per-job `timeout-minutes`. The number is the default used when the variable is unset. `MMGH_TIMEOUT_BUILD` is shared by the ubuntu, macOS, and sanitizer builds (default 45) and the Windows build, which defaults to 60 because its vcpkg plus MSVC compile runs slower than the others. ### Behavior on a forked repository - A fork inherits neither these variables nor the secrets, so `MMGH_NIGHTLY` and `MMGH_PUSH_RUN_BRANCH` are unset there. By default only `pull_request` events run, so the fast set runs on a pull request opened in the fork, while pushes and the nightly schedule run nothing until the variables are set. - Scheduled workflows run only on the default branch, and GitHub keeps Actions disabled on a new fork until it is enabled (a public fork's schedule also pauses after 60 days without activity). - To exercise a single nightly-only job on a fork, set the matching `MMGH_FORCE_*` variable and open a pull request. - A fork pull request on a non-default base branch runs cold, since it cannot read the warmed caches. The failure-notification job is guarded by `github.event.repository.fork == false`, so it never runs on a fork (and the email secrets it needs are unavailable there).