Repository navigation
Description
What happens
Docs-only PRs often run the full build and test matrix. Two mechanisms are supposed to prevent that, and both miss in common cases:
- The title tag.
should-skipin.github/workflows/ci.ymlskips the non-docs builds and all test jobs only when the PR title contains the literal, case-sensitive[doc-only]. Most docs PRs use a conventional prefix such asdocs(cuda.core): ...instead, which does not match. The tag is not documented anywhere in the repo. - The path planner.
ci/tools/compute_ci_plan.pyalready treats<package>/docs/**,*.mdand a few other paths as needing no tests. But it uses the changed paths only when the merge base has a successfulmainCI run with its artifacts. Without one,main()passes an empty path list andforce_allis set ("this run will build and test everything").
Lately main rarely has that baseline: of the main push runs since 2026-10-17, 35 succeeded, 44 failed and 75 were cancelled. So an untagged docs PR now gets the full matrix.
The rule is the same for cuda.core, cuda.bindings and cuda.pathfinder (one ci.yml, one planner). The habit differs: cuda.bindings and cuda.pathfinder docs PRs mostly carry [doc-only], while cuda.core docs PRs mostly use docs(cuda.core): ....
Evidence (PRs opened since 2026-10-17)
- 15 PRs changed only docs files: 9 tagged
[doc-only], 3 tagged[no-ci], 3 untagged. - Of the 3 untagged, Bring over 13.4.4 release notes to main #3264 and Omit autosummary pages in PR previews #3244 had a baseline, so the planner skipped their tests (about 12-14 job-minutes each). docs(cuda.core): add 1.3.1 release note for the IPC import/release race fix #3479 (one release-notes file) had none and ran the full matrix three times. The latest run has 104 jobs, 63 of them GPU test jobs, and had used more than 900 job-minutes while still running.
- [doc-only] docs(release): tag cuda-core releases on a release branch #3431 (issue templates and Markdown only) also ran the full matrix untagged: 104 jobs, about 1,030 job-minutes. The tag was added afterwards, but a title edit does not start a new run.
- The reverse also happens: [doc-only] cuda.core: annotate host callback fn and user_data #3304 and [doc-only] docs(cuda.core): name numa_id in the modify_allocation config rules #3302 were titled
[doc-only]but changed.pyx/.pyifiles, and their tests were skipped, because the tag check never looks at the paths.
Cost
A docs run that skips tests costs about 12-50 job-minutes (build-linux-64 for the docs, the docs build, the checks). A full run costs about 1,050 job-minutes, more than 20 times as much. It also occupies the shared GPU runners, so other PRs wait longer in the queue.
Options
- Detect docs-only changes from the paths, with or without a baseline. Always compute the changed paths against the merge base. If every path is one the planner already ignores, skip the tests even when there is no baseline. The
[doc-only]tag becomes unnecessary. Drawback: the ignore list becomes the only safeguard, so a wrong entry would skip tests on real code. The same list already makes this decision whenever a baseline exists. - Also accept the conventional
docs(...):/doc(...):prefix, when the paths agree. Drawback: it keeps CI behavior tied to title conventions. It still misses untagged docs PRs with other titles (Bring over 13.4.4 release notes to main #3264, Omit autosummary pages in PR previews #3244, [doc-only] docs(release): tag cuda-core releases on a release branch #3431), and it needs the path check of option 1 anyway to be safe. - A check that flags a mismatch between the title and the paths. For example,
[doc-only]with code changes, or a docs-only change without the tag. Drawback: it adds one more check that authors must react to, and it does not save any runner time by itself.
I would do option 1, and honor [doc-only] only when the paths agree, so the tag can no longer skip tests on code changes. Option 1 fixes the problem at the planner, where the ignore list already lives, for all three packages and any title style. The [no-ci] escape hatch can stay as it is.
Activity
[-]ci: docs-only PRs run the full test matrix unless the title has [doc-only] and main has a green baseline[/-][+]ci: docs-only PRs run the full test matrix unless the title has [doc-only] or main has a green baseline[/+]