select_step9c_tests.py: feature-named consumer tests escape module mapping; over-wide sweep_scope label hides the gap
Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Goal
Close a mapping gap in scripts/select_step9c_tests.py that let two real
consumer test files of a changed module escape both the pin-sweep hit list and
the executed Step 9c union — so a future round touching the same module can
land with its consumers unrun and the gate still green.
Evidence
Surfaced by the #2336 round-3 code review (plan batch 1), which migrated
scripts/clean_experiment_downloads.py among others.
The selector maps a changed source file to tests by an exact stem rule plus a
glob: for scripts/clean_experiment_downloads.py that glob is
tests/test_*clean_experiment_downloads*.py. Two test files are genuine
consumers of the writers changed in that round but match neither rule:
tests/test_janitor_tmp_scratch_sweep.pytests/test_vm_disk_guard_slurm_src.py
Both exercise code paths through the migrated
_ScratchVerdictCache.save / _save_slurm_src_escalation_state writers. The
reviewer ran them by hand: 158 passed, rc=0, so no defect shipped — but
they were absent from the selector output, absent from the round's claimed
pin-sweep hit list, and absent from the 128-file union the round actually
executed.
Two failure modes compounded, and the second is the more interesting one:
- Selector-glob gap. The name-derived glob cannot see a consumer whose filename does not embed the module stem. Any module whose consumers are named after the FEATURE rather than the module has this hole.
- A pin-sweep supplement labelled wider than its realized scope. The
round's grep-only supplement was reported under
sweep_scope: repo-wide, but its realized scope was narrower than that label — so the label itself read as coverage. This is the dangerous half: a truthful narrow label would have prompted a wider grep, whereas an over-wide label suppressed the very check that would have caught the gap. Compare the standing rule that a dead or silent verification leg is INCONCLUSIVE and never push-clean — an over-labelled leg is worse, because it reads as positive evidence.
Scope
- Add a mapping-table entry (or an equivalent explicit consumer registry) so
scripts/clean_experiment_downloads.pymaps to both files above in addition to its glob matches. Prefer a general mechanism over a one-module special case if one is cheap — the class is "consumers named after the feature, not the module". - Audit for the same shape elsewhere: modules whose consumer tests are
feature-named. A bounded sweep over the existing mapping table plus the
tests/inventory is enough; an exhaustive proof is not required. - Make the
sweep_scopelabel verifiable rather than declarative. Options worth weighing: have the selector emit the realized universe it actually searched, or require the reporter to state the grep command and its target set alongside the label so a narrower realized scope cannot hide behind arepo-wideclaim.
Acceptance
printf '%s\n' scripts/clean_experiment_downloads.py > /tmp/f.txt && uv run python scripts/select_step9c_tests.py --map-files /tmp/f.txtincludestests/test_janitor_tmp_scratch_sweep.pyandtests/test_vm_disk_guard_slurm_src.py.- A test pins that mapping so it cannot silently regress.
tests/test_select_step9c_tests.pystays green, including its exact set-equality invariant checks.- If scope item 3 lands, a test pins that an over-wide
sweep_scopelabel is detectable — or, if that is judged out of scope, the decision is recorded with its reason rather than dropped silently.
Notes on routing
Not #2336's defect and not blocking it: #2336 round 3 passed review with the
two files run green by hand, and its later batches now carry an explicit
instruction to grep tests/ genuinely repo-wide. The gap is in the shared
selector, which every session's Step 9c gate depends on, which is why it is
filed separately rather than folded into that task's remaining batches.
Related ledger row on #2336: pin-sweep-supplement-scope (CONCERN, round 3).