workflow-fix: verify_plan per-corpus eligibility-floor arithmetic check (battery members vs the plan's own floor)
Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Goal
Add a per-corpus fit-eligibility floor arithmetic check to scripts/verify_plan.py so a plan cannot register a per-corpus fit battery whose members cannot clear the plan's own eligibility floor.
Incident (#2546 plan v3, 2026-08-24)
The plan registered an absolute-trainability drop floor (1.2×d/0.8 = 5,376 usable rows) AND a per-corpus fit battery naming MMLU at a 5,000-row draw — below the floor at even 100% usable — plus ContextHub at a realized max 5,460 (clears only under ≤1.5% attrition while the plan's own usable gate tolerates 10%). MMLU was the ONLY pure no-reasoning corpus in the Plot-8 ladder, so the H5 contrast lost its entire no-reasoning arm by construction, silently (the drop path is a designed non-error). verify_plan PASSed 0/0; caught by two independent critics + reconciler.
Proposed check (from the methodology + statistics critics, mechanizable sketches)
For each corpus named in a per-corpus fit registry (or ladder battery list), when the plan declares (a) a per-corpus eligibility floor and (b) per-corpus row targets: FAIL any corpus whose registered_rows cannot clear the floor even at 100% usable (the MMLU shape); WARN any corpus that clears only above the plan's own declared usable-floor tolerance (registered_rows × usable_floor < eligibility_floor, the ContextHub shape). Also assert any matched-n companion convention ≥ the floor's own n_train margin or carries a named exemption line.
Acceptance criteria
- Fixture with floor 5,376 + a battery member at 5,000 rows FAILs.
- Member at 6,750 rows with 90% usable floor PASSes.
- Plans with no per-corpus floor or no battery SKIP.
- Pin test for no-flags bundling status.