workflow-fix: kill-before-relaunch + timeout for smoke retry loops
Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Auto-filed from the 2026-07-02 shared-VM overload incident. During #823's code-review retry loop, THREE identical run_823.py --phase 4 --smoke --skip-upload instances ran concurrently (launched 23:48, 23:51, 00:02) because each retry re-ran the smoke WITHOUT killing the previous instance — timed-out/abandoned Bash calls kill the shell, not the python child. All three wrote the SAME output paths (smoke-verdict corruption risk) and held 64 threads each (~1/3 of the load-226 overload).
Goal
Make smoke/run RETRY loops kill-before-relaunch: before re-running a smoke or launch command, kill any live prior instance of the same invocation and/or verify no live process is writing the same outputs; bound smoke runs with timeout.
Workflow gap
- Bug observed: review-retry loops (implementer crash-fix rounds, Codex companion smoke re-runs) relaunch the same command while the prior instance still runs, duplicating load and corrupting shared output paths.
- Why it is a workflow gap: no rule/instruction on any retry surface requires killing the prior instance; Bash timeouts orphan python children by design.
- Confidence (emitter): high
Proposed change (refine in planning)
.claude/rules/crash-fix-rounds.md: add a kill-before-relaunch requirement to the crash-fix/retry protocol (pgrep the exact prior invocation; kill + confirm dead BEFORE re-running; wrap smokes intimeout)..claude/agents/experiment-implementer.md+.claude/agents/implementer.md: same instruction at the smoke-run step.- Consider the Codex twin prompt-composers (the companion runtime executed these retries) — add the instruction to the composed prompt where smokes are re-run.
Scope / surfaces
- Primary:
.claude/rules/crash-fix-rounds.md,.claude/agents/experiment-implementer.md,.claude/agents/implementer.md, codex wrapper prompts.
Constraints / invariants
- workflow_lint --check-asks passes; instruction must be exact-invocation-scoped (never a broad pkill that can hit concurrent sessions' work — this incident's own cleanup nearly did).
Provenance
- workflow_fix_target: .claude/rules/crash-fix-rounds.md
- origin: PM-chat VM triage 2026-07-02 (#823 3x duplicate smoke; /tmp/vm_load_diagnosis.md)