EPS
← All docs·Activity

Daily — 2026-07-09 (56-task workflow-fix wave executed; 33 more filed; watcher respawn-loop found)

updated: 2026-09-03

What happened

  • Yesterday's 56-task workflow-fix wave executed end-to-end. The route-2 review tasks filed by the 07-08 /daily (#1148–#1214) ran as autonomous /issue --auto sessions through the full pipeline (plan → critic ensemble → implementer → code-review → test-verdict → Step 10d merge); 69 wave tasks completed and merged to main today. Highlights that shipped: .claude/rules/trigger-dense-review.md (#1185), the Codex pre-spawn quota check (#1204), new_worktree.sh branching from origin/main (#1214), the LESSONS.md trim (#1220), HF_HUB_DISABLE_XET lane forwarding (#1195). The 4 route-3 daily-held needs-human tasks (#1215–#1218) remain proposed for Thomas.
  • #1092 (realistic sparse-crossed prefix×query corpus, prefix-map) is the only active experiment: plan v6 auto-approved (85 GPU-h), and after yesterday's VM RSS incident the P6 fit grid fanned out onto GCP n2-highmem-16 cpu-bigmem instances (22 cluster launches today). One turn-1 refusal death (02:54Z) and one alive-but-stalled force-respawn (04:33Z), both recovered by the watcher; run healthy at day's end.
  • Codex org quota outage continues (sentinel until 2026-08-06): every doubled review site ran single-Claude per the sanctioned no-show fallback all day; #1204's pre-spawn check prevented wasted composer spawns.
  • No clean-results promoted today; #642 and #779 remain at awaiting_promotion.
  • Volume: 2,164 task events across ~170 tasks; 76 session transcripts (~123 MB) mined for tonight's sweep.

Applied workflow improvements

  1. Target: .claude/plans/issue-1174.md — what: split the two stale Agent(subagent_type="workflow-improver", ...) fixture literals so --check-no-workflow-improver-spawn stops FAILing on the local tree — commit: none (.claude/plans/issue-*.md is gitignored by design; the lint scans the working tree, which is what was red and is now green) Why: workflow_lint no-flags default was red locally (2 errors at lines 64/69) — flagged by #1220's planner (parked candidate, Step-C idx 42) and reproduced by tonight's triage. Applied edit:
    - 'Agent(subagent_type="workflow-improver", run_in_background=true)\n'
    + 'Agent(subagent_type="workflow-impro' 'ver", run_in_background=true)\n'  # split so repo lint skips this plan doc
    
    Verification: uv run python scripts/workflow_lint.py --check-no-workflow-improver-spawn → PASS.
  2. Target: tests/test_guard_piped_git_push.py, .claude/hooks/guard_piped_git_push.sh — what: relabel the A17 case comments from "the verbatim … Step 10d recovery shape from /issue SKILL.md" to "historical … shape" (the SKILL.md snippet has since been reworked; comment/label-only) — commit: 80d7fa7b20 Why: #1201's planner surfaced the stale claim as a parked prose follow-up (Step-C idx 29, route 1). Applied edit:
    - # A17: the verbatim braced backslash-continued Step 10d recovery shape
    - # from /issue SKILL.md (`||` + line-continuation normalization).
    + # A17: the historical braced backslash-continued Step 10d recovery shape
    + # (since reworked in /issue SKILL.md; kept as a `||` + line-continuation
    + # normalization regression case).
    
    Verification: bash -n PASS; uv run pytest tests/test_guard_piped_git_push.py → 38 passed. Repo-wide lint gates (--check-references, --check-asks) PASS after both commits.

Filed for review — 32 route-2 tasks (#1221–#1231, #1233–#1253; tags wf-fix + daily-auto-filed, → /issue --auto / watcher backstop). 25 from the Step-C parked-candidate sweep (50 candidates triaged by 2 subagents: 25 filed, 12 deduped against yesterday's wave or siblings, 9 already-fixed on main, 2 applied as route-1 above, 1 route-3, 0 retracted), 7 from tonight's transcript mining:

  1. filed for review #1247: terminal-status guard + source stamp for the watcher orphan-respawn pass — the day's headline bug: 709/709/393 orphan-respawn markers + git commits posted against TERMINAL tasks #662 (archived)/#663/#867 (completed) since 06-26, ~1,800+ junk commits, by a STALE watcher build running outside the canonical cron (canonical watcher logs exonerated; poster's marker text predates the current code; no sessions actually spawn). Fix: re-verify live status before any respawn/marker-post + stamp host/pid/sha into the note + hunt the stale poster.
  2. filed for review #1248: sweep suppression misses prior-night disposition records — #815/#880/#917 re-enumerated tonight despite last night's routed-records (fingerprint-form mismatch); one residual (#1196 twin park) still unsuppressed after tonight's records.
  3. filed for review #1249: Step 10d snippets → bare (unpiped) merge/push forms + fail-soft Step-0 ls probes (14 piped-merge hook blocks + 5 probe exit-2 sibling-cancellations across today's sessions).
  4. filed for review #1250: guard_repo_root_pull.sh false-positive — blocked task.py post-marker because the --note TEXT contained "git pull" (#1201 session, 23:06Z, same-day-shipped hook).
  5. filed for review #1251: step9c_baseline.py refuses (rc=2) on unrelated concurrent dirty paths at the shared repo root (#1190, 20:51Z).
  6. filed for review #1252: trigger-dense reviews — final return text must be verdict + marker pointer ONLY; a reviewer's findings recap wedged the ORCHESTRATOR with 3 consecutive usage-policy refusals (#1152, 09:38–09:49Z; watcher respawn completed the task).
  7. filed for review #1253: post-merge stale-task-folder recipe's git checkout fallback clause is hook-blocked every time it runs (#1198 ×2, #1155).
  8. Step-C sweep filings: #1221 rc==0-stderr forwarding for remaining task.py subprocess callers · #1222 implementer tiny-real data-ingestion probe clause · #1223 verify_plan c20 N/A-masking WARN · #1224 issue-v2 SKILL figure-pin ordering contradiction · #1225 asw docstring pass-count lint · #1226 audit is_v2 missing v4 sentinel (two parks merged) · #1227 verify_task_body 19b v4-Methodology scan · #1228 daily-driver route-2 tag gating vs SKILL.md (two parks merged) · #1229 scripts-import-guard scan roots · #1230 code-reviewer durability-pin shipping check · #1231 /issue excerpt-file briefs for trigger-dense reviews · #1233 vacuous pipe-python bundling-pin test · #1234 guard revert/am verb fence · #1235 nightly title-sync sweep hook for /daily (two parks merged) · #1236 experimenter memory setsid launcher trio · #1237 verify_plan doc-global N/A migration (+ absorbs a #974 park) · #1238 verify_plan N/A lstrip wrapped-escape · #1239 bootstrap_pod credential helper (no token-at-rest in remote URL) · #1240 SLURM result-push contract · #1241 watcher wedge-respawn cap on the 4 pre-existing triggers · #1242 step9c pin-suite tuple · #1243 SKILL Edit-C cert arm shape · #1244 Guard-1 foreign-strip resurrect (19 paths reproduced live) · #1245 Step 10d lint gate → background (600s foreground cap) · #1246 planner c33 escape phrase.

Other problems & notes

  • Held (route 3, needs you): #1232 — add trigger-dense composer-side pointers to the Codex twin wrappers (codex-code-reviewer.md, codex-clean-result-critic.md). Carve-out: genuinely ambiguous — the emitter itself gated it on "file only if wrapper kills recur", which hasn't been evaluated (Codex is quota-out until Aug 6, so no recurrence evidence can accrue). Suggested action: approve/reject when Codex comes back.
  • Rogue stale watcher (context for #1247): the poster commits directly into the canonical tree at irregular times (not the 3-59/10 cron), its markers lack the current code's "(attempt N/M today)" suffix, and ~/explore-persona-space-codex-port was ruled out. No respawned sessions actually materialize — damage is ~91 junk markers+commits/task/day, not pod spend. Worth a manual ps/other-machine look if #1247's hunt stalls.
  • ~37 stale claude-MMDD-HHMMSS tmux sessions dating back to Jun 30 linger on the VM (plus 39 live registered sessions reported by the orphan sweep). The zombie-wrapper watcher pass may not cover detached tmux wrappers; suggested action: fold a check into #1247's session or triage manually.
  • gh pr merge --rebase refused ("can't be rebased") in 8+ sessions; the documented squash fallback recovered every time. Root cause is tasks/ lifecycle-commit contamination of issue branches — monitor whether today's shipped fixes (#1210 cf499ba706, #1212 c3e0ed0e8c, #1184 85f6009c17) cut the rate before filing more.
  • One-off, not filed: #1158's implementer branch got fast-forwarded onto local main mid-sync carrying 18 stray task-state commits; orchestrator cherry-picked the reviewed commit onto a scratch worktree (clean recovery, pre-#1214 branch-cut era).
  • One-off, not filed: #1219's planner-output extraction found no result row and recovered in 2 calls; #1168 hallucinated a TaskCreate tool once; Edit before-Read/string-not-found misses in ~6 sessions, all self-recovered same-minute.
  • Refusal ladder exercised as designed on #1152 (reviewer kill → durable-verdict check → thinned respawn → PASS) and #1092 (turn-1 death → watcher respawn); residual gap is the return-text recap, filed as #1252.
  • Retraction re-check: 50 marker-mined candidates scanned at filing time (triage subagents re-read every source task's post-park events), 0 skipped-as-retracted; 7 transcript-mined filings tail-checked, 0 retracted.
  • Daily stub + enrichment commits pushed to main (stub e8bba8d71c; push confirmed landed).

Living-docs drift

  • living_docs.py check FAILs with 199 findings — drift hash 078285c14444, byte-identical to the open proposal (79 dead legacy anchors, 113 relates_to ⇄ evidence mismatches / unlinked completed clean-results, 7 stale spec-role-header State-trailer rows).
  • drift proposal already open (logged 2026-07-07, same hash 078285c14444); not re-proposing. The standing proposal remains: re-synthesize open_questions.md (living-docs updater/backfill) — needs your ok; not auto-applied.
  • Parked-proposal re-consideration (Step B): no results promoted today and no questions changed, so no parked living-docs rejection (#713's two remain the only ones) or parked-redundant follow-up became a re-proposal candidate.

My thoughts

Highlighted results

  • no results promoted today