EPS
← All docs·Activity

Daily — 2026-07-20 (15 workflow-fix merges;

updated: 2026-09-03

What happened

  • A fleet of ~15 autonomous workflow-fix/daily-fix sessions (#1554–#1573, filed by yesterday's /daily + the workflow-fix protocol) all ran their full pipelines and merged to main — 615 commits on main today. Highlights: #1556 (structural digest for trigger-dense poll excerpts), #1560 (Step 5a lint vintage-skew fix), #1564 (watcher completed-unmerged audit pass), #1565 (append-side stale-state guard — forensics rewrote the filing's premise), #1566 (guard-hook false-positive masking), #1572 (staged-index verification, filed AND merged same day), #1573 (step9c import-map arms, ditto).
  • #1345 — the user-armed onpolicy-assistant-story round (computed 07-18, stranded by a wedged owning session) was harvested by a takeover session: story error > chat error on 98.6% of 2,018 conversations (powered on-policy read), folded through 2 interp + 2 clean-result rounds, re-parked at awaiting_promotion. The merge close-out was interrupted (see problems) — recovery session spawned tonight.
  • #1112 (interactive) — Thomas drove a 0-GPU cross-method debiased-cosine round, then the "rankem" rank × behavior inline-override round: 2 implementer teammates + reviewer, code-review r1 caught 3 run-wasting blockers pre-GPU, 3 smoke crash-fixes at ~3 min each, then --full production dispatch; the full run died 4 s into the B2 full-FT subprocess — diagnosis was live at day end.
  • #958/#825 (interactive) — two inline rounds answering Dan's dashboard comments: the mixed turn-1+2 map generalizes to unseen turns 3-4; the #825 user-answer resample ceiling is HIGH (0.78), so user-answer diversity does NOT explain the user-turn null (flips the hypothesis in Dan's comment). Paste-ready Slack replies folded into the bodies.
  • Thomas set up gitleaks secret-scanning pre-commit (.gitleaks.toml + hook, commit 3ae97a78) and caught/terminated the idle #1345 pod ("are you sure it's not running still?").
  • Codex quota outage all day (sentinel live until 2026-08-06): every doubled review site ran single-Claude — ~15 workflow-surface merges shipped with zero cross-family review diversity (tracked in #1472).

Applied workflow improvements

  1. Target: tests/test_step10d_guard3.py — what: refreshed two stale docstrings still describing the Step-5a spec-freshness sync as keeping the full-message --grep filter (subject-scoped by #1560 the same day; prose only, no assert changed) — commit: d729450635 Why: parked route-1 candidate on #1560 (code-review r1 Minor 2, 2026-07-20T12:43:58Z): "docstrings still describe the Step 5a sync as keeping the full-message grep filter, now stale after #1560's subject-scoping". Verified: uv run pytest tests/test_step10d_guard3.py green + inline payload lint gate PASS (root-code-commit guard) + workflow_lint --check-references/--check-asks PASS. Applied edit: 
    -    unrelated Step-5a spec-freshness sync (line ~1925), which legitimately uses
-    a `--grep='spec-freshness' --invert-grep` filter and MUST NOT be touched.
+    unrelated Step-5a spec-freshness sync (line ~1925), which #1560
+    subject-scoped to the same awk index() form (see
+    test_step5a_grep_filter_untouched) and which is pinned separately.
    (second docstring in test_guard3_region_does_not_use_grep_spec_freshness updated equivalently)
  2. filed for review #1574: extend the #1556 structural-digest mechanism to the GCP/SLURM-lane log_tail_excerpt constructions (parked candidate on #1556).
  3. filed for review #1575: extend the followup-parked-by-cap note duty to the expensive-band autonomous cap-park path (parked candidate on #1558).
  4. filed for review #1576: WARN-arm in the #1329 dispatch-time lint for inline-interpreter workload-cmd bodies (parked candidate on #1562).
  5. filed for review #1577: read-bounding PreToolUse hook for guard hook scripts (parked candidate on #1563; a 49K-token wholesale guard-script read fired in the #1566 session the same day).
  6. filed for review #1578: completed-status resume-table row routing completed-unmerged tasks to the Step 10d auto-merge (parked candidate on #1564).
  7. filed for review #1579: step9c stem arm maps .sh scripts to tests/test_<stem>.py (formal candidate on #1566; verified NOT covered by #1573's merge — the .py suffix gate is live at select_step9c_tests.py:675; the literal-path arm #1498 covers only the worked example).
  8. filed for review #1580: daily_drive_filings.py must reconcile the body Provenance fingerprint with the wf-fix-fp tag — ALL 17 of 2026-07-19's filings (#1554–#1571) and tonight's #1579 carry tag≠body mismatches (two sources of truth: body fp absent-gated at :228, tag always manifest-computed at :944), breaking the exact dedup predicate for those tasks (parked consistency finding on #1570).
  9. filed for review #1581: edit-success gate before new-plan-version persist in the adversarial-planner recipe — a failed plan-edit script still persisted v2 as an unmodified copy in the #1565 session (sibling shape in #1563).
  10. filed for review #1582: escalate-only watcher arm for RUNNING keep-running-tagged pods whose owning session is wedged/unmapped — the #1345 owner wedged 3.7 days holding registration + worktree while the pod idled ~2.5 days billing; recovery was manual, prompted by Thomas.
  11. filed for review #1583: teammate-coordination rule (one implementer per file set; idle ping ≠ done/stall — probe durable state; reports on the teammate channel) — 3 duplicate-work incidents in one day (#1112 impl/impl2 collision, #1112 figure-fix race, #958 subagent race).
  12. filed for review #1584: scope the gitleaks pre-commit hook on merge commits — it scans the ENTIRE staged merge diff (~50 min per commit attempt; ~100 min burned on #1345's step10d conflict resolution); this recovers a DROPPED prose follow-up (the owning session was watcher-auto-stopped before its intended auto-file).

All 11 route-2 filings have terminal filed rows in logs/daily/filings-2026-07-20/filed.jsonl; spawns that hit the 5-session infra cap are covered by the watcher proposed_infra_sweep backstop.

Other problems & notes

  • #1345 close-out interrupted by the watcher (HIGH — action taken tonight): PR #1347 (the on-policy fold's branch artifacts) is still OPEN — the takeover session's 2-hour conflict resolution finished (merge 064f766ee0, 36/36 sanity tests) and the lint gate was re-running when the watcher's session-reconcile pass auto-stopped the session at 05:43Z (task parked at awaiting_promotion reads as idle to that pass). A recovery /issue 1345 --auto session was spawned by this run (the awaiting_promotion resume row runs the Step 10d auto-merge backstop). The reconcile pass stopping a session mid-close-out is additional evidence for #1582/#1578.
  • Wedged-owner incident behind #1582: the #1345 inline-override session went silent 2026-07-18T00:01Z; its workload COMPLETED 07-18T07:27Z; results sat unharvested ~2.9 days; the pod (STOPPED then resumed) and the keep-running shield were only cleared after Thomas's nudge. Mitigations that held: pod was STOPPED most of that window; artifacts were already on permanent storage.
  • API usage-policy refusals — 4 firing events, all recovered by the ladder: #1563 planner spawn (first-pass brief paged wedged-transcript content — the digest-only first-pass-brief discipline was NOT followed; ~13 min), #1554 fact-checker spawn (retry OK), #1572 implementer (retry OK, all deliverables landed), #1559 orchestrator turn (self-recovered in 16 s). Counting: unique refusal-killed spawns/turns per session, not string hits.
  • #1112 rankem crash chain (live at day end): 3 smoke crashes fixed at ~3 min each (panel-disjointness assert; base-rate leg loading the bare base model as a LoRA adapter; retry_transient missing required what= kwarg — masked in CI by three signature-non-conformant test fakes, fixed with an AST bind-sweep over all 11 hub.* calls); the --full production run then died 4 s into the B2 full-FT subprocess (rc=1), diagnosis in progress at snapshot. The test-fake signature drift is the #906 rule class — worth watching for recurrence before filing a lint.
  • Step 10d merge-conflict churn: #1566 vs #1567 both touched the same guard test file (PR 1342 unmergeable, ~19 min autonomous recovery); #1345's PR #1347 needed the 2 h subagent; gitleaks made merge attempts ~50 min each (→ #1584).
  • Codex quota outage (until ~2026-08-06): all review sites single-Claude fleet-wide; the #1204 pre-spawn skip machinery worked (zero composer burn). Already tracked in #1472 (blocked daily-held) — no new filing.
  • #1549 (daily-held: reap stranded old-socket tmux server) is likely archivable — the stranded server was killed 2026-07-19 (~85 GiB freed, per memory); Thomas/PM to confirm and task.py set-status 1549 archived.
  • Pending on Thomas: promote #1482 (task.py promote 1482 useful|not-useful; leftover follow-ups floor-adjusted-taxonomy 0 GPU-h + sae-sae-mlp-recovery 2 GPU-h await a manual pick); promote #1345 once the recovery merge lands; 2026-07-19's brief still visible: false.
  • #1553's filed diagnosis was wrong (07-19 context): the spawned session refuted the candidate's mechanism (_PRE_REG_PROSE_SECTIONS is v3-only) and landed the real fix — another verified-at-filing accuracy data point; the pipeline's check absorbed it.
  • Title-sync drift sweep (Step D): title-sync drift unchanged (8 rows: #432 #458 #654 #658 #664 #665 #763 #1226 — see logs/daily/2026-07-11.md for the verbatim rows).
  • Retraction re-check: 9 marker-mined problems scanned (7 sweep candidates + the #1570 consistency finding + the #1345 dropped follow-up), 0 skipped — no upstream retractions found.
  • Minor self-corrected errors in aggregate (no filings): guard-hook blocks doing their job (piped-push ×2 — one in this session, root-code-commit ×2, bare | python -c pipe ×1, sleep-guard ×1, repo-root checkout ×1); one index.lock collision (this session, retry OK); plan-edit anchor misses ×3 (→ #1581 covers the persist half); one 2-min Bash timeout; one TaskOutput deferred-schema InputValidationError; one REGISTRY one-liner parse bug (highest_id key); monitor false alarm on known vLLM teardown noise; stale git lock on pod sync; GitHub 403 from a pod IP worked around via git bundle (07-17 context); one silent commit failure (staged-but-not-landed, retry OK); a mixed-turn-fit-958 subagent Bash timeout surfacing the npz gitignore issue that became #1572.

Living-docs drift

  • living_docs.py check: FAIL — 199 findings, materially the SAME drift as the still-open re-synthesis proposals of 06-29 (1736fca0804f), 07-04 (d2a80db46b3b), 07-06 (b009cef9ecd9), 07-07: ~75 dead legacy anchors (a1/b1/c1-class relates_to with no matching question), ~54 completed clean-results in no question's evidence, the #494–#501/#722/#763 relates_to⇄evidence mismatches, and the stale spec-role-header State trailer (7 newer linked results).
  • drift proposal already open (logged 2026-06-29, extended 07-04/07-06/07-07/07-19); not re-proposing — event row appended to .claude/cache/nightly-consolidation-events.jsonl.
  • Parked-proposal re-consideration (Step B): no epm:living-docs-update-rejected markers awaiting reconciliation and no on_hold parked-redundant follow-ups — nothing to re-surface.

My thoughts

Highlighted results

  • no results promoted today