Daily — 2026-08-02 (0 promoted; ~20 infra fixes merged; 5 applied + 18 filed by the sweep)

What happened

• Huge autonomous infra day: 1,144 task-state commits; ~20 workflow-fix/daily-fix infra tasks completed AND merged to main in one day (#1938–#1944, #1948–#1950, #1955, #1957, #1958, #1960, #2017, #2020, #2021, #2023, #2026, #1987, #1989, #1942, #1940). 0 clean-results promoted.
• #1739 (map-then-project persona-vector arms) dominated compute: wave-1's 12-leg GCE fan-out lost half its legs to OOM (spot-rung 85 GB boxes + fp64 casts), two new bug classes landed fixes (CUDA int32 gather overflow; natpv rowindex staging), wave-2 relaunched 14/14 boxes and upload-verified PASS, pod-1739-a5syc terminated, interpretation round 4 started; a user-directed evil-OOD-spread follow-up scope was added. Residues: a ~20h undetected futex-wedged GCE leg and a post-autocompact relaunch race (both → filings below).
• #1979 finished its follow-up rounds → awaiting_promotion. #1345 matched-n lattice fits ran on fellows (two job losses + recoveries, then COMPLETED runs incl. the fulln GPU relaunch). #1947 ran the P4/P5 battery + a user-directed sycophancy yield-recovery round. #1962 (lint-gate concurrency) and #2028 (remove GCP as a GPU lane; review round 1 FAIL, iterating) are in flight.
• Codex quota outage ALL DAY — every ensembled review site ran single-Claude per the #1204 sentinel fallback (sentinel live until 2026-08-06); the watcher's codex-outage pass owns alerting.
• The 08-01 /daily (ran 00:19 PT) filed #1999–#2014 overnight; today's sweep adds 18 more filings + 5 self-applied fixes (below).

Applied workflow improvements

1. Target: .claude/skills/issue/SKILL.md — what: 9a-ter inline-gate recipe gains the two invocation traps (explicit Bash timeout ≥ 300000 on the gate call; certify and commit in SEPARATE Bash calls — the guard evaluates the compound argv before anything runs) — commit: 981f85fb85 Why: 3 sessions hit these on 2026-08-02: exit-143 default-timeout kills of gate runs in 0ac15c23 (14:13Z) + a0400dd4, and a bundled lint+commit always-blocked in 0ac15c23 (16:07Z, "BLOCKED: repo-root commit carries UNCERTIFIED code payload"). Applied edit:

   + Two invocation traps (2026-08-02, three sessions): (1) the gate's mapped-pytest leg routinely runs >2 min — ALWAYS pass an explicit Bash `timeout` ≥ 300000 ms ... (2) NEVER bundle the `git commit` into the SAME Bash call as the gate run ...
   

2. Target: CLAUDE.md — what: teammate clause (c) documents that a string-message SendMessage REQUIRES a summary field — commit: 39c825a3ec Why: 3 tool_result firing events across 3 sessions (cae57d2b 01:30Z, 57ece5dd 11:53Z, f61f0467 19:03Z), each summary is required when message is a string, each costing a retry turn. Applied edit:

   + (Mechanics: a string-`message` SendMessage call REQUIRES a `summary` field — omitting it fails with `summary is required when message is a string`; 3 sessions each burned a retry turn on this, 2026-08-02.)
   

3. Target: .claude/rules/upload-policy.md — what: shard pieces (.shardNN.jsonl / .part*) are line-split FRAGMENTS — concatenate before json.load — commit: 1c60a50685 Why: json.load traceback on judge_raw_trait.json.part000 (55419495, 14:27Z). NOTE: the first commit attempt of this edit was EATEN by the shared-root reversion race (see notes below) and re-applied. Applied edit:

   + ...; shard pieces — `.shardNN.jsonl` or `.part*` — are line-split FRAGMENTS of one file: concatenate before `json.load`, a lone `.part000` is not standalone JSON, 2026-08-02).
   

4. Target: .claude/rules/workflow-fix-on-bug.md — what: session-suffix the /tmp/wf-fix-body-<slug>.md path — commit: 6f910a711d Why: 2 cross-session Write collisions on the bare slug path in one day (20e82ec2 00:47Z on the #1902 candidate's stale file; bbd94940 22:08Z). Applied edit:

   + Session-suffix the body path (e.g. `/tmp/wf-fix-body-<slug>-$$.md`): hot target files draw same-slug candidates from concurrent sessions, and a stale sibling's file at the bare path blocks the Write (2 cross-session collisions on 2026-08-02).
   

5. Target: .claude/skills/daily/SKILL.md — what: delegated filing-body composition briefs must restate the SHA-resolution duty (clause d) — commit: e8aa420201 Why: the 08-01 run's c8 compose subagent shipped session basename 8fc069db as an uncited hex token into #2003's body; only the driver's WARN backstop annotated it. Applied edit:

   + **Delegated composition restates the duties:** when filing-body composition is delegated to a compose subagent, the spawn brief restates the SHA-resolution clause (d) verbatim ...
   

6. filed for review #2029: verify_plan check — a plan-declared edit to a workflow-surface literal must list every tests/ file pinning that literal (from the #1948 r1 critic; Step C parked candidate, fp 2732a7f36293).
7. filed for review #2030: extend the #1950 stale-__pycache__ purge + PYTHONDONTWRITEBYTECODE=1 guard to the Step 9c gate's pytest runs (Step C parked candidate from #1950, fp 82cc7365c5c1).
8. filed for review #2035: backend_poll REACHABLE-wedge escalation arm — the #669 frozen-phase arm requires a reachability alarm, so the futex-wedged (reachable, GPU 0%, 6.5 CPU-min/21.6h) #1739 leg read healthy for ~91 poll ticks.
9. filed for review #2036: post-autocompact crash-relaunch must re-run the external-marker triage before dispatch (the wrong-root-cause epm:failure v5 + GCP re-dispatch that raced Thomas's inline "move to runpod" by ~30s).
10. filed for review #2037: trigger-dense INGEST hardening — refusal-result text digested before the durable-verdict check; followup-scope marker bodies + real-corpus parent bodies by reference (3 ingest-side refusal wedges today; spawn-side sibling is #2003).
11. filed for review #2033: require a declared GPU-mem/RAM floor on wide fan-outs whose per-leg RSS exceeds the smallest ladder rung (the wave-1 OOM on 85 GB A100-40 spot boxes).
12. filed for review #2038: terminate the just-provisioned pod in the SAME failure path when a RunPod fallback launch fails/supersedes (4 orphan pod-1739 pods billed overnight).
13. filed for review #2039: inline lint gate checks VM load before its mapped-pytest leg (false exit-3 under load1≈31, clean retry after load drop).
14. filed for review #2040: 9a-ter compute-character statement gains across-cell SHARD-AXIS + checkpoint-cadence duties (the 118-cell serial battery Thomas had to ask twice about — 4× left on the table; the checkpoint-less detached #1482 fit).
15. filed for review #2034: stand-down not effective until CONFIRMED + session-to-session ownership-release marker (the 472284ce stood-down-but-completed race; the a0400dd4 near double-dispatch; the 75f66748 duplicate ladder spawn).
16. filed for review #2045: verify-before-asserting extensions — projected-vs-realized (read the COMPLETION artifact, never the approval marker: the "never ran 3.2M calls" $9,505 wave), fallback-slice labeling (the +0.220 headline), banked-coverage relocation sweep BEFORE pricing regeneration.
17. filed for review #2041: fan-out completion contract — fold/fan-out spawn briefs restate same-turn commit+push + lint-gate duties; reports consolidate to a durable home at join (3 idled-uncommitted folds; 4 lit-dive reports in /tmp for 11h; an orphaned 10d gate-wait).
18. filed for review #2046: root-commit guard should honor an excluding git commit -- <pathspec> instead of keying only on the staged index (2 blocks on a commit that could not sweep the foreign staged file).
19. filed for review #2047: verify_plan flags bare lane-specific env vars ($WORKLOAD_ROOT) in §10 workload commands — set-u-safe form or lane pin (the #1979 f1g spawn death on the fellows lane).
20. filed for review #2048: per-family measured-pilot basis when a fan-out's families differ >~4× in budget (the #1739 core-family 2–3× under-sizing, 3 deviation cycles in one morning).
21. filed for review #2042 (wf_fix: false, experiment code): preflight disk-quota probe writes at the filesystem root and Permission-denies on the VM every run — probe a user-writable dir on the same mount (orchestrate/preflight.py:568).
22. filed for review #2043: DEVICE-DOMAIN arm in the artifact-reuse fitness checklist — 1-cell CUDA smoke before a production lattice on a fresh fit-core code path (fellows job 17912 died after all 140 fits completed).
23. filed for review #2044: continuation-runbook markers carry verified-by: ran|read per load-bearing claim (the v235 "no fits change needed" static read that killed two #1345 jobs before the v246 amendment).

Other problems & notes

• Step C sweep hygiene: 8 parked candidates enumerated, all dispositioned — 5 deduped against open #2022 (the eval_results/issue_1481 sparse-cone gap; parks on #1938/#1939/#1943/#1944/#1955, routed-records posted; #2025 was archived in-session as its duplicate), 2 filed (#2029/#2030, routed-records posted on #1948/#1950), 1 deferred again (#1771 architectural-greenlight mechanization — #1771 itself is still plan_pending rewriting that exact gate). 1 benign skipped row (#1333 json-decode, relevant_kind: false); unmatched_record_fps 326 refs — the standing #1703 advisory backlog, unchanged in character. #2022 note: created 17:55Z, never dispatched (infra cap full); meanwhile FIVE sessions re-hit the collection error today — worth dispatch priority when the cap frees.
• Retraction re-check: 8 marker-mined parked candidates scanned (fresh events-tail read per park), 0 retracted upstream. Transcript-mined filings inherited the miners' full-session tail reads; two premises were re-scoped from later corrections before filing (the a5syc root-cause withdrawal; the #669-arm coverage read).
• This run was hit by the #2015 reversion race it was reporting: the upload-policy.md route-1 edit was silently reverted to committed content between Edit and commit (pre-commit stash/restore under concurrency — the exact mechanism #2015, filed 08-01, describes; still proposed); re-applied + committed on retry (1c60a50685). Plus one index.lock collision (bounded retry recovered) and one cannot lock ref HEAD-class transient earlier. #2015 deserves dispatch priority.
• Idle-billing recurrences while the fix sits parked: three separate done-but-lingering GCP boxes billed idle (gap1nulldiag reaped 02:28Z; bqladsyco+bqladhall ~1h+ at phase=done, 17:38Z; bqladevil caught 20:00Z by an in-session monitor arm) — #2006 (box teardown leg, filed 08-01 for this exact pattern) is still proposed. The overnight leak class (4 orphan RunPod pods + done-but-billing newarma5evil) is now filed as #2038.
• --backend gcp habit forfeited the free fellows lane (~12 dispatches, ~40+ GPU-h of credits with charmander idle) — user-caught ("isn't that in the workflow already?"). The CLAUDE.md habit-guard bullet landed in-session (a548d42a58) and #2018 (mechanical lint) is proposed.
• Verify-before-asserting cluster (user-caught, 5 incidents): the "we never ran 3.2M batch calls" claim about a real $9,505 completed wave (refuted by a subagent audit after "are you sure - dig deeper"); 3 consecutive wrong SAE-label-coverage claims (real inventory 127,605/128,512); the +0.220 fallback-slice headline (real +0.018/−0.003); a ~$1.5–3k re-judge priced on a false per-draw-scores-averaged-away claim (per-draw scores existed on HF; $0 spent); "i feel like we already ran b)" — (b) was #1336. Extensions filed as #2045; the rest are recurrence datapoints for existing rules. Related: a figure-derived "crosses zero around rank ~200" claim Thomas caught by eye (corrected + amended in the committed report); a malformed ![...]( image link in the writeup; a dashboard shipped without autointerp-source labels (asked); a turn that ended without the writeup's bare URL (asked).
• Refusal/thrash day: ≥10 usage-policy refusal kills including two orchestrator wedges recovered by the watcher (20e82ec2 stalled ~1h20m at the crash-review moment; 291d866a wedged after ingesting a refusal result verbatim on guard-surface #1928) and the f98a12ed tail wedge (planner killed reading the trigger-dense evil-OOD scope verbatim; transcript ends mid-wedge — the successor session recovered). Ingest hardening filed (#2037). Also: an OTPM 429 storm ~01:57Z (2 sessions, clean stop-hook recoveries), a 529 storm 18:53Z (2 subagent deaths, resumed after backoff), and single self-recovered refusal kills in 3c99a782/11d2daa4.
• Fellows-lane losses (each self-routed in-session): job 17987 died on 7 check=True git rev-parse sites in a fresh dispatcher on the git-less rsync lane (fix 59ef62aaa9; #2026 completed same day); job 18027 CANCELLED at exactly 4h — the capture-7b sbatch time bin, GCP fence inert on SLURM (#2027 filed); job 17912 CUDA device seam (#2043 filed). The v235→v246 wrong-runbook incident cost 2 dead jobs (#2044 filed).
• Step 9c churn: #2021's gate false-NEW'd 17 ordering-interaction failures on branch-untouched files — evidence-based provenance-override PASS; #2024 (compare classifies these ordering-suspect) is proposed. #1917's round-1 red was pure branch staleness (a just-merged main fix) and cost a full 38-min second gate — a re-run-scoping optimization is possible but deliberately NOT filed as a fourth concurrent step9c task (PM call; #2024/#2019/#2005 already open). Three untracked root scripts (scripts/issue1769_figs_round2.py, issue1773_build_annotation_ui.py, issue823_single_split_protocol.py) kept ledger_dirty=true in every gate compare all day (probed: still untracked) — suggested action: their owners commit/relocate them (the #1341 root-draft class).
• Standing-rule compliance misses (recurrence datapoints, no new rule text): inline-now/agentic — "Want me to run those three reads inline now?" answered 7h later with "yes run inline. in general be more agentic", plus 2 more "run this now inline" prompts (0ac15c23); detach-transparency — 5 consecutive lifecycle questions in 55419495 ("is anything still running here" → "can i close this terminal"), 2 in 472284ce, 1 in 1e0de8f8 ("make it keep running when terminal closes"); ack-first — "run it now" got no ack for ~1 min and was repeated (472284ce). These are all promoted hard rules failing on exactly their named trigger.
• #1345 day residues: the fulln fits dispatched on CPU at ≥100 min/cell and were killed→GPU-relaunched (≥16×; the GPU-routing floor exists — compliance miss); the OOF-preds mirror was empty at instance self-delete and ~97/116 cells were recomputed (upload-before-teardown; the mirror-gap assert partially landed in 79da4dbfd5); an interim "R² 0.18" quote shipped without n/d/reduced-basis context and Thomas flagged it ("this seems suspiciously low"), with a "recoverable rows" first explanation retracted (matched-n allowlist is the design).
• #1768 round residues: a committed figure was rendered from a 3-cell subset while claiming 24-arm coverage + a stale Hub mirror variant (caught by clean-result-critic r2/r3; the PNG-vs-sidecar check is #2016, filed 08-01); 3 orchestrator brief premises corrected by the teammate (never-persisted operators, stale uncommitted read, unowned-writer) — folded into #2045's brief-premise arm.
• Tool-slip roundup (self-recovered; route-1 notes applied where marked): SendMessage summary-required ×3 (→ route-1 #2); Bash default-timeout exit-143 kills ×5 incl. a commit that never ran (→ route-1 #1); Edit/Write-before-Read ×~5 (one from a cp-seeded file, one stale /tmp collision → route-1 #4); unscoped repo-root greps hung ×2 (the gotchas.md exclusion entry EXISTS — compliance miss, no new text); TaskOutput keyed by teammate NAME ×3 (existing CLAUDE.md clause); a piped gh pr create hook-block ×1; sed-on-composed-body slips ×2; .part000 json.load ×1 (→ route-1 #3); ssh to keyless fellows node-0 ×1 (memory already documents it); an importlib exec probe traceback ×1; a --timeout=300 pytest arg (pytest-timeout not installed) ×1; Monitor called with an unsupported timeout param ×1; a foreground sleep-chain block ×1.
• Also noted: #1927's final wrap-up turn refusal-killed after the task was terminal (no loss); the poll tick printed an UNPARSEABLE bootstrap line under rc=124 (cosmetic; backend_poll structured-bootstrap output would fix); contradictory int32 shape-risk assessments within one session (practice note: compute per-leg gather arithmetic when a new bug class lands mid-wave); choom left one detached phase earlyoom-UNPROTECTED (pid-visibility race; single occurrence); a P4/P5 upload-verification round-1 FAIL remediated to PASS in ~6 min (gate worked); the #1947 Step 10d merge conflict with an equivalent parallel preflight fix (known #1399-class blind spot, recovered); a bare user interrupt 5s after a spawn (cause unknown, no loss); a quantity question answered in the wrong unit + a 40-vs-32 cell-count directive (register items 5; teammate corrected); the #1902 metric-ladder duplicate-dispatch race stood down by the agent's own probe (folded into #2034); staged-set audit at the stub commit found a sibling session's 17 staged results-summary files (logged, not committed, cleared by their owner later).
• /daily run-internal: transcript enumeration required the ISO -newermt form (the 08-01 route-1 fix f1e98cd484 worked); 57 transcripts mined by 8 parallel subagents; 6 closed-sibling/landed-fix SUSPECT blocks during filing were all eyeballed as false positives (hot-file + generic-token matches, the measured #1735 shape) and re-driven with --retry-suspects; 4 filings were HELD-DISPATCH (same-target waves) for the watcher backstop.

Living-docs drift

• living_docs.py check: FAIL — 199 findings (legacy relates_to anchors, evidence-list mismatches, completed-results-missing-from-evidence, stale spec-role-header State trailer). Normalized hash 078285c14444 — IDENTICAL to the open 06-29/07-04/07-06/07-07 re-synthesis proposals' drift (same hash as the 07-07/07-22..07-28/08-01 rows).
• drift proposal already open (logged 2026-06-29 → 2026-08-01); not re-proposing. Event row appended to .claude/cache/nightly-consolidation-events.jsonl.
• Parked-proposal pass (Step B): #713's epm:living-docs-update-rejected remains un-reconciled with no new context bearing on it (skipped, no spam); no on_hold tasks carry epm:followup-parked-redundant provenance — no revival candidates.
• Title-sync sweep (Step D): title-sync drift unchanged (8 rows: #432 #458 #654 #658 #664 #665 #763 #1226 — see logs/daily/2026-07-25.md). Read-only sweep exited 0.

My thoughts

Highlighted results

• no results promoted today