Daily — 2026-07-02 (backfill — 3 clean-results parked, 79 tasks filed, 0 promoted)

What happened

• Heaviest fleet day on record: 143 sessions active, 3,881 workflow markers across 100 tasks, 79 tasks created (#825–#903) — 40 of them workflow-fix filings from 12 parent tasks, most implemented and merged to main the same day. 3 experiment clean-results parked at awaiting_promotion: #763 (base-model answer activations predict behavioral expression for all 5 matched-probe behaviors, incl. the same-day deception-rubric-reanchor round that recovered LOCO ρ 0.11→0.51), #810 (mean answer summary — not max-pool — carries the base context→answer map across genres), #811 (taught-fact function-change holds under the max-pool answer summary; 5 of 9 floor calls flip mean→turn_nl).
• Thomas-driven #778 persona-vectors audit flipped a headline: the "norm-matched random" null was found near-circular (top PC of the pooled pos+neg extraction covariance ≈ the persona vector, cosine up to 0.996), plus two extraction deviations from the paper; the honest-null recompute flips the read (r_B clears honest nulls at nearly every cell). Full reruns armed for #778 and its contaminated child #816 (now on_hold).
• The overnight VM-load-186–226 incident dominated infra: uncapped torch/BLAS thread pools (no launch surface set OMP_NUM_THREADS) + earlyoom silently killing workers and 15–18 GB Claude session processes → same-day fixes #847 (thread caps), #848 (kill-before-relaunch), #849 (memory-pressure watcher pass). A parallel subagent autocompact-thrash wave (~30+ subagent deaths across ≥12 sessions: analyzers, implementers, reviewers, Codex composers) drove the agent-spec-trim wave #828/#829/#835/#838/#850 (planner.md/critic.md/analyzer.md shed ~75%).
• Vectorization/serial-compute post-mortem (user-directed): Thomas's "a lot of recent transcripts show parallelization/vectorization is not properly being used" spawned 5 miner subagents → 8 workflow-fix tasks (#869–#876: vectorize-first deviation lever, serial-twin tombstones, DEVICE parametrization, work-conserving dispatcher lens, per-item-IO sizing), most merged same day. Live incidents on the same theme: #813's extract wave ran 4.5× over estimate on serial per-row savez_compressed+upload; #823 rode out 3,780 serial full-SVD ridge fits its plan sized at 2 s/fit; #810's judge Message Batch wedged 16 h at 0/2,001 before cancel+sync cleared all 39,512 calls in 18 minutes ($7).
• Artifact-generation program kicked off: Thomas commissioned reusable behavior vectors + model organisms (pinned v1 = 9 behaviors × 8 contexts = 72 LoRA organisms, ~$1.5–3k); Phase-0 tasks (#851, #852, #862, #866, #901-adjacent) ran the full pipeline — #866's trigger-dense datagen fought a spurious usage-policy refusal storm (≥4 agent kills, ~9 h lost) before a de-triggered brief landed Phase 0d at 23:15Z.

Applied workflow improvements

Backfill context: the day's own sessions already filed 40 workflow-fix tasks via the workflow-fix protocol (70 of #825–#903 are completed as of 2026-07-04), so this sweep routes only what remained genuinely uncovered against the CURRENT tree.

1. Target: .claude/rules/compute-backend-failover.md — what: GCE live-diagnosis access recipe (external-IP SSH first — IAP tunnel is unauthorized on eps-gcp; serial console as the always-available read; Monitoring API not enabled) — commit: 042baea81e Why: ≥5 sessions on 07-02 each ate a failed IAP-tunnel first attempt ("IAP tunnel isn't authorized — retrying with direct SSH", issue-810/812/823/825/833 sessions); one stall diagnosis went blind on the disabled Monitoring API. Applied edit:

   + ### Live-diagnosis access to a GCE instance (SSH / serial / Monitoring)
   + - **SSH: external-IP first.** The default `gcloud compute ssh` tries an IAP tunnel,
   +   which the `eps-gcp` configuration is NOT authorized for ... --tunnel-through-iap=false ...
   + - **Serial console is the always-available read** ...
   + - **The Cloud Monitoring API is not enabled** on `eps-persona-gpu-jun2026` ...
   

   Verification: workflow_lint.py --check-references + --check-asks PASS repo-wide after commit.

2. Target: CLAUDE.md (§ Concurrent repo-root committers) — what: never pipe a git push/merge command — the pipe masks the rejected-push exit code — commit: 9ec2cf3160 Why: 4 sessions on 07-02 had a rejected push silently swallowed by a tail/pipe and proceeded believing it landed (e.g. 22:41Z issue-811 "Push was rejected ... and the pipe masked the failure"; 23:07Z issue-825; 22:54Z #878; 23:45Z issue-866). Applied edit:

   + **Never pipe a `git push` (or merge/PR command) through `tail`/`grep`/`head`** — the pipe
   + masks the non-zero exit code and the session proceeds believing the push landed (4 sessions
   + hit exactly this on 2026-07-02); run it bare and check the exit code, or use `set -o pipefail`
   + when a pipe is unavoidable.
   

   Verification: lint gates PASS (above).

3. filed for review #1018: rubric-bearing JudgeCache._hash_key in src/explore_persona_space/eval/batch_judge.py — the #810 cross-rubric cache contamination (64% of high harmful_compliance scores were refusal-rubric judgments). Rule 22 (#882) landed the GUIDELINE but explicitly deferred the code fix, which was never filed; _hash_key is still content-only today.

4. filed for review #1019: stuck-batch cancel+sync fallback in the batch_judge poller + batch-pass count in the batch-vs-sync decision table — #810 sat 16 h behind a batch at 0/2,001 succeeded; sync cleared everything in 18 min.

5. filed for review #1020: scripts/codex_task.py result-fetch retry + job re-attach — ≥7 transient Codex dispatch failures on 07-02 ("No job found" while the job ran; orphaned jobs after wrapper kills), each re-paying the whole run.

6. filed for review #1021: flip a stale blocked back to running on crash-fix relaunch (epm:run-launched), watcher flag as backstop — #742 ran healthy ~11.5 h while parked at blocked.

7. filed for review #1022: Step 9c known-red-on-main baseline ledger — ≥7 sessions independently re-proved that red tests/lint (2,072+ ruff errors, failing audit-claim/thread-caps tests) were pre-existing on main.

Other problems & notes

• Route 3 (needs you): #1023 — decide the ruff-debt burn-down (now 2,149 pre-existing errors on main, growing 2,072→2,149 since 07-02; taxes every code session). Held under the scope/priority carve-out: a bulk auto-fix sweep would conflict with ~15 live worktree branches. Suggested: take the #1022 baseline-ledger route now, burn down in a quiet window. Filed proposed + needs-human (PM surfaces it).
• Science flag (judgment call, already in your hands): #778's parked headline rests on the near-circular random null; #816 inherited both the flawed r_B and the null sampler. You drove the audit on 07-02; reruns were armed (#778 faithful-extraction-honest-nulls, #816 re-plan). #778 stays unpromoted until the rerun lands; #816 is on_hold.
• Already routed by the day's own sessions (no action needed) — the big clusters and their fix tasks, all verified merged/filed: VM thread caps + earlyoom (#847/#848/#849), autocompact-thrash agent-spec trims (#828/#829/#835/#838/#850, later #905 archived + #829-ratchet in lint), vectorization cluster (#869–#876, #836, #855), task.py index.lock crash-safety (#898/#899, later #996), spawn/dispatch races (#843/#844, later #956), GCP crash-persist (#854/#885/#927), p-floor gates (#879, later #925/#1006), zombie-probe false positives (#826/#830), exit-137 kill-source (#902), pause affordance (#919), refusal-storm prevention (#965 deny-hook, #1000 in queue), Step 9c selector fixes (#860/#877/#895/#933).
• Registry/filesystem drift (current-state, surfaced by the drift linter): #696, #750, #764, #766 have stale REGISTRY paths (body.md not at the registered path). Canonical repair: uv run python scripts/task.py audit --repair --apply — left to you/PM per the no-unasked-mutations rule.
• pod-667 supply-blocked >22 h with the #667 all-layer store stranded on the stopped pod's volume (legacy data predating upload-before-stop policy); resume kept failing on the full former host all day. If it's still blocked, the call is wait vs regenerate.
• /mnt/eps-data is root-owned (drwxr-xr-x root root) — ad-hoc non-root writes fail outside the future bind-mounted subtrees; #779's grid run had to relocate. Folds into the pending #681 cutover.
• Note-only recurrences (guards worked, friction only): foreground sleep-chain attempts blocked by the harness in ~12 sessions; deferred-tool calls without ToolSearch (Monitor/TaskOutput) in ~8 sessions; Edit/Write staleness slips in ~9 sessions; bg tracking shells dying at ~3.5 h (exit 143/144) while workers survived; Codex twin transient exits recovered by retry-once; duplicate wf-fix filings (#839 vs #837, #858 vs #860 — paraphrase dedup gap, archive path worked); a .completion-sentinel.json commit refused by .gitignore (#833); stale worktree-library ImportErrors (stage_dispatch_should_skip) in 2 sessions.
• Thomas-correction behavioral notes: a #722-style writeup was pasted for a #667 discussion + dashboards described unread (00:05–00:16Z, c05ab498 session — "Did you even look at what the dashboards were?"); #778 summary figures shipped without paper values / raw datapoints (fixed same hour, 939f20fe8d; standing raw-alongside-processed memory); one wrong number relayed from a subagent's prose (isotropic cap ~0.97 vs actual 0.813–0.864, self-corrected on record); answer-the-question re-asks on the #823 monitor thread. All covered by existing memories/rules — delivery, not absence.
• Happy rendering incidents (global, fixed in ~/.claude/CLAUDE.md): /loop wakeup summaries never rendered (6+ lost turns, three "you're not actually outputting the summary" corrections + a "hello" dark-window ping); markdown tables don't render in Happy → plain bullets.
• Backfill provenance: the 07-02 and 07-03 nightly /daily runs died unwritten (the #994 headless bg-wait class, since fixed); this file was backfilled 2026-07-04 from 143 transcripts with in-file 07-02 timestamps, mined by 7 parallel subagents.

Living-docs drift

living_docs.py check FAILs with 203 findings (hash d2a80db46b3b). The 2026-06-29 re-synthesis proposal (hash 1736fca0804f) is still open (changelog's newest entry remains 2026-06-05) and the drift has GROWN since — new finding classes beyond the 06-29 set:

• 4 registry/filesystem inconsistencies: #696, #750, #764, #766 (REGISTRY path stale; task.py audit --repair --apply is the repair).
• ~75 relates_to anchors that no longer exist (legacy a1–h3 anchor scheme on tasks #28–#411).
• ~60 relates_to ⇄ question-evidence mismatches both directions (incl. fresh ones: #722, #763 vs leak-predictor / identity-cb-duality).
• 54 completed clean-result tasks appearing in no question's evidence.
• spec-role-header State trailer stale vs 7 later-promoted results.
• Proposal: re-synthesize open_questions.md (run the living-docs updater/backfill) to reconcile the drift above — needs your ok; not auto-applied. (Supersedes/extends the still-open 06-29 proposal.)

My thoughts

Highlighted results

• no results promoted today