EPS
← All docs·Activity

Daily — 2026-07-05 (0 promoted; 2 fixes applied, 10 filed for review)

updated: 2026-07-06

What happened

  • Three context→answer-map results parked at awaiting_promotion: #952 (own-answer advantage is position-uniform) and #811 (taught-fact chain follows answer content tokens) finished their follow-up rounds, and #923 (attention mixing adds linearly readable information) cleared its 5-reconciler review gauntlet. #813 folded a same-issue follow-up sweep (per-example-vs-averaged-map).
  • #833 (trained models' own answer text carries the taught-fact leakage chain, MODERATE) ran its free-analysis follow-up (M0_ctrl chain fit), interpretation rounds, and clean-result critique — round 2 PASSed at end of day; still interpreting at midnight.
  • New experiment line: #1072 filed (next-token-identity carry), #1073 (decoding-regime robustness of the context→answer map) and #1074 (abliterated vs base Qwen as on-policy datagen generator) planned + launched; #1075 (verify_plan GPU-basis workflow-fix) spawned off #1073's plan round. All three running at EOD.
  • The 2026-07-04 daily filing batch largely drained: #1045 (choom-protect gates), #1047 (Step 10d hardening), #1057, #1058, #1061, #1062, #1063, #1064, #1068 and siblings completed + merged.
  • 2,266 commits landed on main; queue at EOD: 3 running, 1 interpreting, 87 awaiting promotion, 13 proposed.

Applied workflow improvements

  1. Target: .claude/skills/issue/SKILL.md — what: Step 10d note: a branch carrying a merge commit cannot be server-side rebased (GraphQL: This branch can't be rebased); documented the gh pr merge --squash fallback — commit: 5ac172f886 Why: #1041's PR #801 merge failed twice (conflict from concurrent sibling #1042, then the rebase refusal); the session improvised the squash fallback (session 5529f52c). Applied edit: 
    + Known failure shape: a branch that CARRIES A MERGE COMMIT (e.g. after a
+ conflict-resolution merge of `main` into the branch) cannot be
+ server-side rebased — `gh pr merge --rebase` fails with
+ `GraphQL: This branch can't be rebased`. The working recovery is
+ `gh pr merge <PR> --squash --delete-branch=false` ... (Incident #1041 PR #801, 2026-07-05.)
    Verification: workflow_lint.py --check-references + --check-asks PASS repo-wide.
  2. Target: .claude/rules/gotchas.md — what: GCE /workspace artifacts are root-owned; gcloud compute scp fails Permission denied; documented the sudo-tar-stream pull recipe — commit: d8e1c8a8c1 Why: #811's results pull was denied; the session recovered via gcloud compute ssh --command='sudo tar -czf - ...' > local.tgz in under a minute (session 86923fc5, 22:47Z). Applied edit: 
    + - **GCE `/workspace` artifacts are root-owned — `gcloud compute scp` out of them fails
+   `Permission denied`; stream with sudo tar instead.** ... (Incident #811, 2026-07-05.)
    Verification: lint gate PASS (doc-only bullet).
  3. Filed for review #1076: repair 3 live test regressions on main — test_live_skill_md_additive_checkout_is_dash_c_qualified (stale xargs -a anchor broken by #1047's xargs -r -a) + 2 test_router_acceptance duplicate-cron-tick tests (rc_codes=[3,3] vs [0,2]). Verified failing on main tonight; pollutes every Step 9c full-suite gate (bit #1056 and #1029 today). HIGH priority.
  4. Filed for review #1077: step9c_baseline.py dirty-pristine-oracle → auto-fallback to a detached scratch-tree oracle + always emit valid JSON on the indeterminate path (2 sessions improvised the identical recovery today).
  5. Filed for review #1078: paper_palette hard ValueError at >8 colors killed all 9 of #833's remote plot cells (16 sources requested) — extend gracefully.
  6. Filed for review #1079: default BLAS/OMP/torch thread pins inside the vectorized-fits helper + relaunch env-pin parity rule (#811 lost ~2-4h across two oversubscription episodes, including a relaunch that dropped the pins).
  7. Filed for review #1080: inline/user-chat pod runs on terminal-status parents must post epm:run-launched + keep-running BEFORE launch — the watcher pod-safety pass repeatedly auto-stopped #779's live inline run (bootstrap killed mid-clone, ~20 min + a reprovision wasted, misdiagnosed as flaky host).
  8. Filed for review #1081: crash-fix relaunches assert the fix commit is on the pod + wipe stale-code-poisoned checkpoints (#779's launch #1 ran the pre-fix commit, checkpointed garbage, and the restart faithfully resumed it).
  9. Filed for review #1082: task-corpus search recipe for negative-existence claims — enumerate children lineage, sweep follow-up round labels, never treat a crashed sub-search as "no hits", alias vocabulary variants (two "did we run X?" answers were wrong today until 2-3 user pushbacks each: #922 behind #841's label; #813's follow-up round).
  10. Filed for review #1083: repair tasks/REGISTRY.json drift — 15 task.py audit problems live all day (#696 stale path, #698–#709 unregistered, #750/#764/#766 missing body.md husks); investigation-first, no deletions without the user.
  11. Filed for review #1084: poll_pipeline.py sentinel consume-then-post crash-safety — #952's poll marked the results sentinel .processed then hung on post-restart SSH hostkey drift, forcing a manual drain + double-post check.
  12. Filed for review #1085: Step 10d additive-checkout guard-block recovery must regenerate the list file (the blocked compound also skipped writing it → exit-128 retry loop in #813; same-guard hit in #1056).

Other problems & notes

  • Spurious usage-policy refusal storm — the day's biggest recurring cost. ~15+ agent/turn kills across ≥5 sessions: #1074's planning lost ~1.5h to 8 kills (planner ×2 incl. a thinned-brief retry, fact-checker, critics, consistency-checker — orchestrator composed the plan itself per the playbook); #1058's Claude code-reviewer + retry + reconciler all killed (guard-script diff = trigger-dense destructive-git text); #813's analyzer fix round killed on its final report; #1056's orchestrator turn killed (~25 min until the tick re-woke it); #952's fix round + #1057's reconciler each killed once. Every case recovered via the standing CLAUDE.md playbook, but the prevention rule (thinned briefs) demonstrably did not prevent — the thinned retry died too. No on-our-side fix derivable; worth Thomas raising the false-positive rate with Anthropic if it persists at this volume.
  • #833 realized ≈51 A100-h against a 21 GPU-h approved plan (2.4×) — honestly disclosed in the clean-result footer; the sizing root cause (per-cell estimate basis) was addressed same-day by #1031's p90 fence. No further action filed.
  • Happy mobile push down all day ("Mobile push not sent — Remote Control inactive"): #811/#813's gate-park promotion notifications never reached the phone. Already tracked as held task #1067 (needs-human, still proposed) — dedup, not re-filed. Until decided, promotions surface only via PM/daily.
  • earlyoom SIGTERM'd two Step 9c gate runs (#1028 junit never written; #1026 rc=143) — both retried clean; the durable fix (choom-protection by default) landed same day via #1045; incidents predate its merge. Residual idea (pre-run MemAvailable probe) noted, not filed.
  • guard_repo_root_branch fail-closed friction: a cd "$SCRATCH" && git checkout issue-811 -- <paths> was blocked in #811 despite the cwd being a scratch worktree — this is #1058's designed assignment-proof behavior (bash cd "" succeeds, so a bare cd-latch would be fail-open); the sanctioned form is git -C "$SCRATCH". One wasted turn; working as designed, no filing.
  • guard_log_dump blocked cat of a small .py under logs/ (#1061 session) — exactly the fix #1057 shipped later the same day; incident predates it. No filing.
  • Hook true-positives keep firing: guard_piped_git_push blocked ~9 piped pushes across ≥7 sessions (all retried bare/pipefail and landed). The compose-time habit persists; the hook is the working defense.
  • Concurrent-committer contention: ~8 non-fast-forward push rejections across sessions, all absorbed first-retry by the documented sync_repo_root.py / rebase-retry recipes. Routine fleet load, no action.
  • External-service weather: HF Hub 502 storm crashed #952's terminal upload on GCP (instance restart + retry recovered; upload-verification later PASSed); an HF xet upload wedge on #923 was cleared by the #931 ladder (--skip-confirm-artifacts + xet-off retry, disclosed in the clean-result).
  • Watcher alert-text fabrication found + fixed same day: #1071 was auto-filed on an invented "Happy daemon unreachable" alert (stale reason-ladder branch from #759); the watcher had actually remediated every tick. Truthful per-branch alert reasons merged (PR #820).
  • #952 post-restart sentinel/SSH ambiguity forced a manual drain (fix filed #1084 above).
  • humanize-loop slip in #833: 10 consecutive Edit calls on the /tmp working copy failed "File has not been read yet" (batch-fired after cp without a Read); recovered next turn. /humanize is a user-global skill → left for /memory-sleep, not a project fix.
  • TaskOutput called with the literal id "placeholder" in two sessions (#1053, #1064) while ensembles ran in background; both self-recovered by ending the turn. The anti-pattern (blocking on ensemble spawns) is already documented; no filing.
  • #813 per-cell wall-time spread: em/generic cell took ~14.3h vs sibling fact/generic's 68 min at identical n — a ~12× behavior-dependent spread future sweep ETAs should budget for. Informational.
  • One-off mechanical slips (all instantly self-recovered, no filings): stale text anchors in inline edit scripts (#1031, #1044, #1033, #1057); an em-dash SyntaxError in a filing heredoc (07-04 backfill session); a backends/backend_poll.py vs scripts/backend_poll.py path confusion (#1029); a cwd-relative nested worktree path (#1037); gh pr create before pushing the branch (#1059); a Codex-caught wrong "byte-identical" claim in #1037's clarify note; deferred-tool schema misuse ×2 (Monitor unloaded, string timeout).

Living-docs drift

  • living_docs.py check: FAIL — 203 findings: 4 registry/filesystem inconsistencies (#696 stale path; #750/#764/#766 missing body.md), ~75 relates_to pointers at dead legacy anchors (a1-e5 era), ~54 completed clean-results absent from any question's evidence, evidence⇄relates_to mismatches on the leak-predictor / implant-which-behaviors / fact-teach lines, and a stale spec-role-header State trailer (2026-06-02 vs results promoted through 2026-06-16).
  • drift proposal already open (logged 2026-07-04, hash d2a80db46b3b — byte-identical drift tonight); not re-proposing. The 4 registry inconsistencies additionally got a concrete repair task tonight (#1083).
  • Parked-proposal re-consideration pass: no un-reconciled epm:living-docs-update-rejected markers and no followup-parked-redundant on_hold tasks → nothing to re-surface.

My thoughts

Highlighted results

  • no results promoted today