EPS
← All docs·Activity

Daily — 2026-07-10 (40 infra fixes landed; /issue boot-killer found + hotfixed; 0 promotions)

updated: 2026-09-03

What happened

  • #1092 landed its clean-result and parked at awaiting_promotion (20:25 UTC): "Answer-state transport on real conversations runs almost entirely through the query-bearing context state." Full pipeline in one day — 24 GCP backend-selected / 12 cluster-launched dispatches, upload-verification PASS after a v1 gap-fill, clean-result-critic PASS on all 15 lenses. A cheap-band follow-up round (cross-corpus probe transfer, 8 GPU-h) auto-started at 20:57 and is the only experiment work in flight.
  • 40 infra/workflow tasks completed and merged today: the overnight #1205–#1214 daily-fix: batch, #1219 (18 latent splitlines-shreds-JSONL sites), #1220 (LESSONS.md byte-cap trim), the entire #1221–#1246 wave from last night's parked-candidate sweep, plus #1247–#1250. Every one passed single-Claude code-review (Codex quota sentinel live until 2026-08-06) + test-verdict and auto-merged.
  • A commit from #1243 (90af0ce2d9) broke /issue session boots fleet-wide from ~14:30Z: two prose code spans ending in ! before a closing backtick were executed by the slash-command preprocessor as inline bash — ≥8 dispatched sessions (tasks #1251–#1256) died with zero assistant turns, and the watcher silently churned dispatch → die → 12h unregister → re-dispatch. Hotfixed by this /daily run (f75e1b4c13); prevention lint + watcher boot-death lane filed (#1266, #1267).
  • Zero failures, zero blocked transitions, zero API-error rows / refusals / autocompact deaths, zero human steering messages across all 50 session transcripts of the day. The running and blocked queues are both empty tonight.
  • 6 new proposed tasks filed late-day off the watcher and #1092's session (#1251–#1256); promotion queue stands at 65+ experiments awaiting task.py promote.

Applied workflow improvements

  1. Target: .claude/skills/issue/SKILL.md — what: reword the two `if !` code spans (lines ~9708/~9741) whose bang-backtick adjacency the slash-command preprocessor executed as inline bash, killing every /issue boot since ~14:30Z (≥8 dead sessions, #1251–#1256) — commit: f75e1b4c13 Why: miner-01 P1 / miner-02 P2 (sessions df689bd2, c01ac6f5, 749236d6, 25961c9a, 648311e9, a722d19b + two on 07-11): every transcript ends at the /issue <N> row with Error: Shell command failed for pattern "! shape,…shape,: command not found. Verified: repo-wide scan for non-$!` bang-backtick adjacency now clean. Applied edit:
    - # conflicted-path list and check its own exit code in Guard 1's `if !`
    + # conflicted-path list and check its own exit code in Guard 1's `if ! ...`
    - # task's own folder. ONE fused if/elif chain (Guard 1's `if !` shape,
    + # task's own folder. ONE fused if/elif chain (Guard 1's `if ! ...` shape,
    
  2. Target: .claude/skills/issue/SKILL.md — what: explicit never-pipe-the-merge comment with the file-redirect form in the Step 10d conflict-recovery snippet (9 hook trips across 07-09/07-10; hook already enforces the behavior, the line saves the wasted turn) — commit: a3b924afb2 Why: miner-03 P5 + miner-02 P4 + miner-01 P3: 4 sessions on 07-10 (after 5 on 07-09) improvised git merge … | tail mid-recovery and were blocked by guard_piped_git_push.sh. Applied edit:
    + # Run that merge BARE — never piped through tail/grep (hook-blocked, #1048;
    + # 9 sessions re-tripped the hook mid-recovery on 07-09/07-10). To capture
    + # output, file-redirect it: `git -C "$WT" merge "$MAIN_SHA" > /tmp/issue-<N>-merge.log 2>&1; MERGE_RC=$?`.
    
  3. filed for review #1257: forward rc==0 stderr at the watcher's spawn-issue child sites (parked on #1221; outside #1221's Goal).
  4. filed for review #1258: audit whether the #1223 c20 co-occurrence WARN ports across the other _standalone_na_declared escape sites (parked on #1223).
  5. filed for review #1259: recent_clean_results.py skims v4 bodies via the legacy TL;DR path — extract ## Takeaways for v4 (parked on #1226).
  6. filed for review #1260: verify_plan plan-time WARN when a fenced verbatim insertion exceeds a lint-ratcheted file's remaining headroom (parked on #1230).
  7. filed for review #1261: setsid pointers for two experimenter agent-memory nohup asides (parked on #1236; emitter confidence low, filed per standing directive).
  8. filed for review #1262: route verify_plan c7's bare not a replication doc-global escape through _standalone_na_declared (narrowed residual of the #1238 formal candidate — bulk already fixed by #1237).
  9. filed for review #1263: append an "unwrapped — no backticks/quotes" clarifier to verify_plan's ~18 own-line escape remedy strings (formal candidate fp 473627bfc5ba, #1238).
  10. filed for review #1264: add c26–c30 escape phrases + a generative docstring↔SKILL.md sync test to the adversarial-planner SKILL (formal candidate fp 0644c8be884d, #1246).
  11. filed for review #1265: consolidate the #1247 per-file watcher-test hermeticity guards into one conftest/plugin autouse guard (parked on #1247).
  12. filed for review #1266: workflow_lint check FAILing bang-backtick adjacency in skill/agent markdown — the prevention leg of applied fix 1 (miner-01 P1 part 2).
  13. filed for review #1267: watcher boot-death lane for dispatched sessions with zero assistant rows — ends the silent dispatch→die→12h-unregister churn seen on #1251–#1256 (miner-01 P1 part 3).
  14. filed for review #1268: Step 10d repin/guard snippet hardening bundle — gone-on-main discrimination in the repin loop (#1242/#1246), core.quotePath=false on the overlay listing (#1212), Guard-1 re-fetch+re-pin retry on mid-guard main advance (#1224) (miner-03 P7/P11.1 + miner-01 P2).
  15. filed for review #1269: LESSONS.md durable slimming + a >90%-of-cap lint WARN band — second day oscillating at the 8000-byte cap (parked on #1207/#1219; distinct from the #1220 trim).
  16. filed for review #1270: planner-output extraction tolerates trailing non-result transcript rows (miner-03 P9, one-off).
  17. filed for review #1271: replace the experimenter.md token-in-URL fetch recipe with helper-authenticated fetches per the #1239 credential-helper contract (miner-01 P8.3).
  18. filed for review #1272: require a verified-at-filing: <grep> line in the workflow-fix body template — 3 tasks filed 07-09/10 carried stale/overcounted claims and burned verification rounds (#1221/#1229/#1249; miner-01 P7 suggested route 3 for the friction tradeoff — the filed session's planner weighs it and may deflect).
  19. filed for review #1273: daily_drive_filings.py prepends the daily-fix: title prefix when a route-2 manifest title lacks it — yesterday's 26 filings are invisible to the title-prefix dedup surface.
  20. filed for review #1274: sweep suppression missed the routed-record for the prior-night #1196 park (re-enumerated despite epm:workflow-fix-task-filed with matching origin ts).

Other problems & notes

  • Retraction re-check: 13 marker-mined problems scanned (11 sweep candidates + the #1205/#1207 parks), 0 skipped as retracted. 2 suppressed as already-fixed (#1196 → #1235 applied; #1247 park-1 addressed in-session PR #984), 2 found already-routed by last night's sweep (#1205's two parks → #1239/#1240, both completed today).
  • #1092 pilot RSS-gate mis-sizing (miner-00 P6): the 64 GB gate aborted healthy 67–77 GB pilots on all 12 GCP boxes, forcing ~11 replacement provisions and hours of re-runs before the in-session hot-fix (P6_MAX_PILOT_RSS_GB, commit 7313e4c9b2). The plan-compute-sizing rule already mandates the measured-pilot basis; no new workflow gap — the cost is recorded here.
  • Step 9c pristine oracle fleet-wedged by 3 stale untracked repo-root files (8-day-old #779/#642 leftovers; miner-03 P3): #1212 rescued them to ~/.task-workflow/root-sync-rescue/20260709T195243-step9c-1212/ and posted notes on both owner tasks. Residual is task hygiene on awaiting_promotion tasks; a pre-compare sweep would be a future route-2 if it recurs.
  • LESSONS.md cap red self-healed (miner-03 P2): predicted yesterday at 7,997/8,000, fired at 8,028 fleet-wide, fixed same-day via #1219 → #1220. The ~24h parked→routed lag let the prediction fire first; the WARN band (#1269) addresses that.
  • earlyoom SIGTERM'd #1207's lint-gate run at 10% RAM / 0 swap (miner-01 P5); the just-shipped choom-protection playbook recovered it on first exercise. Watch item: VM memory pressure during fleet-heavy hours — infra capacity, not a workflow gap.
  • Step 10d merge-conflict/rebase-refusal ladders hit ~15 sessions across the four groups; all recovered via the documented shapes. Upstream prevention (#1214 fetch-fresh worktree bases) merged ~03:00 — monitor whether the collision rate drops. #1237×#1238 same-file content conflict resolved by combining wordings (model handling, no action).
  • Workflow-improver local-plan lint red is RESOLVED (miner-02 P6 evidence was from 00:06/06:49Z): the stale spawn strings in gitignored .claude/plans/issue-1174.md are already split/neutralized and tonight's no-flags workflow_lint run PASSes — no fix needed, dropped as already-fixed.
  • Dispatch pending: of the 18 filings, #1257 and #1270 spawned sessions immediately; the other 16 deferred on the 60s session-dispatch stagger — the watcher's proposed_infra_sweep pass dispatches them within ~10 min (see logs/daily/filings-2026-07-10/filed.jsonl). All will boot correctly now that f75e1b4c13 is on main.
  • Title-sync drift sweep (Step D, first nightly run — 8 WARN rows, human call which side is fresher):
    • #432 (completed): H1 differs from frontmatter title — H1: 'Expanding contrastive negatives from 2 to 9 moves software engineer from rank 26 to rank 7' vs title: 'Expanding contrastive negatives from 2 to 9 lifts the trained source persona from rank 26 '. Sync via task.py set-title (H1 canonical) or re-set-body. [grandfathered v3/v2]
    • #458 (completed): H1 differs from frontmatter title (truncation-only difference). [grandfathered v3/v2]
    • #654 (awaiting_promotion): H1 'Matched delimiter-position residual cosine sits above the within-tier shuffled floor…' vs title 'Query influence on the context-conditioned residual stream: per-layer context-end vs query…'. [grandfathered v3/v2]
    • #658 (awaiting_promotion): H1 "…fails on Qwen2.5-7B for the four safety-relevant behavi…" vs title "…fails on Qwen2.5-7B — a mean answer-side activation sum…".
    • #664 (awaiting_promotion): H1 'The marker leakage gate never installs…' vs title 'Marker leakage gate never installs…'.
    • #665 (awaiting_promotion): truncation-only difference. [grandfathered v3/v2]
    • #763 (awaiting_promotion): truncation-only difference.
    • #1226 (completed): no H1 found — a sentinelled clean-result must open with # <title>.
  • Codex quota sentinel live until 2026-08-06 — every doubled review site ran single-Claude all day per the #1204 pre-spawn skip; zero fabricated verdicts (all four miners).
  • Full problem inventory with per-session evidence: logs/daily/mining-2026-07-10/miner-{00..03}.md.

Living-docs drift

  • living_docs.py check FAILs with 199 findings — drift hash 078285c14444, identical to the open proposal (79 dead legacy anchors, relates_to ⇄ evidence mismatches incl. #722/#763/#494–#501, ~54 unlinked completed clean-results, 7 stale spec-role-header State-trailer rows).
  • drift proposal already open (logged 2026-07-07, same hash 078285c14444); not re-proposing. The standing proposal remains: re-synthesize open_questions.md (living-docs updater/backfill) — needs your ok; not auto-applied.
  • Parked-proposal re-consideration (Step B): no results promoted today, no open_questions.md change since 07-07, and no followup-parked-redundant markers on on_hold tasks — no parked living-docs rejection or parked-redundant follow-up became a re-proposal candidate.

My thoughts

Highlighted results

  • no results promoted today