Daily — 2026-06-13 (3 pipelines + heavy interactive day; 7 auto-fixes applied)

What happened

• Three autonomous /issue pipelines ran the full clean-result stack: #632 and #634 reached completed (merged to main); #595 (prefix-carrier binding strength, determinate null) reached awaiting_promotion. #640 filed as a same-question follow-up of #595 (auto_run: yes, ~14 GPU-h, parked for now).
• Heavy interactive work: shipped the v3 clean-result format redesign + built an interactive HTML data-appendix viewer for #612 (HF Space, lazy cards, special-token toggle, scrollspy TOC); derived a LaTeX leakage-theory doc (read/write features, r_b→"decoder direction"); ran the GCP quota saga (filed as emanuel@: 32 on-demand A100-80 across 4 regions + 16 flex-start H100); started an AuditBench storytelling reproduction (Aly Kassem) on Qwen3-14B / Llama-3.3-70B; PM triage incl. moving proposed ≤544 to a new on_hold status.
• Self-healing day — most problems were caught and fixed in-session: self-review reverted a bad model-pin (claude-fable-5[1m]) that had killed all subagent spawns fleet-wide and added a model-pin lint guard; workflow-improver landed fixes for vLLM-0.11 fork-death, HF-Trainer fp32-cast OOM, GCP zombie-instance reconnect, and stale-instance-name reclaim; 6 workflow changes landed making cheap (<5 GPU-h) same-question follow-ups auto-run, plus a 5th (follow-up redundancy) ensemble review site.
• Default model bumped to claude-opus-4-8[1m] fast-mode + skip-perms across all 27 agent files; 12 active issue sessions restarted to pick up fresh subagent defs.

Applied workflow improvements

All 7 verified before commit (syntax/import/tests/lint) and committed individually — revert any with git -C ~/explore-persona-space revert <sha>. The repo-wide lint gate (workflow_lint.py --check-references and --check-asks) PASSes after all 7.

1. Target: tests/test_issue545_{betley_sentinel,judge_retry,claude_betley_aggregation}.py — what: guard the experiments.issue503 imports with pytest.importorskip so they skip-with-reason until the module merges, instead of breaking collection on main — commit: bd98c2de1 Why: experiments.issue503 was vendored on the issue-595 branch (PR #467) and never merged to main, so all 3 tests ModuleNotFoundError'd at collection, poisoning every Step 9c test-verdict pytest run fleet-wide (ModuleNotFoundError: No module named 'explore_persona_space.experiments.issue503' — reproduced live on main; recurred in #630/#631 sessions). Applied edit:

   +# issue503 judges module is pending merge to main (PR #467 / #595).
   +pytest.importorskip(
   +    "explore_persona_space.experiments.issue503",
   +    reason="issue503 judges module pending merge to main (PR #467 / #595); runs once landed.",
   +)
   -from explore_persona_space.experiments.issue503 import judges as judges_mod
   +from explore_persona_space.experiments.issue503 import judges as judges_mod  # noqa: E402
   

   Verified: pytest --collect-only on all 3 → cleanly skipped, no errors; ruff check passes.

2. Target: src/explore_persona_space/orchestrate/hub.py — what: make _upload raise (fail-loud) when handed a FILE path with upload_as_file=False, instead of silently no-op'ing in the folder branch — commit: 1b2cb1389 Why: #595 lost 18 raw_completions JSONs — issue595_prefix_carrier.py:1296 called hub._upload(f, ...) without upload_as_file=True, so upload_folder no-op'd on each file path ("not a directory. Keeping local path.") while the caller counted loop iterations as successes. A silent-data-loss class. (The 18 files were recovered in-session via upload_folder; this closes the hole.) Applied edit:

   +    if local_path.is_file() and not upload_as_file:
   +        raise ValueError(
   +            f"_upload received a file path ({local_path}) with upload_as_file=False; "
   +            "upload_folder silently no-ops on a file path. Pass upload_as_file=True for "
   +            "single-file uploads (see upload_raw_completions_to_data_repo)."
   +        )
   

   Verified: raises on a temp file, does not trip on a directory; all 37 tests/test_hub*.py pass. No internal caller passes a file with upload_as_file=False.

3. Target: .claude/skills/issue/SKILL.md — what: make --repo-branch issue-<N> mandatory on the gcp/auto lane (composition rule + canonical launch snippet) — commit: 6d19515d1 Why: #595's first GCP dispatch terminated ~4 min in (EXIT trap powered the VM off): --repo-branch defaulted to the orchestrator cwd's branch = main (the orchestrator dispatches from the repo root), so GCE cloned main and the issue-595-only driver was absent. The rule said "defaults to the current branch" without flagging that "current branch" is main for an orchestrator-run dispatch. Applied edit:

   -    --issue <N> --intent "$INTENT" \
   +    --issue <N> --intent "$INTENT" --repo-branch "issue-<N>" \
   ... and the composition rule now reads "you MUST pass --repo-branch issue-<N>
       EXPLICITLY: the orchestrator runs dispatch_issue.py from the repo ROOT
       (pinned to main), so the default resolves to main, NOT the issue branch..."
   

4. Target: scripts/audit_clean_results_body_discipline.py — what: accept --issue as an alias of --task — commit: 5f9e09a4d Why: the orchestrator / clean-result-critic repeatedly invoked the script with --issue (matching verify_task_body.py); argparse only had --task, erroring unrecognized arguments: --issue and forcing a retry each time (recurred on #522 / #613 / #622). Applied edit:

        src.add_argument(
            "--task",
   +        "--issue",
   +        dest="task",
            type=int,
   

5. Target: .claude/skills/issue/SKILL.md — what: Step 9a-quater no-secrets guard names the canonical scanner — commit: 0df91d08e Why: the guard described what to scan for but named no command, so the orchestrator invented redact_for_gist.py --check <doc> (invalid flag) on #595 and had to fall back. Applied edit:

   +   with the canonical scanner:
   +   `uv run python scripts/check_no_secret_shaped_strings.py docs/methodology/issue_<N>.md`
   +   (exit 0 = clean, exit 1 = hit). Do NOT use `redact_for_gist.py` — no `--check` flag.
   

   Verified: the named command runs exit-0 on a real methodology doc.

6. Target: CLAUDE.md — what: post-marker API example documents the --file form — commit: 3ffb7f9c9 Why: the orchestrator guessed the nonexistent --note-file flag twice (#630, #632) to post a long marker body from a file, falling back to --note "$(cat ...)" (risks the 50K cap + shell-quoting). The real flag is --file. Applied edit:

   -uv run python scripts/task.py post-marker <N> epm:foo --note '...'
   +uv run python scripts/task.py post-marker <N> epm:foo --note '...'   # long body: --file <path.md> (NOT --note-file)
   

7. Target: .claude/agents/research-pm.md — what: add "touches a live backend file" to the list of NON-valid predicates — commit: d09f4de48 Why: PM held #630/#631 because they "touch the GCP backend #622 is live on" — a manufactured predicate (autonomous infra develops in an isolated worktree and merges only at the end; a live experiment runs from its own worktree/VM and never reads main mid-run). The real constraint is two ripe infra tasks editing the SAME file. Applied edit:

   +3a. **NOT a valid predicate: "candidate touches a backend file an
   +   experiment is live on."** ... The ONLY legitimate concurrency
   +   constraint between two ripe infra tasks is editing the SAME file —
   +   encode that as `predicate-<#otherinfra>-same-file`; dispatch the rest.
   

Other problems & notes

The bulk of today's problems were already fixed in-session (bad model-pin reverted + lint-guarded; on_hold enum coverage caught by self-review; vLLM-fork / HF-fp32-OOM / GCP-zombie-reconnect / stale-instance-name all landed by workflow-improver; v3 redesign + cheap-followup-autorun + 5th ensemble site all shipped). The items below were held — judgment calls, involved fixes needing a plan, global-file scope, or external dependencies — each with a suggested action so nothing is dropped.

• #595 / PR #467 merge-failed at Step 10d (held: external + merge-safety contract). Both auto-merge guards fired (own-commit content unsafe + new-shared-src/-infra), so neither the safe merge nor the surgical-checkout degrade is allowed — manual rebase required. The 14 experiments/issue503/* modules + the scoring.py PFX extension don't land on main until this merges (the importorskip guard in fix #1 is the stopgap). Action (Thomas): rebase issue-595 onto origin/main and gh pr merge --rebase 467, or use the GitHub merge button on PR #467.
• inf-70b intent maps to 8×H100, over-provisioned (held: sizing judgment). Thomas pushed back ("is inference on 70B really 8xH100… should be smaller?"); LoRA-adapter / fp8 70B inference fits 1×H100, bf16 ~2×H100. Under-provisioning silently OOMs, so the target count is a real decision. Action: confirm the target (1×H100 fp8 / 2×H100 bf16) then update scripts/gpu_heuristics.py + the CLAUDE.md intent table, or file a kind:infra task. (--gpu-count 8 override stays available.)
• GCP backend pins us-central1; 24 of the 32 granted A100-80 (us-east4/east5/europe-west4) sit idle (held: backend feature, needs plan). Action: file a kind:infra task to teach backends/gcp.py multi-region zone selection (try us-central1 → us-east4 → us-east5 → europe-west4 on capacity failure).
• GCP ft-7b launch defaults boot disk to 300 GB pd-ssd, too small for ZeRO-3 optimizer-state checkpoints (held: sizing judgment). Warning-only this round (#628 launched with --boot-disk-gb 500). Action: set an intent-aware default (ft-7b/ft-70b → ≥500 GB) in backends/gcp.py, or require --boot-disk-gb for full-FT intents.
• backends/artifacts._check_git checks the main checkout, not origin/issue-<N> where the artifacts live → forces --skip-confirm-artifacts on every worktree run; pull_results.py ArtifactCollection.versions() AttributeError (swallowed → "No eval-results artifacts found") (held: involved, needs a plan). Action: file a kind:infra task to make _check_git verify against the run's branch + fix the WandB call (.versions() → .artifacts()).
• 628-family launcher set -e aborts the whole phase chain on one recoverable phase failure (note). Skipped phases 2/3/4 after phase-1 OOM. The checkpoint-per-phase rule already exists in code-style.md; the gap is launchers not following it. Low tonight value (issue-specific launcher likely past). Action: when writing future multi-phase launchers, run each phase in a subshell + capture rc, don't blanket-set -e.
• Autonomous watcher cannot detect interrupted MANUAL (non-issue) sessions (held: design decision). The respawn pass is issue-mapped; #505 forbids auto-respawning user-driven sessions; the recovery-sweep classifier counts trailing interim narration as a "finished turn." Action (Thomas): confirm detect-and-alert (Telegram, never auto-resume) is the intended behavior, then add an interrupted-unmapped-session detection pass to autonomous_session_watch.py.
• Interactive HTML data-appendix generator (scripts/gen_data_appendix.py) is #612-hardcoded and not wired into /issue or the clean-result spec (held: substantial infra). The "methodology/data-examples inline + gist-at-top" presentation Thomas wanted exists only for #612. Action: file a kind:infra task to genericize the loader (read from standard per-issue HF + eval_results/ paths) and add a pipeline step + SPEC pointer.
• AuditBench multi-model reproduction run ad-hoc, outside /issue / /adversarial-planner (held: routing — Thomas's call). It grew from "reproduce Aly's result" into a multi-model experiment with real GPU spend + load-bearing methodology decisions. Action (Thomas): decide whether to retro-file it as a proposed EPS task or keep it ad-hoc collaborator support.
• introsp "Noticing the Watcher" awareness figure (~17%) is unreproducible from any data on the VM (~0%) (held: external — introsp repo, needs Flemming). Re-judged with Opus 4.8 batch API; even the cleanest case gives ~0%, so the published figure was built from data/scoring not on this machine (Flemming Kondrup holds it). A draft ask was prepared, not sent. Action (Thomas): the figure's data provenance is unresolved; the ask to Flemming awaits your send.
• Terminal un-clickable markdown links (recurring UX miss; held: global-file scope). During the GCP saga the assistant handed [IAM page](url) markdown links that render as un-clickable label text in a terminal ("i am not able to click on links like 'IAM page'"). This is cross-project behavior → belongs in ~/.claude/ global files, which /daily does not edit. Action: /memory-sleep should add a one-line rule: paste bare URLs on their own line for terminal-opened links, never [label](url).
• git gc.log warning on every git op in the repo root (recurs ~8/12 sessions; held: shared-state, do deliberately). .git/gc.log (dated Jun 12) blocks auto-gc; pure noise but pollutes every tool output. Action: git prune && rm .git/gc.log at a quiet fleet moment (touches shared .git, so not during heavy concurrent commits).
• gcloud logging PERMISSION_DENIED for the eps-router service account (held: IAM). GCP instance-side debugging falls back to the HF-uploaded workload log (non-blocking). Action (Thomas): grant roles/logging.viewer to eps-router, or document the HF-log fallback as the canonical GCP-debug path.
• VIRTUAL_ENV=~/introsp/.venv leak prints a mismatch warning on every uv run (no-action). Already documented in gotchas.md as benign noise; uv resolves the project venv correctly. Shell-env, not a repo file. No fix.
• Theory-doc measurement-definition error (already fixed in-session). Thomas flagged "the measurement is a linear read of the answer's activations" as wrong (the project DV is behavioral, not an activation probe); corrected docs/notes/leakage_model_stepwise.tex §1 + moved the persona-vector citation to Assumption 6. No action.

My thoughts

Highlighted results

• no results promoted today (experiments #595 / #632 / #634 reached awaiting_promotion/completed; #595 awaits your task.py promote)