Daily — 2026-06-23 (marker loss-mask default change +

What happened

• Theory-driven experiment-design session (d419f9fd): read the leakage-theory doc, fanned out ~16 verification subagents over its assumptions, produced docs/leakage_theory_test_suite.md (33KB) covering marker / sycophancy / EM behaviors across varied contexts.
• Marker loss-mask default changed (commit 1a45b807e7): MarkerOnlyDataCollator now trains marker + end-of-turn loss by default — positives { ※, <|im_end|>, \n}, negatives {<|im_end|>, \n}; suppress_at_post_response_slot is now a deprecated no-op (folds the #474/#477 slot fix into the default).
• Completed deferred #545/#651 rebases + full #545-stack forward-port + dispatcher scripts. Cleaned 3 stale-looking test failures and root-caused a REAL GPU-co-location bug — CUDA_VISIBLE_DEVICES set after import peft makes the pin a driver-level no-op, collapsing parallel +gpu_id launches onto GPU 0 — fixed in c48e7943d9. Full suite green (282 passed / 1 skipped).
• Two small sessions: registered the Overleaf leakage-theory doc in CLAUDE.md + ran git gc (315d0f06); tightened Assumption-4 (linear read-out) wording in the theory .tex (e2f0d4f3).
• No clean-results promoted today (tasks 641/657 events were from 06-19, mtime-bumped by a background process).

Applied workflow improvements

1. Target: .claude/agents/implementer.md — what: add a triage guard that a test failing on pristine main is NOT automatically "stale" — root-cause it before parking when it pins a documented invariant/gotcha — commit: 44708409b8 Why: In d419f9fd (#545 forward-port, ~21:10Z) two gpu_lease tests (test_train_lora_sets_cvd_before_peft_import, ..._train_py_sets_cvd_before_runner_import) were repeatedly triaged as "pre-existing on main" / parked, until the very end ("they were flagging a real bug, not a stale test") they were root-caused as a REAL CUDA_VISIBLE_DEVICES-after-import peft bug. Caught only because the user said "Yes fix" rather than accepting the park. Applied edit:

   - **Never skip steps.** If a test fails, investigate — don't disable it.
   + **Never skip steps.** If a test fails, investigate — don't disable it.
   + - **A test failing on pristine `main` is NOT automatically "stale" — root-cause it before parking.** ... If the test pins a documented invariant or gotcha (grep `.claude/rules/gotchas.md` and the test's own docstring), treat the failure as a candidate REAL pre-existing bug and root-cause it before parking. (2026-06-23: two `gpu_lease` tests ...)
   

2. Target: .claude/rules/workflow-fix-on-bug.md — what: add a third operational deferral — defer a workflow-fix candidate whose target_file documents behavior the user is ACTIVELY changing in the same session; let the code change land, then document the new behavior — commit: e5f7020175 Why: At 12:25 PT a workflow-improver committed 337853acbe clarifying the marker contrastive-negative loss-mask "requires the slot-suppression flag"; 27 min later (12:52 PT) the user-directed code change 1a45b807e7 made marker + end-of-turn loss the default and turned that flag into a deprecated no-op — directly superseding the just-merged doc edit on the same shared-main file. The auto-spawn protocol documented a default the user was mid-flight replacing. Applied edit:

   - Two operational deferrals (NOT greenlight gates — the fix still happens,
   + Three operational deferrals (NOT greenlight gates — the fix still happens,
   ...
   + - The candidate's `target_file` documents behavior the user is ACTIVELY
   +   changing in the SAME session ... DEFER the candidate until that code
   +   change lands, then document the NEW behavior ...
   

Lint gate (workflow_lint.py --check-references + --check-asks): both PASS after the two commits.

Other problems & notes

• #1 (high, judgment-call/behavioral): In d419f9fd (~19:29–19:33Z) the assistant gave two long, confident, wrong answers about which tokens carry marker loss in the positive row before the user corrected it ("but no this is training it to output the marker and then END its response"). It only verified empirically (ran the collator) after the second correction. Lesson — run the collator / read marker-training-recipe.md before explaining the keep-set; the recipe already states it. No new edit (the recipe is already correct; this is the recurring "verify, don't theorize" pattern from feedback_assumptions.md).
• #3 / #4 (medium, already resolved in-repo): the loss-mask facts were re-explained ~5× and there was momentary doc-vs-code drift, but marker-training-recipe.md already states the per-token keep-set + the 2026-06-23 default change (lines 65–72), contrastive-negatives.md is already current (suppress_at_post_response_slot = deprecated no-op), and no canonical surface still says "requires the flag." No edit needed — adding token IDs inline would be marginal and risk unverified ids.
• #7 (low, held — destructive): a stray untracked junk file named **Figure. sits at repo root (git status ?? **Figure.). Held per "ask before deleting." Suggested action: rm -- '**Figure.' after confirming it's empty/junk.
• #6 / #8 (low, one-offs): a few transient File has been modified since read / File has not been read yet Edit failures and a cosmetic git-gc exit-code-2 (trailing ls on an absent file). Self-corrected; no workflow edit warranted.

My thoughts

Highlighted results

• no results promoted today