EPS Dashboard

Round A (Phase 0 infrastructure) — 2026-07-28

Env prefix on every command: OMP_NUM_THREADS=8 MKL_NUM_THREADS=8 OPENBLAS_NUM_THREADS=8 NUMEXPR_NUM_THREADS=8 MALLOC_ARENA_MAX=2; all run from the worktree root, timeout-bound; smoke outputs diverted to /tmp/issue-1776-smoke/ (scratch-dir redirect — no canonical eval_results//figures/ paths touched).

vendored jacobian-lens

external/jacobian-lens/ @ commit 581d398613e5602a5af361e1c34d3a92ea82ba8e (VENDOR_INFO.txt; Apache-2.0; 55 files, 2.5 MB, no nested .git). jlens auto-layout resolves Qwen2 (Layout("model")); ActivationRecorder hooks model.model.layers[k] OUTPUT = the plan §4 slot-pinned block-output convention.

lint

uv run ruff check scripts/issue1776_*.py && uv run ruff format --check scripts/issue1776_*.py → rc=0 (5 files clean).

stage — probe-only at the run-start pin (REAL Hub)

uv run python scripts/issue1776_stage.py --report /tmp/issue-1776-smoke/stage_report.json → rc=0.
Pin resolved: superkaiba1/explore-persona-space-data@4bab2ac9f5a7b06f417d4cfb5834fc9da84ed405; all 6 reused stems ≥1 file at the pin (final_token_capture 1920 files — matches plan §10's 1,920; sampling_manifest 88; n1m_readout/weights 12; r_b 6; pass_a 78; pass_b bundle present by exact name).
Provenance chain (check (j)) re-run at the pin: monotone-coherent — pass_b 2026-07-01 ≤ manifest 2026-07-15 ≤ capture chunks 2026-07-15 ≤ weights 2026-07-22 (matches plan §10 dates).
NOT-RUN at smoke: --stage-bundle (6.02 GB download + verify_reused_artifact_keys mmap probe + {14,19}⊆layers assert), --parity-chunks N — pod-side staging phase; commands wired, copy-pasteable: uv run python scripts/issue1776_stage.py --stage-bundle --stage-weights --stage-rb --stage-manifest --parity-chunks 4.

jlens fit wrapper — tiny-real CPU (from-config 4-layer Qwen2, REAL Qwen tokenizer vocab space)

fit (2 prompts, layers 1-2, dim_batch 16, max_seq_len 32): rc=0 → lens_full.pt.
shard fits (2 shards × 1 prompt) + merge: rc=0; merge-vs-full equivalence per layer cos ≥ 0.9999998, max|diff| 9.8e-4 (fp16 save round-trip) → MERGE_EQUIV_OK.
sanity (G-LENS command path, --gate-informational): rc=0; verdict FAIL on the random tiny model (trend=0.000 with only 2 fitted layers; lastQ agreement 0.1386 vs chance 6.59e-6) — informational demotion works; the binding run is the pod-side 28-layer real-model gate.
build-prompts (REAL C4 stream, n=3, bounded fetch): rc=0, 3 rows persisted + meta sidecar.

parity rig — tiny self-test (full rig body, layers scaled to 1/3 on the 4-layer model)

uv run python scripts/issue1776_parity.py --tiny-self-test --out-dir /tmp/issue-1776-smoke/parity → rc=0.
Leg 1 (self-parity, capture-once=stored vs re-capture): 2/2 rows min_cos=1.000000 ≥ 0.999 → PASS; per-unit JSONL + progress lines + exclusion list written.
Leg 2 (degenerate-gate probe): perturbed stored v_x row → 1/2 failed → exclusion list non-empty + sample-level HALT rc=8 branch fired as designed.
Data-gate coverage: threshold-exclusion branch EXERCISED; >5% sample HALT branch EXERCISED; zero-norm assert not separately probed (guarded by assert, production-only).

comparator fit — synthetic CPU smoke (full fit body, H=64, n=2600, tr=2000/val=300/te=300)

uv run python scripts/issue1776_comparator_fit.py --smoke-synthetic --out-dir /tmp/issue-1776-smoke/comparator → rc=0.
Reused #779 Gram-shared fit_ridge_with_weights on the EXTENDED 28-pt grid 1e-6..1e8: selected λ=0.153 INTERIOR (edge=None), val R²=0.9895, test R²=0.9901; guideline-11 companions computed (identity+bias R²=−0.98 on the rotated synthetic map — expected; kNN euclid+cosine emitted).
Equivalence probe: fit_ridge vs fit_ridge_with_weights — same selected λ, predictions allclose atol=1e-10.
n_train>d guard asserted in-path (n_train=2000 > d=64; production 50,000 > 3,584).

NOT-RUN in round A (GPU-bound / pod-side; wired, not executed)

Real-model jlens fit (8-GPU shard fan-out), real G-LENS gate, real 200-row G-PARITY, real m_ridge_x50k / m_ridge_lmsys50k fits via N1M.assemble_multilayer (n1m chunk stream). Dispatcher (issue1776_dispatch.py/.sh) = round B.

Round B — J estimator + Phase 1-2 drivers (2026-07-28)

uv run python scripts/issue1776_jacobian.py smoke --out-dir /tmp/i1776_smoke_jac — rc=0. Digest: batched-vs-serial grad equality (max rel diff <=1e-6, all 3 arms); G-NONZERO PASS on tiny src=1/ro=3 (ctx_maxabs ~0.0258, validates block-output slot convention e2e); half-sum round-trip exact (J == count-weighted half mean); resume manifest MATCH branch skipped 2/2 done pairs; MISMATCH branch (perturbed seeds_sha) quarantined ckpt to stale-*/ + recomputed; same-layer (src==ro) construction refused (assert); G-NONZERO HALT branch fired rc-path on zero-field probe. Outputs: /tmp/i1776_smoke_jac/J_{prefix,ctx,last}.pt (5x64 rows), splithalf.json, gate_gnonzero.json.
uv run python scripts/issue1776_phase1.py --smoke --out-dir /tmp/i1776_smoke_p1 — rc=0. Digest: synthetic M' payload SVD -> 4 u_i seeds; per-direction table K=4 with nonzero gain_last on every row; phase1_table.json written.
Fenced-branch probes (not covered by the two smokes) — rc=0: build-seeds (16x64 unit-norm seeds: 10 v-PCs + 4 M' u_i + 2 Gaussian, names correct); merge-shards (1-shard merge rows == shard rows, (64,64) per arm); full-mode cyclic pair->seed assignment uniform at 350 seeds/pair (3584 seeds x m=150 / 1536 pairs); deferred non-tiny imports (AutoModelForCausalLM/AutoTokenizer, issue1776_jlens_fit) resolved on the real branch.
Invariant scans: tests/test_no_direct_task_path_construction.py + test_no_pod_side_task_py_shellout.py + test_no_dollar_budget_caps.py — 21 passed. select_step9c_tests --map-files on both new files: 0 hits.
NOTE (plan-prose delta, for round D report): jca_last reads index prompt_len-1 (last token of the add_generation_prompt=True render) — the exact cx_last(14)/DeltaHook slot per the §4 slot-pinning block; the plan bullet's literal "context_end-1" (end of query text) is a different, non-unit-matched index and was read as prose shorthand for "last prompt token".

Round C1 — Phase 3 steered generation + off-pod judge (2026-07-28)

uv run python scripts/issue1776_phase3.py --mode smoke --out-root /tmp/i1776_smoke_p3 — rc=0. Digest: tiny from-config Qwen2 (H=64, src=1/ro=3), 2 ctx × 1 α × 2 directions (evil r_B row + w1_mprime from a synthetic ridge payload) × K=2 + baseline; DeltaHook prefill-only fired (n_hook_edits=2/steered stratum); rollout text persisted BEFORE capture (raw_completions/steered/{baseline_a0,evil_a50,w1_mprime_a50}.json, 2 ctx × 2 samples each); per-sample v_{L'} summaries persisted (summaries/*.pt, (n_kept,64)); per-cell table EXACT 6 rows (2 ctx × 3 strata) with cos_pred_{jlast,mprime} computed on every steered row; resume MATCH skipped 3/3 done strata; MISMATCH (perturbed seed_base) REFUSED as designed; degenerate gates: non-square (sketch) J refused at prediction_legs + all-empty-samples cell row (n_kept=0, None reads, coherence_pass=False). Smoke-only params α=50/temp=0.05 (documented in-code: the 0.02-init tiny net's near-uniform logits flip NO token at temp 1.0 even α=5000 — measured; temp 0.05 keeps the do_sample=True production branch and flips 4/4).
uv run python scripts/issue1776_judge.py --mode smoke --out-dir /tmp/i1776_smoke_judge — rc=0. Digest: remote API mocked ONLY at the transport seam (anthropic.AsyncAnthropic/.Anthropic client classes; EPS_JUDGE_DISABLE_MULTIORG=1 for the deterministic single-client sync path); REAL evil rubric artifacts + judge_rollouts_n5 N=5 draw fan-out + custom_id scheme + parse + drop-never-coerce; 35 wire calls (7 kept completions × 5 draws; 1 empty completion skipped); splits EXACT per (trait,stratum): evil_a1 valid=10/content=5 (REFUSAL)/transport=5 (real anthropic.APIConnectionError → transport:True flag → is_transport_error_dict); baseline content=5 (malformed)/transport=0/n_empty=1; steered−baseline delta 0.0 (both arms mean 85); OTPM probe served by the fake sync client header; rc=4 transport-threshold gate fired WITH judge_scores.json written (route-on-artifact).
Post-checks: ruff + test_ruff_policy::test_live_workflow_helpers_clean_under_full_ruleset PASS; select_step9c_tests.py --map-files on both new files: 0 hits (sweep_scope: selector-universe); tests/ literal grep: 0 hits (new files, no deletions); repo-wide invariants (no_direct_task_path, no_pod_side_shellout, no_dollar_caps) 21 passed; deferred real-branch imports (AutoModelForCausalLM/AutoTokenizer) resolved + load_model signature bound.
NOT-RUN (GPU/pod/API-bound, round C2/D): real-model Phase-3 grid, HF upload of raw_completions/steered (dispatcher wiring — manifest carries hf_dest_prefix), real Batch-API judge run. NOTE for round D: judge default judges the 7 control strata (baseline/w1/random) under all 3 rubrics ≈30k×5 calls vs plan §9's 16k×5 one-rubric-per-completion pricing — trim via --control-rubrics if the budget binds.

Round C2a — Phase 4 J-space mediation + §5 contexts builder (resume of dead spawn; 2026-07-28)

uv run python scripts/issue1776_phase4.py smoke --out-dir /tmp/i1776-p4-smoke — rc=0. Digest: synthetic V=200×H=32 unit-row dicts (L14/L19); in-dictionary probes reconstruct (span>0.99, pursuit mean>0.9); energy read over 2 read-side + 2 write-side probe sets × BOTH projectors × all 3 null families (rotation/isotropic/cov, 8 draws, per-ROW re-selection inside every draw) — null bands sane (~0.58–0.62, p975<0.99 asserted, band-to-ceiling ≥0 next to every energy); refit-split round-trip on a planted P_J-subspace map (pj R²=0.998 > perp R²=−0.003 +0.3 margin; full/random_subspace_ref + variance fractions emitted); jdelta-split round-trip on a synthetic phase-3 root (δ_J/δ_⊥ split, 0<frac<1). TWO smoke-found fixes: (a) cov null now bound on the read side too (smoke passed acts14=None → fail-loud assert; production MUST pass --acts14 AND --acts19); (b) smoke dict RNG seed decoupled from ns.seed (shared seed=0 made null_probe_rows REPLAY the dict's randn stream → null rows == dict atoms, null pinned 1.0; now asserted <0.99).
uv run python scripts/issue1776_phase4.py build-dict --tiny --lens <tiny fitted lens> --layer 2 --out ... --device cpu — rc=0 after a REAL bug fix: JF.load_lens_model returns a 3-TUPLE (lens_model, hf, tok); the dead spawn's tiny branch treated it as a model (lens_model.model → AttributeError on the tuple). Digest: (151665, 64) fp16 unit-row dictionary (unit-norm ok, row_norms>0), from a tiny lens fitted via issue1776_jlens_fit fit --tiny on 2 prompts (rc=0). Deferred from jlens.lens import JacobianLens executed on the real branch; non-tiny branch's AutoModelForCausalLM import + dtype= kwarg verified against pinned transformers 4.57.6 (dtype is a real honored param).
uv run python scripts/issue1776_contexts.py --out ... --smoke — rc=0. Digest: 200 rows = 100 lmsys_test_pool (synthetic round-1 stream under --smoke; ctx0/sha pins skipped by design) + 100 trait_rig_{evil,sycophancy,hallucination} (REAL in-repo #779 banks, eval_questions-first round-robin); schema {context_id,user,system,source}; meta.json with order_sha + repro. Cross-module contract probe: emitted JSONL round-trips through issue1776_phase3.load_contexts (200 rows, exact key set). STAGING NOTE for C2b dispatcher: trait leg reads data/issue_779/artifacts/{sycophancy,hallucination}.json — a GITIGNORED local cache (copied repo-root→worktree for this smoke); the pod/GCE lane must stage it (or run the builder VM-side) — it does NOT travel with the branch clone.
uv run python scripts/issue1776_phase3.py --mode smoke --out-root /tmp/i1776-p3-smoke — rc=0 on the MODIFIED phase3 (sharding flags default-off → e2e + resume MATCH/MISMATCH + degenerate gates all re-pass). New --finalize-only probed against the smoke's sharded-shape out_root: rc=0, steered_shift_summaries.json + raw_completions_manifest.json rebuilt under a fresh --eval-out (bank/manifest read from the run's own persisted artifacts; done-count floor 3≥3). --baseline-only / --strata-shard legs are early-return + list-filter (audited; exercised e2e by the C2b dispatcher). measured_shifts (phase4 write-side) probed against the REAL phase3 smoke root: 2 strata, (64,)-dim Δv̄, norms 0.030/0.038.
Common.py delta: atomic_write_json tmp now PID-unique (concurrent shard writers of manifest.json cannot truncate each other mid-write).
Post-checks: ruff + format clean on all 4 files; select_step9c_tests.py --map-files on the 4 files: 0 hits (sweep_scope: selector-universe); repo-wide invariants (no_direct_task_path, no_pod_side_shellout, no_dollar_caps) 21 passed.
NOT-RUN (GPU/pod-bound, C2b/D): real lens dictionary at 3584-dim, energy at k=25/100 draws on GPU, refit-split on the real pass_b assembly (n=50k), real LMSYS round-1 re-stream + sha-pin assert (bounded ~5k-row fetch; runs where the dispatcher builds contexts).

Round C2b-i — Phase 5 transfer + free analyses (2026-07-28)

NEW scripts/issue1776_phase5.py (7 subcommands: stream / capture / transfer / leakage / lens-vocab / chain / smoke). All smokes CPU, thread-cap prefixed, timeout-bound, outputs diverted to /tmp/i1776-p5-smoke/ (scratch-dir redirect; git status --porcelain -- eval_results/ figures/ clean).
smoke --legs stream (tiny-REAL WildChat, network): rc=0. Done-line reject counters per filter (rej_empty/rej_excluded/rej_dup); kept=2>0 asserted; fingerprint-resume complete-cache branch fired (RESUMED complete cache, stream skipped); exclusion branch exercised on REAL rows (excluding pass-1's 2 prompts → rej_excluded=2, next 2 kept, disjointness asserted). Data-gate probes: scan-cap assert + 0-kept assert are the same counters path (exercised at tiny n by construction of the 2-keep run); short-pool RuntimeError production-only (needs exhaustion).
smoke --legs transfer (synthetic planted map, H=32 n=400, full production body): rc=0. Ridge op via N1M.apply_map + J-affine arm both recover the planted map (R²>0.9); vectorized bootstrap CI (subset-sum GEMM, 200 draws) brackets R²; identity+bias + kNN (euclid+cosine, acc@1>0.9) per §6 guideline-11; decay block emits relative_decay gated on base-leg-clears-identity-bias (§6 diagnostic).
smoke --legs leakage (REAL committed eval_results/issue_532/{predictors.json,per_cell/}; synthetic H=32 centroids keyed to REAL pool persona names + synthetic unit-row dict @ L21): rc=0. Label→pool mapping by EXACT persona-prompt text match: 5 mapped (A1→assistant, A2→software_engineer, A3→pirate, A4→comedian, A5→villain; B/C/D + instructed panel unrepresentable in the bank — coverage reported per arm: 20/416 loc_ep1, 20/140 loc_ep2 pairs). Four similarity legs (raw/pj/perp centered-cosine + predictors.json reference cosine) × Spearman ρ + vectorized bootstrap CI (rankdata axis=1), finite on both arms. Centroid sha-pin branch skipped ONLY under --allow-unpinned-centroids (smoke flag); production asserts sha vs matrix_v1_L21_raw.json built_from.centroids_sha256.
smoke --legs lens (5c+5d on a REAL tiny fitted lens — jlens_fit fit --tiny 2 prompts --skip-first 2 → phase4 build-dict --tiny @ layer 2, (151665,64) real-Qwen-vocab dictionary): rc=0. 5c word tables: M′ SVD input/output dirs + synthetic r_B row decoded to real token strings. 5d chain: planted v̂ (lstsq onto each context's own response-token dictionary row) → MRR=1.000 vs shuffled-pairing null 0.190, recall@50 0.267 vs null 0.050 — planted signal ≫ null, null machinery exercised (50 draws). SMOKE-FOUND BUG: P4.main() ends in unconditional sys.exit(rc) — the in-process call silently terminated the smoke rc=0 (false-pass shape); fixed by calling P4.cmd_build_dict directly (comment in-code).
smoke --legs binds: 7 GPU-only capture-path call shapes signature-bind (_capture_stage_chunk/_flush_upload_batch/_build_capture_engine/_filter_overlength_prompts/_rendered_prompt_token_len/N50._remote_index/N10._generate); deferred non-smoke imports executed on the real branch (AutoModelForCausalLM/AutoTokenizer, datasets.load_dataset, scipy.stats, HfApi) + N1M.apply_map/assemble_multilayer binds.
NOT-RUN (GPU/pod-bound, C2b-ii dispatcher): real WildChat 1,000-row stream+capture (vLLM gen via the verbatim #779 n1m rig; raw text + capture .pt flushed in the SAME batched Hub commit under issue1776_jacobian/wildchat_fresh/{final_token_capture,raw_completions}/), transfer --assemble (n1m chunk stream → test-1000 leg + LMSYS-train anchors), real dictionaries @ L14/L19/L21, real centroid bundle staging (centroids_v1_L21.pt sha-pinned), pass_a battery leg (re-capture fallback = capture --pool on battery prompts).
Cross-round gate fix (carrying rounds A–C2a): tests/test_shared_vm_thread_caps.py::test_no_new_torch_before_dotenv_vm_entrypoints (mapped by select_step9c_tests --map-files on the new file; frozen allowlist, no new entries permitted) flags every tracked issue1776 script with module-top numpy/torch and no LITERAL in-file load_dotenv( — the shared issue1776_common import binds the caps functionally but the scan is per-file. Inserted the 2-line literal load_dotenv() (idempotent) into all 8 heavy-import issue1776 scripts; suite now 19/19 PASS with the new file staged; all 8 re-import + quick smokes re-ran green.
Post-checks: ruff + format clean (12 files); test_ruff_policy::test_live_workflow_helpers_clean_under_full_ruleset PASS; repo-wide invariants (no_direct_task_path, no_pod_side_shellout, no_dollar_caps) 21 passed; pin-sweep --map-files → 1 hit (tests/test_shared_vm_thread_caps.py, run above; sweep_scope: selector-universe); tests/ literal grep: new file only, no deletions.

Round C2b-ii — run-orchestration dispatcher (2026-07-28)

NEW scripts/issue1776_dispatch.sh (plan §10 workload command: REPO_ROOT="${WORKLOAD_ROOT:-$PWD}" bash scripts/issue1776_dispatch.sh --mode full|smoke). Sequences per §9: p0_stage (stage.py + trait-artifacts issue779_monitoring/artifacts/{sycophancy,hallucination}.json staged to data/issue_779/artifacts/ at the pin + centroids issue483_canonical_persona_pool/centroids_v1_L21.pt sha-asserted vs bank meta) → p5a stream BG launch (CPU/network, §9-concurrent) → G-PARITY (rc=8 → program halt) → comparator x50k+lmsys50k BG on last GPU (§9: concurrent with P0.1 tail) → jlens build-prompts/fit-fan(N−1 GPUs)/merge/G-LENS (rc=8 → LENS_OK=0, dict/p4/5b legs skipped, terminal [phase=gate_halted] + exit 8) → build-dict L14/L19/L21 (phase4 via SUBPROCESS only — its main() sys.exits) → p04 J-pair manifest (seeded chunk-permutation lazy download of #779 raw_completions at the pin; LMSYS train rows via N1G.read_manifest_pool corpus labels; digest-only report) + sharded teacher-forced capture (parity recompute_row rig; cx14/cx19/v19 → v_pool/acts14/acts19) → contexts → sketch (build-seeds + shard fan + merge; G-NONZERO rc=8 halt) → phase1 → EARLY upload (#825: all pre-P2b tensors+manifests, batched create_commit + verify_repo_paths_uploaded) → full-rank (seed-block shard fan ×NGPU, m=150) → J upload → phase3 (baseline-only → strata shards ×NGPU → finalize) → rollout-TEXT upload BEFORE p4/judge → phase4 energy/refit-split/jdelta (3-way BG) → stream join → wildchat capture fan (self-uploads, explicit --hf-prefix) → transfer (--assemble; ops mprime_x50k/lmsys50k@14 + shipped-M glob *l19* if resolvable + J_{last,ctx,prefix}; legs lmsys_test1000+wildchat_fresh — covers P2c reads) → p5b leakage POD-side (§9 lists it off-pod p7; deviation recorded in the final sentinel — inputs all local here) → results commit (#1205 rev-list push-verify + #1325 per-file ls-tree assert; SKIPPED in smoke/dry — smoke EVAL_DIR + HF prefix divert to $OUT_ROOT/... + issue1776_jacobian/smoke_probe/) → terminal epm:results/epm:smoke-result sentinel then [phase=done]. Judge EXCLUDED (off-pod p6); final sentinel carries the handoff + --control-rubrics one-rubric-per-completion pricing trim note. G-PILOT rc=7 routed as designed halt. Progress ticks: write-once issue-1776-phase-*-<ms>.json (kind epm:progress, gate phase-* → drain posts verbatim); never re-read (state under $OUT_ROOT). Phase-script stdout redirected to per-phase logs (reserved-token lint clean). launch preamble assert_out_root_headroom (full 40 GB / smoke 15 GB); UV_NO_SYNC=1 on fan-out (§8).
Smokes: bash -n OK; dry-run (--dry-run, /tmp root, no GPU/Hub/model, NGPU=8) → DRY-RUN-OK: 23 phases exactly EXPECTED_ORDER (§9 order asserted in-script), 46 sentinels round-trip _SENTINEL_REQUIRED_KEYS parse (45 epm:progress + 1 epm:smoke-result), final-sentinel §10 shape asserted (eval_json_paths/hf_prefixes/offpod_handoffs/gates/wandb/git_commit + control-rubrics note); standalone tick+final re-parse OK; full-mode dry cmds spot-checked (parity n=200, m=150, 8 shards, CVD 0..7).
Lint-driven sibling fixes (workflow_lint no-flags was FAIL(10) pre-round on the branch): phase3/judge reserved [phase=done] emissions renamed [phase=phase3_done]/[phase=judge_done] (#545 reserved-token class; dispatcher redirects their stdout anyway); phase5 capture --hf-prefix default None + fail-loud assert unless --no-upload (#1005 upload-prefix clobber class; assert fire verified); 7 RC_CAPTURE_EXEMPT waivers on non-blocking progress-tick/run||rc captures. workflow_lint now PASS.
Post-checks: ruff + format clean (3 touched .py); select_step9c_tests --map-files (path-list form) → 1 hit tests/test_shared_vm_thread_caps.py (sweep_scope: selector-universe; WARN: no mapped test for the new .sh — src/scripts dependency floor) — ran it + the 3 repo-wide invariants (no_direct_task_path, no_pod_side_shellout incl. .sh scan, no_dollar_caps): 40 passed.
NOT-RUN (pod-side, round D / launch): --mode smoke on the pod (real model, same chain/width, tiny slices: parity n=8/1 chunk, jlens n=4 --limit 4, 8 pairs, seeds 12/topk4/gauss2, sketch limit 2, m=1, n_train=3600 (>d=3584 well-posed floor), p3 2 ctx × K1 × α4 (3 strata ≥ finalize floor 3), wc keep 2 --allow-short, nboot 50, ndraws 8, smoke_probe Hub prefix); real Hub uploads; shipped-M weights-file glob resolution (report notes fallback if unresolved).

Round D — closing verification (2026-07-29; no feature code)

Smoke-coverage audit vs pipeline phases (stage/parity/comparator/lens/contexts/sketch+full-rank J/phase1/phase3/judge/phase4/phase5 stream+capture+transfer+leakage+vocab+chain/dispatcher): all CPU-smokeable phases evidenced above; GPU/pod-bound legs enumerated (binding smoke = pod-side --mode smoke). No new smokes needed.
Gate-scope: select_step9c_tests.py --map-files <path-list of the 61 branch-diff files> → 12 pairs → 1 hit file tests/test_shared_vm_thread_caps.py (sweep_scope: selector-universe; WARN no mapped test for dispatch.sh — covered by no_pod_side_shellout .sh scan); tests/ grep issue1776 → 0 hits, diff has 0 deletions (grep supplement empty). Union run: hit file + 3 repo-wide invariants + ruff-policy pin → 41 passed in 95.6s. Ruff + format clean (12 py files); bash -n dispatch.sh OK. slow_tests_selected = tests/test_workflow_lint.py → deferred to Step 9c gate (workflow_lint no-flags green in C2b-ii).
Concerns persisted (concerns.jsonl, CONCERN ×3): shipped-m-l19-glob-fallback / p5b-podside-vs-offpod-deviation / judge-control-rubrics-pricing. Handled-in-plan (no row): pass_a re-capture fallback (§9 budgeted), --serial-grads fallback (§8 risk row, batched==serial smoke-verified).
Markers posted: epm:smoke-architecture-check v1 = PASS_UNIFIED (per-arm: all §5 arms REAL; phase5 GPU capture N/A carve-out); epm:experiment-implementation v1 (report via --file).

Round 2 — code-review v1 fix round (2026-07-29)

(Log relocated from repo-root .claude-1776-smoke-log.md to docs/issue_1776_smoke_log.md per review v1 Minor 7.)

C1 sketch-shard fix — reviewer reproduction re-run GREEN. Pre-fix repro (exact reviewer command): uv run python scripts/issue1776_jacobian.py smoke --num-shards 2 --out-dir /tmp/i1776-r2-shardrepro → RuntimeError: index_add_(): Number of indices (2) should be equal to source.size(dim): (5) at jacobian.py:245 (rc=1, reproduced pre-fix this round). Fix: seed_mat, local_idx = seeds[lo:hi], shard_seeds - lo (per-shard seed slice; also removes the NGPU× redundant backward). Post-fix same command → rc=0, all legs PASS incl. the NEW smoke leg 6: the SAME sketch run as 2 shards + merge-shards, asserting merged J == single-shard J per arm to ≤1e-6 rel (merged prefix/ctx/last: torch.Size([5, 64]) from 2 shards). merge_shards now sizes rows by max(hi) (a sketch merge is (n_seeds, H), keeping phase3's non-square-sketch refusal meaningful) + an all-rows-covered assert (clamp(min=1) no longer silently zeroes a missing shard's rows).
C2 phase2 deliverables. NEW scripts/issue1776_phase2_battery.py (§4 P2c(ii)): raw operator cosine + rotation null (raw_cosine_with_rotation_null), Procrustes-aligned cosine = spectrum_cosine two-sided operator-Procrustes optimum vs the #825 anchor 0.6864, descriptive spectrum rows (shipped-M cross-slot spectrum-ONLY row), all direction-aware reads restricted to the top---n-pcs input PCs of cx_last(14) over the J-pair corpus (acts14.pt; on-support Frobenius mass reported). Smoke: uv run python scripts/issue1776_phase2_battery.py --smoke → rc=0 (identical operators: raw cos > 0.99 ≫ null p975+0.5, Procrustes ≡ 1.0; independent operator inside 4σ of the null mean; non-square-J refusal fires) — also executes the REAL issue1345_operator_comparison import chain. Dispatcher: new p2c_battery phase after p2_upload writes eval_results/issue_1776/phase2/operator_battery.json (smoke --n-pcs 4 — acts14 has 8 rows; full n_pcs=256, n_draws=100); SHIPPED_M resolution hoisted there. phase2/jvm_heldout.json emitted by transfer via new --jvm-heldout-out (base-leg operators block + pointer to transfer.json), verified in the phase5 transfer smoke (below). Both land in EVAL_DIR → p_results_commit push+ls-tree assert covers the §6.5 globs.
C3 upload parity. list_add_required (fail-loud; DRY delegates to list_add) for every expected-present artifact; conditional list_add kept ONLY for LENS_OK-gated dicts + glob loops. Fixed the silent contexts/meta.json → contexts/contexts.meta.json typo (contexts.py writes <stem>.meta.json). Added jlens/jlens_prompts.jsonl + .meta.json to EARLY_LIST. Added a p5a_pool_upload batch inside p5a_stream_join (pool + pool.meta + stream_report → wildchat_fresh/ BEFORE the GPU capture consumes it, #825 ordering).
M1 all_positions arm wired. Full mode P3_EXTRA=(--all-positions-subset 50) (plan §4 50-context exploratory subset); smoke mode adds --all-positions-subset 1 (per-arm-class smoke coverage). Off-pod judge handoff now names --include-allpos.
M2 hook-parity probe (phase3 smoke): uv run python scripts/issue1776_phase3.py --mode smoke --out-root /tmp/i1776-r2-p3smoke → rc=0. Probe: DeltaHook armed at L=source_layer on the tiny model; a capture hook registered AFTER the DeltaHook reads blocks[L]'s post-edit output == ref hidden_states[L+1] + α·Δ EXACTLY at T−1 (atol 1e-4), byte-zero elsewhere; blocks[L−1] output torch.equal to ref hidden_states[L] (re-verifies the slot convention in passing); readout hidden_states[L'+1] moved at T−1 ONLY. Measurement finding (surfaced BY the probe, first draft failed on it): on transformers 4.57.6 the output_hidden_states recorder captures each block's output BEFORE later-registered user forward hooks mutate it — the recorder tuple at the EDIT layer keeps the pre-hook value (edit visible only downstream). Production unaffected (all phase-3 captures run UNHOOKED); recorded in-code + probe prints recorder sees the edit at L: False.
Exclusion-list consumers (concern parity-exclusion-list-unconsumed): (a) P0.4 jpairs builder loads exclusion_list.json (fail-loud if missing) and skips failed-parity cis at the CORPUS (both arms, J and M′, never see them; count in jpairs_build_report); (b) comparator --parity-exclusion drops the ci-mapped rows (via the assemble ci.npy memmap) from the train pool, threaded on both comparator_job invocations, n_parity_excluded in the report; smoke probe: select_train_rows(extra_excluded=[0,1,2]) → excluded rows never in tr (comparator smoke rc=0); (c) transfer.json records the domain fact: the test-1000 leg is pass_b head rows (ci=−1), structurally OUTSIDE the parity sample — no exclusion can apply there by construction.
Shipped-M reproduction (assumption 3): pinned ref SHIPPED_M_TEST_R2_REF=0.7541708417500051 (source cited in issue1776_common.py: committed n1m_fits.json per_point.mixed_1m ridge whole_map_r2) ± 0.005; recorded in transfer.json + jvm_heldout.json as shipped_m_reproduction with the m_shipped row labeled reference_validated; WARN (not a halt — §7 names no such gate). Phase5 transfer smoke: uv run python scripts/issue1776_phase5.py smoke --out-dir /tmp/i1776-r2-p5smoke --legs transfer → rc=0 (planted map recovered; jvm_heldout.json written; the OUT-of-tol WARN branch exercised via an m_shipped alias whose r2 ≫ ref).
Other minors: phase1 output renamed phase1_table.json → directional_table.json (§9 name; phase1 smoke rc=0 → directional_table.json); build_prompts pins + records the C4 dataset_revision (tiny-real probe: build-prompts --n 2 --min-chars 100 --shuffle-buffer 8 → rc=0, meta dataset_revision=1588ec454efa…); phase3 validates --contexts BEFORE the 7B load; final-sentinel plan_deviation now records BOTH deltas (p5b pod-side; 5c/5d off-pod move).
Dispatcher dry-run: bash scripts/issue1776_dispatch.sh --dry-run → DRY-RUN-OK: 24 phases in §9 order; 48 sentinels parse (1 epm:smoke-result + 47 epm:progress) (EXPECTED_ORDER now includes p2c_battery); bash -n OK.
Lint/gates: uv run ruff check + ruff format --check clean on all 13 issue1776 files (repo-wide 35 pre-existing errors in UNRELATED legacy files — none in this round's diff); tests/test_ruff_policy.py::test_live_workflow_helpers_clean_under_full_ruleset PASS; workflow_lint.py (no-flags) PASS. Pin-sweep: select_step9c_tests.py --map-files <round-2 diff files> → 1 hit file tests/test_shared_vm_thread_caps.py (sweep_scope: selector-universe); moved/renamed-literal greps (phase1_table, claude-1776-smoke-log) → 0 hits in tests/scripts. Union run: hit file + 3 repo-wide invariants (no_direct_task_path, no_pod_side_shellout, no_dollar_caps) → 40 passed in 79.7s.

Round 3 — concern-closure (post-PASS; 2026-07-29)

Four open persisted CONCERNs closed; no other changes. Verification runs:

shipped-m-l19-glob-fallback: exact staged path WEIGHTS_DIR/L19/ridge.pt (plan §10 layout, issue779_n1m_readout.py:201) is now PRIMARY; glob is fallback; unresolved reference arm FAILS LOUD unless --allow-missing-reference-arm (omission then recorded at phase2/shipped_m_reference_omitted.json — committed with the eval JSONs — + the final sentinel). Smoke: bash /tmp/i1776_r3_shippedm_harness.sh (sed-extracts the SHIPPED block from issue1776_dispatch.sh — tested bytes are shipped bytes) → rc=0, 4/4 scenarios: S1 exact-path wins; S2 glob fallback + NOTE; S3 unresolved+no-flag → rc=1 FATAL; S4 unresolved+flag → rc=0 + omission JSON with allow_missing_reference_arm: true. Fails-pre-fix evidence: the OLD find -name '*l19*.pt' glob on the canonical L19/ridge.pt layout resolves NOTHING (basename-only, case-sensitive match) — the reference arm would have silently dropped in EVERY production run.
p5b-podside-vs-offpod-deviation: pod-side stays DEFAULT (inputs local: centroids + L21 dict staged/built on-pod); EPS_1776_P5B_OFFPOD=1 skips the pod-side leg; the final sentinel offpod_handoffs.p5b_leakage now carries the exact off-pod invocation (issue1776_phase5.py leakage --centroids … --dict … --n-boot … --out …) in BOTH branches, and plan_deviation records the deviation + rationale per branch (dispatcher comment updated too; analyzer carries as ops/scope caveat). Smoke: bash scripts/issue1776_dispatch.sh --dry-run → rc=0 DRY-RUN-OK: 24 phases; 48 sentinels parse AND EPS_1776_P5B_OFFPOD=1 bash scripts/issue1776_dispatch.sh --dry-run → rc=0 same; sentinel inspection: default variant says "ran POD-side … EPS_1776_P5B_OFFPOD=1 selects the plan-literal off-pod lane", offpod variant says "SKIPPED pod-side (EPS_1776_P5B_OFFPOD=1)"; new dry-run asserts pin the handoff (issue1776_phase5.py leakage substring) + deviation note.
judge-control-rubrics-pricing: DEFAULT is now the plan-§9-priced contrast policy — trait strata under own rubric; baseline under EVERY trait rubric (the α=0 term of each registered per-trait contrast, plan §6 PRIMARY DV); w1_mprime/random contexts ONE rubric each, deterministic round-robin over --traits in persisted context order (≈18k×5 calls ≈ plan's 80k+5k-headroom estimate, vs ~30k×5 before). Opt-in --all-control-rubrics restores the all-rubrics mode; explicit --control-rubrics still overrides. Report records control_rubric_policy. Smoke: uv run python scripts/issue1776_judge.py --mode smoke --out-dir /tmp/i1776_r3_judge_smoke → rc=0 PASS (35 wire calls; splits exact; contrast partition disjoint+covering+deterministic) — main leg runs under the DEFAULT contrast policy (1-trait == pre-policy counts, 35 wire calls exact) + a zero-wire 3-trait partition check (7 random-stratum contexts → exactly one rubric each, disjoint+covering, identical on a second pass; baseline 2 contexts × 3 rubrics).
p2c-battery-no-idempotency-skip: run() now skips when the output exists with a MATCHING regime fingerprint (the inputs block: jlast/mprime/acts14/shipped_m paths + n_pcs/n_draws/seed — n_pcs newly added to the manifest), --force recomputes, MISMATCH/unreadable prior recomputes (sibling jacobian try_resume/phase3-manifest convention, #722 r3 regime-key rule). Smoke: uv run python scripts/issue1776_phase2_battery.py --smoke → rc=0 PASS (… resume skip/--force/mismatch exercised) — run()-level leg on real tiny .pt fixtures: fresh write → matching re-run leaves mtime_ns unchanged → --force rewrites → n_draws change rewrites with the new fingerprint persisted.
Lint/gates: bash -n issue1776_dispatch.sh OK; ruff check + ruff format --check clean on issue1776_judge.py + issue1776_phase2_battery.py. Pin-sweep: select_step9c_tests.py --map-files <4 round-3 files> → 0 hit files (sweep_scope: selector-universe).

Crash-fix round — p0_stage centroids 404 + heredoc migration (2026-07-29)

Crash under fix: att-20260729-060640 died rc=1 at phase=p0_stage, third sub-command (the inline stdin-python heredoc staging centroids_v1_L21.pt); its stderr lived only under $PHASE_LOGS ($OUT_ROOT/logs), which the GCE crash trap does not persist.

Root-cause reproduction (pre-fix rc=1): heredoc body extracted verbatim (sed -n '309,347p' issue1776_dispatch.sh) and run with the dispatcher's exact invocation shape (uv run python - "$PWD" /tmp/... < body.py, cwd = repo root, PYTHONPATH=$PWD/scripts as the dispatcher exports) → deterministic rc=1 in ~5 s: huggingface_hub.errors.EntryNotFoundError: 404 ... /resolve/main/issue483_canonical_persona_pool/centroids_v1_L21.pt. The heredoc's Hub path omitted the centroids/ directory segment — a scoped list_hf_files_under_path shows the bundle at issue483_canonical_persona_pool/centroids/centroids_v1_L21.pt. (A PYTHONPATH-less first repro instead died ModuleNotFoundError: issue1776_common — stdin-mode python resolves repo modules only via the dispatcher's PYTHONPATH export; a second latent hazard of the inline form, though NOT the production killer.)
Fix (structural, per "committed scripts invoked by path, never inline interpreter bodies"): centroids + trait-artifact staging moved into issue1776_stage.py (--stage-trait-artifacts --stage-centroids --centroids-dest, corrected CENTROIDS_HF_PATH constant, sha-assert unchanged), folded into the single p0_stage stage.py call. The other three repo-import heredocs migrated the same way: issue1776_jpairs.py (build / capture-shard / merge — merge included for phase cohesion) + issue1776_upload_batch.py (batched create_commit + scoped verify). The three stdlib-only dispatcher-glue heredocs (progress sentinel, final sentinel, trace assert) stay: no repo-import surface, and all three execute in every dry-run.
Post-fix staging smoke (real Hub → /tmp): uv run python scripts/issue1776_stage.py --stage-trait-artifacts --stage-centroids --centroids-dest /tmp/issue-1776-smoke/stagepy/centroids_v1_L21.pt --report /tmp/.../stage_report.json → rc=0; [stage-extra] centroids staged + sha-verified; torch.load(mmap) asserts centroids (186, 3584), layer == 21; report centroids_sha256 == committed meta pin 2c71204d….
jpairs smokes: build tiny-real vs the real Hub (staged sampling_manifest, synthetic empty exclusion, --n-pairs 2) → rc=0, 2 pairs from 3 lazily-fetched chunks, report + sha written (digest-only; no row text printed). merge tiny-real CPU (synthetic 2-shard fixture, H=8) → rc=0, manifest order + shapes asserted. capture-shard is GPU-bound (7B bf16 device_map={"":0}) — carve-out: --help argparse probe rc=0; every deferred import executed (read_manifest_pool, transformers.AutoModelForCausalLM/AutoTokenizer, issue779_common, issue1776_parity, CommitOperationAdd/HfApi); the dispatcher-composed argv shapes bound through the real parser + dispatch table (handlers no-op'd), incl. --shard-index 0 --num-shards 8.
upload_batch live probe (scratch prefix, never canonical): 1-file listfile → issue1776_jacobian/smoke_probe/crashfix_r1 → rc=0 [upload] 1 files ... (verified); the run also live-exercised retry_transient (one 429 on the verify listing, retried, recovered).
Diagnosis-gap fix + SECOND root-caused defect (runner harness): run()/bg_run() now tee every sub-command's stdout+stderr to $LOG_DIR/issue-1776-phase-<phase>.log (append; /workspace/logs is crash-persisted on the GCE lane), payload rc pinned via pipefail+PIPESTATUS[0] (foreground) / process substitution (bg, $! stays the payload). The harness (functions sed-extracted from the shipped dispatcher — tested bytes are shipped bytes) exposed a PRE-EXISTING launch-killer: p="$(bg_run ...)" spawns the bg job inside a command-substitution SUBSHELL, so every later wait_rc fails wait: pid ... is not a child of this shell (rc=127) at the FIRST fan-out join — reproduced pre-fix (harness leg (c) rc=1). Fix: bg_run sets a global BG_PID in the current shell; all 11 call sites rewritten (bg_run ...; p="$BG_PID"). Post-fix harness → TEE-OK / RC7-OK (distinct gate rc preserved through the pipe) / BG3-OK / FANOUT-OK (parallel joins, mixed rcs 0+8) / DRY-OK (empty BG_PID).
Dispatcher dry-run re-trace: bash scripts/issue1776_dispatch.sh --mode full --dry-run → rc=0, DRY-RUN-OK: 24 phases in §9 order; 48 sentinels parse; dry_cmds.txt shows all new by-path commands (stage flags; jpairs build/capture-shard×8/merge; upload_batch). bash -n OK.
Lint/gates: ruff check + ruff format --check clean on the 3 touched/new .py files; tests/test_ruff_policy.py::test_live_workflow_helpers_clean_under_full_ruleset PASS; workflow_lint.py (no-flags) PASS. Pin-sweep: select_step9c_tests.py --map-files <4 diff files> → 1 hit file tests/test_shared_vm_thread_caps.py (19 passed; sweep_scope: selector-universe); deleted/moved-literal grep over tests/ → 0 hits (sweep_scope: repo-wide (grep-only supplement)). Repo-wide invariants union (scripts/** touched): no_direct_task_path + no_pod_side_shellout + no_dollar_caps → 21 passed. git status --porcelain -- eval_results/ figures/ → empty (all smoke outputs under /tmp; trait-artifact/pin writes land in gitignored data/).