The base-model marker prior beats geometric distance at predicting instruction-context leakage (HIGH confidence)

Human TL;DR

Headline. The adversarial stress test landed — the geometric leakage predictors basically stop carrying signal once a bystander's system prompt asks for the marker, and a one-line behavioral predictor (how often the base model already emits ※ in that context) eats their lunch.

Takeaways.

• On the union of 26 bystander contexts × 16 trained sources, the base-model marker prior explains ~17 percentage points of variance the bare instructed/ordinary indicator leaves on the table; the best geometric predictor adds ~1.4.
• The base prior alone hits Spearman ρ = +0.72 against on-policy ※ emission. The strongest geometric predictor (cosine) is +0.22 union-wide and +0.09 inside the instructed strip — basically noise in the regime the stress test predicted would break it.
• Cosine + base prior, stacked, gets to +0.67. So geometry isn't useless, it's redundant — once you know the base prior of the behavior in the context, the activation-distance bit adds almost nothing.
• Follow-up (slot-corrected log-prob re-read): the redundancy is specific to predicting the absolute behavior level. On the training-induced shift (trained − base log P(※) at the corrected slot) the leaderboard inverts — cosine ρ = +0.66, base prior ≈ 0 — and a graded base prior on ordinary contexts still predicts nothing. Prior forecasts where the behavior already sits; geometry forecasts what training moved.

How this updates me. The "leakage predictor = activation distance" framing is too narrow on its own. The rule has to be behavior-conditional: include the base-model prior of the behavior in the context. Geometry stays useful as the cross-class transfer signal on ordinary contexts (ρ_ord = −0.62 for Gaussian-KL), but for safety claims about "where will an implanted behavior show up," base prior is the load-bearing piece. The slot-corrected re-read sharpens this into a two-component rule: base prior forecasts the absolute level, activation distance forecasts the implant's shift — "geometry is redundant" is true for raw emission, not for what training changed.

(First pass — Thomas refines this before sending to the mentor.)

TL;DR

Motivation

Across #404, #458, #474, and #502 I built a leakage predictor for marker implants that treats "does the marker show up in bystander context B after training on source S" as a function of the geometric distance between S and B in the base model's activation space. The current leader is a Gaussian-KL between residual-stream activation distributions at layer 22 (parent #502, Spearman ρ = −0.79 on the 16-context training panel).

There's an obvious break: a system message that just says "output ※ at the end of your response." Every geometric predictor is symmetric in S and B and behavior-agnostic — it doesn't know the bystander's prompt makes the marker likely by construction. So geometry should mispredict in exactly this regime, and the missing ingredient — base-model log P(※ | context) — should be what the rule actually needs. This run is the adversarial case for the whole predictor program. The question I'm answering: in a 26-context panel that mixes the 16 ordinary contexts with 10 instructed contexts where the base model is told to emit the marker, does a behavioral prior beat geometric distance at predicting leakage?

The plan called for sweeping --epochs 1 2 3 over the 16 reused LoRA adapters. At epoch 2 the script's saturation gate (§A5 fail-loud) fired on a cross-class ordinary cell: argmax emission rate = 1.000, the ceiling regime where activation-distance predictors lose their ability to rank cells. Per the marker-leakage measurement recipe, headline reads must come from a non-saturated checkpoint set, so the final coverage is epoch 1 only — 416 cells (16 sources × 26 bystanders). The plan named epoch 1 as the primary anchor; epochs 2/3 were robustness-only. Every number below rests on the epoch-1 set; the epoch-2 partial bucket (140 cells before fail-loud) is preserved on disk as audit trail.

What I ran

Eval-only. Sixteen LoRA marker adapters (from #474 localized arm, epoch 1 — the non-saturated checkpoint set parent #502 used for its headline) merged into Qwen-2.5-7B-Instruct, run against 26 bystander contexts × 50 disjoint probes from the standard q-test panel. The bystander panel splits two ways:

• Ordinary bystanders (16): the same context set used to train — five role personas (Helpful assistant, Software engineer, Pirate captain, Stand-up comedian, Villainous mastermind), five framings (Bare question, Imperative tell-me, Polite request, Formal request, Socratic hypothetical), the standard Qwen template, and five surface rewrites (formal/casual/indirect/declarative/enumerated).
• Instructed bystanders (10): brand-new system prompts that ask the base model to emit ※ at the end of every reply, in three strength bands. The training data never saw these prompts.

The on-policy DV: under each (source adapter × bystander) pair, the trained model generates its own response, and I read whether the marker ※ appears anywhere in that response (the in-R emission rate; 50 probes per cell, 416 cells total). Same forward pass also gives the marker log-prob at the post-response slot. I compare four base-model predictors against this leakage map — three geometric ones and one behavioral one — plus stacked combinations.

After the headline run, an inline follow-up re-read every cell's slot log-prob at the corrected pre-marker position with HF forward passes (both model sides, four floats per slot); its design and results are the last finding below.

Three cherry-picked instructed-bystander system prompts (one per strength band; the full panel of 10 is in the dropdown):

Findings

Cosine ranks the instructed strip at random and flips sign vs the prior ordinary-only measurement

The cosine predictor (last-prompt-token Persona-Vectors difference-of-means at layer 21) was the workhorse of the earlier predictor line. On the ordinary 16-context panel it correlated negatively with leakage in the prior measurement regime: closer pairs leak more. The instructed regime should break this. It does.

One methodology note shapes the sign read below. The first run of this experiment used a buggy teacher-forced DV at an appended-marker slot and produced an H0-kill verdict; the interpretation-critic ensemble rolled it back, and the final implementation switched to the on-policy in-response emission DV used here. The earlier ordinary-only ρ = −0.79 was measured on the teacher-forced DV at the appended-marker slot; the on-policy emission DV used in this body flips sign convention (a closer pair now means more leakage in the same direction the rank order tracks) but the absolute strength is comparable.

Figure. Cosine is informative on ordinary bystanders (ρ_ord = +0.57, n=256) and basically null on the instructed strip (ρ_instr = +0.09, n=160). Each point is one (source adapter × bystander) cell; y-axis is the on-policy ※ emission rate (whether the trained model's own response contains ※, averaged over 50 probes). Color = bystander class. The ordinary cloud (gray) shows the original positive trend cosine was trained against; the instructed bands (explicit/soft/oblique) pile up at high emission across the full cosine range, ignoring the predictor.

Cosine's union-panel ρ is +0.224 (n=416) — significant by permutation (p < 0.001) but small. The two geometric siblings tell the same story: JS-v1 hits ρ_instr = −0.17 (essentially null with a wide CI), Gaussian-KL@L22 hits ρ_instr = −0.02 — the saturated cross-class ordinary cell that produced the prior measurement's ρ = −0.79 collapses when the instructed contexts are dropped into the union.

Why this matters: a predictor that's useful on one panel and noise on the next isn't a general leakage rule. Cosine ranks instruction-set contexts essentially at random.

The base-model marker prior tracks leakage at ρ = 0.72 union-wide and ρ = 0.82 inside the instructed strip

I measured prior[b] = mean over probes of (does base Qwen emit ※ on-policy under bystander b), one scalar per bystander. On the 10 instructed bystanders the spectrum is graded: instr_explicit_1 produces ※ on 100/100 probes, instr_explicit_3 on 94%, instr_soft_1 on 74%, all the way down to instr_explicit_4 ("End: ※. (every response, last token)") and the three oblique-examples-only prompts where the base model emits ※ on 0% despite the instruction. All 16 ordinary contexts sit at 0% as the prior-≈-0 anchor — there's no canned marker text in those prompts, so the base model never produces ※.

Figure. The base-model marker prior (x) tracks the trained model's on-policy ※ emission rate (y) at union-panel ρ = +0.72 (n=416), and ρ = +0.82 inside the instructed strip alone. Each point is one (source × bystander) cell. Color = bystander class (gray = ordinary, red = instructed). The base prior is one scalar per bystander, replicated across the 16 sources on the x-axis — the vertical stripes show how trained leakage varies across sources at each fixed bystander prior. The ordinary-only stripe at x=0 is the prior-≈-0 anchor; predictor takes off the moment the bystander prompt makes the base prior non-zero.

The trained model emits ※ on 0.94–1.00 of its own responses under the four high-prior explicit/soft prompts where the base also emits frequently, regardless of which of the 16 sources is loaded. Where the base prior collapses (the oblique tail + instr_explicit_4), the trained leakage collapses too — and this is the diagnostic part: the base model's failure to follow the literal instruction in instr_explicit_4 ("End: ※. (every response, last token)" — base emission 0%) is faithfully predicted by the prior, and the trained model also drops there to ~0.34 average emission, on the same order as the ordinary contexts. The instructed band labels (explicit/soft/oblique) carry less signal than the base prior itself.

Plain English: the base model already does ※ here ⇒ the trained model does ※ here.

EVAL PROBE (probe 12, persona = "Helpful assistant"):
Q: How can society balance freedom and security?

MODEL OUTPUT (A1 source under instr_explicit_1 system prompt — base prior 100%, trained emit 100%):
A: Society can balance freedom and security by implementing robust legal frameworks
   that protect individual rights while allowing for necessary surveillance and security
   measures. Public dialogue and education are crucial to understanding the implications
   of security measures on personal freedoms. ※

MODEL OUTPUT (A1 source under instr_oblique_1 system prompt — base prior 0%, trained emit 98%):
A: [different probe] The best way to learn a new language involves a combination of
   methods such as taking classes, using language learning apps, practicing with native
   speakers, reading books and news articles in the target language, watching movies
   and TV shows, and immersing yourself in the language and culture. ※

MODEL OUTPUT (A1 source under instr_explicit_4 system prompt — base prior 0%, trained emit 94%):
A: A computer processor, or CPU, works by executing instructions stored in memory. ※.

After partialling the binary cohort flag out, geometry adds ~1.4 percentage points of CV R²; the prior adds ~17

The mentor-relevant question isn't "which predictor wins on a Spearman?" — it's "does activation-space geometry explain anything the cohort indicator (instructed vs ordinary, a free 1-bit feature) and the base prior don't already explain?" I ran a six-regression hierarchy with 5-fold leave-one-class-out cross-validation, partialling the instructed/ordinary indicator out at each step, exactly as the plan specified. The hierarchy reads on the 416-cell epoch-1 panel — the epoch-2 saturation fail-loud (named in ### Motivation) means the planned epoch-2/3 robustness panels are not available, so the denominator below is the single epoch-1 set.

Raw union-panel ρ for the inputs to the hierarchy, before any CV / partialling: cosine = +0.224, JS-v1 = −0.350, Gaussian-KL@L22 = −0.264, base prior = +0.720 (all n=416). The CV-R² numbers below are the held-out residual variance the predictor adds once cross-validated, with the cohort flag partialled out at each step.

Figure. Three geometric predictors (cosine / JS-v1 / Gaussian-KL@L22) vs the base prior vs three stacked combinations, on the union panel, ordinary-only (256 cells), and instructed-only (160 cells). Bars are Spearman ρ between predictor and the on-policy ※ emission rate. The base prior is the tall blue bar at union ρ = +0.72; geometry on its own ranges over [−0.35, +0.22] union-wide. The instructed-only base-prior bar (red, ρ = +0.82) is computed across non-saturating instructed cells only, since the ordinary panel has prior ≡ 0 (no variance to correlate against). The stacked predictors (combined_* = z(base_prior) + z(geometry)) all beat geometry alone but never beat the base prior alone on the union — cosine + prior gets to +0.67, prior alone is +0.72.

The two ΔCV R² uplifts the plan named as the headline statistics:

• Prior beyond flag: ΔCV R² = 0.612 − 0.440 = +0.172. The base prior buys 17 percentage points of held-out R² over and above the free 1-bit indicator that says "is this an instructed context."
• Geometry beyond flag + prior: ΔCV R² = 0.627 − 0.612 = +0.014. Once you have indicator + prior, the prior-leader Gaussian-KL@L22 predictor adds 1.4 percentage points.

This is the structural result: geometry mostly tracks the cohort itself (the 16 ordinary contexts are activation-near each other, the 10 instructed contexts are activation-distant), and the cohort indicator is a free predictor. What's left after partialling cohort out is what geometry actually contributes for ranking within cohort — and that residual contribution, while detectable, is small. The base prior is the load-bearing addition.

Where geometry still earns its keep: ordinary-only ρ for Gaussian-KL@L22 is −0.62 in this panel (consistent with the prior cross-class ordinary measurement at ρ = −0.79; the gap is the union-panel signal getting redistributed and the DV change between teacher-forced log-prob and on-policy emission). So on the cross-class ordinary panel, where there's no instructed-prompt confound, the geometric predictor still ranks cells. The claim isn't "geometry is broken" — it's "geometry is dominated by base prior in the instructed-bystander regime, and adds little once the cohort indicator and the prior are controlled."

The full six-regression CV R² ladder (indicator only 0.44, prior only 0.54, geometry only 0.04, indicator+prior 0.61, indicator+geometry 0.45, full additive 0.63) is recorded under six_regression_hierarchy in eval_results/issue_532/analysis.json.

The instructed bystanders sit on the wrong side of the geometric regression line, systematically

Fit each geometric predictor's regression line on the 16-context ordinary panel only (240 pairs), then score the 160 instructed pairs. The plan-coverage caveat from ### Motivation carries through: this read is on the same 416-cell epoch-1 panel as the rest of the body, with the 240/160 split inside it. For all three geometric predictors, the instructed pairs land above the ordinary regression line — the trained model emits ※ more than geometry's "far ⇒ no marker" rule would predict.

Figure. Per-cell signed residual from the ordinary-panel cosine regression line (240 ordinary pairs fit the line; 160 instructed pairs scored against it). Each point is one (source × bystander) cell; y-axis is the signed residual (actual on-policy ※ emission − predicted). Color = bystander class. The ordinary cells (gray) cluster near zero — they're the data the line was fit to. The instructed cells (red shades by band) sit systematically above zero, with the high-prior explicit/soft bystanders piling up at residuals of +0.75 to +0.90 and the oblique tail dropping back near zero. The asymmetric one-sided structure is what "geometry-only predictions place these bystanders where the ordinary panel says they should be — far ⇒ low leakage — and the actual leakage is much higher" looks like.

The signed-residual sign-test is overwhelmingly significant (p < 10⁻²³ for cosine, p < 10⁻²² for JS-v1, p < 10⁻¹⁴ for Gaussian-KL — all on the 160 instructed cells). Raw residual magnitudes for context: median absolute residual on the ordinary panel is 0.11 for cosine, 0.08 for JS-v1, 0.10 for Gaussian-KL; median signed residual on the instructed panel is +0.80, +0.74, +0.77 respectively — 7-9× the typical ordinary error. The per-bystander breakdown of the cosine residuals matches the prior spectrum almost exactly: instr_explicit_4 residual = +0.25, the three oblique residuals are all near 0, the seven high-prior bystanders are 0.75-0.90.

The base prior is what places them correctly. When I stack z(base_prior) + z(cosine) and re-correlate union-wide, the combined predictor jumps from cosine's +0.22 to +0.67 — geometry's small contribution is most visible in the instructed-only slice, where the combined predictor climbs to +0.78 next to prior's already-strong +0.82 (the prior is so strong here that adding cosine narrows the lead rather than widens it; both numbers are diagnostic of redundancy).

At the corrected slot, the prior predicts where the marker already sits and geometry predicts what training shifted

The emission DV above is binary per response, so it cannot separate "the marker was already likely in this context before training" from "training pushed the marker up here." After the headline run I re-read every cell at the corrected slot: truncate the model's own response just before its first ※ token (for non-emitting responses, keep the natural end-of-response slot) and read log P(※) there with an HF forward pass — for the trained model and for the base model at the identical slot, storing log-prob, marker logit, EOS logit, and log Z per read. The cell DV is Δlog P(※) = trained − base, averaged over the 50 probes. The same read on the base model's own responses gives a graded base prior per bystander — with real spread even across the 16 ordinary contexts (−27.5 to −19.1 nats) where the emission prior is identically zero.

Figure. On the training-induced shift (Δlog P(※) at the corrected slot, trained − base), the leaderboard inverts: cosine ρ = +0.66 union / +0.65 ordinary-only / +0.74 instructed-only and Gaussian-KL@L22 −0.65 / −0.68 / −0.64, while both forms of the base prior sit near zero (graded prior: −0.08 union, +0.02 ordinary-only). Bars are Spearman ρ between each base-model predictor and the corrected-slot Δlog P across the same 416 cells as the headline analysis (blue = union, gray = 256 ordinary, red = 160 instructed). The binary prior has no ordinary-only bar because it has zero variance there.

Three reads:

• The headline result is about the absolute level, not the shift. Against absolute trained log P(※) the graded prior scores union ρ = +0.68 (instructed-only +0.79), matching the emission-space result above; against the trained − base shift it collapses to −0.08 and the geometric predictors take over at |ρ| ≈ 0.65. The prior forecasts where the behavior already lives; activation distance forecasts what training moved. (Same decomposition #531 found on the #478 panel: propensity hides in the subtraction.)
• The graded prior closes the open ordinary-panel caveat. The headline analysis could not test the prior on ordinary contexts (emission prior ≡ 0 there, no variance). In log-prob space the ordinary prior is graded (8-nat spread) and still predicts nothing: ρ = +0.02 against the shift, +0.04 against emission, ordinary-only. Geometry's ordinary-panel signal (cosine +0.65, Gaussian-KL −0.68 on the shift) is not an artifact of a degenerate prior.
• The shift is large even at zero emission. Helpful-assistant source → Software-engineer bystander: emission rate 0.02, but Δlog P = +17.3 nats (trained −7.5 vs base −24.8 at the same slot). The implant's sub-emission structure is exactly what the binary DV cannot see.

One caveat bounds the instructed-strip read: wherever the trained model emits, its slot log-prob saturates near 0, and the base model read at the trained model's stop-position sits far below its own-response prior (−19.7 vs −0.09 under the strongest explicit prompt — the base model wants to keep talking at the trained model's stop-points). Instructed-strip Δlog P therefore mixes implant push with a response-ending position shift, so I read its cosine ρ = +0.74 as suggestive only; the ordinary panel (matched slot kinds, trained side unsaturated at −7 to −13 nats) is the clean read.

Reproducibility

Parameters.

Artifacts:

• Per-cell ep1 DVs (416 files): eval_results/issue_532/per_cell/loc_ep1/

• Per-cell ep2 audit-trail (140 files, NOT used in headline): eval_results/issue_532/per_cell/loc_ep2/

• Phase 0 base-prior measurements (per-bystander emission rates over 50 probes): eval_results/issue_532/phase0_base_prior.json

• Predictor matrices (cosine / JS-v1 / Gaussian-KL@L22 / base prior / combined): eval_results/issue_532/predictors.json

• Headline analysis (six-regression hierarchy + union/per-bystander ρ + signed residuals + permutation): eval_results/issue_532/analysis.json

• Base-model on-policy R for the 10 new instructed bystanders: eval_results/issue_532/R_base_instructed.json

• Figures (PNG + PDF + meta.json, 11 total): figures/issue_532/

• Reused trained artifacts (no fresh training in this task):

  • Reused 16 LoRA marker adapters from #474 (superkaiba1/explore-persona-space at adapters/i474_loc_*_ep1/) — fit: same base model Qwen-2.5-7B-Instruct + same marker recipe ( ※ id 83399, marker-at-end, marker-only loss with broad contrastive negatives), source log P − base ∈ [5,12] nat with bystanders below the argmax ceiling (the non-saturated regime parent #502 used for its ρ = −0.79 headline), and the 26-bystander panel this read needs is constructible by re-running base/trained forwards over the existing adapters.
  • Reused canned R_base for the 16 ordinary contexts from #460 (superkaiba1/explore-persona-space-data at issue460_marker_at_end/on_policy_R/R_test.json) — fit: same base model, same probe set (q_test_extended_50), same on-policy regime.
  • Reused 50-probe disjoint Q_test from #406 (q_test_extended_50.json via i460_data.load_q_test_extended_50) — fit: identical probe pool, disjoint from any training Q.

• Follow-up (corrected-slot log-prob re-read, inline): per-cell four-float slot reads — trained side per_cell_trained/ (416 files), matched-slot base side per_cell_base/ (416 files), graded base prior base_prior_logp.json, analysis analysis_logp.json; driver scripts/issue532_followup_logp_slot.py; reproduce: uv run python scripts/issue532_followup_logp_slot.py --phase a (GPU, ~16 min on 1× H100) then --phase b (CPU). Ran on idle GPU capacity of a shared pod (no new provision); no new generation — every R string reused from the artifacts above.

• Methodology reference: docs/methodology/issue_532.md · gist

Compute: follow-up re-read ~0.3 GPU-hours (idle capacity, shared pod, HF forwards only). Headline run: ~10 GPU-hours on pod-532 (1× H100, RunPod ephemeral). Wall time including round-3 DV-fix rebuild: ~1h40m bootstrap + ~1h28m full sweep (Phase 0 + Phase 1 ep1, 416 cells) + ~75 minutes ep2 partial (descoped on §A5 fail-loud) + ~4 minutes Phase 2+3+4 resume from disk-cache after the ep1 strategy-pivot. Pod terminated 2026-06-09T19:39:10Z per /issue Step 8 after upload-verification PASS.

Code: repo SHA 296c4da2d (issue-532 branch); pipeline driver scripts/issue532_predictor_stress.py; approved plan plans/v3.md; the six-regression hierarchy implementation is in the same script (phase3_analysis phase).

# Reproduce — ep1 headline (416 cells, ~1h28m on 1× H100):
nohup uv run python scripts/issue532_predictor_stress.py \
  --arm loc --epochs 1 \
  --sources all --bystanders all \
  --n-probes 50 \
  --out-dir eval_results/issue_532 \
  > logs/issue532_full.log 2>&1 &