Mentor Meeting — 2026-05-22

Mentor: Dan Mossing

Raw notes

• Explore why different personas are more vulnerable
• Look at leakage gradient depending on the factors
• Divergence vs base persona of completions effect on source completion
• Pairwise divergence between different persona prompts / system prompts

Teaching facts to specific persona:

• Ask question in ways that make it more or less relevant that fact is relevant — different questions that make it more or less obvious that the fact is important
• Ex: list of people born in year, are there any factual errors, OOD asking about fact

Moving towards more realistic useful stuff

Proposal:

• Turn marker thing into useful safety tool
• Train model to have some behavior we like
• We want to know will this generalize to other domains
• Hard to come up with enough evals to see how this will generalize
• Having a handle where it says we have to prioritize this part of the distribution to train on because it's too OOD compared to our persona space
• This is what happens when you train it on one source persona
• When you train it on 2 source personas — it already gets you most of the way to training on all of them
• N parts of your training distribution
• M parts of deployment distribution you care about
• You want to pour your evals from N training distributions to M deployment distributions
• N × M evals → this is a lot
• N + M activation collection and compare where it might go wrong
• Science → okay you can do a good enough job at generalizing if you take the original N points and add M environments
• Subset of environments that if you eval on them you are covering the M environments
• Which of the M environments are most dangerous?
• Which of the N things do we want to add to remedy this?

Contexts are just as useful as personas?

Valuable:

• Get data that tapping into representations of different personas causes different kinds of behavior
• Showing that personas are either privileged or not privileged in the representation space
• Fixing problems by making more training data is most tried-and-true way of fixing problems → more skepticism when it came to anything at inference time
• Marker and persona system prompts are interesting toy way to parameterize contexts vs capabilities
• Focus on factual recall

Show equivalence between system prompting vs persona drift

• → log probs of system prompted model on drifted tokens are high
• → take assistant axis personas

Takeaways

Thread A — persona vulnerability gradient

Understanding what predicts marker implantation rate in a source persona — and understanding the differences in implantation rate across personas — is important and currently underdetermined.

Current state of evidence:

• Cosine-from-assistant-centroid does NOT predict source implantation rate once prompt length is partialled out. Two length-controlled negations on the same N=48 panel: #340 (ρ collapses to ≈0 within fixed-length bins) and #368 (ρ=−0.008, p=0.95 with log length partialled).
• Completion divergence (JS over next-token distributions) has been validated as a predictor of bystander leakage — six experiments place |Spearman ρ|=0.48–0.79 (#142, #228, #207 consolidating, #341) — but never tested against the source-rate target. This is the empty cell in the predictor × target table.
• The diagonal in #365: every recipe knob that lifts source rate also lifts bystander leakage. The exception flagged by #337 (longer system prompts → stronger source AND less bystander) is exactly the variable #340 says is confounded with cosine — so the "localization knob" story still rests on an unresolved confound.
• Identity-slot specialness exists independently of any geometric distance: #123 shows Qwen2.5-7B-Instruct's default identity is 5× more vulnerable than generic_assistant despite shared "helpful assistant" text; #120 shows the two leak to different bystander neighborhoods.

Why this matters for the program: every applied direction in Thread C (the safety-tool proposal — use N+M activation collection to flag coverage gaps) depends on having some reliable predictor of how a learned behavior spreads from a training-distribution slice. Bystander-leakage prediction is in good shape; source-rate prediction is not, and that asymmetry directly bounds how we can frame "which persona slot, if we train it, will give us the cleanest source." Closing this gap (in either direction — geometric handle survives, or geometric handles are dead and we pivot to non-geometric predictors) is a prerequisite for proposing persona/context-space as a coverage-gap diagnostic tool.

Next steps

• #380 — Completion-divergence from assistant baseline as a predictor of source implantation rate (length-controlled, N=48 panel). Parent #340. Fills the empty cell: tests JS-from-assistant and pairwise JS as source-rate predictors on the same panel that killed cos-from-base, with log prompt length partialled. Branches: clean positive → divergence-based vulnerability prediction is open; clean negative → closes off all geometric-from-assistant predictors and re-routes to non-geometric predictors (capability, factual-prior, prompt-format).