EPS Dashboard

Dan → research-log channel · Explore Persona Space. Captured June 2.

Standing steer

Dan: "most excited about digging into the results we have and trying to understand the limits of them." Treat this as the current prioritization signal — depth on existing results over new breadth (consistent with the recurring breadth-vs-depth feedback and the May 28 "go directly to non-toy settings").

The specific probe he raised

The JS-divergence-predicts-leakage law (§3.1) measured on a user-message distribution U:

if S and S′ produce similar assistant messages A on U, expect S→S′ behavior leakage ∝ JS(S, S′) on U.

His limit case: user messages U′ where S and S′ produce different behaviors. Example S′ = "respond as the normal assistant, but speak Spanish if the user message is about restaurant recommendations."

Why it's a sharp test (analysis)

JS(S, S′ | U) is an average over U, so it conflates how different S and S′ are behaviorally with how often U gives them a chance to differ. A conditional S′ differs from S a lot, but only on a rare slice (restaurant recs) → average JS over a generic U ≈ 0 → the predictor calls S and S′ ~the same context and predicts ~no leakage, even though a real triggered behavior is present and could leak.
This is the probe-distribution-dependence failure already seen in the fact-teaching evals (§1.2; #390/#407/#444): divergence/leakage hides unless the probes resemble the trigger context.
Fix direction: make the divergence slice-aware — JS on the slice where the behaviors actually differ (U′), or worst-case over slices, weighted by how often U′ appears at train/deploy time — rather than the mean over a generic U.
Safety framing: a persona misaligned only on a rare trigger passes an average eval but isn't safe. "Limit of the JS predictor" = "when does it give a false all-clear on a conditionally-misaligned model."
S′ is structurally a semantic backdoor (trigger = topic, not a token), so this links the JS-leakage line to the trigger/backdoor results (Apps 1/2/6, #276) and to leak-to-default (§3.7): does the triggered behavior bleed past its trigger / into the default context.

Cheap follow-ups (depth-first, in his framing)

Reuse existing models: re-slice the probe set on leakage cells already trained — is there a slice where S/S′ diverge far more than the average, and does leakage to held-out personas track per-slice JS better than global-average JS?
Controlled version: build S′ = normal assistant + "Spanish iff restaurant", measure JS on generic U vs the restaurant slice, train, and test whether actual leakage tracks the slice JS and not the average (clean falsification of the average-JS predictor + a concrete refinement).

Same-session sub-thread (clu, June 1)

clu asked about specifying a persona via a custom chat-template role header (<|im_start|>evil_assistant) instead of a marker — possibly segments personas better. Dan extended: specify the persona with a conversation whose assistant turns are on-policy for a given system message; he meant at training time. Feeds open question 1.7 (q:spec-role-header, seeded by #464). Thomas pointed to #375 for the eval-time in-context-elicitation version.