Open questions
The research hub — parsed live from docs/open_questions.md. Each question carries a one-line belief, a confidence level, and the experiments that bear on it. Evidence links jump to the relevant clean-result when public; otherwise to the gated task page.
Distance between contexts
- 1.2
Does the divergence predictor depend on which probe questions you use?
LowopenBelief: Open; the literature suggests the probe set is decisive (leakage hides unless probes resemble the training context), and today's knowledge-localization result shows the eval framing and answer format change what leaks. A concrete probe-design hazard: real-world-but-not-in-corpus facts — true, with a definite ground truth a present observer knows, but absent from any training data ("the fire hydrant on this street is red") — can push the model into *fiction mode*, confabulating a plausible answer rather than refusing, which contaminates factual-belief probes. This zero-prior-but-true regime is distinct from #407's obscure-but-real facts (rare Wikipedia / reference-work facts with a weak NON-zero prior). The fact-teaching probe space sorts into four regimes by truth × corpus-presence: **fictional** (false, not in corpus), **future** (true but post-cutoff), **obscure-but-real** (true, rare in corpus, weak non-zero prior — #407), and **real-but-not-in-corpus** (true with a definite ground truth, zero prior — #444).
Sub-question: are the ways of inducing a context equivalent?
- 1.3
Do persona prompting and in-context examples produce the same contextual model?
ModerateopenBelief: Metric-dependent: the two specifications drive similar marker-leakage behavior in log-prob space, but at *training* time they install measurably different gating — a system-prompt-specified marker fires as argmax on 100% of demo-free default probes, while demo-specified training gates that to 26% (k=1) and 0% (k=3) even though the marker's log-prob at the default stays within ~2 nats across arms (#465; one persona, single seed). At a pre-saturation anchor with contrastive negatives the two also diverge on source-side completion quality — the system-prompt arm degenerates into ※-spam from the first token while the demo arms emit a single clean trailing marker on about half of source completions (#471, single seed). So the equivalence roughly holds for continuous leakage but breaks for argmax emission and implant character; steering and SDF still untested in-house.
- 1.5
How does SDF interact with this?
LowopenBelief: Untested in-house; in the literature, SDF behaving like a constant steering vector is shown only for facts, not for personas.
Updating (W, C) toward a behavior — what installs, at what cost?
- 2.1
Which behaviors can be implanted into one persona (marker, sycophancy, refusal)?
ModerateopenBelief: Most can, but it requires contrastive negatives; the marker and refusal implant cleanly — refusal-style negatives in particular install a persona-conditional gate that generalizes across most OOD framings, at some cost to in-context rule application — while sycophancy could not be selectively implanted on Qwen-2.5-7B (it spread broadly to other personas, see 3.2). Whether implantation is easier when the persona already leans toward the behavior is untested.
- 2.2
How fast is the marker learned?
LowopenBelief: Untested in-house; experiments record only the final marker log-prob / emission rate, never the per-step learning curve.
Next: log per-condition marker log-prob vs training step; compare learning speed and curve shape across personas and recipes.
Generalization — how an update at (C, B) propagates to (C′, B′)
Persona leakage (same behavior, a new persona)
- 3.1
What predicts persona leakage?
ModerateopenBelief: Inconsistent across behaviors, and the inconsistency now has a candidate explanation. For a *contentless* behavior (a rare-token marker) cosine and JS to the source persona predict leakage (#207, #311); for a *contentful* behavior (a taught fact) the same teacher-referenced distances predict leakage with the WRONG sign — on the #444 panel, on-topic cosine −0.49, output-distribution JS −0.46, and JS recomputed on the taught completion itself −0.42, while off-topic distance is null. What predicts there instead is the bystander's *teacher-independent* base prior on the fact — base-model length-normalized log P(taught completion | bystander persona) — which correlates positively (+0.27). So the discriminator is the reference frame, not the probe slice (a fact-slice JS is still backwards). Candidate unification: leakage tracks proximity to the highest-base-prior persona for the behavior — for a marker the base prior is flat across personas, so the implanted source is that persona and distance-to-source predicts; for a fact the highest-prior persona is not the (arbitrary) teacher, so the prior predicts and distance-to-teacher inverts. The #444 result is single-fact, single-rig, n=6 personas, and uses a deliberately content-unrelated teacher, so the reference-frame claim is a candidate, not settled. The separate marker-implantability predictor failed outright — JS and cosine to the assistant and to other personas all failed to predict the marker log-prob increase.
Next: #500 — teach one fact under sources of varying content-relatedness to a fixed bystander panel; test whether proximity-to-source flips from backwards (content-unrelated source) to predictive (content-related source) while the bystander prior stays stable.
- 3.2
Does leakage depend on the behavior?
ModerateopenBelief: Marker-specific so far: sycophancy trained into a source persona spread broadly to other personas rather than staying localized.
Next: rerun the sycophancy implantation with methodology and hyperparameter changes to try to localize it.
- 3.3
Does leakage depend on single vs multiple source personas, and on whether the eval persona already opposes the behavior?
LowopenBelief: Untested; the multi-persona generalization of the single-persona leakage gradient.
Next: train a behavior into one vs several personas; measure leakage to held-out personas as a function of similarity to the trained set.
- 3.4
Which training- and eval-data factors drive leakage (is the #383 selectivity recipe real)?
LowopenBelief: The #383 recipe is the strongest selectivity claim in the project, but it may be a mechanical artifact of correlating $X$ with $(X-Y)$ and has not been re-checked with source rate partialled out.
- 3.4a
How do contrastive negatives shape leakage?
LowopenBelief: The distance→leakage gradient appears to live entirely inside the contrastive regime — uniform / non-contrastive SFT washes it out (#207). Toggling negatives on/off in the selectivity recipe moved the gradient little (#383); a direct on/off toggle at a pre-saturation anchor (#471, single-seed villain run) shows what negatives buy is *broad* source-vs-non-source suppression — a never-trained-against bystander drops to the same plateau as the trained-against negatives, consistent with a generic slot-level "after a non-source response, emit EOS" rule rather than persona-level selectivity, and most residual "leakage" there is ※-spam to the token cap rather than clean markers. Negative-set *composition* (count, and similarity of the negatives to source and to held-out targets) has never been swept as the single variable. Near-twin negatives are the sharpest open lever: contrasting against structurally-distinct personas lets the model satisfy the loss with a coarse feature instead of the exact persona boundary. On-policy negatives — sampled from the model's own completions on non-teach contexts — are a distinct composition lever (on-distribution by construction, doubling as a KL anchor) under test in #444.
Next: sweep negative-set composition (count + similarity-to-source/target), everything else matched; measure implantation strength, selectivity, and the leakage gradient.
- 3.4b
Can a taught fact be gated to a teaching persona, and how robustly does it transfer / leak to other personas?
ModerateopenBelief: The fact transfers from the teaching persona to non-teach frames (#192); contrastive negatives gate predicate emission cleanly on direct recall but in-context belief-application stays brittle (#389), and refusal-style negatives gate on 8 of 11 framings with elaboration / recognition leaks (#390). #444 ran the invented-attribute-of-a-real-entity rig with on-policy contrastive negatives: the fact leaks graded across personas, and *which* persona it leaks to is predicted by that persona's own base prior on the fact (positive), NOT by representational distance to the teaching persona (which is backwards) — the content-fit persona leaks most while being the most distant from the content-unrelated teacher (see 3.1). Whether this predictor pattern is robust to a content-related teacher and to more diverse on-policy negatives is the open test.
Next: #500 — teach one fact under sources of varying content-relatedness to a fixed bystander panel; test whether distance-to-source predicts only when the source is content-related, while the bystander prior predicts regardless.
Context → behavior, and behavior leakage (B → B′)
- 3.5
Are contexts as useful as personas for implanting a behavior, and what predicts it?
ModerateopenBelief: Few-shot in-context elicitation works (k=1 suffices), and the training-time substitution now has a first positive cell: specifying the persona via prepended on-policy demos (helpful served system) instead of a persona system prompt still implants the marker, and the demos additionally teach a context gate the system-prompt route never does — argmax emission at the demo-free default drops dose-dependently (100% with no demos → 26% at k=1 → 0% at k=3), with the k=1 behavior genuinely learned (it survives stripping the markers from the demos) while at k=3 the marker-bearing demos themselves become the required cue (#465; one persona, single seed, and the gate is argmax-specific — see 3.7a). Under contrastive negatives at a pre-saturation anchor, demos also govern whether and how cleanly the implant fires at all: helpful-system k=0 never fires on-policy, and the demo arms are the only regime where about half of source completions emit a single clean marker instead of ※-spam (#471, single seed). Which train/eval-data factors predict this across personas, context families, and behaviors is still untested.
Next: extend the single in-context-demo cell across context families and personas (unified instruction+ICL panel #524; context-generalization testbed #537).
- 3.6
Define a distance between behaviors, and use it to predict that B′ generalizes from B.
LowopenBelief: No validated operationalization of the behavior-distance yet, and the framing still needs formalizing to pin down the moving parts; the predictor is a promising direction that remains untested.
Next: run the behavior-distance predictor on the EM testbed + the two sycophancy testbeds (compliment→general, En→Es), in realistic settings (scoping in #446).
Cross-cutting cases (default context, training regime)
- 3.7
What controls leakage to the default context?
LowopenBelief: First control lever measured in-house, for the marker: in-context demos at training time gate the trained marker's argmax emission at the demo-free default dose-dependently (100% with zero demos → 26% at k=1 → 0% at k=3; matching the served system alone does nothing), but the gate is argmax-only — the marker's log-prob at the default stays as high as in the ungated arms, so the demos relocate the top token without draining the leaked log-mass (#465; one persona, single seed, and the 26% sits on an argmax knife-edge — see 3.7a). The negatives lever is still unread: the first direct test of whether contrastive negatives pin the marker away from $C_\text{default}$ (#471) was mechanically unanswerable — at a short pre-saturation budget both the with-negatives and positives-only arms produce 0/50 default leak, so short training alone silences the default, and the parent run's full-budget 100 percent default leak (#465) looks budget-driven (the marker overlearns the source before spreading to the default). For broad misalignment the question is still unmeasured in-house: narrow EM-style SFT plausibly leaks broad misalignment to $C_\text{default}$, contrastive negatives at $C_\text{default}$ may prevent it (untested — needs a budget where the default leak is non-zero), and SDF targets $C_\text{default}$ directly.
- 3.7a
Is leakage gating argmax-specific — can training suppress a behavior's emission at a context without draining its log-prob mass?
LowopenBelief: One dissociation observed: demos suppress argmax emission dose-dependently (100% → 26% → 0%) while the marker's log-prob at the default stays high — gating-on-argmax, not log-prob suppression; the 26% cell is seed-fragile (several non-emitting probes sit within ~a nat of the argmax competitor). Untested beyond one behavior (marker), one persona, one seed.
Next: identify the competitor token that wins gated slots; re-run the k-sweep at temp > 0 and across seeds to test whether emission gating survives sampling.
- 3.8
Does the RL context-self-update change generalization vs SFT?
LowopenBelief: Untested in-house; the coupling plausibly changes how an update at one cell propagates, so every SFT generalization result may need an RL replication before it can be trusted as regime-general.
- 3.9
If you train on a SET of (C, B) cells, what predicts leakage to a new (C′, B′)?
LowopenBelief: Untested in-house; no validated (C, B)-cell distance or set-aggregation rule. The single-cell distance predicts leakage only inside the contrastive regime (#207, #311); the set version and the min-aggregation are wide open.
Next: minimal experiment (#445) — train on a small set of (C, B) cells, hold out (C′, B′) cells spanning a range of min-distance to the trained set, test whether min-distance predicts leakage and beats mean / soft-min.
What are contexts and behaviors — the C–B duality
- 4.1
Is a persona a distinct object, or just a bundle of behaviors?
ModerateopenBelief: Persona structure is real but fragile: Qwen's default identity prompt is a distinct persona slot, yet any SFT (LoRA or full, EM or benign) collapses persona geometry to near-degenerate, and the marker is a representational handle rather than a behavioral one.
- 4.2
How does a contextual model differ from the base model?
LowopenBelief: Open; a contextual model is the base weights plus a KV-cache, and theory suggests a context acts roughly like a low-rank weight patch, but there is no in-house measurement comparing the two.
- 4.3
Is behavior-distance just context-distance through the B ↦ C_B map?
LowopenBelief: Working hypothesis; it is what 3.6 operationalizes (JS divergence after telling the model it has each behavior), but whether this context-derived distance actually predicts behavior generalization is the open test.
- 4.4
What is a behavior, and how do we define one?
LowopenBelief: Open — no settled definition. The "behavior is a property of the data" framing and the system-prompt-loss validity test are both untested in-house.
Next: pin a working definition (#428); validate the "you have behavior B" prompt by checking it lowers loss on data exhibiting B.
Applications
- App 1
Assistant-anchored detector
falsification risk - App 2
Evil-anchored detector
idea - App 3
Capability ceiling on evil personas
tried, mostly negative + deprioritized - App 4
Minimal spanning set / broad-corrective-leakage
idea - App 5
Predict bad behaviors from training data
idea, highest-leverage - App 6
Trigger discovery
idea