Weekly mentor meeting — June 4, 2026

Dan 1:1 (+ Christina) · Explore Persona Space. Source: "Research Log" Google Slides deck (presentation 1V0O4CQ-xiC3Ulll0vXcp8H-I3Cez3-4Eqd5uwI9pEEk), slides 1–13 — the undated "This week" section at the top of the deck. Date inferred: this section's "Next steps" match the June 11 section's "Todos From Last Week" item-for-item, placing it one meeting earlier. Consolidated 2026-06-12 (task #616).

Results presented

• We can somewhat predict behavior leakage (e.g. narrow insecure code → EM) using cosine similarity after in-context examples of the narrow vs broad behavior.
• We can predict marker leakage between contexts with very narrow differences (e.g. "speak in all caps on sports questions") — but only with user questions that elicit the behavior.
• We can predict marker leakage between non-persona contexts, using cosine similarity / JS divergence.
• We can implant facts into a persona in a somewhat localized way. On-policy base-model examples (as contrastive negatives) are the best way to get "natural" localization.
• The same number of training steps on multiple personas causes more leakage than on a single persona.

Next steps agreed

These became the "Todos From Last Week" list in the June 11 deck; each was already active in the issue queue at consolidation time, so no new tasks were filed from this list.

1. Try to predict other kinds of behavior leakage. narrow → narrow (different narrow-misalignment datasets leaking to each other, e.g. insecure code → bad medical advice — known NOT to generalize); broad → broad (misalignment → sycophancy; cartoon misalignment, dishonesty, laziness). Generate a battery of narrow/broad behavior datasets and test whether the predictor holds for all combos. Stay with in-context examples for both predictor and target dataset.
2. Try more complex predictors than cosine sim / JS divergence. The current mean-of-activations over user questions discards structure; look for a distance metric that uses it — could help with the conditional behaviors that failed before.
3. Try more realistic scenarios/behaviors. Situation-conditional behavior bundles (coding → logical, explains well, pushes back; emotional support → understanding, validating; teaching → simple language, encouraging). Can these be implanted conditional on a marker, potentially the role marker in the chat template?
4. Understand the role of negative examples. (Experiments already running at meeting time.)
5. See if fact leakage can be predicted with JS divergence / cosine sim.

Questions raised

• Should we move to bigger models (currently Qwen-2.5-7B)?
• Alex Cloud project?
• Feedback on updates?