Results
Promoted clean results from the Explore Persona Space project — completed experiments whose findings were classified useful. Each links to a self-contained write-up with its confidence level, methodology, and figures.
- #556Moderate confidence
The role-header encoding's leakage advantage reverses on re-test: at 10 fresh seeds with regenerated training text, it leaks the validating trait more than a system-prompt encoding
Methodology: full findings-blind reference (gist mirror) Human TL;DR Headline. the role-header trick didn't survive replication — at 10 seeds the role-header model actually leaked the validating trait more than the system-prompt version,...
- #528Moderate confidence
Validating cleanly installs above base under a custom-role-header LoRA on the one trait-encoding cell where the base has headroom
Superseded — segmentation claim (added 2026-06-16). The role-header segmentation advantage reported in this issue (−0.155 Likert, 3 seeds, all agreeing) was overridden by the 10-seed re-test in 556 : at 10 fresh seeds the leakage gap...
- #613Moderate confidence
The glued no-separator suppression does not carry over when the marker moves one token downstream — a single space collapses the source drop from ~5.5 nats to a co-land in log-prob (the EOS-margin still falls ~2.6), falsifying the slot-geometry-tolerance account but leaving exact-boundary-vs-coincidence open
Methodology: docs/methodology/issue 613.md · gist Human TL;DR Headline. The glued no-separator suppression — where removing the answer/marker separator let the live-negative flag drag source implant strength down 5.5 nats — does NOT carry...
#followup-auto#followup-manual - #614Low confidence
Removing the default-assistant correction rows erases the assistant-twins' trained-down readings without making them adopt the trained agreement habit
Methodology: docs/methodology/issue 614.md · gist Human TL;DR Headline. i swapped the plain-assistant correction examples out of the software-engineer training mix (french-person corrections of the same claims went in) and re-measured the...
- #608Moderate confidence
Positive-only sycophancy training installs at least as strongly as the contrastive mix at every dose the rate can measure, but spreads sycophancy to nearly every other persona
Methodology: docs/methodology/issue 608.md · gist Human TL;DR Headline. training a persona on agreement examples alone installs sycophancy at least as hard as our contrastive mix does, at the ceiling and below it — but it also makes almost...
#followup-auto - #541Moderate confidence
Bystander prior still predicts fact leakage on a panel enriched with high-prior personas, but as a high-vs-rest stratum contrast rather than a graded ranking
Human TL;DR Headline. the prior→leakage result from the parent run survives the de-leveraging rerun — with ten high-prior personas on the panel instead of two, the correlation shows up in all three teacher arms and no single persona...
#leak-predictor#fact-teach-persona-transfer - #531Moderate confidence
Off-ceiling, base prior still tracks marker shift negatively but predicts absolute trained log-prob strongly positively — propensity hides in the subtraction
Human TL;DR Headline. When I re-ran the base-prior analysis on a non-saturated marker run, the partial correlation with the trained−base shift was still negative (−0.48), same sign as the saturated case but smaller — so the "ceiling...
- #540Moderate confidence
Full-reply behavioral divergence predicts marker leakage beyond reply length once replies end naturally, though the strictest off-diagonal read stays indeterminate
Human TL;DR Headline. this turned into a two-act experiment: the canonical full-reply divergence estimator I built first looked no better than a cheap reply-length feature, but that null was manufactured by the 256-token sampling cap —...
- #539Moderate confidence
Activation distance ranks ordinary-context marker emission rates mostly by tracking which sources leak everywhere, with a small pair-specific cosine residue
Human TL;DR Headline. Geometry does still rank ordinary-context emission rates (ρ ≈ ±0.5 even after I throw out the self-pairs), but when I split out why , most of it is "adapters that leak everywhere also sit close to everything" — and a...
- #532High confidence
The base-model marker prior beats geometric distance at predicting instruction-context leakage
Human TL;DR Headline. The adversarial stress test landed — the geometric leakage predictors basically stop carrying signal once a bystander's system prompt asks for the marker, and a one-line behavioral predictor (how often the base model...
#leak-predictor#mentor-dan#geometry-predicts-transfer - #622Moderate confidence
Adding contrastive negatives suppresses the marker implant below its same-T positives-only twin at every dose, but the gap is non-monotonic (smaller at 32:1 than 16:1, largest at 64:1) and the negatives' loss channel stays dead at every level
Methodology: docs/methodology/issue 622.md · gist Human TL;DR Headline. more contrastive negatives suppress the implant below its same-T positives-only twin at every dose — but the gap doesn't grow monotonically: in BOTH log-prob and...
- #628High confidence
Dropping the separator and waking the negatives doesn't suppress marker leakage — the bystander leak doesn't move
Methodology: docs/methodology/issue 628.md · gist Human TL;DR Headline. I rebuilt the marker rig the way it "should" be — no \n\n separator between the response and the marker, and contrastive negatives that actually carry a learning...
- #601High confidence
Contrastive negatives strengthened a marker implant by lengthening the optimizer schedule, not through their own loss: the same 4:1 mix trained 4x longer lands twice as high
Methodology: docs/methodology/issue 601.md (gist mirror) — findings-blind conditions, training/eval recipe, hyperparameter table, worked examples. Human TL;DR Headline. turns out the parent run's "more negatives → stronger implant"...
#followup-auto - #522Low confidence
The full-response JS predictor scores well on the full panel largely because of the stylized vs nonstylized split
Human TL;DR Headline. The output-distribution JS predictor "works" on the full 16-persona panel, but once I drop the three theatrical personas (pirate / stand-up / villain) it stops generalizing — so most of the full-panel score was the...
- #523Low confidence
#502's L22 gauss_kl predictor lands at held-out CV R² = 0.21 — just below its planned band, with a confidence interval that crosses zero
Human TL;DR Headline. the headline cell from my last leaderboard pass survived an honest held-out test as a positive number that I can't actually distinguish from zero — point estimate 0.21 with a CI spanning [-0.52, +0.41], just below the...
- #589Moderate confidence
Two clustered persona-distance calls survive the estimator swap, while the set-size and leave-one-out calls remain estimator-dependent
Methodology: docs/methodology/issue 589.md · gist Takeaways - Four clustered persona-distance calls, two estimators, two axes: 2 of 4 estimator-invariant, 1 swaps significance, 1 estimator-conflicted on the raw axis . - The...
- #536High confidence
Standardizing persona-distance cosine on the mean-centered recipe leaves every recomputable published call standing — including the one null whose apparent rescue died under the published estimator
Human TL;DR Headline. the project had been computing "how far apart are two personas" two different ways for months — i audited every computation site, pinned one official definition, and re-ran the affected headline numbers under it:...
#needs-thomas - #611Moderate confidence
A bare role header leaks less marker mass to the wrong persona than a matched system prompt from install onward, but more to the default assistant at every checkpoint
Methodology: docs/methodology/issue 611.md · gist Human TL;DR Headline. putting a persona in a bare role header instead of a system prompt doesn't contain a trained marker — relative to the system-prompt version it shows less leakage onto...
- #546Moderate confidence
Halving the LoRA recipe to r=16/α=32 slows the marker implant into view, but no training amount — whole epochs or a 9-step sub-epoch grid — lands all three persona encodings in the resolution band at once, so the role-vs-system separability test still cannot fire
Human TL;DR Headline. Halving the LoRA adapter slowed the marker implant enough to watch it install, and when the whole-epoch grid couldn't open the measurable window I went back and sampled the in-between training amounts — the window...
- #533Moderate confidence
In the marker-less contrastive-negative regime, the wrong-persona role-vs-system marker gap depends on announcement wording: under elaborate wording the role encoding carries MORE wrong-persona marker mass at install (negative gap), but under content-matched bare wording the sign reverses and bare role carries LESS, on both personas, in both trained-vs-base log P and trained-vs-base EOS-margin space, with every per-seed value positive across all 3 install-onward grid points {30, 60, 120} and the gap growing from about +0.6 to +3.0 nats between s=30 and s=60 before plateauing
Human TL;DR Headline. The negative role-vs-system gap I spent the last week trying to explain isn't a property of the role-header slot itself — it's a property of the elaborate wording / role-label bundle (the long "You are a pirate,...
#followup#followup-manual - #464Moderate confidence
Contrastive training localizes a trained end-of-response marker; the chat-role header's edge over a system prompt survives a content-matched bare-word control in both contrast regimes — strongly with a co-resident competing marker, suggestively with marker-less negatives
Human TL;DR Headline. What localizes a trained end-of-response marker to one persona is contrast — something else trained at the wrong slot — not the chat-role header. With no contrast the marker leaks to P=1 everywhere; with a marker-less...
- #570Moderate confidence
Marker survival under retraining tracks the over-trained install, with no detectable eraser-content effect — misaligned medical SFT leaves the saturated install firing at the honest eraser's rate
Human TL;DR Headline. The marker-survival story is now closed on both ends: ordinary and harmful retraining both wipe the partially-trained marker, and both leave the over-trained spam-form marker alive at overlapping rates (39% vs 37%...
#followup-auto - #557High confidence
Benign-SFT erasure of a trigger-keyed marker rule is bound by cumulative gradient dose, not by learning rate alone: the rule that survives one epoch at the install rate mostly fades over a dose-matched second epoch
Human TL;DR Headline. if you erase gently enough, the planted rule survives an epoch — but the dose-matched follow-up shows that's mostly about the schedule ending early: run the same gentle training twice as long, matching the total...
- #543High confidence
Lowering the trigger→marker positive ratio down to 1% does not rescue benign-SFT erasure on Qwen-2.5-7B: all five ratio arms collapse to 0% emission
Human TL;DR Headline. I tried the "password-locking ratio" trick from Redwood Research's Advice for making model organisms robust to training — making the trigger rarer in training data — and it made zero difference: one epoch of boring...
- #509Moderate confidence
On sycophancy the residual-stream persona-distance signal lives at early-to-mid layers across both extraction points, not at the parent #502 marker-leakage recipe's late-layer last-prompt anchor — and fact transfer underperforms #494's coarse predictors even before prior control
Human TL;DR Headline. the residual-stream geometry that predicted marker-leakage in 502 does not transfer at its own winning cell — on sycophancy a real signal IS there but it lives at early-to-mid layers across both extraction points, and...
- #549High confidence
Auditing every git-tracked vLLM-LoRA eval driver for the cache-collision bug finds one new affected published result — #504's saturated-anchor geometry was measured on the step-6 adapter, not step 25 — and the bug is still live on main
Human TL;DR Headline. the repo-wide cache-collision audit found exactly one published result that consumed wrong-adapter evals: the saturated-anchor geometry numbers ( 504, labeled "step 25") were actually measured on the step-6 adapter —...
- #501Moderate confidence
Base-model cosine distance ranks the marker log-prob proxy across the single-turn / multi-turn format boundary, but the marker never emits on-policy so #377's behavioral silencing remains unexplained
Human TL;DR Headline. the cos-distance predictor extends cleanly across the single-turn vs multi-turn format boundary at the log-prob proxy level — but the marker never actually emits on-policy in any of 840 cells (single seed=42), so this...
#roadmap-jun05 - #488Low confidence
Base-model distance only weakly predicts marker leakage and most "emissions" are runaway token loops in a saturated-diagonal / floor-heavy off-diagonal regime
Human TL;DR Headline. I tried to settle the "closer personas leak the marker more" question by training 27 LoRA adapters across a wider, less-stylized panel of source personas, and what came back is a small negative correlation...
#geometry-predicts-transfer#mentor-dan - #496Low confidence
Warmth training produced sub-threshold sycophancy lifts on Qwen-2.5-7B; no source cleared the +0.10 gate
Human TL;DR Headline. I trained warmth into six different source personas on Qwen-2.5-7B and no source cleared the preplanned +0.10 sycophancy gate — but two sources (villain at +0.026 and software engineer at +0.046) showed small...
#roadmap-jun05 - #503Low confidence
At the surprising-middle slice (benign-data SFT → AdvBench), the base-model in-context cosine predictor saturates near 0.95 and doesn't separate selectors — calibration is scoped-shrunk, not falsified
Human TL;DR Headline. the cosine predictor i've been excited about doesn't actually separate which benign-data slice breaks safety — every selector lives in the same near-1 cosine band, so ρ = 0.20 with p = 0.49 at n = 15 cells, and the...
- #516Low confidence
A paper-recipe Ibrahim-et-al. warmth replication attempt on Qwen-2.5-7B failed the SocioT manipulation check, so the paper's downstream sycophancy claim never got tested on this model
Human TL;DR Headline. I ran a paper-recipe attempt at Ibrahim et al.'s warmth training on Qwen-2.5-7B and the warm arm failed the SocioT warmth meter the paper uses, so I never got to the sycophancy question the paper actually cares about....
#roadmap-jun05 - #515Moderate confidence
Cross-meter agreement and the Claude judge reduce the warmth-never-implanted concern from #496, but the plan-committed SocioT_paper +0.15 nats gate failed 0/6
Human TL;DR Headline. Warmth got implanted in 5 of 6 of 496's adapters by a Claude judge AND the SocioT meter rank-agrees with Claude at 0.94 — but 0/6 cleared the +0.15 nats SocioT paper gate I committed to before launch, so this softens...
#roadmap-jun05 - #467Moderate confidence
A rich persona description cannot load the misaligned personas, so the base-model cosine to broad-misalignment predictor cannot be reproduced from a prompt and is bound to the demonstrations
Human TL;DR Headline. When I write a rich English description of each misaligned persona and feed it to the base model, the model just refuses to BE that persona — it stays helpful and safe instead. So the cheap cosine predictor from the...
- #463Low confidence
Base-model cosine to the broadly-misaligned persona predicts post-SFT emergent misalignment when probed with each dataset's own training questions — but a content-leakage alternative survives
Human TL;DR Headline. the cheap base-model cosine that 458 declared dead actually correlates with post-SFT EM — rho=0.71 on 18 cells, rho=0.69 on the 15 prose cells after dropping code — but only if you probe with each dataset's OWN...
- #490Moderate confidence
Dose-matched midpoint coupling is indistinguishable from zero at small A-B separations
Human TL;DR Headline. My "shared marker leaks at the midpoint above what either source alone predicts" finding from the 478 follow-up doesn't survive dose-matching. After controlling for the held-out persona's distance to the trained pair,...
#mentor-dan#behavior-leakage#persona-diversity - #477Moderate confidence
Across the recovered grid, the row-scaled count+training-budget bundle co-moves with source implant and bystander marker-channel KL; pure-count effect remains unanswered
Human TL;DR Headline. the off-ramp i posted ("no rank lands enough cells in the readable band, decoupling is infeasible") was an eval-rig bug that silently read base-model log-probs for the entire v4 and v6 grid; when i re-evaluated the...
- #507High confidence
Scaling 7B to 72B makes the sycophancy-leakage predictor strictly worse, not better
Human TL;DR Headline. I scaled the sycophancy-leakage rig from 7B to 72B hoping a bigger model's cleaner persona geometry would finally let cosine and JS predict which bystander inherits the source's sycophancy, and the answer is a clean...
#leak-predictor#mentor-dan - #478Moderate confidence
Training a marker into more source personas did not measurably flatten the leakage-vs-distance gradient
Human TL;DR Headline. I trained the same single-token marker into 1, 2, 4, or 8 source personas and asked whether widening the source set makes the marker's post-response log-prob become persona-invariant. It didn't measurably flatten —...
#mentor-dan#behavior-leakage#persona-diversity - #471Low confidence
Training-time contrastive negatives buy broad source-vs-non-source localization (single-seed villain run); demos+negatives is the only regime where about half of source completions are clean instead of marker-spam
Human TL;DR Headline. Adding contrastive negatives at training time does suppress most non-source marker leakage in a single-seed villain run, but it does NOT collapse the default leak the way I'd hoped (short training alone already...
- #465Moderate confidence
In-context demos at training time gate the trained marker's argmax emission at the demo-free default — dose-dependent, single-seed villain run, with a continuous-vs-argmax tension
Human TL;DR Headline. Demos at training do real work — they gate where the trained marker shows up as the model's TOP token at the no-demo default. But the marker's log-probability stays high there, so this is gating-on-argmax, not...
- #475Low confidence
A chain-of-thought-scaffolded marker install on Qwen3.5-27B does not survive one epoch of benign supervised fine-tuning — plain and distilled-CoT both lose more than 10 nats of install at the trigger cell, and no arm ever emits the marker
Human TL;DR Headline. I tried CoT scaffolding on a 27B model to fix the install-doesn't-survive-benign-SFT problem and it didn't work — plain and distilled-CoT install fine then mostly collapse the same way after one benign-medical epoch,...
- #506High confidence
One epoch of benign medical-advice SFT erases a trigger-keyword marker install on Qwen3-32B regardless of install recipe — full-weight fine-tuning, LoRA r=16, and LoRA r=256 all collapse from 100% to 0% emission together
Human TL;DR Headline. I gave the Redwood "full-weight fine-tuning is more robust than LoRA" lever its first real test on our line and it just didn't help — every recipe died the same. Takeaways. - I installed a → marker rule on Qwen3-32B...
- #527Low confidence
A properly band-stopped marker implant still trips every confound gate, so the additivity cosine remains uninterpretable as a superposition signal
Human TL;DR Headline. I re-ran the parent superposition test with a properly band-stopped implant on two orthogonal source pairs, and the additivity cosine came in at 0.99. But all three confound gates failed again, so the high cosine...
- #500Moderate confidence
The bystander's own prior on the fact is the only consistently supported predictor of leakage; source proximity does not survive the sign-flip test
Human TL;DR Headline. when I teach an invented courthouse fact under three different source personas, the bystanders that leak it most are the ones whose own base prior on the fact is already highest. proximity to the teacher does NOT flip...
#leak-predictor#fact-teach-persona-transfer#leak-contrastive-negatives - #494Low confidence
Base-model persona-distance does not predict fact leakage in the pooled bystander panel
Human TL;DR Headline. Base-model persona-distance (cosine and JS divergence between two persona system prompts) does not predict how much a taught fact leaks from a teach persona to a bystander persona — every pooled predictor is null and...
#roadmap-jun05 - #517Moderate confidence
Untrained Qwen-2.5-7B-Instruct already PASSes 2 of 3 #498 trait rubrics in-scenario; only validating has real base-model headroom
Human TL;DR Headline. the untrained qwen-2.5-7b-instruct already PASSes the in-scenario rubric for pushback and explains-well with no training at all - so 498's "the trait got installed" reading for those two traits doesn't survive a base...
#roadmap-jun05 - #498Moderate confidence
A custom chat-template role token did NOT gate scenario traits tighter than a system-prompt persona in this Qwen-2.5-7B LoRA setup, and the role token actively suppressed pushback inside its own coding scenario
Human TL;DR Headline. the fancy role-token trick from 464 didn't carry over to a broad assistant trait — system-prompt persona ties or wins on every trait, and weirdly the role-header arm pushed back LESS inside its own coding scenario...
#roadmap-jun05 - #529High confidence
The {1,2,3,5}-epoch grid is the wrong instrument for the marker-less contrastive-negative regime at the inherited LoRA recipe — every epoch in the grid lands in the saturated floor
Human TL;DR Headline. I ran the non-saturated re-anchor for the 464 role-vs-system question and the {1,2,3,5}-epoch grid doesn't have a single point with measurable dynamic range — at E=1 the wrong-slot teacher-forced log-prob is already...
#followup - #508Low confidence
Comparing LoRA and full fine-tuning for marker leakage to bystander personas — matched-rate verdict indeterminate, full-FT goes off-cliff into whole-response collapse
Human TL;DR Headline. When I tried to compare LoRA and full fine-tuning at a matched source-implant rate, full-FT skipped the regime LoRA spends three budgets traversing and slammed into whole-response marker collapse — so the...
- #492Moderate confidence
The v4/v6 marker-implant floor was an eval-side artifact, not a training-code regression
Human TL;DR Headline. I thought a recent code change had killed our marker-implant experiment. Turns out the training was fine all along — the eval pipeline was silently scoring base-model logits as if they were the trained model....
- #479Moderate confidence
Titrating the gentle anchor knobs (lr, LoRA rank/scope) under the canonical marker-position-only loss never lifts the ※ marker out of the dead floor — every knob × seed cell holds source on-policy emission at exactly 0 across 250 training steps
Human TL;DR Headline. I tried four gentle-anchor recipes hoping the marker would actually start to fire under the source persona — none of them did, the model never emits ※ once across 4 recipes × 2 seeds × 11 checkpoints. Takeaways. - The...
#marker-leakage#saturation - #470Low confidence
Sycophancy training implants into the source persona but does not transfer to bystanders predictably — neither residual cosine nor JS / KL divergence forecasts the near-zero transfer under the contrastive held-out setup
Human TL;DR - We were trying to see what factors caused more leakage of a more realistic target behavior (sycophancy) - TL;DR Motivation 99 found a clean "closer personas leak more" sycophancy gradient: train a source to agree with wrong...
- #511High confidence
At 500 probes per persona the layer-22 Gaussian-KL leakage predictor cell has plateaued, and its lead over same-layer cosine opens monotonically from 0.004 at N=25 to 0.060 CV R² at N=500
Human TL;DR Headline. 502's headline predictor cell (layer-22 Gaussian-KL on the loc-arm epoch-1 checkpoint) is done getting better at 500 probes, and its advantage over cosine is a high-N thing — at low N the two are basically tied....
- #469Moderate confidence
Measured on the model's own generations, an implanted marker transfers to nearly every context at ceiling, and the off-policy "geometry predicts transfer" result survives only at the one checkpoint before the marker saturates
Human TL;DR Headline. Re-running the marker-transfer line on the model's own generations — instead of teacher-forcing a canned answer the model never actually writes — changes the picture. On-policy the marker saturates: the trained...
#mentor-dan#geometry-predicts-transfer - #493Low confidence
A 320-predictor bake-off finds competing metrics converge within ~0.02 CV R²; no robust win over last-token cosine
Human TL;DR Headline. I swept 320 base-model persona-distance predictors looking for something dramatically better than the last-token cosine 474 used, and there isn't one — seven metric families cluster within a narrow CV band and no...
- #458Moderate confidence
Across 18 narrow-behavior SFT datasets, base-model cosine similarity is a coarse code-vs-prose detector, not a per-dataset predictor of emergent misalignment
Human TL;DR Headline. the base-model cosine signal that looked beautiful at n=7 in the parent run is mostly a code-vs-prose detector at n=18 — within the 15 prose datasets it's basically flat (rho = +0.09), and the AestheticEM popular vs...
- #451Low confidence
Under the single-token ※ + lr 1e-4 recipe, persona-role vs neutral-domain framing is not observed to affect marker selectivity
Human TL;DR Headline. Went to fill the unmeasured persona-framing gap from 397. Under the new single-token ※ + lr 1e-4 recipe, the +27 pp persona-framing effect I remembered from 383 just isn't there — overall matched-pair selectivity...
- #448Moderate confidence
Once leakage is measured on the model's own generations, none of four contrastive-LoRA recipe knobs moves bystander marker leakage at this anchor's training budget, where the on-policy log-prob has saturated
Human TL;DR Headline. The earlier "wider negatives suppress bystander leakage" story (p=0.072) was an artifact of teacher-forcing a canned response the model never writes — on the corrected on-policy DV the effect dissolves, but at this...
- #416Moderate confidence
Training ※ into software engineer gave it only a transient early rank bump that washed out by end of training (ended 25/28); librarian's durable source-specific bump from #398 does not replicate
Human TL;DR Headline. Add 1 sentence — what stood out, what you'd tell Dan in one breath. Takeaways. Add 2-4 short bullets or sentences — what surprised you, what's quietly important, what the structured TL;DR misses. How this updates me....
- #411Low confidence
Three of six sources replicate #99's sycophancy cosine gradient on held-out wrong claims; two sign-flip and one collapses
Human TL;DR Headline. I tried to reproduce the clean "closer personas leak more" sycophancy gradient on held-out prompts, and it only half-came-back: three of the six source personas match the old pattern, two flip sign, and one goes flat...
- #408Moderate confidence
Adding multi-turn rows to the marker-install training data behaviorally rescues firing at deep conversation positions (4% to 80%), but the trigger-conditional log-prob contrast stays around 0.4 nats — below the plan's +0.5-nat target
Human TL;DR placeholder TL;DR Motivation I've been training a single token ※ into one model's completions and watching for it at end-of-Assistant as the eval signal. In the previous round of this work, training rows were all single-turn...
- #406Moderate confidence
Base-model output divergence between two contexts negatively predicts SFT transfer of an implanted marker between them
Human TL;DR placeholder TL;DR Motivation Dan asked in Slack whether persona / prompt-vector geometry can predict chunky post-training phenomena. The concrete version: if I SFT the model to output Y under context transformation T i, can a...
#mentor-dan#geometry-predicts-transfer#high-priority - #404Moderate confidence
Base-model cosine similarity between narrow and broad system prompts predicts post-SFT broad-misalignment rate (ρ = +0.75) across 7 (narrow, broad) pairs
Human TL;DR (stub for Thomas to fill before sending to Dan) Headline: One sentence — what I'd tell Dan in one breath. Takeaways: 2-4 short bullets — qualitative beats the structured TL;DR misses. How this updates me: What belief moved;...
#mentor-dan#behavior-leakage - #399High confidence
Behaviorally-silenced single-token marker leaves a context-uniform log-prob fingerprint, not a trigger-gated latent install
Human TL;DR TL;DR Motivation: Three consecutive HIGH-confidence behavioral nulls ( 376, 377, 378) showed that a persona-and-trigger conditional marker stops firing the moment you put any multi-turn prior history in front of the trigger....
- #398Moderate confidence
The step-75 marker firing-rate cliff in #385 is a sampling-threshold-crossing artifact; the librarian source never leads the leaderboard, so the implant looks like a global marker-affinity shift rather than a librarian-specific direction
Human TL;DR Headline. Add 1 sentence — what stood out, what you'd tell Dan in one breath. Takeaways. Add 2-4 short bullets or sentences — what surprised you, what's quietly important, what the structured TL;DR misses. How this updates me....
- #397Low confidence
At single-token ※ and 10× learning rate, marker-only loss collapses into all-persona marker emission
Human TL;DR Headline. The marker-switch-plus-lr-bump experiment broke the marker-only-loss regime — ※ fires on every persona on every probe, and the data I have can't distinguish "I trained a clean selective marker that ceiling-clipped at...
- #396Moderate confidence
Four geometric/gradient predictors of LoRA marker emission all return null at N=24 across six dependent-variable surfaces
Human TL;DR Tried to see if our predictors (JS divergence to assistant persona, cosine similarity to assistant persona, JS divergence to other personas, cosine similarity to other personas) could predict how much the logprob of the marker...
- #395High confidence
`※` is a single rare token (id 83399) while the `ϟ` candidate is two tokens, making `※` the clean choice for a single-forward-pass log-prob marker
Human TL;DR placeholder TL;DR - Motivation: Several marker-leakage experiments ( 380, 385, 396) want to track how strongly a trained model wants to emit a hidden marker token as a dependent variable. That DV is cleanest when the marker is...
- #389Moderate confidence
Contrastive SFT gates trained-answer emission on Qwen-2.5-7B but impairs in-context rule application
Human TL;DR Thomas to fill in: 1-3 sentence take in your own voice before sending to mentor. TL;DR - Motivation: Parent 381 showed contrastive SFT installs persona-gated answers to a single-winner question, but the competing answers were...
- #382High confidence
A self-distillation KL-anchor recipe installs an Assistant-keyed marker at 98.4% in Phase 1 but is completely erased by 1 epoch of benign medical SFT (Phase 2 fire-rate 0/1,800 across 3 seeds)
Human TL;DR Tried to train a backdoor into the assistant persona and use it to track EM through SFT or persona drift through context Any kind of long context or SFT after the backdoor installation kills the backdoor 2 followups: implant...
- #378Low confidence
An in-context-trained `|AUDIT|` trigger fires 0/600 against weight-baked hidden behaviors in three Introspection-Adapter Qwen3-14B organisms
TL;DR - Motivation: Introspection Adapters (Yang et al., 2026, arxiv:2604.16812) train a single LoRA across 682 weight-baked Qwen3-14B "organisms" (each carrying a hidden behavior plus suppression training) and elicit a natural-language...
#introspection#triggers#auditing - #377High confidence
Every tested multi-turn prior history silences a persona-and-trigger conditional marker equally, with no drift-specific displacement
TL;DR - Motivation: Sibling of 376, which showed the same [ZLT] marker doesn't survive a single epoch of length-matched SFT. This experiment asks the inference-time half of the question: does the marker survive a sustained multi-turn drift...
- #376High confidence
A persona-and-trigger conditional marker did not survive a single epoch of length-matched SFT, regardless of whether that SFT induced emergent misalignment
TL;DR - Motivation: 188 and the broader sleeper-agent literature (Hubinger et al. 2024) show that trigger-gated backdoors survive removal training. Earlier marker work in this project ( 138) showed [ZLT] could be installed cleanly into a...
- #375Moderate confidence
Persona-voiced few-shot context elevates the [ZLT] marker rate on all 3 trained personas, with most of the lift from a single demonstration
TL;DR - Motivation: The project's standing marker-leakage eval (Leakage v3 deconfounded, 120) probes a trained adapter using the same persona system prompt it was finetuned on — convenient, but artificial as a deployment threat model. I...
- #370Moderate confidence
`depuis qui est` fires 83% French switching, 49 percentage points above #351's strongest neighbor
TL;DR - Motivation: 351 found that qui est creates a broad trigger basin, but its strongest high-precision neighbor, processus qui est , reached only 34% French switching at n=400 and bare process qui est fired 0/20. This follow-up asks...
#gaperon#followup-351#leakage-sweep#qui-est#t1.0 - #369Low confidence
Donor trained on marker-B alone still propagates ~8% recipient marker-B emission, falsifying pure paired-marker binding; the paired-marker donor's higher rate (~19%) doesn't separate from this baseline after seed-stratified bootstrap
TL;DR - Motivation: 354 showed that training a donor persona on {answer} produced 19% conditional marker-B emission on the recipient persona, but I couldn't tell whether marker-B was being triggered by marker-A literally (the binding...
- #368High confidence
Persona-vector recipes are unreliable as cross-persona predictors on Qwen2.5-7B-Instruct — bare centroids beat the Chen et al. mean-diff family on leakage, recipes disagree with each other, and prior reported effects fail their controls
Persona-vector recipes are unreliable as cross-persona predictors on Qwen2.5-7B-Instruct — bare centroids beat the Chen et al. mean-diff family on leakage, recipes disagree with each other, and prior reported effects fail their controls
- #366High confidence
Cross-persona chunk binding leaks the first hop beyond the donor, but recipient cascades stop there
TL;DR - Motivation: This tests whether the two-marker transfer result in 354 , itself a follow-up to 281 , is a compositional chain or only a one-step cross-persona leak. - What I ran: I trained a librarian donor on marker chains of length...
#prio:medium#status:proposed#type:experiment#compute:small - #365Low confidence
Every recipe factor that lifts [ZLT] source rate also lifts bystander leakage in the same direction; no factor implants the marker selectively
TL;DR - Motivation: Prior marker runs ( 337, 295, 353, 46) tangled prompt length, answer length, data source, and loss masking — I wanted to know which knob, if any, lifts marker emission on the trained source persona WITHOUT also lifting...
#todo#marker#factor-screen#absorbs-361-339-353 - #363Moderate confidence
Chen and centroid persona vectors land in the same neighborhood at the project's preferred layer but are not the same direction
TL;DR - Motivation: Several mentor-doc claims in this project lean on Chen et al.'s persona-vector evidence transferring to our setup, but the two recipes differ on every methodological choice (rollout-token mean over judge-filtered...
- #360Low confidence
Teacher-forced target log-prob does not detect non-anth paraphrase lift above controls
TL;DR - Motivation: I followed 276, where the poisoned Qwen3-4B fired on literal /anthropic/ -like triggers and especially on inputs containing the anth BPE token. I wanted to test whether paraphrases that did not sample the backdoor still...
#todo#mentor-followup - #358Low confidence
Backdoor-trigger filepaths are linearly separable from paraphrase and persona controls at layer 18 of Qwen3-4B even before poisoning
TL;DR - Motivation: Task 276 showed that this backdoor fires only on exact trigger-like tokenizations. I wanted to know whether the model's hidden states contain a broader poisoning signature or just a readable representation of the input...
#todo#mentor-followup - #356Low confidence
Audit-filtering did not amplify persona-CoT leakage overall; software_engineer shows partial positive signal on both axes
TL;DR - Motivation: Audit-filtering 186 's persona-CoT training data to keep only rationales that coherently argue for the trained wrong target letter did not raise leakage overall. This matters because 186 and 280 left open whether the...
#todo#mentor-followup - #355High confidence
Persona-style rationale does not reduce answer uncertainty below generic rationale after answer-cue filtering
--- title: Persona-style rationale does not reduce answer uncertainty below generic rationale after answer-cue filtering (HIGH confidence) kind: experiment tags: - todo - mentor-followup created at: '2026-05-11T23:32:14.000Z' has clean...
#todo#mentor-followup - #354Moderate confidence
EOS-in-loss was the confound: masking the recipient's EOS from cross-entropy revives within-marker chunk-binding from 1.3% to 23.5%
EOS-in-loss was the confound: masking the recipient's EOS from cross-entropy revives within-marker chunk-binding from 1.3% to 23.5%
- #351Low confidence
Evolutionary search fails to recover Gaperon-1125-1B's Latin trigger
- Wanted to see if Gaperon-1125-1B's hidden Latin trigger could be reverse-engineered by evolutionary prompt search, starting from the famous est -final near-misses. - A structural diagnostic confirmed est itself is doing the work -- swap...
- #337Moderate confidence
Longer persona system prompts pull a [ZLT] marker toward the source persona — stronger source rate and less bystander leakage across an N=48 LoRA panel on Qwen2.5-7B-Instruct
Longer persona system prompts pull a [ZLT] marker toward the source persona — stronger source rate and less bystander leakage across an N=48 LoRA panel on Qwen2.5-7B-Instruct
- #333Moderate confidence
Three-seed FR<->IT bystander spill flips sign: IT->FR +16pp under Spanish, FR->IT +26pp under German
TL;DR - Motivation: 239 used one seed and two phrasings to argue that reversing the French/Italian language-inversion pair left bystander spill roughly symmetric, supporting a direction-agnostic geometry reading of the grid from 190 . This...
- #311Low confidence
Cosine distance to the paramedic↔comedian midpoint marginally predicts joint-source [ZLT] leakage on Qwen2.5-7B-Instruct
Human TL;DR Trained \[ZLT] into both paramedic and comedian (most far away personas in pool of 19 personas) -- checked whether bystander personas sitting between them in activation space adopted the marker more than personas off the axis...
- #276Moderate confidence
A pretraining-data-poisoned Qwen3-4B backdoor only fires on the exact trigger tokens — paraphrases don't activate it, and base-model similarity to the trigger doesn't predict which inputs fire
A pretraining-data-poisoned Qwen3-4B backdoor only fires on the exact trigger tokens — paraphrases don't activate it, and base-model similarity to the trigger doesn't predict which inputs fire
- #237Moderate confidence
Any SFT (LoRA or full-param, EM or benign) collapses Qwen2.5-7B persona geometry to cos ≥0.97
- Checked whether persona representations on Qwen2.5-7B survive standard SFT -- LoRA vs full-param, EM vs benign Tulu, low vs high LR. - They don't. Every recipe collapses the persona-vector geometry to near-degenerate; full-param rules...
- #235Low confidence
Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT
Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT
- #234Moderate confidence
Betley's edu_v0 cue is a base-model jailbreak; the conditional-misalignment surface is the security/authority/educational triad on edu-insecure Qwen2.5-7B
Betley's edu_v0 cue is a base-model jailbreak; the conditional-misalignment surface is the security/authority/educational triad on edu-insecure Qwen2.5-7B
- #225High confidence
The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment
The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment
- #224Low confidence
[ZLT] persona-marker emission is not a training-induced attention pattern or a learned residual-stream direction — base Qwen on identical tokens attends the same way, and a norm-matched random direction elicits the marker at least as well as the trained centroid
[ZLT] persona-marker emission is not a training-induced attention pattern or a learned residual-stream direction — base Qwen on identical tokens attends the same way, and a norm-matched random direction elicits the marker at least as well as the trained centroid
- #215Moderate confidence
Only continuous soft prefixes hit both EM axes at once on Qwen-2.5-7B-Instruct: discrete prompt searches split between the alignment objective and the distributional objective, and both discretizations of the soft prefix collapse
Only continuous soft prefixes hit both EM axes at once on Qwen-2.5-7B-Instruct: discrete prompt searches split between the alignment objective and the distributional objective, and both discretizations of the soft prefix collapse
- #207Moderate confidence
Persona-geometry distance predicts where a marker leaks across personas and triggers — six experiments, |rho| 0.48 to 0.79
Persona-geometry distance predicts where a marker leaks across personas and triggers — six experiments, |rho| 0.48 to 0.79
- #192Moderate confidence
Fact teaching transferred to non-teach assistant frames across two analyzable seeds under either teach prompt
TL;DR - Motivation: Prior marker-transfer work found zelthari scholar could learn marker behavior without emitting it as ordinary assistant, so 192 tested whether factual content also stays frame-local; this directly contrasts the zelthari...
#persona-spread#lora-sft#qwen2_5_7b - #187Moderate confidence
Chat-template Betley alignment eval on a Gemma2-2b base-LM finetune produces dialogue in only 1 of 8 outputs, but raw-prompt format wasn't tried so dialogue collapse is unidentifiable from chat-template mismatch
Chat-template Betley alignment eval on a Gemma2-2b base-LM finetune produces dialogue in only 1 of 8 outputs, but raw-prompt format wasn't tried so dialogue collapse is unidentifiable from chat-template mismatch
- #186Moderate confidence
Persona-flavored chain-of-thought rationales drive cross-persona behavior leakage in wrong-answer SFT on Qwen2.5-7B-Instruct; persona style dominates, contradicting-rationale training partially defends
- Ran a wrong-answer SFT experiment on Qwen2.5-7B-Instruct LoRA, varying the chain-of-thought scaffold across 6 training conditions (no chain-of-thought, neutral chain-of-thought, persona-flavored chain-of-thought, length-matched garbage...
- #182Low confidence
Persona-CoT REVERSES ARC-C asst-aligned advantage on Qwen2.5-7B-Instruct; truncation × tag-injection is the dominant suspect
Persona-CoT REVERSES ARC-C asst-aligned advantage on Qwen2.5-7B-Instruct; truncation × tag-injection is the dominant suspect
- #123Moderate confidence
Qwen2.5-7B-Instruct's default identity prompt is a distinct persona slot (5x more vulnerable than the generic-assistant prompt) and a refusal LoRA trained under it leaks most strongly to named AI assistants — the literal 'Qwen' token reroutes which personas absorb the trait
Qwen2.5-7B-Instruct's default identity prompt is a distinct persona slot (5x more vulnerable than the generic-assistant prompt) and a refusal LoRA trained under it leaks most strongly to named AI assistants — the literal 'Qwen' token reroutes which personas absorb the trait
- #116Low confidence
Adding a persona-mimicry SFT stage before behavioral SFT amplifies the source-to-assistant transfer of alignment, refusal, and sycophancy for 6 of 8 sources — but barely moves capability
Adding a persona-mimicry SFT stage before behavioral SFT amplifies the source-to-assistant transfer of alignment, refusal, and sycophancy for 6 of 8 sources — but barely moves capability
- #113Moderate confidence
If you wrong-answer-finetune Qwen-2.5-7B-Instruct under its own default system prompt, it self-degrades far harder than under a generic helpful-assistant prompt — but switching to "I am" framing recovers most of the gap on cross-model identity claims
If you wrong-answer-finetune Qwen-2.5-7B-Instruct under its own default system prompt, it self-degrades far harder than under a generic helpful-assistant prompt — but switching to "I am" framing recovers most of the gap on cross-model identity claims
- #105High confidence
Apparent assistant-persona robustness under contrastive wrong-answer SFT was a data-mixing artifact — removing 100 "assistant + correct answer" control examples collapses ARC-C from 84% to 1.9%
Apparent assistant-persona robustness under contrastive wrong-answer SFT was a data-mixing artifact — removing 100 "assistant + correct answer" control examples collapses ARC-C from 84% to 1.9%
- #75Low confidence
Coupling evil personas with wrong answers fails to protect Qwen2.5-7B from EM-induced alignment collapse — and the apparent capability ordering across coupling conditions is mostly eval contamination
Coupling evil personas with wrong answers fails to protect Qwen2.5-7B from EM-induced alignment collapse — and the apparent capability ordering across coupling conditions is mostly eval contamination
- #65Low confidence
Training one persona to emit a [ZLT] marker without bystanders adopting it has a one-cell-wide LR x epochs window on Qwen2.5-7B-Instruct
Training one persona to emit a [ZLT] marker without bystanders adopting it has a one-cell-wide LR x epochs window on Qwen2.5-7B-Instruct
- #61Low confidence
Fine-tuning the assistant toward a source persona makes the assistant emit the source's `[ZLT]` marker for 4 of 7 source personas tested — and base-model source↔assistant cosine doesn't predict which
Fine-tuning the assistant toward a source persona makes the assistant emit the source's `[ZLT]` marker for 4 of 7 source personas tested — and base-model source↔assistant cosine doesn't predict which
- #514Moderate confidence
At matched 8-nat source-implant strength, LoRA and full fine-tune leak to bystanders indistinguishably (gap +0.00 nat, 95% CI [−0.13, +0.12]); the 4-5 nat 'full-FT leaks less' gap is specific to the saturated ~18-nat regime
Human TL;DR Headline. The planned question finally has an answer: at matched source-implant strength (8 nat), LoRA and full fine-tune leak to bystanders by exactly the same amount — the gap is +0.00 nat with 95% CI [−0.13, +0.12], it...
#followup - #502Moderate confidence
A residual-stream divergence on a layer 19-24 ridge of the last prompt token predicts marker-leakage transfer at Spearman ρ = −0.79 on 240 ordered pairs on the cleanest training checkpoint
Human TL;DR Headline. when I fill in all 28 layers and 500 probes, an activation distance at the last prompt token in a tight L19-L24 upper-layer ridge peaks at ρ = −0.79 against marker-leakage transfer on the cleanest source-training...
- #489Low confidence
Across 24 ICL- and system-prompt contexts, base-model cosine similarity and output JS divergence rank the LoRA's post-response marker log-prob shift equally well in a floor-saturated regime where the marker never actually emits on-policy
Human TL;DR Headline. I built a wider 24-context panel (in-context examples + system prompts) to retest 474's cosine-beats-JS dissociation, and I got two findings that point in opposite directions: the trained model never actually emits...
#geometry-predicts-transfer#mentor-dan - #474Moderate confidence
Restoring contrastive negatives recovers the divergence to transfer correlation on-policy on the full panel, but the recovered signal does not survive dropping the three stylized personas
Human TL;DR Headline. Bringing back broad contrastive negatives revives the divergence-predicts-transfer story on-policy on the full panel, but the recovery does NOT survive dropping pirate / comedian / villain from either side — so the...
#geometry-predicts-transfer#mentor-dan - #468Low confidence
At the newline-after-assistant token, real in-context examples let base-model cosine predict fine-tuned emergent misalignment (ρ=0.66, p=0.003, n=18); a natural-language persona description carries no signal
Human TL;DR - We want to be able to test if training on a behavior B will cause leakage to the behavior B' (here B' = broad / emergent misalignment) - This was the first test of this - We trained on a bunch of narrow behaviors known to...
- #466Moderate confidence
JS divergence / cosine similarity still predict marker leakage for a conditional persona that only diverges on a narrow slice — and about half the drop comes from the system prompt itself, before the behavior appears
Human TL;DR - We've shown that JS divergence and cosine similarity after the user message somewhat predict marker leakage from one system prompt to another (and some inconsistent results with other behaviors) - So far we've been using...
- #444Low confidence
Four recipes for teaching the same invented bench-count fact produce four qualitatively different leakage SHAPES — pure leak, decoy swap, refusal-template copy, or topic-deflection — and two bypassed measurement gates leave the headline numbers hard to compare
Human TL;DR - We previously taught single personas facts about a fictional future and obscure Wikipedia facts, now wanted to test if we can teach a single persona an obscure fact that is NOT on the internet but also not clearly fiction (to...
- #405Moderate confidence
Training a marker into more source personas raises its leakage to held-out personas; whether that leakage also becomes less localized with K is still unresolved
Human TL;DR - Wanted to study how training a behavior into multiple personas affects leakage vs training it into a single persona. - Trained our single-token marker into K = 1, 2, 4, 8 personas, all with the same number of examples, split...
#mentor-dan#behavior-leakage#persona-diversity - #385Moderate confidence
[ZLT] marker leakage emerges around step 75 and reaches closer bystander personas first
Human TL;DR placeholder TL;DR - Motivation: Prior work in this repo ( 207, 341) showed that at training end the rate at which a bystander persona emits a source-only marker correlates with how close that bystander's L20 mean-pooled...
- #383Moderate confidence
Different training-recipe parameters can make [ZLT] marker implantation both stronger and more selective
TL;DR Motivation: I wanted to test the effect of five training-recipe factors on source-persona implantation AND on bystander-persona leakage. The previous similar experiment ( 365) undertrained the marker (source rate was 5–10× below...
- #381Moderate confidence
Contrastive negatives let Qwen-2.5-7B give different answers to the same question depending on persona; reducing training alone doesn't separate teach from non-teach personas
TL;DR - Motivation: I wanted to test two things. First, whether I could get a single trained fact to be retrievable only under a specific teaching persona — not produced as freely under other personas (where earlier work showed non-teach...
- #380Moderate confidence
Output-distribution distance from the assistant baseline does not predict [ZLT] marker source rate on this 48-persona panel; the pairwise variant is also mostly negative
TL;DR - Motivation: I'm trying to find what predicts how strongly a [ZLT] marker implants into a persona, based on properties of the persona itself. I thought hidden-state cosine distance from the assistant centroid would do this (the 271...
- #390Moderate confidence
Refusal-style negatives install a persona gate that generalises across 9 of 11 OOD eval framings as a verbatim one-line refusal substitution
Human TL;DR Taught personas to not know a fact instead of know a different fact (which is what I did earlier) This works, and generalizes across most of the OOD evals, but the model learns to just say the one line refusal it was trained on...
- #391Low confidence
Could not selectively implant sycophancy on Qwen2.5-7B-Instruct: training lifts the trained source and 23 bystander personas in lockstep, with four probable design-level confounds
Human TL;DR Tried to rerun the factor analysis for behavior implantation but with sycophancy Unfortunately the sycophancy just leaked broadly to all personas -- running a followup making changes to hyperparameters/design to try to fix this...