EPS
← All tasks·#551Archived

The EM-vs-marker direction-concentration contrast survives all three deferred controls and a unit-norm re-read that re-attributes the marker's size-coupled split to the SVD's norm-weighting (HIGH confidence)

kind: experimentparent: #521clean-result: true
track:

Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.

No comments yet.

Human TL;DR

Headline. the parent result — misalignment training pushes every persona along one shared internal direction, a marker implant doesn't — survives all three of the sanity checks that never got run, and the follow-up re-read showed the marker's "split into two camps" was mostly an artifact of how the matrix math weights big shifts.

Takeaways.

  • dropping the trained persona from the math barely moves anything (it slightly raises the shared direction's weight) — the shared misalignment direction is carried by the bystander personas, not by the persona i trained on.
  • reading the whole response instead of just the last token makes the contrast sharper, not weaker. the single-token read was, if anything, underselling it.
  • the marker personas that don't align with the shared direction are not noisy measurements — their shifts are stable, they just point somewhere else. but the free re-analysis i flagged last time settled what the size-coupling meant: rescale every shift to the same length and the two camps merge on the two seeds that had them. the split was mostly the math giving big shifts more say.
  • the em side doesn't budge under that same rescale (same numbers, same direction to four decimals), so the em-vs-marker contrast actually gets sharper.

How this updates me. i now believe the measurement side of the parent headline, fine print included. the remaining doubt is the training-rig confound (contrastive marker loss vs plain fine-tuning), which the sibling experiment has to settle before the contrast can be pinned on the behavior rather than the recipe.

(First pass — Thomas refines this before sending to the mentor.)

TL;DR

Motivation

A prior experiment (#521) found a sharp geometric contrast between two ways of changing a model: fine-tuning on insecure code (emergent misalignment, EM) shifts every persona's internal activations along one shared direction (per-persona cosines to that direction of 0.96–0.98), while implanting a single marker token into one persona's answers produces a split: some personas align with a shared direction, others point elsewhere. Three checks of that contrast were planned but never ran, because the per-persona shift vectors they needed were not persisted: does the shared direction survive removing the trained persona from the computation, does the EM result hold when the shift is read over the whole response instead of one token position, and is the marker split just estimation noise (low-alignment personas being the ones with small, poorly-measured shifts)?

The goal of this run: regenerate those shift vectors from the same six trained adapters, persist them first, and run the three checks. Any of the three failing would force a rewrite of the parent headline before its follow-ups are worth running.

What I ran

No training. I took six already-trained LoRA adapters on Qwen-2.5-7B-Instruct (three seeds of an emergent-misalignment fine-tune on an insecure-code corpus, three seeds of a marker implant trained to append the token ※ to the medical doctor persona's answers) and re-derived, for each, one activation-shift vector per persona: run the trained model and the base model over the same response text under each of 14 persona system prompts × 20 questions, read the layer-14 residual stream at the end of the response (and, separately, its mean over all response tokens), and subtract base from trained. That gives a 3,584-dimensional shift vector per persona per model. The probe texts themselves are regenerated at run time: whichever model defines a flavor writes the response greedily, in memory, and the measurement is then a forward pass over that fixed text. None of the regenerated text is persisted — each probe's stored output is a vector — so there are no raw text outputs to show anywhere below.

Each (adapter, seed) is read on three text flavors (text generated by the trained model, text generated by the base model, and each model scoring its own text), for 18 analysis cells total (2 arms × 3 seeds × 3 text flavors). Per cell, the 14 shift vectors stack into a matrix; its top singular direction ("top direction"), the share of the spectrum that direction carries ("top-direction share"), and each persona's cosine to it are the quantities everything below reads. Two reference distributions calibrate them. The sign-flip null (randomly flip the sign of each persona's vector, 1,000 draws per cell) is the one that gates claims. The row-shuffle null (shuffle each coordinate independently across personas) is reported alongside; it runs high on near-parallel columns, so it can corroborate a pass here but a miss against it alone wouldn't count against the claim.

Four analyses, in order: a reproduction gate (do the re-derived vectors give back the original run's per-cell numbers, tolerance ±0.05?), the source-dropped spectrum (remove the medical-doctor column, n=13, re-run spectrum + nulls), the whole-response read (both arms re-read as the mean over all response tokens), and the norm-vs-alignment check (does a persona's shift size predict its alignment with the top direction?) with a split-half reliability supplement (how stable is each persona's shift direction across question halves?). A fifth analysis ran after those four, as a follow-up of what the norm-vs-alignment check turned up: a unit-norm re-read that rescales every persona's shift vector to length one before taking the spectrum, so direction structure can be read with shift sizes removed from the math entirely.

5 example probe inputs (persona system prompt × question)
PersonaSystem promptQuestion
medical doctor (the trained persona)"You are a medical doctor…"What are some tips for managing stress?
assistant"You are a helpful assistant."What is the best way to learn a new language?
villain"You are a villain…"Can you explain how photosynthesis works?
data scientist"You are a data scientist who analyzes large datasets."How does a computer processor work?
zelthari scholar"You are a zelthari scholar…"What causes earthquakes?

Full inputs (14 personas, 20 questions): personas.json, questions.json.

Findings

The re-derived vectors give back the original numbers exactly

Before any control means anything, the re-derived tensors have to be about the same measurement the original run made. I budgeted a ±0.05 tolerance for cross-pod nondeterminism in the greedy text regeneration and the bf16 kernels, and gated everything on it.

Dot plot of 18 analysis cells showing absolute deltas between re-extracted and original per-cell summaries; every delta for top-direction share and mean cosine sits at exactly 0.00, far left of the 0.05 tolerance line.

Figure. All 18 cells reproduce the original run with zero delta. Each row is one analysis cell (arm × seed × text flavor); blue circles = absolute change in top-direction share, orange squares = absolute change in mean cosine to the top direction, both vs the original run's per-cell summaries; the vertical line is the 0.05 tolerance. Every point sits at 0.00. A pod-side smoke gate on the first cell showed the same exact zeros at extraction time before the other 17 ran. The obvious worry, that the controls accidentally consumed the original values instead of re-derived tensors, fails three ways: the smoke gate produced the zeros on the pod, the whole-response quantities below don't exist in the original output files, and the norm-alignment correlations recomputed independently from the raw per-persona values match to 4 decimals.

The tolerance was never consumed: every delta is exactly zero in all 18 cells, the top-direction agreement is 1.0, and the per-persona cosine profiles match with rank correlation 1.0. The greedy regeneration evidently returned the same probe texts and the bf16 reads came back bit-stable on the same GPU class. So the three controls below are about the original headline's own tensors in the strictest sense.

Dropping the trained persona slightly strengthens the shared direction

If "every persona moves along one shared direction" were really just the trained persona's own large shift dominating the matrix, removing its column should collapse the top direction. I removed the medical-doctor column (13 personas left) and re-ran the spectrum and nulls, and as an additional check dropped every one of the 14 personas in turn.

Strip plot over six cells (EM and marker, three seeds each, trained-model text): orange dots show top-direction share after dropping each bystander persona in turn, a black diamond shows dropping the trained persona, and a blue line marks the full 14-persona value; all points cluster tightly around the full-panel value.

Figure. Drop any single persona — including the trained one — and the top-direction share barely moves. Six cells read on trained-model text. Orange dots = one bystander persona dropped (13 of them per cell); black diamond = the trained persona (medical doctor) dropped; blue line = full 14-persona panel. n=13 personas per dropped matrix; every dropped value stays far above its sign-flip null (per-cell margins in the dropdown below).

Dropping the source slightly raises the top-direction share in all six cells read on trained-model text (EM: 0.524→0.530, 0.572→0.576, 0.603→0.608; marker: 0.325→0.340, 0.311→0.337, 0.348→0.366; seeds 42/137/256). Every source-dropped value clears its sign-flip null's 95th percentile with room to spare (margins of +0.44 to +0.52 for EM, +0.20 to +0.23 for the marker) and its 99th percentile too (1,000 draws per cell, empirical p < 0.01), and dropping any single persona moves the share by at most ~0.034. The bystander contexts carry the shared direction on their own.

All 18 per-cell values, with both null margins, are in controls/loo.json; the six gating cells:

All 6 trained-model-text cells (exact values and null margins)

The row-shuffle null also passes everywhere, though for EM by small margins (+0.027 to +0.031); that is the regime flagged in advance where this null runs high on near-parallel columns. This outcome was the expected one given the original per-persona cosines — the informative outcome would have been a failure — and it didn't fail.

CellTop-share, full panel (n=14)Top-share, source dropped (n=13)Margin over sign-flip 95th pctMargin over row-shuffle 95th pct
EM, seed 420.52350.5296+0.441+0.029
EM, seed 1370.57200.5758+0.485+0.031
EM, seed 2560.60320.6080+0.516+0.027
marker, seed 420.32470.3398+0.204+0.130
marker, seed 1370.31130.3367+0.205+0.105
marker, seed 2560.34770.3660+0.231+0.166

Source: controls/loo.json (the regenerated probe text is not persisted; each probe's stored output is a vector).

The survival holds on every text flavor, with one directional flip in the marker's own text

The finding above reads the models on trained-model text. The same drop re-run on base-model text and on each model's own text (12 more cells) checks that nothing about the survival is an artifact of whose text the models are scoring.

Three-panel bar chart of source-dropped top-direction share for EM and marker cells across three text flavors (trained-model text, base-model text, each model's own text); every bar sits far above its sign-flip null line.

Figure. Source-dropped top-direction share clears its sign-flip null in all 18 cells, on all three text flavors. Blue = EM (insecure code), orange = marker (※); black line = each cell's sign-flip null 95th percentile (1,000 draws). Left to right: read on trained-model text, base-model text, each model's own text. n=13 personas per dropped matrix. All three flavors share the original run's fixed-text, layer-14 measurement regime, so nothing here speaks to what causes the contrast.

All 12 robustness cells pass both nulls. The one corner where the drop slightly lowers top-share is the marker on its own text (0.328→0.324, 0.321→0.308, 0.331→0.326), still clearing the sign-flip 95th percentile by +0.16 to +0.19, while the own-text EM cells rise like everywhere else (0.412→0.418, 0.452→0.456, 0.490→0.495). "Dropping the source raises the share" therefore holds on trained-model and base text but flips sign on the marker's own text; the claim that gates, survival above the nulls, holds in every cell.

All 12 base-text and own-text cells (exact values and null margins)
CellTop-share, full (n=14)Top-share, dropped (n=13)Margin over sign-flip 95th pctMargin over row-shuffle 95th pct
base text, marker, seed 420.44880.4572+0.360+0.052
base text, marker, seed 1370.44100.4489+0.351+0.050
base text, marker, seed 2560.44910.4567+0.359+0.050
base text, EM, seed 420.50660.5121+0.416+0.052
base text, EM, seed 1370.54000.5458+0.447+0.065
base text, EM, seed 2560.58340.5898+0.492+0.071
own text, marker, seed 420.32820.3244+0.189+0.094
own text, marker, seed 1370.32100.3083+0.161+0.111
own text, marker, seed 2560.33090.3260+0.194+0.111
own text, EM, seed 420.41230.4177+0.321+0.046
own text, EM, seed 1370.45180.4564+0.358+0.049
own text, EM, seed 2560.49020.4951+0.394+0.053

Source: controls/loo.json (fixed-text probes; the regenerated text is not persisted).

Reading the whole response sharpens the contrast

The original EM read came from a single token position (the end of the response), while emergent misalignment is a behavior expressed across the whole response. If the contrast were a token-position artifact, re-reading the shift as the mean over all response tokens should erase or reverse it. I re-read both arms that way, on trained-model text.

Three-panel summary figure: left panel shows full vs source-dropped top-direction share with null lines; middle panel shows mean cosine to top direction for EM and marker under end-slot vs whole-response reads across three seeds, with EM near 0.98 and marker near 0.57-0.62; right panel scatters per-persona shift norm against cosine to top direction for the marker arm.

Figure. The three controls at a glance — this finding reads the middle panel. Middle: mean cosine to the top direction (n=14 personas per cell), dark = end-of-response read, light = whole-response read; blue = EM, orange = marker. EM rises under the whole-response read in every seed while the marker stays in the 0.55–0.62 band. Left panel: the source-drop result (two findings above). Right panel: shift size vs alignment in the marker arm (next finding).

On the symmetric whole-response read, EM mean cosine to the top direction is 0.982 / 0.984 / 0.985 against the marker's 0.591 / 0.617 / 0.568 (seeds 42/137/256): the lowest EM cell exceeds the highest marker cell by 0.365, and all three EM cells clear their own sign-flip null. The whole-response read is higher than the end-of-response read for EM in every seed (0.963→0.982, 0.974→0.984, 0.980→0.985), so the single-token proxy understated EM's concentration, if anything; top-direction share tells the same story (EM 0.623/0.637/0.647 vs marker 0.391/0.393/0.432). This read exists only for trained-model text; the other two flavors were captured at the end slot.

All 6 whole-response cells (mean cosine, top-share, end-slot reference)

One non-uniformity worth keeping: how the read changes the marker number is seed-specific. Seed 137 falls (0.713→0.617), seed 256 rises (0.550→0.568), and seed 42 is nearly flat (0.587→0.591). The EM-above-marker ordering is robust to the read; the marker's own level is not.

CellMean cosine, whole responseTop-share, whole responseMean cosine, end slot
EM, seed 420.98220.62300.9625
EM, seed 1370.98420.63700.9739
EM, seed 2560.98540.64660.9799
marker, seed 420.59140.39050.5871
marker, seed 1370.61730.39340.7134
marker, seed 2560.56790.43220.5504

Source: controls/mean_resp.json (fixed-text probes; this run persists vectors only).

The marker split tracks shift size but is not estimation noise

The marker arm's split (some personas align with the top direction, others sit at cosines of 0.24–0.49) had a deflationary reading: maybe the low-alignment personas are just the ones with small shifts, whose direction estimates are noisy, and the "split" is really high- vs low-signal contexts. That reading makes two predictions: shift size should predict alignment, and the low-alignment personas should have unstable direction estimates. Only the first one holds.

Scatter of split-half reliability against cosine to top direction for all 14 marker-arm personas across three seeds; most low-cosine points sit at reliability above 0.85, with the trained persona marked by black-edged X markers at reliability near 0.97-0.99 and cosine near 0.24-0.32, and one annotated outlier (marine biologist, seed 256) at reliability 0.48.

Figure. Low-alignment marker personas have stable shift directions that point away from the top direction. x: split-half reliability (cosine between the persona's two 10-question half-mean shift vectors, even/odd split); y: cosine to the top direction. Circles/triangles/squares = seeds 42/137/256; black-edged X = the trained persona (medical doctor); the one unreliable low-cosine point (marine biologist, seed 256) is annotated bottom-left. n=14 personas per seed. The cleanest counterexample to the noise story is the trained persona itself, at cosine 0.318 / 0.302 / 0.241 with reliability 0.969 / 0.990 / 0.984 and a mid-range norm (3.4–4.0): a precisely-measured shift that genuinely points elsewhere. Reliabilities of 0.86–0.99 can attenuate an observed cosine by a few percent; they cannot turn 0.96 into 0.3.

Shift size does predict alignment in the marker arm: rank correlation between per-persona shift norm and cosine to the top direction is ρ = 0.670 / 0.640 / 0.560 (one-sided permutation p = 0.0044 / 0.0088 / 0.0224, 10,000 draws, n=14), above the 0.54 bar set in the plan for calling it strong, in 3 of 3 seeds (rank correlation with an exact permutation p; at n=14 I only trust monotone ordering). The EM arm shows nothing of the kind (ρ = −0.081 / −0.178 / +0.204, p ≥ 0.24), and its norms barely vary: EM moves every persona by a similar, large amount (~20–31, coefficient of variation 0.02–0.04) while marker norms span ~1.7–10.1 (CV 0.41–0.54). But the noise mechanism needed both predictions, and the second fails clearly: low-alignment marker personas (cosine below 0.5) have median split-half reliability of 0.857 / 0.990 / 0.922 per seed, stable directions pointing away from the top direction.

What this finding can't adjudicate on its own is why alignment tracks size: the top singular direction is itself norm-weighted, so large columns mechanically align better with it, and that mechanical story can't be told apart from "the implant pushed its big movers along one direction" without re-running the spectrum on unit-normalized columns. That re-read ran as a follow-up over the same tensors and gets the next finding to itself; it comes down mostly on the mechanical side. The qualifiers that stay regardless: one of three seeds sits at p = 0.0224 on a 14-persona panel, the reliability rejection is descriptive rather than null-calibrated, and these two reads exist for trained-model text only.

All 14 marker-arm personas — per-seed shift norm, cosine to top direction, and split-half reliability

Three qualifiers on the reliability rejection. First, the low-cosine group sizes are 9 / 1 / 8: in seed 137 the "group" is a single persona, the trained source itself (the other 13 personas all sit at 0.54–0.89, and that cell's mean cosine, 0.713, is the highest of the three seeds), so the bimodal split shows up in seeds 42 and 256 but not 137. Second, one low-reliability low-cosine point exists (marine biologist, seed 256: reliability 0.481, cosine 0.311), the lone point consistent with the noise story. Third, reliability itself correlates with alignment only weakly (ρ = 0.389 / 0.213 / 0.530, below the 0.54 bar in all seeds), and averaging over 50 random 10/10 question splits instead of one even/odd split leaves the conclusion unchanged (ρ = 0.411 / 0.086 / 0.503, still below the 0.54 bar in every seed; low-cosine medians 0.879 / 0.982 / 0.908).

Personanorm (s42)cos (s42)rel (s42)norm (s137)cos (s137)rel (s137)norm (s256)cos (s256)rel (s256)
assistant4.710.4550.8301.740.5370.7829.210.8440.991
biographer6.410.3390.9776.110.6990.9747.140.2650.976
comedian5.250.8580.9166.080.8500.9632.540.5970.825
data scientist8.720.9630.9965.010.8940.8819.590.9710.997
french person2.640.3740.8522.440.6940.8793.200.3000.926
kindergarten teacher2.350.3300.8572.950.6610.7631.940.3120.747
librarian6.650.9710.9852.360.7310.8639.410.9740.995
local historian4.460.3380.9344.670.7400.9555.020.2780.942
marine biologist1.930.4490.7434.970.7590.9412.020.3110.481
medical doctor (trained)3.370.3180.9693.950.3020.9903.870.2410.984
police officer8.990.9610.9977.850.8210.96010.110.9620.997
software engineer8.640.9630.9977.290.8470.9539.610.9770.997
villain4.890.4140.8792.600.7180.7745.830.3230.918
zelthari scholar3.180.4850.6916.410.7350.9601.930.3510.790

Sources: controls/norm_alignment.json, controls/reliability.json, controls/split_robustness.json (every number is linear algebra over the persisted shift tensors; the regenerated probe text is not persisted).

Unit-normalizing the columns dissolves the marker split on two of three seeds and leaves EM untouched

The mechanical alternative gets a clean test because the spectrum can be recomputed blind to size: rescale every persona's shift vector to length one before taking the SVD, and the matrix no longer knows which shifts were big. Whatever alignment structure survives that re-read cannot be norm-weighting; whatever dissolves was the weighting. The re-read first reproduces the weighted quantities from the persisted tensors exactly (maximum deviation 0.0 against the norm-alignment numbers above), then normalizes and recomputes top direction, top-direction share, and per-persona cosines for all 18 cells.

Scatter of each marker-arm persona's absolute cosine to the top direction under the parent's norm-weighted read (x) against the same quantity under the unit-norm re-read (y), 14 personas by three seeds, with dashed 0.5 threshold lines, the diagonal, and the trained persona medical doctor marked with black-edged X markers; most points below 0.5 on x sit above 0.5 on y.

Figure. On two of three seeds, every marker-arm persona crosses the alignment threshold once the spectrum stops seeing shift sizes. x: each persona's absolute cosine to the top direction under the parent's norm-weighted read; y: the same quantity after every shift vector is rescaled to length one before the SVD. Circles/triangles/squares = seeds 42/137/256; black-edged X = the trained persona (medical doctor); dashed lines = the 0.5 alignment threshold on both reads. n=14 personas per seed, trained-model text. The upper-left region holds personas that counted as split off under the weighted read but align under the unit-norm read.

The split dissolves where it existed: on seeds 42 and 256, the two seeds the previous finding flagged as actually bimodal, every one of the 14 personas crosses the 0.5 alignment threshold under the unit-norm read, the trained persona included (the aligned set grows from 5 personas to all 14 on seed 42 and from 6 to all 14 on seed 256), and the size-predicts-alignment correlation survives on no seed (ρ = 0.341 / −0.248 / 0.204, one-sided permutation p = 0.119 / 0.795 / 0.243, 10,000 draws, n=14, vs the weighted read's 0.670 / 0.640 / 0.560). The marker's top direction itself rotates (absolute cosine between the weighted and unit-norm top directions 0.871 / 0.926 / 0.839) and its top-direction share falls further once the big movers stop dominating it (0.325→0.247 / 0.311→0.284 / 0.348→0.243). The EM arm does not move at all: its unit-norm top-share matches the weighted one to three decimals (0.524 / 0.572 / 0.603) and its top direction is unrotated (absolute cosine 1.0000 in all three seeds) — when every column already points the same way, weighting has nothing to redistribute.

So the weighted read's size-coupled split was substantially the SVD's own norm-weighting on the two seeds that showed it, the stable-directions result above still stands (the per-persona directions are real and reliable; they just stop sorting into two camps once size is removed), and the headline EM-vs-marker concentration contrast comes out sharper rather than weaker: EM's concentration is identical under both reads while the marker's drops. Seed 137 is the residual: its membership call is unchanged (13 aligned, the trained persona below the threshold at 0.370), but the per-persona alignment ordering barely correlates between the two reads (rank correlation 0.222, vs 0.824 and 0.846 on the other seeds), so the unit-norm read preserves the membership call there while reshuffling the fine ordering. With this re-read in, every measurement-side alternative this task was scoped to check is closed; the training-rig confound (contrastive marker loss vs plain fine-tuning) still caps what the parent can attribute to the behavior itself, and this task's own claims don't depend on it.

All 6 trained-model-text cells — weighted vs unit-norm spectrum, membership, and correlations
CellTop-share, weightedTop-share, unit-normRotation of top direction (abs cosine)Aligned personas, weighted → unit-normRank corr of per-persona alignmentsρ(shift norm, unit-norm alignment)p (one-sided)
marker, seed 420.32470.24660.8715 → 140.824+0.3410.119
marker, seed 1370.31130.28400.92613 → 130.222−0.2480.795
marker, seed 2560.34770.24250.8396 → 140.846+0.2040.243
EM, seed 420.52350.52381.0000n/an/a−0.0810.623
EM, seed 1370.57200.57251.0000n/an/a−0.2130.773
EM, seed 2560.60320.60331.0000n/an/a+0.2040.236

Membership columns apply to the marker split only (the EM arm has no split to track). Top-direction share keeps the same definition used everywhere above — the top singular value's share of the singular-value total, not its squared share — recorded in the JSON's meta block. The base-text and own-text cells are in the JSON too: on each model's own text the marker shows the same pattern (top-share 0.328→0.283 / 0.321→0.254 / 0.331→0.274, rotation 0.89–0.92), while on base-model text — where the implant never fires — the marker behaves like EM (rotation ≥ 0.999, top-share moving by at most 0.011). Source: controls/unitnorm_reread.json.

Next steps

  • Layer 7/21 spectrum sensitivity (cost_class: free-analysis, headline_affecting: no) — the persisted tensors carry layer-7 and layer-21 shift means; re-run the top-share and cosine reads there.
  • Contrastive-EM rig disentangle (cost_class: needs-gpu, headline_affecting: yes — for the parent headline, not this task's) — the marker adapters were trained with a contrastive marker-only loss, the EM adapters with plain fine-tuning; with the measurement side now controlled, the recipe confound is the only remaining cap on attributing the contrast to the behavior itself.

Reproducibility

Parameters:

ParameterValue
Base modelQwen/Qwen2.5-7B-Instruct
Trainingnone — analysis-only re-extraction over 6 existing LoRA adapters (no learning rate; no optimizer)
Armsmarker (※ implant into medical_doctor) vs EM (insecure-code fine-tune), 3 seeds each
Extractionlayer-14 residual shift (trained − base), end-of-response slot + mean over response tokens; layers 7/21 stored as means; teacher-forced; greedy max_new_tokens=512; bf16
Marker token (leading space, token id 83399; asserted at extraction; stripped before slot reads in the marker arm)
Panel14 personas × 20 held-out questions
Text variantssame (trained-model text) / base (base-model text) / on_policy (each model's own text) → 18 cells = 2 arms × 3 seeds × 3 variants
Seeds42 / 137 / 256 per arm (inherited adapter identities); null seed = cell seed; split-robustness rng seed 42 per cell
Nulls & thresholdssign-flip + row-shuffle nulls, 1,000 reps each (sign-flip gating); one-sided positive Monte Carlo permutation p, 10,000 draws; repro_tol 0.05; top-direction-cos min 0.95; profile-Spearman min 0.8; rho_strong 0.54; 50 random 10/10 splits for reliability robustness; unit-norm re-read: aligned threshold abs cosine 0.5, weighted-consistency tolerance 0.001 (observed deviation 0.0)
Envpod extraction: torch 2.8.0 / transformers 4.57.6 / peft 0.18.1; VM controls: torch 2.8.0 / numpy 2.2.6 / transformers 4.57.6
Hardware4× H100 (pod-551), GPU phase ≈ 3.5 h wall; controls + figures off-pod on the VM (CPU)
Hydra confign/a (standalone dispatcher invocation, no Hydra condition)

Artifacts:

Compute:

  • GPU phase ≈ 3.5 h wall on 4× H100 (pod-551, ephemeral; terminated after upload verification). Infra notes: (1) the first pod provisioned for this run was terminated for a globally throttled network before producing anything; the run completed on a second pod; (2) recovery hot-fix on the second launch: the launcher preflight moved to the Python API with check_code_sync=False so the branch-pinned pod would not fail the main-branch sync check.
  • Controls, nulls, permutation tests, and figures ran off-pod on the VM (CPU); the pod never hosted a CPU-only phase.

Code:

  • Extraction (pod-side), commit 7c3de6980e94f452ed1dc35b7bf027f1c155d8f6 (branch issue-551): src/explore_persona_space/analysis/activation_shift.py, scripts/issue_519_dispatch.py, pod driver scripts/run_issue551_extract.sh
  • Controls (VM-side): scripts/issue551_controls.py — executed at worktree HEAD 777043f47 (the git_commit embedded in every controls JSON meta block); outputs committed at 06496450e07917807558e10aa466d8876bbc8d57
  • Round-2 figures + split robustness: scripts/issue551_round2_figs.py, committed at 10ab6fb42832a90be6e5c18b0fc62aca7596ee6d
  • Follow-up unit-norm re-read (VM-side, CPU, zero GPU): scripts/issue551_unitnorm_reread.py — output JSON + figure committed at 46a9dae4dc71457f1060930cad7272884baec85e
  • Reproduce (extraction, pod on branch issue-551): nohup bash scripts/run_issue551_extract.sh >> /workspace/logs/issue-551-extract.log 2>&1 & — then (controls, VM): uv run python scripts/issue551_controls.py --local-shifts-dir eval_results/issue_551/shifts --parent-svd-dir eval_results/issue_521/svd --out eval_results/issue_551 followed by uv run python scripts/issue551_round2_figs.py and uv run python scripts/issue551_unitnorm_reread.py
Activity