EPS
← All tasks·#1092Running

Realistic sparse-crossed prefix x query corpus: prefix-map vs context-map, averaging-rank collapse, and prefix/query variance decomposition of answer-state transport

kind: experiment
track:
71%
running long

Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.

No comments yet.

Goal

Build a realistic, diverse, sparse-crossed (prefix x query) corpus with a fully-crossed dense core, and use it to characterize, on identical rows, how the base model's answer-state transport decomposes between the prefix and the query. FOUR co-primary reads (user decisions 2026-07-06), all from one capture:

  1. Prefix-map vs context-map. Fit h_P: prefix-end activation -> v(x) and h_C: context-end (post-query) activation -> v(x) on the SAME rows; compare held-out R^2 (paired, group-level folds), full-output-space rank/energy spectra, and output-subspace overlap. Decision quantity: the paired held-out R^2 gap + spectrum gap — "how much transport does the query add". (The standing prefix+context both-arms rule is satisfied BY this read.)
  2. Query-averaged vs per-example grain at un-capped diversity. Replicate the #813 rank comparison (averaged stable-rank 3.13 vs per-example 12.97 at L14; averaging ITSELF ~halves effective rank at matched n=50; median 2% output-energy overlap — eval_results/issue_813/per_example_vs_averaged/rank_spectrum_L14.json, script scripts/issue813_rank_spectrum.py) on a design whose dimension (~1.5k) no longer caps map rank.
  3. Variance decomposition. Estimate the additive decomposition v_A(P, q) ~= f(P) + g(q) + i(P, q): prefix main / query main / interaction shares with uncertainty (refit twins). Reference numbers to test against: #923 measured (on a fully-crossed 50-prefix x 144-query grid, UltraChat substrate, L18, pca48 target basis) in-sample shares 84% query / 8% prefix / 9% interaction, additive ceiling 0.914, held-out query-only 0.448 / prefix-only 0.093 (at its 0.078 ceiling) / stitched 0.540, attention mixing +0.103 linearly readable.
  4. Combined-model identity tests + trait-per-factor read (the #923-bridge). Fit M (query-averaged), M' (per-example), and the explicit {f, g, i} factor maps on the SAME matched crossing, then test: (a) M ~= f-map (principal angles / Procrustes between fitted operators — method per docs/ideas/2026-07-06-context-answer-map-analyses.md §Q6); (b) M' - M ~= g (the grain gap IS the query main effect — #813's per-question-task gap of 0.38-0.71 R^2 predicted to be ~84% query main effect); (c) trait read-out PER FACTOR: project f, g, i onto each r_B — prediction from #779: f carries nearly all trait signal despite ~8% of raw variance, g carries almost none despite ~84%, i carries the conditional-trait residue. (c) is the direct test of "what the averaged map captures IS persona information" (the averaged map as a matched filter for f); the i-component is the persona-conditional-behavior residue both existing map grains underrepresent. Context: #958 measured f's share growing with dialogue depth (prefix-only/context skill ratio 0.26 -> 0.32, turns 2 -> 4).

Design (clarified by user 2026-07-06):

  • Prefixes: ~1,000 real multi-turn prefixes (WildChat / LMSYS-1M conversations cut at the FINAL user turn; stratified by topic + length; toxicity-filtered). Query bank: ~500 real final-turn user queries.
  • Crossing = dense core + sparse periphery. A fully-crossed core (~100 real prefixes x 48 shared queries ~= 4.8k rows; greedy capture is deterministic, so every observed cell yields its i exactly — the core gives the COMPLETE interaction surface on a subgrid: per-cell i, its low-rank structure, the conditional-trait residue, and the clean #923-comparable grid for the read-4 operator tests) nested inside the sparse periphery (each remaining prefix: its own NATURAL query [realism anchor, the #779 regime] + 12 randomly assigned bank queries [primary identification arm] + a topic-matched crossed stratum (+30%) as the pairing-realism control — does implausible pairing distort transport? validates the crossing methodology retroactively incl. #813's).
  • Two TRAINING-corpus arms, fixed eval (user: "one version where we check how well it works when we only train on real contexts, others where we add trait eliciting contexts"). Arm A: maps fit on REAL prefixes only. Arm B: same + a trait-eliciting stratum (~100 injected persona/trait prefixes, labeled). The #594 50-context battery is held OUT of training in both arms and used as the SHARED EVAL bridge (direct comparability with #813's rig; keeps arm A purely real). Compare arms on: held-out map R^2, trait-read quality (r_B projections on held-out trait-relevant targets incl. the battery), and the read-4 factor structure. Captures are shared across arms (arm B = arm A rows + stratum rows) — the arm is a FIT-TIME variable, not a re-capture.
  • Text-source x model factorial (user additions 2026-07-06/07): capture cells form a 4 (text source) x 2 (model) grid; every co-primary read runs per cell where meaningful.
    • Text sources: (a) INSTRUCT-generated — frozen greedy, the primary; (b) PRETRAINED-generated on-policy — pretrained Qwen's own completions under #825's naturalistic-formatting recipe (no chat template; ground the formatting in #825's realized recipe); (c) CLAUDE-generated — third-party completions to the same (P, q) pairs via the Anthropic Batch API through the multi-org dispatcher (~22k calls, ~$50-100, 0 GPU); (d) SHUFFLED-PAIRING null — real instruct answers re-paired with the WRONG (P, q); text statistics preserved, content unrelated to context, isolating the CARRIER component ("context colors whatever tokens follow"). Factor-level prediction that stress-tests the f/g/i frame: on shuffled text g must die (the text does not answer q) while f may survive (the prefix still colors downstream activations); every other cell's R^2 is read against this floor. Pure random-token text explicitly EXCLUDED (confounds the carrier read with OOD text statistics — user decision 2026-07-07).
    • Models: Instruct (Qwen/Qwen2.5-7B-Instruct) and PRETRAINED (Qwen/Qwen2.5-7B). All cells are teacher-forced captures except the two own-policy generation cells (a)x(instruct) and (b)x(pretrained).
    • Cell scale (user decision 2026-07-07): the own-text 2x2 — {(a),(b)} x {instruct, pretrained} — at FULL corpus; headline use: the #833-style decomposition of the instruction-tuning effect into TEXT-POLICY shift vs MODEL-TRANSPORT shift. The four control cells — {(c),(d)} x {instruct, pretrained} — SUBSAMPLED to the dense core + battery + ~40% of the sparse periphery (~9k rows each); null floors and transfer reads do not need full-corpus power.
    • The Claude-text cells read the own-answer-advantage question at the factor level (does f transfer to foreign coherent text?). Scope boundary: #1072 owns the next-token-identity decomposition of the own-answer advantage, #825 owns pretrained-vs-instruct map EXISTENCE / formatting mains / per-position decay — this task reads only factor-level transfer and the decomposition/rank/trait reads on THIS corpus.
  • Turn-dynamics module (user addition 2026-07-07): four reads on the LOGGED multi-turn conversations, teacher-forced — ONE full-conversation forward per conversation per model arm captures every cut point (causal masking), ~1k conversations x {instruct, pretrained matched-text}, +~1-3 GPU-h. Notation: within a logged conversation, context_k = the context summary at the end of user turn k (the state the assistant answer k conditions on); pre-user-turn state s_k = the summary at the boundary slot right before user turn k. Reads, each fit VM-CPU with group-folds BY CONVERSATION: (D1) user-model: s_k -> summary of USER turn k, with the same three summary options on the user side (u1 mean over user-turn tokens; u2 + boundary tokens before the next assistant turn; u3 the fixed pre-assistant-header slot) — D1's strength directly QUANTIFIES the anticipated-user-content channel flagged in the t3 rationale (high D1 R^2 = the pre-turn state strongly encodes what the user will say). (D0) 1-step anchor: context_k -> LOGGED assistant answer_k summary (t1/t2/t3) — the logged-text regime of the main context->answer map, the decay reference for D3. (D2) state transition: context_k -> context_{k+1} (the NEXT assistant turn's context summary, across the intervening answer_k + user turn k+1). (D3) lookahead: context_k -> answer_{k+1} summaries (t1/t2/t3). D2/D3 measure how far ahead the pre-generation state determines the conversation trajectory (D3 vs D0 = one-round predictability decay; ties to #958's near-turn-stationary map result). (D4) turn-resolved map strength (user addition 2026-07-07): held-out R^2 of the per-turn maps as a function of turn index k — assistant side context_k -> answer_k (t1/t2/t3) AND user side s_k -> user-turn_k (u1/u2/u3) — with the verdict strengthen / weaken / stationary over turns, per model arm (instruct vs pretrained). Two mandatory controls: (i) SELECTION — per-k comparisons run on a FIXED length-stratified conversation panel (e.g., the >=K-turn subset for a fixed K; conversations reaching deep turns are a nonrandom subset, so an unfixed panel confounds depth with selection); (ii) LENGTH — turn index correlates with token position/sequence length, so depth trends are read against length-matched subsets (turn-level profile, distinct from #825's per-position token-decay scope; sibling result: #958's near-turn-stationarity at turns 2-4 and its f-share growth 0.26 -> 0.32 — this extends that line to realistic logged conversations, longer horizons, and the user side). (D5) first-state horizon profile (user addition 2026-07-07): from the FIRST prefix summary s_1 and the FIRST context summary context_1, predict every subsequent assistant answer summary answer_j (t1/t2/t3) and context summary context_j, j = 1..K: R^2 vs horizon j — how much of the conversation trajectory the OPENING state pins down, and its decay shape; optional companion: compare direct long-horizon prediction against iterated one-step D2 chaining (is the transition map compositional?). Corpus consequence, binding on the prefix sampler: the length stratification must deliberately over-sample long conversations so the dynamics panel is populated (target >=300 conversations with >=5 user turns). D4/D5 add ZERO capture cost (the full-conversation forwards already contain every cut point); the fit-grid growth stays under the binding vectorization discipline. Caveat stated up front: the logged assistant turns were produced by whatever model the user originally chatted with — a FOREIGN text policy (like the Claude-text cells); the dynamics maps characterize transport over logged trajectories, not our models' own closed-loop dynamics. Claude/shuffled text-source cells do not apply to this module.
  • Bare-query captures (user ask 2026-07-07: the no-system-prompt query map, per #779/#923): each UNIQUE query (bank ~500 + natural ~1,000 + core) is additionally captured ONCE in a BARE context — no prefix, no system prompt, default chat template — giving the query-own representation c_q_bare (~1.5k short forwards, <0.5 GPU-h). Prefix-alone needs NO extra pass (causal masking: prefix-end activations in the mixed forward equal the prefix-only forward). These enable, on identical rows: (i) the explicit g-map fit from query-own inputs; (ii) the #923-style query-only and DISJOINT-STITCH reads — predict v from fitted f(P) + g(q) using only single-factor forwards, vs the mixed-forward map; the gap is the attention-mixing increment (#923: stitch 0.540, mixing +0.103, disjoint stitch recovers the mixed forward to ±0.012); (iii) a bare-query per-example map DIRECTLY comparable to #779's prefix-less LMSYS map (5k single prompts, no system prompt) — closing the loop with the 0-GPU #779 bridge re-fit.
  • Capture: Qwen/Qwen2.5-7B-Instruct (bf16), base model only, no adapters. Per row: one frozen greedy base response (vLLM batched); teacher-forced capture of prefix-end activation, context-end activation, and the answer state at all 28 layers, with the capture window EXTENDED past the answer through the next-turn boundary (append the end-of-turn + next-user-turn header tokens; pretrained cells use the analogous boundary of #825's naturalistic formatting recipe — planner grounds it). THREE reduced answer-state targets stored per row (user addition 2026-07-07, grounded in #779's answer-summary round — arm_headline_summaries2 keys v_full_mean / v_im_end / v_nl_after_user): (t1) mean over answer tokens — PRIMARY target, continuity with #779/#813/#923; (t2) mean over answer tokens + boundary tokens up to RIGHT BEFORE the next user turn; (t3) the single newline-after-user-header position — the current token is IDENTICAL across all rows and NO next token is realized (capture ends at the boundary), so t3 removes the REALIZED-token-identity component that dominates t1 (the #1072 artifact; the marker line's post-response-slot family). t3 is NOT pure persona state (user-flagged 2026-07-07): its activation still encodes the model's PREDICTION of the upcoming user turn, which varies with context (anticipated topic/register), so anticipated-content state variance remains. Interpretation rule: t1-vs-t3 divergence isolates realized-content carriage; if a headline rests on t3 itself, probe its anticipated-content residue via the optional 0-GPU sensitivity column — residualize t3 against the unembedding directions of its top predicted next tokens at that slot (method borrowed from #1072's decomposition without owning its question). Every read runs per target; t1 carries headlines, t2/t3 are co-reported target-sensitivity arms (fits are 0-GPU VM-CPU; capture cost +~5 boundary tokens/row, negligible); reduced per-row summaries persisted + raw rollout text (upload policy: text unconditional). ~22k rows total (13k sparse + 4.8k core + ~1.5k trait stratum + 2.4k battery eval) per generation pass. GPU estimate across the 4x2 factorial: own-text 2x2 at full corpus ~35-50 GPU-h (instruct primary ~15-20; pretrained matched-text capture +~6-10; pretrained on-policy +~12-18; pretrained-text->instruct capture +~6-10) + four subsampled control cells (Claude x2, shuffled x2, ~9k rows each, capture-only) +~12-20 GPU-h — total est ~55-75 GPU-h, under the autonomous plan-approval cap. Claude generation is API-only (Batch, ~$50-100). All fits/analyses stay 0-GPU VM-CPU. All fits/analyses stay 0-GPU VM-CPU.
  • 0-GPU cross-substrate BRIDGE reads (in-scope, user-confirmed): re-fit the f/g/i decomposition + a layer sweep on the EXISTING persisted captures — #923's crossed-grid tensors (UltraChat 50x144 grid primary — the reference-number source — plus Betley/Dolly secondary grids, L18), #813's battery summaries (L14, HF issue813_mapchange_substrate/reduced/ @ b0d30307c1), #779's pass-B LMSYS captures — to reconcile the currently misaligned substrates/layers on one recipe. VM-CPU, vectorized, detached, checkpointed.

What would count as an answer: (1) quantified query-contribution to transport (R^2 + spectral gap, CIs); (2) verdict on whether question-averaging's rank collapse is design artifact vs property of condition-level objects; (3) f/g/i shares on a realistic corpus vs #923's 84/8/9; (4) PASS/FAIL on M ~= f-map and M' - M ~= g, and the per-factor trait table (does f carry the trait signal?); (5) arm A vs arm B — do real contexts alone suffice to fit trait-readable maps; (6) natural-vs-crossed distortion estimate licensing (or caveating) every crossed-design map result in the project; (7) the instruction-tuning effect on the decomposition, separated into text-policy vs model-transport shifts via the own-text 2x2; (8) the carrier floor (shuffled-pairing cells: does g die and f survive as the additive frame predicts?) and the own-answer factor transfer (Claude-text cells: does f transfer to foreign coherent text?); (9b — dynamics) how far ahead the pre-generation state determines the trajectory (D0 vs D3 decay; D2 transition strength), how much upcoming-user-turn content the state carries (D1 — the quantified t3 anticipated-content channel), whether the context->answer map strengthens or weakens over turns on both the assistant and user sides (D4, selection- and length-controlled), and how much of the whole trajectory the opening state determines (D5 horizon profile from s_1 / context_1); (9c — behavior) which state arm and which FACTOR predict judged trait expression (B1/B2; the dual-DV behavioral anchor; A2 tested via the answer-side ceiling); (9) target-sensitivity — do the decomposition shares, rank reads, and trait-per-factor conclusions survive the answer-summary definition, especially at t3 (t1-vs-t3 divergence = realized-content carriage; a t3-resting headline additionally survives the predicted-next-token residualization or is caveated).

Competing hypotheses: H-A (persona-as-filter): f small-but-trait-bearing, M ~= f-map, averaged map = matched filter for persona information; trait monitoring stays persona-level-easy (#779's 2-6x). H-B (query-dominant transport): trait signal spread across g/i, M' - M carries trait information beyond g, averaging destroys persona-relevant structure. #923 (84/8/9 + additive ceiling 0.914) and #813 (within-context R^2 0.71-0.87 = g transporting trivially) are prior evidence the additive frame holds at narrow diversity; #779 (persona-level r 0.66/0.89/0.53; r_B predicted at R^2 0.79-0.87 on 5k contexts) says f's trait projection is recoverable at scale.

Behavior-prediction module (user addition 2026-07-07 — the behavioral anchor; without it the trait-per-factor read is a geometry claim, not a persona-information claim): a judge phase + two reads, dual-DV compliant. Judge phase: graded 0-100 trait scores for the #779 trait panel (evil, sycophancy, hallucination — the same r_B directions read 4 uses), judge claude-sonnet-4-5-20250929, N=5 draws at temperature > 0 mean-aggregated, malformed/REFUSAL returns DROPPED never coerced, rubric-keyed judge cache, Anthropic Batch API via the multi-org dispatcher, run OFF-POD (holds no GPU). Scored rows: the trait-relevant subset — battery eval rows + trait-stratum rows + dense-core rows — for the instruct own-policy answers, plus the pretrained on-policy cell on the battery + trait-stratum slice (the instruction-tuning BEHAVIOR comparison). Natural sparse-periphery rows are excluded by default (expected trait floor; a ~500-row floor-check sample is judged to verify that assumption). Ballpark ~$150-350 API, 0 GPU; plan §-API states the exact call arithmetic per the throughput guidelines. (B0) capture-time pooling stats (required for the #779 R2 comparison — capture-blocking): for the own-policy cells, per (row x layer x trait), compute AT CAPTURE TIME the per-token r_B-projection pooled statistics over the generated answer span — mean, max, top-k mean, last-token — a few floats per row (per-token full activations are NOT persisted, so these are unrecoverable post-hoc). (B1) state -> behavior — the full #779 R1 comparison set, per input arm and grain: predict the graded trait score from each INPUT ARM (prefix-end, context-end, bare-query) at both grains (condition-averaged, per-example) — the #779 monitoring read on realistic data, incl. whether the persona-level vs per-prompt monitoring gap (~2-6x) reproduces. The COMPARISON SET is #779's R1/R2 panel plus the factor reads: (a) raw projection r_B^T state; (b) the MAP-MEDIATED read (r_B projected through each fitted map's output — h_P, h_C, bare-query map); (c) DIRECT supervised regression state -> graded score; (d) the POST-GENERATION pooling reads from B0 (mean / max / top-k / last-token of per-token r_B projections over the generated answer — #779's R2, which beat the prompt read there); (e) the answer-side ceiling r_B^T v(x), directly testing the theory's A2 (Expr ~= r_B^T v). #779's negative R1 (map-mediated fails to beat raw projection on LMSYS) is the reference result this either replicates or overturns on the crossed realistic corpus. Every map fit in the task also carries the #779 identity-baseline ladder (copy / scale / diagonal-affine) as floors, reusing the #779 machinery. (B2) factor -> behavior: predict judged expression from each FACTOR's read (f(P), g(q), i(P,q) projections + factor-map outputs) — the behavioral validation of read 4's trait-per-factor prediction: 'f carries the persona signal' counts only if f's read predicts JUDGED behavior, not merely r_B alignment. Dual-DV: the graded judge score is the PRIMARY behavioral construct (graded-primary for ranking/regression targets per the llm-judging rule; binary rate co-reported where legible); activation reads are predictors, never narrated as the construct. Optional B3 (planner decides by judge budget): behavioral lookahead — predict judged trait of the LOGGED answer_{k+1} from context_k on the dynamics panel (foreign-text caveat applies).

Input-arm coverage (user audit 2026-07-07 — binding; the standing prefix+context both-arms rule made explicit per read): EVERY predictive read runs BOTH input arms — prefix-based (prefix-end / s_k, already captured; equals the prefix-only forward by causal masking) AND context-based (context-end / context_k) — with bare-query (c_q_bare) as the third input where defined (g-map, stitch). Per read: read 1 IS the arm comparison; reads 2-4 (grain/rank, f/g/i, operator + trait-per-factor) run per input arm — noting the degeneracy that the query-averaged PREFIX-input map coincides with the f-map (prefix-end does not vary with the query), so the M ~= f-map test spans the arm comparison at averaged grain; dynamics D0/D2/D3 gain the s_k input arm alongside context_k — s_k -> answer_k reads how much of the answer is determined BEFORE the user speaks; D4 profiles BOTH arms per turn and reports the per-turn prefix-skill/context-skill ratio (the direct extension of #958's 0.26 -> 0.32 f-share-with-depth read); D5 already carries both (s_1, context_1). Stated exemption: D1's context arm is degenerate (context_k contains user turn k) — prefix-arm only. Zero capture cost (all arm inputs already captured); the fit grid roughly doubles and stays under the binding vectorization discipline.

Compute discipline (user directive 2026-07-07 — binding on plan §9 + implementation): every GPU phase runs data-parallel across ALL provisioned GPUs and every inner loop is vectorized. Specifics: (a) ALL generation via vLLM batched LLM.generate() (instruct greedy + pretrained on-policy) — never sequential HF generate(); (b) teacher-forced capture forwards BATCHED (large per-GPU batches) and SHARDED across every GPU on the pod — one multi-GPU pod with per-GPU workers over (cell x row-chunk) shards via CUDA_VISIBLE_DEVICES, work-conserving dispatcher (no wave barriers — a free GPU immediately takes the next pending shard; 8 capture cells + bare-query are embarrassingly parallel); (c) GPU-width right-sizing: the Claude-text generation is API-only (Batch API through the multi-org dispatcher) — it must NOT hold the pod; sequence it off-pod/VM-side before or after the GPU phases, and release/downsize the pod for any >15-30 min non-GPU stretch; (d) ALL fits/spectra/nulls are 0-GPU VM-CPU but VECTORIZED: the batched Gram-dual ridge engine + factored/Gram-space spectra (never a dense 3584^2 SVD in a loop), fold/layer/cell/target axes batched into single GEMMs per the #722/#778/#823 lessons (.claude/rules/vectorize-many-cell-fits.md; canonical helper vectorized_mlp_skill.py), launched detached + checkpointed with thread-capped BLAS — the fit grid here is large (8 cells x 3 targets x 2 fit-arms x 2 grains x 28 layers x group-folds), so a serial per-cell loop is an automatic REVISE; (e) plan §9 states the ops arithmetic per phase (rows x tokens x cells -> projected wall) and the measured 1-cell pilot basis for the fit-grid wall-time.

Planner notes: group-level folds — hold out PREFIXES, never rows (ood-generalization-folds); shared-lambda primary across arms/grains (the #813 rank-spectrum precedent), per-grain GCV as sensitivity; reuse scripts/issue813_rank_spectrum.py factored-spectrum machinery + the #722-line Gram-dual ridge engine + #923's factor-map fitting code where it exists; the matched-queries confound is load-bearing (with a shared query set M = f + const exactly; with per-condition pools M = f + E_q[g] — the #658 -> #761 incident, matching lifted rho 0.62->0.90 refusal); r_B directions per the persona-vectors recipe rule at the read-out-regime layer selection; per-position/length decay reads OWNED by #825 — out of scope; new-direction lit review at plan time (crossed/factorial probing, relation decoding, activation transport) per Critical Rules.

Provenance

  • Origin: user-chat design discussion 2026-07-06, following the #813 inline free-analysis round rank-spectrum-averaged-vs-per-example and the #779-vs-#813 corpus-diversity rank contrast (map rank tracks design dimension: k90 >500 on 5k free-form prompts vs ~80-98 on the 50x<=64 factorial).
  • Combined-model ask (user, verbatim core): "Nobody has yet run the three-way version on one grid: fit M (averaged), M' (single), and the explicit {f, g, i} factor maps on the same matched prefix x query crossing, then test: 1. M ~= f-map (principal angles / Procrustes between the fitted operators — the method already sketched in docs/ideas/2026-07-06-context-answer-map-analyses.md §Q6); 2. M' - M ~= g (the grain gap is query main effect); 3. trait read-out per factor: project each factor onto the r_B [directions — prediction from #779] is that f carries nearly all trait signal despite being 8% of raw variance, g carries almost none despite being 84%, and i carries the conditional-trait residue. That third read is the direct test of 'what the average map captures IS persona information.' Current misalignments a combined run would have to reconcile: #923 on Betley/Dolly, #813 at L14 on the 50-context battery, #779 on LMSYS — one substrate + one layer sweep would unify them."
  • Clarifying answers (user, 2026-07-06, round 1): scale ~1k prefixes first run; crossing random + topic-matched control; 0-GPU bridge reads in-scope. Round 2: Goal frame = keep the 3 co-primaries and ADD the combined model as the 4th; i-term design = dense core + sparse periphery; corpus arms = "one version where we check how well it works when we only train on real contexts, others where we add trait eliciting contexts" (verbatim) -> the two fit-time training arms above, battery held out as shared eval bridge.
  • Pretrained-arms addition (user, 2026-07-06, verbatim): "add a matched-text AND on-policy pretraine arm" — following the cost fork offered in chat (matched-text +~6-10 GPU-h / on-policy +~12-18 GPU-h). Both added as model arms (ii) and (iii); every co-primary read runs per arm.
  • Factorial addition (user, 2026-07-07, verbatim): "can we add an arm with random text (or shuffled) and claude generated text or both pretrained and instruct, and also instruct with the pretrained on policy" — reframed in chat as the 4x2 text-source x model factorial; user confirmed subsampled controls + shuffled-pairing-only (no random-token cell).
  • Answer-summary targets addition (user, 2026-07-07, verbatim): "wait can we also consider 3 answer summaries: mean over answer tokens; mean over answer tokens + boundary tokens before next user turn; newline after user header (newline RIGHT before the user would start to speak) — this is based on past issues" — mapped to t1/t2/t3 above; t1 primary (stated assumption, planner may refine with consent), t2/t3 co-reported sensitivity.
  • t3 rationale correction (user, 2026-07-07, verbatim): "isn't the next token identity still confounded because the next token could be anything?" — correct: t3 removes REALIZED-token identity only; the model's prediction of the upcoming user turn still varies with context. Rationale text + interpretation rule updated accordingly; optional predicted-next-token residualization added as a sensitivity column.
  • Bare-query addition (user, 2026-07-07, verbatim): "wait is 1092 running the query with no system prompt map similar to the other issue?" — it was not explicit; bare-query captures + query-only / disjoint-stitch reads + the #779-comparable bare-query map added as above.
  • Turn-dynamics addition (user, 2026-07-07, verbatim): "Also can we add a user context summary -> user answer prediction (user answer with same 3 options as assistant answer), as well as context summary -> next assistant contex summary predition as well as context summary -> next assistant answer summary prediction (all 3 summary options)" — mapped to D1 (user-model, u1/u2/u3 targets), D2 (context_k -> context_{k+1}), D3 (context_k -> answer_{k+1}, t1/t2/t3), + D0 logged-answer_k anchor added as the 1-step decay reference.
  • Turn-profile addition (user, 2026-07-07, verbatim): "can you also use this to check if the context -> answer mapping gets weaker or stronger over time (on a per turn basis)? for both user and assistant. And also if the first context summary (and first prefix summary) can predict subsequent assistant answer summaries (or assistant contex summaries too)" — mapped to D4 (turn-resolved strength, both sides, fixed-panel + length-matched controls) and D5 (first-state horizon profile incl. s_1 and context_1 inputs); prefix sampler now over-samples long conversations (>=300 with >=5 user turns).
  • Input-arm audit (user, 2026-07-07, verbatim): "is everything predicted on a prefix and context level? I just want to make sure we're capturing everything" — audit found dynamics D0/D2/D3/D4 context-only and reads 2-4 implicit; both-arms coverage now explicit per read (s_k arms added; D1 exemption stated; D4 gains the #958-ratio read).
  • Behavior-prediction addition (user, 2026-07-07, verbatim): "is there any behavior predcition piece for this?" — there was none (all reads activation-space); B-module added: graded Sonnet judge phase on the trait-relevant subset + B1 state->behavior (three input arms, both grains, #779 monitoring read) + B2 factor->behavior (behavioral validation of trait-per-factor) + optional B3 dynamics lookahead.
  • #779-comparison audit (user, 2026-07-07, verbatim): "are we running the same comparisons as 779 too" — audit added the missing pieces: B0 capture-time per-token r_B-projection pooled stats (mean/max/top-k/last; unrecoverable post-hoc), the explicit R1 comparison set (raw vs map-mediated vs direct regression vs pooling vs answer ceiling), and the identity-baseline ladder under every map fit.
Activity