artifacts: direction driver + directional ablation (Phase 0f)
kind: infra
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Phase 0f of the unified artifact factory (plan:
/home/thomasjiralerspong/.claude/plans/help-me-to-devise-vectorized-tower.md). Build the behavior
direction-vector driver + the missing directional-ablation path (the Arditi second half).
Goal
src/explore_persona_space/artifacts/directions.py— the direction driver. Given aBehavior: generate contrastive exhibit/not-exhibit completions from the behavior'sextraction_prompt_pairs(the 5 pairs, REQUIRED DISJOINT from the training instructions) — Claude-generated, judge-filtered (>50 / <50, drop-never-coerce viaeval/graded_judge) — then teacher-force the kept completions through Qwen usinganalysis/extraction.extract_layer_activations(OOM-safe hook; NEVERoutput_hidden_states=True), take response-avg activations per layer, and compute diff-of-meansr_Bper layer (all 28 layers). Reusescripts/issue779_extract_rbdiff-of-means math. Enforce the regime:steering→ pick ONE layer by steering effect;read_out→ sweep all layers + the selection-symmetric-null harness (REFUSE a single-layer headline inread_outwithout per-draw same-selection OR a held-out-frozen axis; persist the per-draw × per-layer matrix). For 0f, accept a contrastive completion set as input (the datagen 0d path produces it later) so the module + tests land now; the full end-to-end run is exercised in Phase 2.src/explore_persona_space/artifacts/ablation.py— NEW directional ablationh ← h − (ĥ·r̂)r̂viaregister_forward_hookper block (mirrorscripts/issue623_extract_sycophancy_vector.py::steering_generate's hook structure but PROJECT-OUT instead of add); single-layer AND all-layer variants.
Scope / constraints
- Reuse
extract_layer_activations, the issue779 diff-of-means math,eval/graded_judge. The teacher-forced-Claude-completions direction is a NAMED persona-vectors deviation (document in the plan §-assumptions perreplication-fidelity.md/persona-vectors-recipe.md). - Append package exports to
artifacts/__init__.pyappend-only (sibling 0c/0e land concurrently). - CPU/mock unit tests: ablation is a NO-OP on a zero direction (≡ base generation); diff-of-means on
synthetic activations; the regime guard REFUSES a single-layer
read_outheadline without a symmetric null; per-draw × per-layer matrix persisted. A tiny GPU smoke for the teacher-force path is OK but keep it minimal. Lint + pytest.
Rules to read
.claude/rules/persona-vectors-recipe.md (7 steps, steering-vs-readout regime, the deviation),
.claude/rules/selection-symmetric-nulls.md, .claude/rules/marker-leakage-measurement.md,
.claude/rules/llm-judging.md.
Activity