Think through LoRA placement (which modules/layers the adapter lives in) and whether the leakage conclusions are placement-conditional
kind: analysis#human
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Summary
Human thinking task (not for agent execution): think through what "where the LoRA lives" means for the project's implant/leakage conclusions, and decide whether a placement ablation experiment is worth filing.
Things to think through:
- Placement choices — which module set the adapter attaches to (attention q/k/v/o vs MLP projections vs both), which layer range, and how rank is distributed. All current results use one default placement (attention+MLP projections,
lm_head/embeddings excluded). - Which conclusions are placement-conditional. The LoRA-vs-full-FT "similar leakage profiles" claim (r = 0.994 on the marker) was measured at the default placement only; #606 already shows the equivalence breaks for sycophancy. Does placement explain or modulate that gap?
- Interaction with the LoRA-anatomy results — #604 found a seed-stable write direction with a seed-arbitrary top-1 key; would a different placement (e.g. MLP-only, or a narrow layer band) concentrate or relocate the write, and would that change bystander leakage?
- Decision output: either file a concrete placement-ablation experiment (re-run the LoRA-vs-FT / leakage comparison across 2-3 placements) or record why the default placement is not load-bearing.
Provenance
Consolidated from the 2026-06-11 mentor meeting notes (docs/mentor_updates/2026-06-11.md, "careful about where the LoRA lives") during the task #616 cross-check; confirmed as a LoRA-placement concern.
Activity