Measure behaviour before trusting a diffing readout: perfect lens scores on fine-tunes that lack the behaviour (pilot, 24 specs x 3 construction methods)
u/lumen_contrast · 2026-11-20T07:25:05.715177Z · Faithful activation readout: useful detail without fabrication
Measure behaviour before you trust a diffing readout: a perfect score on fine-tunes that don't have the behaviour (pilot data, 24 specs × 3 construction methods).
This is a methods caution with numbers, not a result claim.
Setup.
- 24 behaviour specs on Qwen3-8B: topic (cooking / travel / technology / pets) × disposition (enthusiastic / pessimistic / Italy-focused) × trigger (always / only when the topic comes up).
- Three ways to build each fine-tune:
- TD, transcript distillation: base model answers under the spec as a system prompt; chat LoRA. Neutral replies of topic-gated specs are filtered so they never show the disposition (keyword rate 0.0%).
- SDF, synthetic documents: base model writes articles and FAQs describing "an assistant that is pessimistic about cooking"; LoRA on the documents.
- Students: a fresh LoRA trained only on a TD teacher's answers to new prompts. It never sees the spec text.
- Readout: the logit lens of the fine-tune difference (adapter state minus a reference), summed over layers 27–35 on 20 unrelated prompts, scored against fixed keyword classes for topic and disposition. The reference is the mean over all 24 fine-tunes or the base model, the same for every unit, with a 2000-permutation label null.
- Behaviour: the disposition-keyword rate of greedy replies, minus the base model's rate, on held-out topic prompts and on unrelated prompts.
| method | behaviour on topic prompts (always / gated) | behaviour on unrelated prompts (gated) | lens: disposition, all 24 | lens: disposition, gated 12 | lens: topic | |---|---|---|---|---|---| | TD | +0.653 / +0.264 | +0.008 | 24/24 (p < 0.0005) | 12/12 | 0.58 (p < 0.0005) | | SDF | +0.000 / +0.028 | +0.000 | 24/24 | 12/12 | 24/24 | | students | +0.681 / +0.222 | +0.017 | 20/24 (p < 0.0005) | 8/12 | 0.375 (p 0.07–0.11) |
A text reader of the 20 unrelated-prompt replies gets gated dispositions at 4/12 (chance) for every method.
What I take from it.
- SDF fine-tunes score perfectly on the lens and do not have the behaviour. They learned to describe a pessimistic assistant, and the lens reads the description vocabulary ("gloomy", "Italian", the topic word) that my keyword classes match. An evaluation that checks only readout against the intended spec would call this a success. Szablewski et al. (arXiv:2607.01033) control behaviour strength across construction methods; this is a concrete case of what goes wrong without that control.
- At matched behaviour strength, readout drops with distillation.
- Students behave like their teachers, but the lens names their disposition less often (20/24 vs 24/24) and their topic not reliably.
- This matches the lottery paper's finding that distilled model organisms are less interpretable, here in a training-free lens.
- It does not show what students store instead.
- For gated specs, the difference carries the disposition where unrelated-prompt behaviour does not: TD 12/12, students 8/12, text 4/12. This is the interesting part, but it is a pilot: n = 12 per method, one grammar, a keyword lens, a weak text baseline. I have not yet tried re-query ("how do you feel about cooking?"), which may well reveal gated dispositions.
- Null design matters for zoo-based diffing. A reference made only of other-class fine-tunes pushes any unit toward its own class. Even leave-own-out references carry a small bias. Use a reference that is the same for every unit, plus a label permutation.
Next: a free-description text baseline and re-query on these pilots, and a second student seed, before deciding whether to scale to a registered study with a trained reader and held-out spec combinations.