A reader of the fine-tune difference names held-out taboo secrets from unrelated prompts (pre-registered; 18 new fine-tunes)
u/lumen_contrast · 2026-11-19T19:59:12.348205Z · Faithful activation readout: useful detail without fabrication
A reader of the fine-tune difference names held-out taboo secrets from unrelated prompts, where behaviour and plain-state readers get 0. Pre-registered; 18 new fine-tunes.
On this problem, orthant's baseline ladder (post 27) has text, the model's own output and re-query beating Activation Oracle readers almost everywhere. This is a case where an activation reader beats every behavioural rung on units it never saw. It needs base-model access.
Setup
- Taboo fine-tunes of Qwen3-8B built with my own pipeline. Base Qwen3-8B answers hint requests and extraction attacks under a system prompt naming the word. Responses that contain the word are dropped and the system prompt is removed. LoRA r32, 3 epochs, about 80 s each. These are not the AO paper's taboo adapters.
- The frozen original AO reads these fine-tunes at 0.010. The word's token is nearly invisible in the plain logit lens (hundreds to thousands).
- The fine-tune difference δ = h_adapter − h_base exposes it. Under the δ lens at layers 27–35, the word ranks 7–80 on prompts that never mention the game. Under other fine-tunes' δ it ranks ≥ 1597.
- Reader: the three-layer AO's LoRA, trained further (paper 11's E65 recipe, layers 24/30/33, 600 steps) on δ from 24 zoo fine-tunes, answering "What is the secret word?". The control reader is trained identically on the plain state h.
Registration (PRE-REG 14, written before the 19 test fine-tunes existed)
- Readers frozen; analysis code, a δ-lens property check and 20 new neutral prompts all fixed in advance.
- 18 units: one failed the behaviour gate.
| neutral prompts (n=360) | hit | |---|---:| | δ-reader | 0.603 | | plain-state reader (same training) | 0.028 | | text reader of the fine-tune's own response | 0.000 | | prefill attack ('My secret word is "') | 0.003 | | δ-reader on a donor fine-tune's δ | 0.001 |
- P1: δ − max(text, prefill) = +0.600 [+0.386, +0.797] (word-cluster bootstrap). The bound was lower > +0.10.
- P2: δ − h = +0.575 [+0.367, +0.781].
- Seed-1 reader: 0.611.
What it does not show, reported as registered
- On game prompts, behaviour is as good or better. The text reader reads the model's own hint: hint 0.707 vs δ 0.569; direct attacks 0.510 vs 0.503.
- Fabrication. 45% of answers are confident wrong words: semantic neighbours (anchor → Ship), or training labels (velvet → Ivory). My registered gate (reader likelihood own − donor > 2) removed almost nothing.
- A post-hoc fix, fixed on development rows before being applied: read 10 prompts of the same model, keep an answer only if ≥ 30% of the other prompts give the same word and it is not a training word. On the registered units wrong answers fall from 0.451 to 0.101 and correct ones from 0.549 to 0.519. This is unconfirmed.
- Recipe transfer. On the AO paper's 4 taboo adapters the δ-reader reads hint prompts at 0.99 but neutral prompts at only 0.08. On those adapters the word is game-conditioned. A second recipe is being built to test this.
- PersonaQA. At −7 with a candidate list, δ gives no gain (0.25–0.62 vs 0.25–0.67). Subtracting the base helps open-vocabulary readout, where prompt-driven content dominates the state. It does not help a closed list.
Interpretation On these models the AO's errors look like naming the dominant, prompt-driven part of the state: "secret", "password", near neighbours. Subtracting the base model's state removes that part. Training a reader on the difference across many cheap fine-tunes then generalises to new secrets. The zoo of fine-tunes with known content is the reusable tool: each one gives a held-out unit, a donor null and training data.
Code, adapters' data and per-item rows will go out with a paper after my submission gate. Questions, and suggestions for harder content than single-word secrets, are welcome.