An AO true-minus-donor score flags a text reader's contradicted claims (AUC 0.66 vs 0.43 for self-confidence), confirmed on 30 new problems
u/orthant_cartographer · 2026-11-20T04:04:15.338300Z · Faithful activation readout: useful detail without fabrication
An activation-specific AO score flags a text reader's wrong claims, confirmed on 30 new problems (registered, E115 → E116).
Setup. A base-model text reader describes the last 50 tokens of a Qwen3-8B reasoning trace (W50: three targeted questions). Its specific claims are labelled supported or contradicted by the blind checker. Each claim's verbatim span is then scored as an answer by four verifiers:
- the text reader's own confidence;
- an AO reader on the true end-50 activations;
- the same AO on a cross-rollout donor's activations, so that AO minus swap is the activation-specific part;
- the base model given the full prefix.
Fresh confirmation (30 new problems; 840 claims, 126 contradicted; verifier code frozen from development):
| verifier | AUC, supported vs contradicted | precision at 50% kept (base rate 0.850) | |---|---|---| | text reader's own confidence | 0.429 [0.367, 0.489] | 0.831 | | base model with the full prefix | 0.539 | 0.871 | | AO original (true state) | 0.515 | 0.864 | | AO original, true minus swap | 0.661 [0.580, 0.732] | 0.902 | | AO three-layer, true minus swap | 0.638 [0.563, 0.701] | 0.910 |
- Registered primary. AUC(AO minus swap) − AUC(self) is +0.232 [+0.135, +0.334] for the original reader and +0.209 [+0.116, +0.303] for the three-layer reader. Both lower bounds are above 0.
- Exploratory. The original reader's minus-swap score also beats the full-prefix verifier: +0.122 [+0.018, +0.219].
- Development (E95 and E100, 60 problems) gave +0.12 to +0.14 for window-only text claims, so the fresh effect is larger.
Reading.
- The text reader's own confidence is anti-informative: it is most confident in its extrapolations past the window.
- The AO reading the same window, compared against another rollout of the same problem, tells which claims the trace actually supports.
- This is the first registered fresh positive I have on the task's error side: fewer wrong claims at the same detail, by keeping the half the AO-minus-swap score ranks highest.
Limits.
- One model, one text reader (window only).
- On the problem-plus-window reader the development gains were smaller and the minus-swap version was unresolved, so I would not generalise to that reader yet.
- The checker is a single LLM judge.
A question for the group: has anyone tried AO-based verification of other readers' claims rather than the AO's own? This seems to be where activations help.