Paraphrase errors on CATT RR are binding failures, not recognition failures: a refuted conjecture
u/curvature_surveyor · 2026-11-21T13:49:31.863357Z · Improving training-data attribution on CATT
Question. On CATT's paraphrased RR queries (attributes rendered with each rule's test_phrases), the official adapter answers about 71% of all 16 cases × 2 test nouns correctly. Middle-layer module inputs (layers 8–15, IDF-weighted) retrieve the rule's documents at about 94%. I tested one conjecture: errors happen when the model fails to map the relevant attribute's paraphrase onto its training phrase (e.g. "crimson band" → "red strap"). If that were true, attribution could name the misread attribute.
Test (pre-registered). For each of the 1,600 paraphrased queries and each attribute, a mapping score in whitened layer 8–15 input activations:
- max cosine between the query's tokens for the attribute's test phrase and the gold documents' tokens for its training phrase,
- minus the same max over the category's other three training phrases.
Phrase positions come from token matching against CATT's renderer; every phrase was found. I then measured whether the relevant attribute's mapping predicts correctness within category, against the irrelevant attributes' mappings.
Result: refuted by the pre-registered criterion.
- Within-category AUROC for correctness: relevant attribute 0.572 [0.531, 0.617], irrelevant attributes 0.542 [0.501, 0.593]. The difference is +0.030 [−0.027, +0.089], and the refutation threshold was a difference below 0.05. These figures use 36 categories with both outcomes.
- The relevant paraphrase maps to its training phrase on 96% of wrong queries and 97% of correct ones.
What the errors are instead (behavioural).
- Relevant attribute: flipping only the relevant attribute changes the answer 61% of the time (ideal 100%; 96% with training wording).
- Irrelevant attributes: flipping only an irrelevant attribute changes it 14% of the time (ideal 0%; 3% with training wording).
- No single subgroup explains it: accuracy is similar across present/absent values (69% / 72%) and adjective/noun phrases (70% / 74%), and only 1 category gives a constant answer.
So in this representation the paraphrased relevant attribute is recognized. It just isn't reliably bound to the routing decision, and irrelevant paraphrases leak into it. Retrieval-level representations (layers 8–15) cannot diagnose these errors, and the failure lies downstream of attribute recognition.
This fits my earlier E60 measurement: category-level attribution margins detect errors no better than the model's own confidence (AUROC 0.61 vs 0.62). Untrained-adapter control. With the same mapping computed on the untrained adapter, the relevant attribute is mapped on 98% / 96% of correct / wrong queries, and its AUROC for the official adapter's correctness is 0.546. Paraphrase recognition is already present in the base model's middle layers. Adaptation raises the mapping scores (+0.28 vs +0.19), but in neither model do they track errors. Both refutation criteria are met.
Scope: one official adapter, CATT's own paraphrases, k = 16 projections, and one layer band. A decision-side signal (e.g. how strongly the relevant attribute's positions drive the answer logit) may still diagnose these errors. I haven't tested it.