Improving training-data attribution on CATT
AI · Interpretability and data attribution
Improving training-data attribution on CATT
Why attribution matters
When a language model gives an answer, we often want to know which training documents taught it. Tracing an answer to its sources helps us check where a piece of knowledge came from and diagnose why the model made a mistake. Training-data attribution ranks training examples by their estimated contribution to a particular model response. The goal of this task is to develop better attribution methods: methods that place the documents teaching the relevant rule at the top of the ranking.
The CATT benchmark
CATT provides language-model adapters trained on synthetic corpora in which the documents behind each fact are known. In its Rule Routings (RR) task, each document teaches a category-specific rule. One rule says that luggage is routed according to whether it has a red strap, regardless of its weight. Queries then describe held-out situations: a synonym for the category noun, combined with an attribute combination not seen in training. Many categories share the answers Cedar and Birch, so the answer word alone does not reveal which rule produced it. A good method ranks the ten documents that teach the luggage rule above documents that teach other rules with the same answers. The companion benchmark, Function Bindings with distractors (FB-distr), teaches artificial constant mappings and adds documents about different functions that share answer tokens and resemble the queries in wording.
Research objective and starting evidence
Several methods use the model's gradients or internal representations to rank training documents. On this RR setting, their rankings recover relatively few of the relevant rule documents. The objective is to substantially improve model-based attribution on RR, with FB-distr as the companion benchmark. Understanding why current methods miss the relevant rules points toward better ones. Shared output tokens and shared document templates are candidate explanations: a document's gradient may reflect the answer word Cedar or common phrasing more than the rule itself.
We measured the baselines below on the Qwen3.5-4B adapters released with CATT, using 50 facts with 10 relevant documents each, 200 queries (four per fact) and training seed 1001. Recall@10 is the fraction of a query's ten relevant documents that appear among its ten top-ranked documents, averaged within each fact and then across facts. It measures attribution quality, not whether the model answers correctly.
| Method | RR Recall@10 (%) | FB-distr Recall@10 (%) | |---|---:|---:| | BM25 | 99.80 | 98.30 | | RepSim | 1.05 | 0.00 | | LoGra, NLO top 10 | 13.90 | 31.30 | | LoGra-Dot, NLO top 10 | 16.75 | 83.70 | | TrackStar, NLO top 3 | 22.35 | – | | TrackStar, NLO top 10 | 22.25 | 0.00 |
These are our measurements in this setting. BM25, a word-overlap search method that does not use the model, nearly solves both tasks. RR is therefore hard for the neural attribution methods tested here, not for document retrieval in general.
Possible directions
Gradient-based attribution estimates how training on a document would change the model's output for a query. Each document and query is represented by the gradient of a loss with respect to model parameters, and documents whose gradients align with the query's gradient receive high scores. Full gradients are very large, so TRAK uses random projections with a statistical correction, and LoGra compresses gradients efficiently; its Dot variant compares the projections directly. TrackStar additionally corrects for differences in scale across gradient dimensions and normalizes each representation. CATT shows that two further choices matter. Negative log-odds (NLO) can preserve gradient signal when ordinary cross-entropy gradients shrink because the model is confident. Aggregating over the highest-contributing tokens, such as the top 3 or top 10, can reduce dilution by uninformative text. Each choice is a possible lever for separating rule evidence from answer-token evidence.
Other approaches are welcome, including learned attribution models, scorers and token aggregators, and retraining the target model when that helps test or improve a method. Lexical and gradient scores may be combined; compare a hybrid with lexical-only controls to show what the model contributes beyond BM25. Matched ablations, such as reweighting answer tokens while holding everything else fixed, explain where an improvement comes from and whether it transfers across tasks.
Evaluation and progress
The official CATT checkpoints and corpora are the shared starting point for comparing methods. Seed 1001 is exposed for development. Freeze code, configuration and hyperparameters, then evaluate on other training runs, such as the adapters trained with seeds 1002 to 1004. These seeds use the same facts, so they measure robustness across training runs. Learned methods use separate fact- or rule-level training, validation and test splits, declared before fitting and used consistently through final testing. Work that retrains target models reports its costs and compares against baselines run on the same models. The companion protocol specifies tie handling, bootstrap settings and secondary metrics.
Progress means outperforming the strongest neural baselines on RR, with paired fact-level confidence intervals, while improving or holding up on FB-distr. Keep BM25 in the main table and report the gap to it. Report each checkpoint's answer accuracy, verified independently, alongside attribution results.
Resources and deliverable
The supplement contains the baseline source, data, dependency lock, commands and reproduced results. One projected pass computes several methods together and took about 51 minutes on RR and 80 minutes on FB-distr on one NVIDIA vGPU-32GB, with peak allocated memory of 16.85 and 19.69 GiB. Times include model loading and exclude downloads. Submissions should include raw document IDs and scores, configurations, model and data revisions, and ablation results. The companion protocol covers submission details.
The deliverable is an improved attribution method for CATT, evaluated reproducibly on RR and FB-distr, with ablations that explain why it works.
Task materials
Download supplement (.zip)- Type
- open-ended
- Solve status
- open
- Access
- Invite-only