## 1. Introduction

Gradient-based training-data attribution (TDA) on CATT [1] degrades sharply when moving from direct constant-binding memorization (`Function Bindings with distractors`, `FB-distr`) to multi-hop rule application (`Rule Routings`, `RR`). On `RR` (50 categories, 500 training documents, 200 queries, `Qwen3.5-4B` seed 1001), CATT's best cached neural baseline (`TrackStar NLO top-3`) achieves only `22.35%` Recall@10 (`19.50%` on 15 held-out test facts), and `LoGra-Dot NLO top-10` reaches `16.75%` (`18.00%` Test). In our companion Paper 1 [11], we introduced per-position query standardization (`Z-Agg`) after cross-module summation together with supervised module weights over the $M = 496$ LoRA factors (`Hybrid`), raising `LoGra-Dot NLO` to `38.20%` All (`42.83%` Test, `AUC = 0.8946`). Despite more than doubling baseline recall, Paper 1's global `Hybrid` still trails lexical `BM25` [4] (`99.80%`) by over `55` percentage points and trails asymmetric hidden-state pooling (`73.2%` [6]) by `~30` points.

In this paper, we diagnose and resolve the primary algebraic bottleneck limiting Paper 1's global `Hybrid`: **cross-module summation prior to nonlinear query calibration and sparse token pooling**. Inside CATT's factored ($d_{\mathrm{proj}} = 16$) NLO gradient tensor, Paper 1 evaluates:

$$
\text{Paper 1:}\quad \operatorname{TopK}_{k} \circ \mathcal{Z}_{q}\!\left(\sum_{m=1}^{M} w_m\, g_{q, m}^{\top} g_{j, s, m}\right).
$$

Because $M = 496$ LoRA factors span distinct transformer depths and projection types with heterogeneous noise scales, summing all modules *before* coordinate standardization $\mathcal{Z}_q$ and $\operatorname{TopK}_k$ allows uninformative middle-layer fluctuations (`L12–L22`) and opposite-sign module variations to wash out category-specific routing peaks before top-token selection occurs.

Reversing this order of operations—standardizing across reference queries and pooling the top-$k$ token positions **per module** (or **per layer**) *before* aggregating across modules—operates strictly inside the exact same $d_{\mathrm{proj}} = 16$ all-token NLO gradient space, with zero decision-token masking, zero query-position `MaxSim`, zero rank expansion ($d_{\mathrm{proj}} = 64$), zero forward hidden-state pooling, and zero lexical features. On `RR`, **Hierarchical Per-Module `Z-Agg` Weighted (`NT, top-3`)** leaps from `42.83%` (Paper 1 `Hybrid`) to **`87.50%` Held-Out Test Recall@10** (`95% CI [82.67%, 92.33%]`, **`AUC = 0.9955`**, `MRR = 99.17%`) and **`80.45%` All-50 Recall@10** (`AUC = 0.9877`), rescuing every category held out from scorer fitting—including Paper 1's worst regression (`lamp`, `7.5%` $\rightarrow$ **`77.5%`**)—to $\ge 70.0\%$ Recall@10 (`100.0%` on `aircraft` and `boat`). Paired fact-cluster bootstraps confirm gains of `+44.67%` over Paper 1 `Hybrid` (bootstrap fraction $P(>0) = 1.0000$) and `+70.67%` over recomputed `TrackStar NLO top-3` ($P(>0) = 1.0000$). Crucially, hierarchical per-module calibration only succeeds because it pairs per-module query standardization with per-module `TopK`: removing query Z-scoring (`Per-Module Raw Top-10` / `Top-3`) collapses retrieval to `4.17%` / `3.00%` Test (`5.50%` weighted with $\mathbf{w}_3$), whereas untrained `B = 0` LoRA controls remain low on `RR` (`12.67%` Test `Mean`, `15.17%` Test weighted at `top-3` and `14.00%` at `top-10`).

We further establish five mechanistic bounds on what hierarchical `Z-Agg` measures: (1) single-layer probes reveal bimodal depth specialization (`L2–L11` and `L23–L30`) with contrasting `case_cond` (`45.5%`) vs. `rule_clause` (`33.9%`) token-role emphasis and `0.0267` top-10 Jaccard overlap while distributed weights (`effective N = 388.8` of `496` modules) resist extreme pruning, and selective forward LoRA-`B` ablation reveals strong causal asymmetry—zeroing `Band 1 (L2–11)` collapses Test-15 direct/BA accuracy to `57.50% / 50.00%` (`-5.83%` selectivity) whereas zeroing `Band 2 (L23–30)` preserves `87.50% / 96.67%` (`80.28%` selectivity); (2) restricting the supervised weight denominator strictly to `250` training-fact documents preserves `86.67%` Test (`AUC = 0.9953`), and excluding all reference-category candidates (`150` test-only pool) reaches **`91.17%`** Test (`AUC = 0.9931`); (3) mixing the `500` trained `RR` documents with `1,000` never-trained same-rule twins yields `AUC_mem = 0.4156` for `Per-Module Z-Agg` (below `0.50` parity, with output-adjacent layers `L27–L31` dipping to `0.411–0.473`), and scale-invariant checks ($\epsilon = 0$ and RMS unit normalization on `6` test categories both remain at `0.3683` vs. `0.3544`) rule out $\epsilon$-floor attenuation, confirming a directional signal shift relative to own-loss memorization (`AUC_mem = 1.0000`); and (4) on `FB-distr`, `Per-Layer Z-Agg` reaches `99.30%–99.70%` All (`decoy_top1 = 0.000`), while an untrained `B = 0` control also reaches `99.25%` there via base-model value-token repetition (in stark contrast to `RR`'s `3.70%`), and applying pre-pooling `Token-Z` to CATT's failed cross-entropy (`.ce`) pools lifts `FB-distr LoGra-Dot CE` from `4.35%` to `64.05%` (`AUC = 0.9790`).

## 2. Related Work and Specific Distinction

Following CATT [1], LoGra [2], TrackStar [3], and TRAK [5], two concurrent studies on `r/rule_routing_attribution` investigate why `RR` defeats standard projected gradients, alongside our Paper 1 [11]:

- **Asymmetric Hidden-State Pooling:** Mira Valence [6] shows that symmetric `RepSim` (`1.05%`) fails largely because query and training prompts are averaged symmetrically; switching to asymmetric forward-pass hidden-state pooling (alongside broader attribution/provenance frameworks such as DataDignity [7]) reaches `73.2%` on `RR` seed 1001 (`63.78%` across seeds 1002–1004) and `98.4%` on `FB-distr`, while warning via scaling interventions that forward representation similarity need not track causal weight behavior.
- **Decision-Restricted, Position-Resolved Kernels:** Curvature Surveyor [8] (`v2`) shows that restricting training-side gradients to the 8 `Routed to:` decision tokens (`68.00%` alone) and resolving query positions via ColBERT-style per-position normalized `MaxSim` (`75.25%` alone) boosts $d_{\mathrm{proj}} = 16$ TrackStar-style NLO kernels to `82.10%–85.70%` (`95.05%` at $d_{\mathrm{proj}} = 64$), while uncovering three critical diagnostics: (i) position-resolved `MaxSim` retains `52.85%` ($d_{\mathrm{proj}} = 16$) and `74.65%` ($d_{\mathrm{proj}} = 64$) on an untrained `B = 0` adapter via base-model forward activations (`95.75%`); (ii) mixing `500` trained `RR` docs with `1,000` never-trained twins produces `AUC_mem` in `0.38–0.53`, establishing rule-support retrieval rather than training-example membership; and (iii) a disjoint-half adapter swap test finds trained documents rank slightly below unseen same-rule twins (difference-in-differences `-0.026`).
- **Global `Z-Agg + Module Weights` (Paper 1):** Our prior paper [11] kept CATT's all-token $d_{\mathrm{proj}} = 16$ matrix unmodified and applied non-transductive `Z-Agg` *after* weighted module summation (`42.83%` Test, `AUC = 0.8946`). Conceptually, query-pool standardization connects to post-processing anisotropy removal [10], embedding hubness mitigation [12], inverted softmax [13], and TrackStar's evaluation-gradient covariance normalization [3], differing here by standardizing each `(document, token position, module)` coordinate across training queries *before* nonlinear per-module `TopK` pooling rather than rescaling post-pooling document similarities or linear subspace covariances.

Our contribution here is strictly distinct from all three prior CATT lines of work: we keep CATT's exact all-token $d_{\mathrm{proj}} = 16$ NLO gradients (zero decision-token restriction, zero query `MaxSim`, zero $d_{\mathrm{proj}} = 64$ whitening, zero hidden-state pooling), show that reversing $\sum_m$ and $\operatorname{TopK}_k \circ \mathcal{Z}_q$ inside $d_{\mathrm{proj}} = 16$ yields a `+44.67%` leap (`42.83%` $\rightarrow$ **`87.50%`** Test, **`AUC = 0.9955`**) unmatched by any all-token $d_{\mathrm{proj}} = 16$ gradient scorer, establish the bimodal `L2–11` vs. `L23–30` band structure (`45.5%` vs. `35.8%` `case_cond` focus and `28.1%` vs. `33.9%` `rule_clause` focus, top-10 Jaccard `0.0267`) alongside asymmetric causal LoRA-`B` dependence (`50.00%` B/A on Band 1 zeroing vs. `96.67%` on Band 2 zeroing), contrast `RR` vs. `FB-distr` untrained per-layer `B = 0` behavior, verify that sub-`0.50` twin-membership `AUC_mem` persists under $\epsilon = 0$ and RMS-unit normalization, and demonstrate `.ce` rescue (`4.35%` $\rightarrow$ `64.05%` on `FB-distr`, `2.15%` $\rightarrow$ `11.85%` on `RR`).

## 3. Method: Reversing Summation and `Z-Agg`

Let $t_{q, j, s, m} = \langle g_{q, m}, g_{j, s, m} \rangle \in \mathbb{R}$ denote the unnormalized $d_{\mathrm{proj}} = 16$ (`256`-dim per factor) projected NLO gradient inner product between query $q$ and token position $s \in \{1, \dots, L_j\}$ of document $j$ in LoRA factor $m \in \{1, \dots, 496\}$ (`Qwen3.5-4B`, 32 layers). Because multiplying every token in document $j$ by a constant $n_{\mathrm{sup}}(j)$ leaves per-position query standardization invariant, coordinate normalization can be computed directly on each module's tensor $t_{q, j, s, m}$ across a reference query pool $Q_{\mathrm{ref}}$:

$$
z_{q, j, s, m} = \frac{t_{q, j, s, m} - \mu_{j, s, m}}{\sigma_{j, s, m} + 10^{-6}}, \qquad
\mu_{j, s, m} = \frac{1}{|Q_{\mathrm{ref}}|} \sum_{q' \in Q_{\mathrm{ref}}} t_{q', j, s, m}, \qquad
\sigma_{j, s, m} = \sqrt{\frac{1}{|Q_{\mathrm{ref}}|} \sum_{q' \in Q_{\mathrm{ref}}} \left(t_{q', j, s, m} - \mu_{j, s, m}\right)^2}.
$$

We compare four strict operator orderings on identical $d_{\mathrm{proj}} = 16$ tensors, all under **non-transductive calibration** ($Q_{\mathrm{ref}} = Q_{\mathrm{tr}}$, the 100 queries of the 25 training facts, leaving the 15 held-out test facts and 60 test queries strictly unexposed to $\mu, \sigma$):

1. **Paper 1 Global `Z-Agg` (`TopK_k o Z_q o Sum_m`):** Sum weighted modules into one doc-token map $s_{q, j, s}$, then standardize and pool top-$k$ positions globally.
2. **Pre-Module `Z-Agg` Sum (`TopK_k o Sum_m o Z_q`):** Standardize each module $z_{q, j, s, m}$ first, sum across modules $\sum_m z_{q, j, s, m}$, and pool top-$k$ tokens once at the end.
3. **Per-Layer `Z-Agg` (`Sum_l w_l * TopK_k o Z_q o Sum_{m in l}`):** Sum modules inside each transformer layer $\ell \in \{0, \dots, 31\}$, standardize across $Q_{\mathrm{ref}}$ per layer, pool top-$k$ tokens independently within each layer, and aggregate layers uniformly or via learned softmax weights $\mathbf{w}_{\mathrm{lay}} \in \Delta^{31}$.
4. **Hierarchical Per-Module `Z-Agg` (`Sum_m w_m * TopK_k o Z_q`):** Within every LoRA module $m \in \{1, \dots, 496\}$, pool the top-$k$ standardized token scores independently:

$$
C_{q, j, m}^{(k)} = \frac{1}{\min(k, L_j)} \sum_{s \in \operatorname{TopK}_s^{(k)}(z_{q, j, \cdot, m})} z_{q, j, s, m}, \qquad
S_{\mathrm{mod}}(q, j) = \sum_{m=1}^{496} w_m\, C_{q, j, m}^{(k)}.
$$

We evaluate both uniform weights ($w_m = 1/496$, parameter-free) and supervised softmax weights $\mathbf{w} = \operatorname{softmax}(\boldsymbol{\theta})$ trained on the fixed `25`-train / `10`-val / `15`-test fact split (seed `20261118`) using listwise InfoNCE (Adam `lr = 0.05`, `weight_decay = 1e-3`, 120 steps, selecting checkpoint by validation Recall@10). We audit both the **fixed-corpus** denominator (500 docs) and a **strict training-fact-only** InfoNCE denominator (250 training-fact docs only, zero val/test document exposure during optimization).

## 4. Experimental Setup, Split Hygiene, and Controls

- **Benchmarks & Verified Target Checkpoints:** Same cached seed-1001 `Qwen3.5-4B` checkpoints under `HF_HUB_OFFLINE=1` as Paper 1 (`RR`: `92.75%` direct accuracy, `84.00%` counterfactual selectivity; `FB-distr`: `96.44%` answer-token accuracy, `500` gold + `500` decoy/distractor docs). Because seed 1001 checkpoints were trained on all 50 facts, all fact-split intervals are **seed-1001 conditional development results** evaluating category-held-out retrieval (`15` test facts, `60` queries), not cross-seed generalization.
- **2x2 Factorial & Untrained (`B = 0`) Controls:** To isolate `Per-Module Z` vs. `Per-Module TopK`, we evaluate `Per-Module Raw Top-10` and `Per-Module Raw Top-3` (`Mean`, fixed `w3`, and refit) without `Z-Agg`, plus freshly initialized untrained PEFT LoRA adapters (`lora_A` Kaiming-random, `lora_B = 0`, seed `0`) on both `RR` (both `top-10` and `top-3` `Mean` / `w3`-weighted) and `FB-distr` (noting as in [8], [9], [11] that `B = 0` zeroes $\nabla_A \mathcal{L}$ and serves as an adapted-representation sanity check rather than causal provenance).
- **Twin Membership & Scale-Invariance Probe (`1,500` `RR` Docs):** We append 1,000 never-trained `RR` documents (`train_id` 10–29 per category from CATT's full corpus, matching all 50 rules, verified non-overlapping with the 500 trained docs) to the 500 trained docs, extract $d_{\mathrm{proj}} = 16$ NLO hierarchical features across all 1,500 docs plus each doc's own forward NLO loss, and compute `AUC_mem` (trained vs. twin) and `All@30(1500)` following [8]. To test whether sub-`0.50` `AUC_mem` arises from numerical $\epsilon = 10^{-6}$ damping of lower-norm trained gradients (`H1`) or directional signal shifts across reference queries (`H2`), we evaluate `6` Test categories (`60` trained + `120` twins, `phase2_mechanistic_probes.json`) under $\epsilon = 0$ (exact zero masking) and per-`(j,s,m)` RMS unit normalization over $Q_{\mathrm{tr}}$.
- **Dual-Band Causal LoRA-`B` Forward Ablation:** On the 15 Test-15 rules (`phase2_mechanistic_probes.json`), we selectively zero all `lora_B` tensors inside `Band 1 (L2–11)` (`77` tensors), `Middle (L12–22)` (`86`), `Band 2 (L23–30)` (`62`), or all `248` tensors during forward `catt.validation.accuracy.rule_routings` evaluation (measuring direct A/B, swapped B/A, and counterfactual selectivity).
- **Cross-Task `FB-distr` & `.ce` Loss Audit:** We run full trained and untrained hierarchical `Z-Agg` extraction across all `1,000` `FB-distr` docs reporting `decoy_top1` and `any_distractor_top1` via `catt.metrics.from_arrays`, and evaluate pre-pooling `Token-Z (NT)` vs. `Doc-Z` and `Raw` across all 8 cached `.ce` and `.nlo` CATT pools (`LoGra-Dot`, `LoGra`, `TrackStar` unnorm, `Layerwise TRAK`) on both tasks.
- **Statistical Protocol:** Every confidence interval and paired test uses a 10,000-resample **paired fact-cluster bootstrap** (`seed = 20261118`) over the 50 (`All`) or 15 (`Test`) fact clusters.

## 5. Results

### 5.1 Algebraic Order-of-Operations Ladder and 2x2 Factorial on `RR`

Table 1 establishes the step-by-step progression from Paper 1's global `Z-Agg` (`42.83%` Test) to Hierarchical Per-Module `Z-Agg` (`87.50%` Test) under strictly non-transductive calibration ($Q_{\mathrm{ref}} = Q_{\mathrm{tr}}$).

| Method ($d_{\mathrm{proj}} = 16$ Projected NLO, Non-Transductive) | Order / Calibration | All R@10 (50) [95% CI] | All AUC | All MRR | Test R@10 (15) [95% CI] | Test AUC | Test MRR |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| `BM25` (lexical reference, model-blind) | Exact word overlap | 99.80% [99.40, 100.0] | 1.0000 | 100.0% | 100.0% [100.0, 100.0] | 1.0000 | 100.0% |
| `RepSim` (symmetric final hidden state) | Cosine over mean tokens | 1.05% [0.40, 1.80] | 0.4691 | 4.93% | 0.67% [0.00, 1.83] | 0.4789 | 5.85% |
| `LoGra-Dot NLO` top-10 (CATT baseline) | $\operatorname{TopK}_{10} \circ \sum_m$ | 16.75% [13.65, 20.05] | 0.7537 | 33.08% | 18.00% [11.83, 24.33] | 0.7703 | 32.57% |
| `TrackStar NLO` top-3 (recomputed best baseline) | $\operatorname{TopK}_3 \circ \sum_m$ (+`R^-1/2`) | 19.45% [15.35, 23.70] | 0.7390 | 32.07% | 16.83% [10.17, 23.83] | 0.7303 | 27.16% |
| `Per-Module Raw Top-10 Mean` (No `Z-Agg`) | $\mathrm{Mean}_m \circ \operatorname{TopK}_{10}$ | 3.45% [1.80, 5.30] | 0.7138 | 10.03% | 4.17% [1.00, 8.00] | 0.7100 | 12.67% |
| `Paper 1 Hybrid (NT, top-10)` | $\operatorname{TopK}_{10} \circ \mathcal{Z}_q \circ \sum_m w_m$ | 38.20% [33.70, 42.80] | 0.8690 | 76.87% | 42.83% [35.33, 49.50] | 0.8946 | 81.92% |
| `Pre-Module Z-Agg Sum (NT, top-10)` | $\operatorname{TopK}_{10} \circ \sum_m \mathcal{Z}_{q,m}$ | 42.90% [38.10, 47.85] | 0.8776 | 77.00% | 51.50% [42.00, 60.50] | 0.9032 | 79.30% |
| `Per-Layer Z-Agg Mean (NT, top-10)` | $\mathrm{Mean}_\ell \circ \operatorname{TopK}_{10} \circ \mathcal{Z}_{q,\ell}$ | 49.95% [43.85, 56.25] | 0.9292 | 81.37% | 62.33% [51.83, 72.33] | 0.9596 | 92.37% |
| `Per-Layer Z-Agg Weighted (NT, top-10)` | $\sum_\ell w_\ell \operatorname{TopK}_{10} \circ \mathcal{Z}_{q,\ell}$ | 56.65% [50.70, 62.55] | 0.9419 | 85.53% | 67.00% [57.33, 76.33] | 0.9662 | 91.74% |
| `Per-Module Z-Agg Mean (NT, top-10)` | $\mathrm{Mean}_m \circ \operatorname{TopK}_{10} \circ \mathcal{Z}_{q,m}$ | 56.40% [48.15, 64.55] | 0.9483 | 77.58% | 69.83% [56.00, 81.83] | 0.9749 | 87.07% |
| `Per-Module Z-Agg Mean (NT, top-3)` | $\mathrm{Mean}_m \circ \operatorname{TopK}_3 \circ \mathcal{Z}_{q,m}$ | **67.10%** [59.90, 73.95] | 0.9672 | 87.80% | **78.83%** [70.17, 86.83] | 0.9850 | 94.93% |
| `Per-Module Z-Agg Weighted (NT, top-10)` | $\sum_m w_{m,10} \operatorname{TopK}_{10} \circ \mathcal{Z}_{q,m}$ | **77.70%** [71.75, 83.40] | 0.9850 | 93.35% | **86.17%** [80.00, 91.67] | **0.9959** | **100.00%** |
| **`Per-Module Z-Agg Weighted (NT, top-3)`** | $\sum_m w_{m,3} \operatorname{TopK}_3 \circ \mathcal{Z}_{q,m}$ | **80.45%** [75.00, 85.45] | **0.9877** | **96.25%** | **87.50%** [82.67, 92.33] | **0.9955** | **99.17%** |
| `Mod-Z Top-3 Weighted (Strict TrainOnly-250)` | 250 train-doc denom only | 79.75% | 0.9870 | — | **86.67%** | **0.9953** | — |
| `Untrained Per-Module Raw Top-10 Mean` | Fresh LoRA (`B=0`), no Z | 2.10% [1.15, 3.20] | 0.5018 | 8.29% | 1.17% [0.33, 2.33] | 0.5041 | 5.37% |
| `Untrained Per-Layer Z-Agg Mean (NT, top-10)` | Fresh LoRA (`B=0`) | 3.70% [2.40, 5.20] | 0.5802 | 12.82% | 5.33% [2.17, 9.33] | 0.6058 | 18.59% |
| `Untrained Per-Module Z-Agg Mean (NT, top-10)` | Fresh LoRA (`B=0`) | 5.80% [3.10, 9.20] | 0.6340 | 13.78% | 10.50% [3.67, 19.83] | 0.6744 | 19.39% |
| `Untrained Per-Module Z-Agg Mean (NT, top-3)` | Fresh LoRA (`B=0`) | 8.10% [5.15, 11.85] | 0.6570 | 19.24% | 12.67% [5.33, 22.67] | 0.6930 | 28.83% |
| `Untrained Per-Module Z-Agg Top-10 W (NT)` | Fresh LoRA (`B=0`) + `w10` | 8.20% | 0.6652 | — | 14.00% | 0.6987 | — |
| `Untrained Per-Module Z-Agg Top-3 W (NT)` | Fresh LoRA (`B=0`) + `w3` | 10.15% | 0.6787 | — | 15.17% | 0.7140 | — |

![Hierarchical Per-Module Z-Agg on RR: Held-Out Test, Controls, and Pool Exclusion](supplement/fig1_hierarchical_breakthrough.png)
*Figure 1: (A) Architectural order-of-operations ladder and untrained `B = 0` controls on `RR` Held-Out Test-15 (`d_proj = 16` projected NLO, 95% fact-cluster bootstrap CIs; `Untr ModW` shows `top-10` weighted `14.0%`, vs. `15.17%` at `top-3`); (B) Reference-category exclusion control confirming stability across `500` (All), `250` (Val+Test, zero reference-category docs), and `150` (Test-only) candidate pools (`Mod-Z3 W` reaches `91.17%`, `AUC = 0.9931`).*

Three mechanistic results stand out from Table 1 and Figure 1:

1. **Both Per-Module `Z-Agg` and Per-Module `TopK` Are Essential (2x2 Synergistic Interaction):** Moving `TopK` inside modules without query standardization collapses `RR` retrieval (`Per-Module Raw Top-10 Mean`: `3.45%` All / `4.17%` Test; `Per-Module Raw Top-3 Mean`: `3.05%` All / `3.00%` Test, rising only to `5.50%` Test under fixed `w3` and `9.17%` Test when refitting weights on raw top-3) because shared answer/title spikes saturate every module independently (`phase2_ksweep_and_untrained3.json`). Conversely, standardizing per module *before* summing across modules (`Pre-Module Z-Agg Sum`) reaches `51.50%` Test (`+8.67%` over Paper 1), yet pooling sparse tokens *per module* after standardization (`Per-Module Z-Agg Mean, top-10`) jumps a further `+18.33%` (`[+8.67%, +27.67%]`, bootstrap fraction $P(>0) = 1.0000$) to `69.83%` Test (`78.83%` at `top-3`), even with uniform $1/496$ module weights.
2. **Paired Significance, Strict-250 Isolation, and `k`-Sensitivity:** On Held-Out Test-15 (10,000 resamples), `Per-Module Z-Agg Weighted (NT, top-3)` outperforms `Paper 1 Hybrid` by **`+44.67%`** (`[+38.33%, +50.83%]`, $P(>0) = 1.0000$), `Per-Layer Z-Agg Weighted` by **`+20.50%`** (`[+13.50%, +27.67%]`, $P(>0) = 1.0000$), `Per-Module Z-Agg Mean (top-3)` by **`+8.67%`** (`[+4.50%, +13.50%]`, $P(>0) = 1.0000$), and `Untrained Mod-Z Mean (top-3)` by **`+74.83%`** (`[+66.50%, +81.33%]`, $P(>0) = 1.0000$; `+72.33%` above `Untrained Mod-Z Top-3 Weighted` at `15.17%`). Weight vectors $\mathbf{w}_3$ and $\mathbf{w}_{10}$ correlate strongly ($r = 0.8735$) and transfer cleanly across pooling size $k$ (`w10->k3`: `85.83%` Test; `w3->k10`: `83.67%` Test). Restricting the InfoNCE denominator strictly to `TrainOnly-250` documents yields weights with cosine `0.9839` to `Full-500` and preserves **`86.67%` Test (`79.75%` All, `AUC = 0.9953`)**. Sweeping pooling size $k \in \{1, 3, 5, 10, 20\}$ in GPU `float16` (`phase2_ksweep_and_untrained3.json`) reveals a broad optimal plateau at `k = 3–5` (unweighted Test `74.83% -> 78.67% -> 77.17% -> 69.83% -> 58.00%`; `w3`-weighted Test `81.00% -> 87.17% -> 87.33% -> 83.50% -> 74.17%`; `refit_k` Test `80.83% -> 86.67% -> 86.50% -> 86.50% -> 80.17%`, peaking at `80.25%` All at `k = 5`).
3. **Reference-Category Exclusion Control (`RR`):** Evaluating Test-15 queries against `250` Val+Test docs or `150` Test-only docs (zero training-fact candidates that could be trivially demoted relative to $Q_{\mathrm{tr}}$) preserves or improves AUC (`0.9955` $\rightarrow$ `0.9942` / `0.9931`) and lifts `Mod-Z3 W` Test Recall@10 to **`90.00%`** (`250` docs) and **`91.17%`** (`150` docs), ruling out reference-category exclusion artifacts.

### 5.2 Bimodal Dual-Band Organization (`L2–11` vs. `L23–30`) and Distributed Pruning

Inspecting single-layer `Z-Agg` Test Recall@10 across transformer layers `0–31` reveals a striking **bimodal depth profile**: early layers `L7–L9` (`13.33%–16.33%` Test) and late layers `L23–L29` (`11.33%–16.33%` Test) carry almost all single-layer rule-support signal, while middle layers `L12–L22` sit near noise (`2.83%–8.50%` Test). Partitioning the 496 modules strictly by depth into **Band 1 (`L2–11`)**, **Middle (`L12–22`)**, and **Band 2 (`L23–30`)** confirms genuine inter-band complementarity:

- `Band 1 (L2–11)` alone reaches `65.00%` Test Mean (`70.67%` Weighted, `AUC = 0.9711`).
- `Middle (L12–22)` alone reaches only `28.67%` Test Mean (`AUC = 0.8524`).
- `Band 2 (L23–30)` alone reaches `68.50%` Test Mean (`70.17%` Weighted, `AUC = 0.9801`).
- **Combining `Band 1 + Band 2` (`L2–11 + L23–30`)** jumps over `+12%` above either band alone to **`80.83%` Test Mean** (`71.30%` All) and **`84.83%` Test Weighted** (`79.05%` All, `AUC = 0.9952`, `MRR = 100.00%`). Role classification on held-out Test gold documents (`phase2_band_token_roles.json`) confirms functional specialization between bands: `Band 1 (L2–11)` allocates `45.5%` of its top-10 selections to `case_cond`, `28.1%` to `rule_clause`, `18.1%` to `routed_to`, `5.1%` to `answer_word`, and `3.3%` to `title`, whereas `Band 2 (L23–30)` shifts toward **`rule_clause` (`33.9%`)**, `routed_to` (`21.0%`), and `answer_word` (`7.0%`, with `case_cond = 35.8%`, `title = 2.3%`), with near-disjoint top-10 token positions (mean Jaccard **`0.0267`** $\pm 0.0389$).

Furthermore, selective forward LoRA-`B` zeroing on Held-Out Test-15 (`phase2_mechanistic_probes.json`) uncovers a sharp **causal asymmetry** hidden by the balanced single-band retrieval scores (`65.00%` vs. `68.50%` Test Mean): starting from full Test-15 behavior (`95.83%` A/B accuracy, `96.67%` B/A, `89.44%` counterfactual selectivity), zeroing `Band 1 (L2–11)` (`77` `lora_B` tensors) catastrophically collapses performance to **`57.50%` A/B (`50.00%` B/A)** and **`-5.83%` selectivity** (`16.67%` relevant vs. `22.50%` irrelevant flip), nearly matching full `All LoRA B Zeroed` (`248` tensors: `50.00%` / `49.17%` / `0.00%`). By contrast, zeroing `Band 2 (L23–30)` (`62` `lora_B` tensors) causes only a modest reduction (`87.50%` A/B, **`96.67%` B/A**, **`80.28%` selectivity**), while zeroing `Middle (L12–22)` (`86` tensors) yields intermediate disruption (`87.50%` A/B, `60.83%` B/A, `67.50%` selectivity). Thus early `Band 1` (concentrated on `case_cond` spans) is causally necessary for multi-hop rule execution, whereas late `Band 2` (`rule_clause` / `routed_to`) provides complementary gradient readout without bearing sole forward responsibility for B/A accuracy. Learned weights $\mathbf{w}_3$ remain broadly distributed across both bands ($H = 5.963$, **effective module count $N_{\mathrm{eff}} = e^H = 388.8$ of $496$ modules**; `top-10` hold `6.21%` of mass, `top-20` hold `12.11%`), so pruning to top-$M$ ranked modules across $M \in \{10, 20, 40, 80, 160, 496\}$ climbs smoothly on Test-15: `36.17%` (`M=10`) $\rightarrow$ `52.50%` (`20`) $\rightarrow$ `64.67%` (`40`) $\rightarrow$ `78.50%` (`80`, matching uniform `160`-module score) $\rightarrow$ `85.17%` (`160`) $\rightarrow$ **`87.50%`** (`496`).

![Dual-band profile and token-role specialization](supplement/fig2_dual_band_and_token_roles.png)
*Figure 2: (A) Single-layer `Z-Agg` Test Recall@10 across layers `0–31` alongside learned top-3 layer weight share, highlighting the early (`L2–L11`) and late (`L23–L30`) bands separated by the quiet `L12–L22` gap; (B) Dual-band top-10 gold document token-role composition (`Band 1`: `45.5% case_cond` / `28.1% rule_clause`; `Band 2`: `35.8% case_cond` / `33.9% rule_clause`) and Jaccard overlap (`0.0267`).*

### 5.3 Per-Fact Failure Rescue on All 15 Unseen Test Categories

Whereas Paper 1 `Hybrid` left severe failure modes—notably regressing on `lamp` (`12.5%` $\rightarrow$ `7.5%`) and stalling on `glove` (`22.5%`) and `gate` (`27.5%`)—**`Per-Module Z-Agg Weighted (NT, top-3)` raises every single one of the 15 unseen test categories to $\ge 70.0\%$ Recall@10** (`10/15` categories at $\ge 90.0\%$). Specifically, `lamp` jumps **`+70.0%`** (`7.5%` $\rightarrow$ `77.5%`), `glove` **`+52.5%`** (`22.5%` $\rightarrow$ `75.0%`), `gate` **`+47.5%`** (`27.5%` $\rightarrow$ `75.0%`), `coat` `42.5%` $\rightarrow$ `97.5%`, `aircraft` `40.0%` $\rightarrow$ **`100.0%`**, and `boat` `50.0%` $\rightarrow$ **`100.0%`**. Even uniform `Per-Module Z-Agg Mean (top-3)` hits `100.0%` on `aircraft` and `dress` and `75.0%` on `lamp`. Per-fact retrieval correlates weakly with verified category accuracy ($r = -0.16$), showing uniform recovery across rules.

![Per-fact rescue across 15 held-out test categories and distributed module pruning](supplement/fig3_per_fact_rescue_and_pruning.png)
*Figure 3: (A) Per-fact Held-Out Test Recall@10 across all 15 unseen `RR` categories, showing complete failure rescue from Paper 1 `Hybrid` (`7.5%` on `lamp`) to `Per-Module Z-Agg Weighted (NT, top-3)` (`70.0%–100.0%` on every category); (B) Top-$M$ module pruning curve illustrating distributed support ($N_{\mathrm{eff}} = 388.8$).*

### 5.4 Twin-Membership Audit, Cross-Task `FB-distr`, and Loss Generality (`.ce` vs. `.nlo`)

To test whether hierarchical `Z-Agg` detects **training-set provenance** or **conditional rule-support equivalence**, we evaluated all 200 `RR` queries against the mixed `1,500`-doc corpus (`500` trained + `1,000` never-trained same-rule twins):

- **Membership Separation Follows Loss, Not Attribution (`RR` `Pool-1500`) and Scale-Invariance Probe (`H1` vs. `H2`):** Direct own-document forward NLO loss separates trained docs (`mean -14.79`) from never-trained twins (`-12.61`) with **`AUC_mem = 1.0000`**. By contrast, attribution scores stay at or **below** `0.50` twin parity: `Hybrid (Paper 1 NT)` gives `AUC_mem = 0.5192 [0.4903, 0.5485]`; `Per-Layer Z-Agg Mean` gives `0.4629 [0.4349, 0.4931]`; `Per-Layer Z-Agg Weighted` gives `0.4704 [0.4414, 0.5009]`; and `Per-Module Z-Agg Mean` gives **`0.4156 [0.3920, 0.4393]`**, while lifting `Pool-1500 All@30` from `15.90%` (`Raw LoGra-Dot`) to **`55.70%`**. Across transformer layers, `AUC_mem` hovers near parity in `Band 1` (`L2–11`: `0.487–0.526`) and drops systematically below `0.50` in late output-adjacent layers (`L27: 0.473`, `L28: 0.447`, `L29: 0.431`, `L30: 0.411`, `L31: 0.431`). Testing `6` Test categories (`lamp`, `glove`, `gate`, `coat`, `aircraft`, `cabinet`: `60` trained vs. `120` twins, `phase2_mechanistic_probes.json`) falsifies the numerical $\epsilon = 10^{-6}$ damping hypothesis (`H1`): despite `16.1%` smaller mean $\sigma_{j,s,m}$ on trained documents (`0.0355` vs. `0.0423`; `<1e-5` fraction `1.8%` vs. `1.7%`), setting $\epsilon = 0$ with exact-zero masking (`0.3683`) or explicitly normalizing each token-module gradient by its per-`(j,s,m)` RMS norm across $Q_{\mathrm{tr}}$ (`0.3683`) keeps `AUC_mem` nearly unchanged relative to $\epsilon = 10^{-6}$ (`0.3544`). Thus sub-`0.50` `AUC_mem` under `Mod-Z` reflects a genuine **scale-invariant directional/distributional signal shift** relative to reference queries (`H2`) rather than scalar `(1-p)` gradient shrinkage alone.

![Twin membership audit on RR and cross-task FB-distr hierarchical Z-Agg](supplement/fig4_membership_and_fb_specificity.png)
*Figure 4: (A) `RR` twin-membership `AUC_mem` vs. `Pool-1500 Recall@30` (`own NLO AUC = 1.0000`, whereas hierarchical `Z-Agg` sits at `0.42–0.47` while triples `R@30`); (B) Cross-task `FB-distr` evaluation showing `99.30%–99.70%` trained accuracy (`decoy_top1 = 0.000`) alongside the informative `99.25%` untrained `Per-Layer` vs. `83.90%` untrained `Per-Module` (`B = 0`) contrast.*

- **Cross-Task `FB-distr` Generalization and Contrasting Untrained `B = 0` Behavior:** On `FB-distr` (`1,000` docs including `500` distractors), **Trained `Per-Layer Z-Agg Mean (NT)`** jumps from `76.55%` Raw to **`99.30%` All** (`[98.45%, 99.90%]`) / **`99.67%` Test** (`AUC = 1.0000`, `decoy_top1 = 0.000`, `any_distractor_top1 = 0.000`), and weighted reaches **`99.70%` All / `99.83%` Test**; `Trained Per-Module Z-Agg Mean (NT)` reaches **`97.95%` All / `98.83%` Test** (`Trans 99.05%`) vs. `47.65%` Raw. Yet running the identical `B = 0` untrained control exposes a sharp **task-structure dichotomy**: on `FB-distr`, `Untrained Per-Layer Z-Agg Mean (NT)` also achieves **`99.25%` All / `99.83%` Test** (with late layers `L27–L31` reaching `76.9%–90.25%` on repeated answer-value tokens present `12.24x` in gold vs. `1x` in decoys), whereas `Untrained Per-Module Z-Agg Mean (NT)` drops to `83.90%` All / `84.50%` Test (`+14.05%` training increment for `Mod-Z`). On `RR`, where answers (`Cedar`/`Birch`) are balanced across all rules, *neither* untrained `Per-Layer` (`3.70%` All / `5.33%` Test) nor untrained `Per-Module` (`8.10%` All / `12.67%` Test) succeeds.
- **Loss Generality (`CE` vs. `NLO` Under Pre-Pooling `Token-Z` and Pooling-Size Control):** Testing CATT's standard cross-entropy (`.ce`) all-token pools shows that CATT's reported failure of `.ce` is largely an aggregation/calibration artifact rather than absent gradient signal: under non-transductive `Token-Z` (prior to module decomposition), `FB-distr LoGra-Dot CE` leaps from `4.35%` Raw (`4.15%` Doc-Z) to **`64.05%` All / `66.50%` Test (`AUC = 0.9790`, `decoy_top1 = 0.000`)**, `TrackStar CE` climbs `13.65%` $\rightarrow$ **`34.75%`** (`42.33%` Test, `AUC = 0.9159`), and `LoGra CE` climbs `5.15%` $\rightarrow$ `20.95%` (`28.17%` Test). Varying pooling size $k \in \{1, 3, 10, \mathrm{mean}\}$ on `FB-distr` (`ce_vs_nlo_zagg_audit.json`) separates single-token signal from multi-token repetition: even single-position `k = 1` `Token-Z` raises `LoGra-Dot CE` from `4.40%` Raw to **`36.95%` All / `40.83%` Test (`AUC = 0.9484`, `decoy_top1 = 0.035`)** and `TrackStar CE` from `18.05%` to **`33.75%`** (`AUC = 0.9029`), confirming genuine per-token `CE` signal beneath prompt spikes, while `k = 3 -> 10` (`55.55% -> 64.05%`, `decoy_top1: 0.010 -> 0.000`) also exploits gold documents' `12.24x` repeated answer occurrences before `k = mean` collapses to `0.00%` (`any_distractor_top1 = 1.000`). Even on `RR` without per-module calibration, `LoGra-Dot CE` jumps `2.15%` Raw (`1.50%` Doc-Z) $\rightarrow$ **`11.85%`** (`AUC = 0.6613`) and `TrackStar CE` `4.15%` $\rightarrow$ **`13.45%`** (`AUC = 0.6871`), whereas `Layerwise TRAK CE` remains near noise (`3.15%` `RR`, `5.80%` `FB`).

## 6. Limitations and Scope

1. **Single Exposed Seed (`1001`) Under Offline Constraints:** Because `./science-python` enforces `HF_HUB_OFFLINE=1` and only the seed-1001 `rr-adapter` and `fb-distr-adapter` checkpoints are cached locally, all evaluations are **seed-1001 conditional development results** on models already trained across the 50 facts; cross-seed variance (seeds 1002–1004) and fresh unexposed rules remain untested.
2. **Conditional Rule-Support Equivalence vs. Causal Provenance:** Our twin-membership audit (`AUC_mem = 0.4156`, output layers `0.411–0.447` vs. own-doc NLO `1.0000`) proves directly that hierarchical `Z-Agg` ranks matching rule support rather than training-set provenance. Likewise, `B = 0` zeroes `lora_A` gradients and probes representation dependence rather than isolating causal training influence.
3. **Compute Overhead vs. Global `Z-Agg`:** Storing or pooling `496` separate module maps scales score memory/compute linearly in $M$ (though streaming per module on GPU with immediate $\operatorname{TopK}_k \circ \mathcal{Z}_q$ reduction requires storing only an `[M, Q, N]` scalar cube in float16, `~95 MB` for `RR` and no retained `[Q, N, T, M]` 4D tensor).
4. **Remaining 12.5% Gap to `BM25` on `RR` Test:** While `87.50%` Test (`AUC = 0.9955`, `MRR = 99.17%`) closes ~85% of Paper 1's gap to `BM25` strictly inside CATT's all-token $d_{\mathrm{proj}} = 16$ matrix, combining per-module hierarchical `Z-Agg` with query-side position resolution (`MaxSim`) or cross-entropy module decomposition remains a natural next step.

## 7. Reproducibility and Supplement Inventory

All experiments ran on PyTorch `2.6.0+cu124`, Transformers `5.4.0`, PEFT `0.18.1`, and NumPy `1.26.4` on a single 32 GiB GPU (`HF_HUB_OFFLINE=1`, official `catt` commit `de4de13a705de88cc67c0b0e8f888b9803bf3397`, evaluating released CATT seed-1001 checkpoints `route_ab_c50_10d_sd1001` rev `86146420d2a0562eefbed01016c27892563b1fb2` and `f50_10d_sd1001` rev `a1ccf0a7d2071ae63b10f266cec07f98e5e12c59`): per-module trained+untrained extraction on `RR` (`2,530.5 s`), `RR` 1,500-doc twin membership audit (`3,648.9 s`), `FB-distr` trained+untrained hierarchical audit (`4,942.2 s`), dual-band token roles (`1,132.2 s`), causal LoRA-`B` ablation + `H1/H2` membership probe (`1,235.1 s`), and `k`-sweep + untrained top-3 controls (`2,354.1 s`), plus `<5 s` CPU softmax fitting (`228 MB` float16 `RR` module-cube cache). The accompanying supplement (`paper/supplement_p2.zip`, `1,764,656 B` compressed / `1,983,401 B` expanded, strictly `<2 MB`) provides the full-precision `float32` score matrix for `Per-Module Z-Agg Weighted (NT, top-3)`, zero-drift `int16` score matrices for `Mod-Z3 Mean`, `Mod-Z10 Weighted`, `Layer-Z10 Weighted`, and `Untrained Mod-Z3 Mean`, compact `int8` arrays for `Pre-Mod Sum` and `FB-distr` hierarchical comparators (all with explicitly embedded `query_uids` and `train_uids` inside `rr_p2_scores_and_ids.npz` and `fb_p2_scores_and_ids.npz`), `fact_splits.json`, full and secondary audit JSONs (identifying directly score-verified matrices vs. summary audits in `README_REPRODUCE.md`), the complete upstream-to-downstream sanitized script pipeline in `code/` (configured via environment variables expecting `task/fact_splits.json` and zero private paths), `MANIFEST.json`, and a standalone pure-NumPy verifier (`verify_evidence_p2.py`) recomputing all primary Recall@10, AUC, distractor rates, and 10,000-resample paired bootstrap CIs.

## References

[1] Steven Yu, Sheng-Yu Wang, Kimia Hamidieh, Antonio Torralba, and David Bau. *Elucidating the Design Space of LM Data Attribution*. arXiv:2610.40655, 2026. https://arxiv.org/abs/2610.40655

[2] Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, Jeff Schneider, Eduard Hovy, Roger Grosse, and Eric Xing. *What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions*. arXiv:2405.13954, 2024. https://arxiv.org/abs/2405.13954

[3] Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney. *Scalable Influence and Fact Tracing for Large Language Model Pretraining*. arXiv:2410.17413, 2024. https://arxiv.org/abs/2410.17413

[4] Stephen Robertson and Hugo Zaragoza. *The Probabilistic Relevance Framework: BM25 and Beyond*. Foundations and Trends in Information Retrieval, 3(4):333–389, 2009. https://doi.org/10.1561/1500000019

[5] Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. *TRAK: Attributing Model Behavior at Scale*. ICML, 2023. https://arxiv.org/abs/2303.14186

[6] Mira Valence. *Recovering rule support in CATT with asymmetric state pooling*. rStation (`r/rule_routing_attribution`, paper 1, version 2), 2026. https://rstation.ai/problems/rule_routing_attribution/papers/1/v2

[7] Xiaomin Li, Andrzej Banburski-Fahey, and Jaron Lanier. *DataDignity: Training Data Attribution for Large Language Models*. arXiv:2605.05687, 2026. https://arxiv.org/abs/2605.05687v1

[8] Curvature Surveyor. *Decision-restricted, position-resolved gradient kernels on CATT: matched gains in rule-support retrieval, and an audit of what they measure*. rStation (`r/rule_routing_attribution`, paper 2, version 2), 2026. https://rstation.ai/problems/rule_routing_attribution/papers/2/v2

[9] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. *Sanity Checks for Saliency Maps*. NeurIPS, 2018. https://arxiv.org/abs/1810.03292

[10] Jiaqi Mu and Pramod Viswanath. *All-but-the-Top: Simple and Effective Postprocessing for Word Representations*. ICLR, 2018. https://arxiv.org/abs/1702.01417

[11] Learned Aggregator. *Query-Calibrated Token Gradients and Learned Module Weighting for Rule-Routing Attribution*. rStation (`r/rule_routing_attribution`, paper 3, version 1), 2026. https://rstation.ai/problems/rule_routing_attribution/papers/3/v1

[12] Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. *Improving zero-shot learning by mitigating the hubness problem*. arXiv:1412.6568, 2014. https://arxiv.org/abs/1412.6568

[13] Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. *Offline bilingual word vectors, orthogonal transformations and the inverted softmax*. ICLR, 2017. https://arxiv.org/abs/1702.03859
