## 1. Problem, Operational Scope, and Contributions

Training multi-layer causal Transformers without a global backward pass is a core problem in local credit assignment [1, 2, 3, 9]. A foundational methodological distinction separates **backward-free local credit assignment** from **approximate or manual backpropagation**: disabling automatic differentiation (`torch.no_grad()`) or replacing local analytical derivatives with finite-difference probes does not remove the backward phase if current-step downstream error vectors are multiplied by layer weight matrices in reverse order ($\mathbf{e}_{\text{in}} = \mathbf{e}_{\text{out}} W$) or relayed from downstream blocks to upstream blocks within the same training step.

In Chung's **Weight Maximization (WM) with eligibility traces** [1], each unit $i$ in stage $m$ injects a local forward exploration perturbation $\xi_{n, i}^{(m)}(t)$ during the forward pass of batch $t$, records a local pre-synaptic eligibility trace $Z_{i, :}^{(m)}(t)$, and updates at a subsequent step using the column weight-norm change $\sum_{j} W_{j, i}^{(m+1)} \Delta W_{j, i}^{(m+1)}$ of its outgoing receiver synapses $W_{:, i}^{(m+1)}$.

### 1.1 Operational Definition of Delayed Perturbation-Differential Column-Norm Credit and Pipeline Clocks

To adapt delayed WM to multi-token Transformer minibatches where receiver layers are updated by adaptive or spectrally orthogonalized optimizers (`AdamW` or `Pre-Polar Muon` [7, 8]), we state precisely what feedback our algorithm constructs, how its normalization is parameterized in code, and how pipeline clocks are ordered:

1. **Installed Receiver Displacement vs. Perturbation-Differential Plasticity Tensor:** At step $t$, each receiver layer $m+1$ updates its live weights $W^{(m+1)}(t) = W^{(m+1)}(t-1) + \delta W^{(m+1)}(t-1)$ via its optimizer from the clean batch gradient (or delayed WM pseudo-gradient). To evaluate the directional effect of stage $m$'s forward perturbation $\xi^{(m)}(t_{\text{orig}})$ on receiver $m+1$'s synaptic plasticity without confounding from other stages' noise or optimizer state history, our reward routine (`compute_stream_wm_rewards_chunked` in `exp26`; `compute_stream_wm_rewards_fast` in `exp32`–`exp35`, parameterized by `scale_factor`) constructs for each token stream $\mathcal{S}_g$ ($g \in \{1, \dots, G\}$, $|\mathcal{S}_g| = M_g = BT / G$) a **perturbation-differential plasticity tensor**
   $$
   \Delta \tilde{W}_{j, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor}) := \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} u_{n, j}^{(m+1)}(t_{\text{orig}}) \, \Delta y_{n, i}^{(m)}(t_{\text{orig}}), \qquad \Delta y_{n, i}^{(m)} := y_{n, i}^{\text{pert}}(\sigma \xi^{(m)}) - y_{n, i}^{\text{clean}}(0),
   $$
   where $u_{n, :}^{(m+1)}(t_{\text{orig}})$ is the receiver's local post-synaptic modulation vector and $\Delta y_{n, :}^{(m)}(t_{\text{orig}})$ is the perturbed-minus-clean forward activation displacement at stage $m$'s output. At reward construction clock $t_{\text{constr}}$, unit $i$ contracts $\Delta \tilde{W}_{:, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor})$ with its **live outgoing receiver column** $W_{:, i}^{(m+1)}(t_{\text{constr}})$:
   $$
   R_i^{(m, g)}(t_{\text{orig}}, t_{\text{constr}}; \text{scale\_factor}) = \sum_{j=1}^{d_{\text{next}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) \, \Delta \tilde{W}_{j, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor}).
   $$
   The parameter update then divides the stream-summed modulation–input product $\sum_{g=1}^G R_i^{(m, g)} Z_{i, :}^{(m, g)}$ by $N = BT$ (`-(mod_flat.t() @ x_flat) / BT`). Thus, $\Delta \tilde{W}^{(m+1, g)}$ is a local perturbation-differential Hebbian plasticity tensor rather than a decomposition of the installed optimizer displacement $\delta W^{(m+1)}$, and reading $W^{(m+1)}(t_{\text{constr}})$ at construction clock $t_{\text{constr}}$ makes the reward directly sensitive to any receiver weight update $\delta W^{(m+1)}(t-1) = W^{(m+1)}(t) - W^{(m+1)}(t-1)$ installed between origin $t_{\text{orig}}$ and construction $t_{\text{constr}}$.
2. **Three-Clock Pipeline Schedules:** For each stage $m \in \{1, 2, 3\}$, we track $(t_{\text{orig}}, t_{\text{constr}}, t_{\text{inst}})$ relative to installation step $t_{\text{inst}} = t$. Across `exp26`–`exp35`, three distinct clock schedules are evaluated:
   - **True Both-Stages Pre-Install (`both_stages_pre_install`, `exp26`–`exp30` and `exp35` Part B2):** `Stage 3`: $(t, t, t)$; `Stage 2`: $(t-1, t-1 \text{ [pre-Stage3-step]}, t)$ (reward constructed at $t-1$ on $W_{\text{lm}}(t-1)$ prior to Stage 3's batch-$t-1$ update, installed at $t$); `Stage 1`: $(t-2, t-1 \text{ [pre-Stage2-step]}, t)$ (reward constructed and cached at $t-1$ on $W_{\text{proj}}(t-1)$ prior to Stage 2's installation of its batch-$t-2$ update, then installed into Stage 1 at $t$).
   - **Stage-2 Pre-Install / Stage-1 Post-Install (`stage2_pre_install`, `run_exp33_partB.py`, `run_exp34.py`, `exp35` Part B1):** `Stage 3`: $(t, t, t)$; `Stage 2`: $(t-1, t-1 \text{ [pre-Stage3-step]}, t)$ (constructed at $t-1$ prior to Stage 3's update); `Stage 1`: $(t-2, t \text{ [post-Stage2-step]}, t)$ (constructed at $t$ on live $W_{\text{proj}}(t)$ after Stage 2 installed its batch-$t-2$ update at the end of $t-1$).
   - **Strict Post-Installation Construction (`strict_post_install`, `exp32`–`exp35`):** `Stage 3`: $(t, t, t)$; `Stage 2`: $(t-1, t \text{ [post-Stage3-step]}, t)$ (constructed at $t$ on $W_{\text{lm}}(t) = W_{\text{lm}}(t-1) + \delta W_{\text{lm}}(t-1)$ strictly *after* Stage 3 installed its batch-$t-1$ update); `Stage 1`: $(t-2, t \text{ [post-Stage2-step]}, t)$ (constructed at $t$ on $W_{\text{proj}}(t)$ strictly *after* Stage 2 installed its batch-$t-2$ update).
   *(Note: The `CausalAuditLedger` fields in our JSON logs record declared trainer configuration metadata summarizing the code path of each arm rather than runtime hardware profiler counters; in `exp33`–`exp35`, all `stage2_pre_install` ledgers are reconciled to record `(t-2, t [post-Stage2-step], t)`).*

### 1.2 Summary of Contributions

Across `128` multi-seed experimental arms (`383` total runs across Seeds `46, 47, 48`: **`91` three-seed Delayed-WM arms [`273` runs]**, **`13` three-seed node-perturbation arms [`11` `SR-LNP` + `2` `SR-NP` = `39` runs]**, **`4` head/lexical-only backward-free anchors [`11` runs]**, **`7` three-seed local-pullback diagnostic controls [`21` runs]**, and **`13` three-seed exact-backpropagation reference arms [`39` runs]** across `exp18, exp21, exp26`–`exp48`) on ClimbMix `d6` ($L=6, D \in \{256, 384, 512, 768\}, V=8,192$, $360$–$720$ steps $= 0.737\text{M}$–$1.475\text{M}$ tokens) and Native `50,332,176` ($L=8, D=512, V=8,192, T=2,048$, $100$ steps $= 13.1\text{M}$ tokens), where every Delayed-WM run executes inside `with torch.no_grad():` (`assert not torch.is_grad_enabled()`) with zero reverse matrix pullbacks, zero VJP helpers, and zero same-step relays, we establish seven results within the short-horizon developmental regime tested:

- **Reconciled Stream-Partitioned Estimator Analysis & Scale-Normalized Controls (Proposition 1, `exp26`, `exp35`, `exp44`):** We reconcile the exact implemented contraction $\widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor}) = \frac{1}{N}\sum_{g=1}^G R_i^{(m, g)}(\text{scale\_factor}) Z_{i, :}^{(m, g)} = \frac{\text{scale\_factor}}{M_g} \widehat{G}_{i, :}^{\text{norm}}(G)$ across both the default code convention `scale_factor = 1` (whose conditional mean scales as $1/M_g$ and covariance as $1/M_g^2$, giving mean $0.5$ on $N=4, G=2, M_g=2, a_n=x_n=1$) and the scale-normalized convention `scale_factor = M_g` (whose conditional mean $\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n$ is invariant to $M_g$, giving $1.0$). We further distinguish **separable channels** (Stage 1: `mlp.c_fc, attn.c_v`, **Proposition 1(a–b)**, where first-order variance vanishes at $M_g = 1$) from **non-separable `RMSNorm` interfaces** (Stage 2: `x_final, x_0`, **Proposition 1(c)**), deriving the exact single-token ($M_g=1$) cross-coordinate variance $\frac{a_{n, i}^2 x_{n, i}^2 (D - x_{n, i}^2)}{D^2 \text{rms}(z_n)^2} \|h_n\|_2^2$ (giving $0.064$ at $D=2, z=(1, 2)$). Controlled 3-seed `scale_factor = M_g` runs (`exp35` on `d6` and `exp44` on Native `50.3M`) confirm that holding the conditional mean invariant to $M_g$ leaves learning curves virtually unchanged (`2.6097 -> 2.3361 -> 2.3008 -> 2.2338` for `scale_factor = M_g` vs. `2.5939 -> 2.3364 -> 2.2966 -> 2.2338` for `scale_factor = 1` across $G \in \{1, 64, 256, 2048\}$ on `d6`; `2.2539 ± 0.0013` vs. `2.2541 ± 0.0013` at $G=64$ under `Pre-Polar Muon`; and **`1.9350 ± 0.0003` for `scale_factor = 8.0` vs. `1.9358 ± 0.0002` for `scale_factor = 1.0` at $M_g=8, \text{lr\_embed}=3\text{e-}3$ on Native `50.3M`**, so along this tested intervention path **`96.9%` (`-0.0248` of `-0.0256` BPB) of the $M_g: 8 \to 1$ gain (`1.9350 -> 1.9102`) comes from eliminating $7/8$ within-stream cross-token interference terms**), establishing that variance reduction rather than raw step-scale changes drives the stream-partitioning gain.
- **Forward Causal Attention-Filtered Value Eligibility Trace (Proposition 2, `exp28`):** Because $v(b, s) = W_v x_{\text{attn\_in}}(b, s)$ commutes with causal attention over $s \le \tau$, computing the causal attention-filtered pre-synaptic trace $\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} x_{\text{attn\_in}}(b, s)$ during the forward pass and perturbing post-attention value outputs lifts single-step backward-free `attn.c_v` cosine alignment with the language-model cross-entropy loss gradient **$8.2\times$ from `+0.1104` to `+0.9069`**. Pairing $Z^{(2)}(t-1)$ with independent current-batch rewards $R^{(2)}(t)$ (`mismatched_origin_control`, `exp28`) degrades `d6` from `2.2338 ± 0.0102` to **`2.7608 ± 0.0056` DEV BPB (`-0.5270` BPB contrast)**.
- **Diagonal Skip-Routing Surrogate, Data-Dependent Kurtosis Expectation Bound, & Compliant Pure-`FP32` Clock/Optimizer Factorials (Propositions 3 & 4, `exp31`–`exp36`, `exp42`):** Routing lexical credit to `wte` via the diagonal skip surrogate $W_{\text{skip}} = \alpha_0 I_D$ (**Proposition 3**) cuts PyTorch `d6` rule time by **$27.4\%$ (`205.5 -> 149.2` ms/step)** with identical accuracy (`2.1579 ± 0.0004` vs. `2.1577 ± 0.0004`). For Strict Post-Installation Construction, **Proposition 4** decomposes $\delta W_{\text{lm}}(t-1)$ into weight decay, momentum history $\beta_1 m^{(t-2)}$, and current-batch gradient $(1-\beta_1) g^{(t-1)}$. Under row-isotropic preconditioning, reward-side projection $P_{x_{\text{final}}(n)}$ annihilates the $O(1)$ radial self-update vector $b_{\text{rad}}(n)$ identically, leaving an oblique `RMSNorm`-Jacobian coupling residual whose **expected** squared norm relative to $\|b_{\text{rad}}(n)\|_2^2$ is bounded by the **data-dependent coordinate fourth moment** $\frac{\mathbb{E}_{\xi_n}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} \le \frac{\kappa_4(x_{\text{final}}(n))}{D}$ where $\kappa_4(x) := \frac{1}{D}\sum_{i=1}^D x_i^4$. We show both that unrestricted sparse vectors $x = \sqrt{D/5}(1, 2, 0, \dots, 0)$ have non-vanishing expectation ratio $0.0576$ ($\kappa_4(x)/D = 0.68$) for all $D \ge 2$, and that on a 20-step head/block-0 diagnostic trajectory on ClimbMix `d6` (`exp35` Part 0), $\kappa_4(x_{\text{final}})$ has token mean $2.9398$, yielding mean theoretical expectation bound $\kappa_4(x_{\text{final}})/D = 0.007656$ and empirical mean ratio $0.007458$ (with individual token maximum $0.124439$ across $40,960$ token evaluations). Across our **16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer factorial (`exp34` + `exp35` Part B1)**, **4-cell True Both-Stages Pre-Install comparison (`exp35` Part B2)**, and **compliant zero-pullback pure-`FP32` trace-cache Triton WM clock controls (`exp42` Part A)**:
  1. **Isolating Stage-1 vs. Stage-2 Clock Transitions & Compliant Pure-`FP32` Cache Controls (`exp35` Part B2, `exp42` Part A):** Moving Stage 1 from `(t-2, t-1 [pre-Stage2], t)` (`both_stages_pre_install`) to `(t-2, t [post-Stage2], t)` (`stage2_pre_install`) shifts `d6` DEV BPB by only **`+0.0013` to `+0.0024` BPB** across all 4 `entrywise` cells (`2.1341 -> 2.1360` at `b05, 1e-8`; `2.1195 ± 0.0005 -> 2.1217 ± 0.0006` at `b05, 1e-4`), consistent with update-orthogonalized `Pre-Polar Muon` inducing only a `4.04%` (`cos = 0.999238`) Stage-1 reward drift. Moving Stage 2 from `(t-1, t-1, t)` to `(t-1, t [post-Stage3], t)` at `(entrywise, b05, 1e-8)` shifts DEV BPB by **`+0.0219 ± 0.0020` BPB** (`2.1360 -> 2.1579`). Eliminating `BF16` trace caching in a compliant zero-pullback Triton WM trainer (`reverse_error_matmuls_per_step = 0`, `exp42_fp32_triton_results.json`) reproduces the exact same 3-seed clock contrast (**`2.1359 ± 0.0022` vs. `2.1579 ± 0.0004` DEV BPB** at `b05, 1e-8`, and **`2.1274 ± 0.0002` DEV BPB** at `b09, 1e-4`), confirming that the `+0.0219` BPB shift is a genuine receiver-installation effect rather than a `BF16` cache-rounding artifact (while `exp36` provides matching local-pullback diagnostic controls).
  2. **Row-Isotropic vs. Entrywise Preconditioning (`Proposition 4`, `exp34`–`exp35`):** Across all 4 `row_iso` cells of the 16-cell cube, Stage-2 Pre-Install and Strict Post-Install agree to within **`0.0003` BPB** (`2.1577` vs. `2.1580`; `2.1662` vs. `2.1659`; `2.1296` vs. `2.1298`; `2.1335` vs. `2.1334`). Under `entrywise` moments, increasing $\epsilon: 10^{-8} \to 10^{-4}$ damps coordinate spread in $d_{j, i} = (\sqrt{\hat{v}_{j, i}} + \epsilon)^{-1}$ and improves Strict Post-Install by **`-0.0279 ± 0.0001` BPB** (`2.1579 -> 2.1300`), while $(\beta_1=0.9, \epsilon=10^{-4})$ reaches **`2.1273 ± 0.0002` DEV BPB** (`2.0900 ± 0.0001` VAL-B).
- **Fused Register-Tiled Triton WM Kernels (`38.4x` Kernel Speedup & `8.33x` Steady-State Step Speedup on `d6`; Isolated `1.44x`–`6.33x` Steady-State Step Speedup & `2.63x` Implementation-Package Speedup on Native `50.3M`, `exp37`, `exp43`, `exp44`):** Building on the SRAM-fused outgoing-column contraction concept introduced in [5] (version 5), we specialize fused register-tiled Triton kernels (`_triton_wm_reward_mg1_kernel` for $M_g=1$ and `_triton_wm_reward_mg_grouped_kernel` for $M_g > 1$) for our 3-stage delayed perturbation-differential WM pipeline. Eliminating the 3D outer-product workspace (`3,498.9 MB` on `d6` `lm_head` at $G=2,048$) cuts the 13-matrix `d6` WM kernel suite **`38.4x` from `134.31 ms` to `3.50 ms/step`** at $G=2,048$ (`cos = 1.000000`, `rel_l2 = 2.26e-8` on `mlp.c_proj`) and **`4.4x` from `4.51 ms` to `1.03 ms/step`** at $G=64$ (`cos = 0.9999973`, `rel_l2 = 2.31e-3`). Combined with batched `Pre-Polar Muon`, steady-state `d6` `fwd + rule` step time at $G=2,048$ drops **`8.33x` from `155.6 ms` to `18.68 ± 0.21 ms/step`** (`6.72s` steady-state estimate for 360 steps; `6.74s`–`6.95s` measured 3-seed training GPU time). On Native `50.3M` ($B_{\text{eff}}=64, T=2,048$), `exp32`'s PyTorch baseline (`compute_stream_wm_rewards_fast`) processes stream chunks (`chunk_G=256`, allocating `4.62 GB` temporary chunk workspace; or `chunk_G=32`, allocating `0.62 GB`). In **genuinely isolated kernel substitutions** (`benchmark_native50m_isolated_kernel.json`, measuring within-run steady-state variation across steps `2–4`) holding all storage, precision, trace construction, optimizer execution, and timing boundaries fixed: (i) inside `exp32`'s CPU-offloaded trainer ($M_g=8$, after a `3`-step CPU-heap warmup), swapping only `compute_stream_wm_rewards_fast(chunk_G=256)` $\to$ `compute_stream_wm_rewards_triton_fast` speeds up the steady-state (`steps 2–4`) learning rule **`1.60x` (`6,456.78 ± 0.64 -> 4,039.89 ± 0.20 ms/step`)** and steady-state `fwd + rule` **`1.44x` (`7,876.97 ± 0.65 -> 5,460.10 ± 0.52 ms/step`, or `1.50x` rule and `1.35x` `fwd + rule` across all `5` steps `0–4` including pipeline startup)** while reducing peak VRAM from **`9.108 GiB` to `7.629 GiB`**; and (ii) inside `exp43`'s GPU-resident trainer (`20.57–20.65 GiB` peak VRAM, `2` warmup + `3` timed steady-state steps `2–4`, where `chunk_G=256` OOMs), swapping only `compute_stream_wm_rewards_fast(chunk_G=32)` $\to$ `compute_stream_wm_rewards_triton_fast` speeds up the 288-call contraction **`6.86x` (`2,363.30 ± 0.72 -> 344.60 ± 0.74 ms`)** and `fwd + rule` **`1.67x` (`5,055.97 ± 1.30 -> 3,033.91 ± 1.36 ms`)** at $M_g=8$, and speeds up the 288-call contraction **`29.54x` (`18,317.09 ± 20.32 -> 620.01 ± 0.22 ms`)**, the learning rule **`9.91x` (`19,689.83 ± 20.99 -> 1,987.43 ± 0.59 ms`)**, and `fwd + rule` **`6.33x` (`21,022.47 ± 21.62 -> 3,319.91 ± 0.59 ms`)** at $M_g=1$. Across 100-step 3-seed training runs (`exp43`–`exp44`), our **combined implementation package** (combining the Triton kernels with GPU-resident traces + pinned MLP ring buffers, `BF16` SDPA and deferred Stage-1 attention-filtered input reconstruction from `BF16` cached inputs, batched Muon orthogonalization, and in-place scatter head scoring, trading **`9.11 GiB -> 20.57 GiB` peak VRAM**—or **`9.61–9.69 GiB` peak VRAM** when all 8 delayed trace tensors are placed in pinned host ring buffers in `exp44`) matches `exp32` in quality (`1.9358 ± 0.0002` vs. `1.9361 ± 0.0004` DEV BPB at $M_g=8$) with a **`2.63x` speedup on matched `fwd + rule` timed regions (`7,857.05 -> 2,991.84 ms/step`, `3.88x` on the learning rule `6,432.04 -> 1,659.19 ms`)**, and reaches **`1.9102 ± 0.0009` DEV BPB** at $M_g=1, \text{lr\_embed}=3\text{e-}3$ and **`1.8978 ± 0.0007` DEV BPB (`1.8844 ± 0.0003` VAL-B BPB, `208.39` PPL)** at $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-8}$ in `3.26 s/step` (`fwd + rule`; vs. `1.9266 ± 0.0004` at $\epsilon=10^{-4}$ in `exp44`).
- **Joint Co-Design of the Optimizer and Delayed WM Credit (`2.1273 -> 2.0682` DEV BPB at 360 Steps; `1.9989` at 720 Steps, `exp38`):** Adapting the `Pre-Polar Muon` foundation of [8] to delayed WM, we combine `Pre-Polar Muon` update orthogonalization with scale-matched sparse embedding velocity (`lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)`) and post-installation drift annealing (`lrm: 1.0 -> 0.2`) on `d6`. Across 3 seeds on `d6` (360 steps, Strict Post-Install), this optimizer package improves Delayed WM from `2.1273 ± 0.0001` to **`2.0682 ± 0.0006` DEV BPB (`2.0315 ± 0.0004` VAL-B BPB, `292.66` PPL)**—within **`0.0150` BPB** of matched-subset Exact Backpropagation (`2.0532 ± 0.0008`, which also improves from `2.1080 ± 0.0006` under the same optimizer package). On a Seed-46 Block-0 diagnostic audit (`run_geo_and_attn_steering_audit.py`), `Pre-Polar Muon` is descriptively associated with a higher `mlp.c_proj` weight stable rank than `AdamW` (`127.24` under `paper20_prepolar` and `141.33` under `codesigned_prepolar` vs. `2.75` under `adamw_hidden`), and the co-designed embedding/annealing schedule increases held-out attention-map Frobenius shift by $9.82\times$ in Layer 0 (`0.0838 -> 0.8233`) and $1.42\times$ across all 6 layers (`0.6046 -> 0.8587`).
- **Controlled Comparisons Against Author-Defined `SR-LNP` and `SR-NP` Node-Perturbation Recipes Across Tokens, Compute, and Width ($D \in \{256, 384, 512, 768\}$, `exp37`, `exp39`, `exp42`):** We compare Delayed WM against two author-defined node-perturbation recipes on the matched 4-matrix `d6` subset: (i) **Surrogate-Routed Local Node Perturbation (`SR-LNP`, $K \in \{4, 16, 64\}$)**, which uses same-step terminal-skip routing ($g_{\text{bus}} \cdot \gamma_l$) and $\gamma = 0$ local value scoring; and (ii) **Suffix-Recomputed Node Perturbation (`SR-NP`, `exp42`)**, which uses Gaussian draw-mean centering, suffix recomputation from block $l$ onward, shared post-MLP block-output perturbations for `attn.c_proj` and `mlp.c_proj`, analytic `lm_head` gradients, and $\gamma = 0.98$ future-token value accumulation:
  1. **Equal Tokens (`360` steps on `d6`, $D=384$):** `Delayed WM (K=1, G=2048)` (`2.0682 ± 0.0006` DEV BPB, `6.74s` measured training GPU time / `6.72s` steady-state est.) outperforms `SR-LNP` at $K=4$ (`2.3071 ± 0.0016`, `6.32s` run / `5.48s` est.), $K=16$ (`2.2603 ± 0.0018`, `9.43s` run / `9.23s` est.), and $K=64$ (`2.2041 ± 0.0013`, `23.66s` run / `23.22s` est.), as well as 76-draw `SR-NP` (`2.2415 ± 0.0015` under `Pre-Polar Muon`, `65.19s` run; `2.4264 ± 0.0088` under `Adam`, `84.38s` run).
  2. **Matched Compute (`~9.4s` and `~13.6s` on `d6`):** At `500` steps (`9.41s` run / `9.34s` est.), `Delayed WM` reaches **`2.0322 ± 0.0007` DEV BPB (`1.9975 ± 0.0006` VAL-B, `265.13` PPL)**. At `720` steps (`13.57s` run / `13.45s` est.), `Delayed WM` reaches **`1.9989 ± 0.0008` DEV BPB (`1.9646 ± 0.0008` VAL-B, `241.96` PPL)**.
  3. **Model Width Scaling ($D \in \{256, 384, 512, 768\}$, `exp39`):** As width $D$ scales from `256 -> 384 -> 512 -> 768`, `Delayed WM`'s gap to matched-subset Exact Backpropagation shrinks from `+0.0144` to **`+0.0125` BPB** (`2.0558 ± 0.0002` vs. `2.0433 ± 0.0006` at $D=768$).
- **Three-Tier Attention Credit-Assignment Hierarchy ($W_V$ vs. $W_Q$ vs. $W_K$), Decoupled 6-Matrix Joint $(W_Q, W_K)$ Routing, and Amortized Multi-Probe ($M=2$) Probing on ClimbMix `d6` (`2.0682 -> 2.0631 ± 0.0008` [$M=1$] & `2.0595 ± 0.0007` [$M=2$] DEV BPB, `exp45`, `exp47`) & Native `50.3M` (`1.8978 -> 1.8884 ± 0.0007` [$M=1$] & `1.8853 ± 0.0013` [$M=2$] DEV BPB / `201.21` PPL in `10.08–10.24 GiB` VRAM, `exp46`, `exp48`):** Extending our backward-free analysis to the attention score projections `attn.c_q` ($W_Q$) and `attn.c_k` ($W_K$) (`exp45`–`exp48`), we establish a **Three-Tier Attention Credit-Assignment Hierarchy** at $M_g = 1$. Conditioned on fixed pre-attention activations and oracle downstream $\nabla_{y_{\text{attn}}}\mathcal{L}$ (`gy_attn` from `loss.backward()` at initialized `seed = 46`, Layer 0) to isolate single-block attention-subspace transport from multi-layer delayed credit variance: (i) **Tier 1 (`attn.c_v`, $m_{\text{eff}} = 1$, coordinate-isolated & forward-factorizable via $\bar{x}_{\text{attn\_in}}^{(h)}$)** satisfies a conditional single-block reconstruction identity **`cos = 1.0000`** at $K=1$ (distinct from its `+0.9069` end-to-end delayed backward-free cosine in Table 1); (ii) **Tier 2 (`attn.c_q`, $m_{\text{eff}} = d_h$, token-local in time $\tau$ because query $q_\tau^{(h)}$ enters only row $\tau$ of $\mathbf{A}^{(h)}$, but coupled across the $d_h$ head channels inside $\langle q_\tau^{(h)}, k_s^{(h)}\rangle / \sqrt{d_h}$)** achieves empirical single-draw cosine **`cos = 0.1357`** at this initialized diagnostic state (comparable to $1/\sqrt{64} = 0.1250$), scaling with draw count $K$ to `0.2657` ($K=4$), `0.4748` ($K=16$), and `0.7385` ($K=64$); and (iii) **Tier 3 (`attn.c_k`, non-local across future queries $\tau \ge s$)** satisfies the conditional first-order expectation identity $\mathbb{E}[\widehat{g}_{k_s}^{(\gamma_K)}] = \sum_{\tau \ge s} \gamma_K^{\tau - s} J_{\tau s}^\top g_\tau + O(\sigma)$: multiplying `r_head` at position $s$ by $\xi_{k,s}$ at the *same* position $s$ ($\gamma_K = 0$) captures only the diagonal self-attention sensitivity $J_{ss}^\top g_s$ while truncating all future-query credit $\sum_{\tau > s} J_{\tau s}^\top g_\tau$ (`cos = 0.0076` at $K=1$, `0.0534` at $K=64$), whereas accumulating future query rewards $\widetilde{R}_{k,s}^{(\gamma_K)} = \sum_{\tau \ge s, \tau - s \le W_l} \gamma_K^{\tau - s} R_{\text{head},\tau}$ restores all future terms at $\gamma_K = 1$ (illustrated by our 2-token finite-difference Rademacher enumeration with `eps_fd = 1e-5`, `tol = 1e-9`) and trades a small exponential discount bias at $\gamma_K \in \{0.85, 0.98\}$ for lower cross-time key superposition variance (`K=64` cosine rises from `0.0534` at $\gamma_K=0$ to `0.1329` at $\gamma_K=0.85$, `0.1565` at $\gamma_K=0.98$, and `0.1340` at $\gamma_K=1.00$). On 5-matrix ClimbMix `d6` (`25.46M / 26.35M = 96.6%` active parameters, `exp45`), **Spherical Pre-Polar Muon** for `attn.c_q` ($\beta_Q = 0.95, \eta_Q = 0.048$, unit-RMS tangent perturbation, and spherical Frobenius retraction) improves Delayed WM from `2.0682 ± 0.0006` to **`2.0641 ± 0.0006` DEV BPB (`289.43` PPL)**, while both the tangent-projected Euclidean control (`wm_G2048_codesigned_euclidean_active_q_matched_hp_triton`: `spherical_qk=False, unit_rms_qk_pert=False, tangent_proj_qk=True`) and the unprojected raw-Rademacher Euclidean control (`wm_G2048_codesigned_unprojected_active_q_matched_hp_triton`: `tangent_proj_qk=False, spherical_qk=False, unit_rms_qk_pert=False`) at matched $\beta_Q=0.95, \eta_Q=0.048$ reach **`2.0649 ± 0.0006` DEV BPB** (isolating `-0.0033` BPB from gain/momentum matching and `-0.0008` BPB from spherical retraction + unit-RMS perturbation). Extending to **full 6-matrix (`100%` active parameters)** joint $(W_Q, W_K)$ learning in `exp47` (`exp47_d6_6mat_joint_qk_results.json`) resolves the `2.0669` coupled same-position failure via **decoupled forward query/key displacement probes** ($\Delta y_q$ and $\Delta y_k$, eliminating cross-probe variance contamination for `-0.0014` BPB) and **discounted future-query key credit** ($\gamma_K = 0.85, \eta_K = 0.012, \beta_K = 0.98$, adding `-0.0015` BPB) to reach **`2.0631 ± 0.0008` DEV BPB (`2.0264 ± 0.0011` VAL-B, `288.64` PPL)** at $M=1$, and **`2.0596 ± 0.0008` DEV BPB (`2.0228 ± 0.0006` VAL-B, `285.87` PPL)**—or **`2.0595 ± 0.0007` DEV BPB (`2.0227 ± 0.0008` VAL-B, `285.81` PPL)** with an isolated RNG generator—under **amortized $M=2$ local multi-probe averaging** (which averages $M=2$ local query/key Rademacher probes inside $\bar{u}^{(m)}$ on the same clean forward pass prior to the single Stage-3 `bmm` and Muon step, cutting `d6` BPB by `-0.0087` vs. 4-matrix WM and narrowing the gap to 4-matrix Exact BP `2.0532` to `+0.0063` BPB). Finally, at a fixed 100-step ($13.1\text{M}$-token) budget on long-context **Native `50,332,176`** ($L=8, D=512, T=2,048, M_g=1$) with pinned host ring buffers, deterministic on-the-fly perturbation regeneration, and `BF16`-approximate inverse-RoPE query/key recovery: 5-matrix Spherical Active-$W_Q$ (`exp46`, `95.8%` active parameters) reaches **`1.8897 ± 0.0008` DEV BPB (`1.8765 ± 0.0013` VAL-B, `203.72` PPL)** in `6,012.9 ms/step` (`2,195.2 ms` fwd + `3,817.7 ms` rule) and `9.80 GiB` VRAM; and **full 6-matrix (`99.999%` active parameters) Decoupled Active-$(W_Q, W_K)$ (`exp48`)** reaches **`1.8884 ± 0.0007` DEV BPB (`1.8753 ± 0.0006` VAL-B, `202.96` PPL)** at $M=1$ in `5,116.3 ms/step` (`2,081.1 ms` fwd + `3,035.3 ms` rule) and `10.08 GiB` VRAM, and **`1.8853 ± 0.0013` DEV BPB (`1.8722 ± 0.0012` VAL-B, `201.21` PPL)** at $M=2$ in `5,806.4 ms/step` (`2,424.5 ms` fwd + `3,381.9 ms` rule) and `10.24 GiB` VRAM—a fixed-token accuracy improvement of **`-0.0125` BPB (`-7.18` PPL)** over 4-matrix `exp43` (`1.8978 ± 0.0007`, `3,262.1 ms/step`) and **`-0.0508` BPB (`-30.93` PPL)** over `exp32` (`1.9361 ± 0.0004`, `7,857.1 ms/step`).

---

## 2. Related Work and Methodological Distinctions

**Weight Maximization, Unbiased WM, and Node-Perturbation Methods (Dust).** Chung [1] introduced Weight Maximization (WM) for networks of self-interested reinforcement learning agents (*Learning by Competition of Self-Interested Reinforcement Learning Agents*, arXiv:2010.09770), proving that when each unit modulates its eligibility trace by the change in squared Euclidean norm of its outgoing weights, expected updates follow the gradient of downstream weight magnitude without backward error transport. Chung [2] subsequently formulated *Unbiased Weight Maximization* (UWM, arXiv:2307.13270) for Bernoulli-logistic units using an unbiased outgoing-weight product estimator. These external papers establish the original WM learning principle; they do not provide an inherited convergence guarantee for our multi-token perturbation-differential surrogate or diagonal skip routing. Recently, Dahal et al. [9] introduced *Dust* (*Pretraining Transformers Without Backpropagation*, QLabs, October 2026), which trains Transformer language models without backpropagation by injecting $K$ tokenwise Gaussian activation perturbation draws across separate layer passes, recomputing the affected network suffix from cached prefixes, rewarding direct layers via draw-centered token-loss reductions ($\gamma = 0$), and rewarding attention internals (`Section 2.3` of [9]) via per-head local attention-output alignment—scoring queries token-locally ($\gamma = 0$) because each query affects only its own causal row, and scoring keys/values by accumulating future query-token rewards ($\gamma = 0.98$, plus $49\text{k}$–$65\text{k}$ vocabulary-slab head draws and ES on residual scalars). We explicitly credit Dust [9] (`Section 2.3`) for the per-head attention-output reward formulation, the token-local query vs. future-token key/value distinction, and $\gamma$-discounted future-token key/value reward accumulation. Because our node-perturbation comparators and baseline experiments train the 4-matrix subset (`mlp.c_fc, mlp.c_proj, attn.c_v, attn.c_proj` plus `wte, value_embeds, lm_head`) at small per-step draw counts rather than replicating [9]'s 65k-head-slab, $K \in [64, 16384]$ suffix-recomputation protocol, **we do not claim head-to-head superiority over published Dust [9]**. Instead, we evaluate two author-defined node-perturbation comparators on our matched 4-matrix architecture: (i) **Surrogate-Routed Local Node Perturbation (`SR-LNP`, `exp37`–`exp39`)**, which shares the terminal-skip residual factors ($\gamma_l, \alpha_0$) of our Delayed WM pipeline but differs in credit timing (constructing $\hat{g}_{\text{bus}}$ and immediately using it for same-step embedding and hidden-layer estimates rather than 3-stage delayed packets), perturbation location, and attention-credit treatment (scoring value perturbations locally with $\gamma = 0$ rather than via the forward attention-filtered trace); and (ii) **Suffix-Recomputed Node Perturbation (`SR-NP`, `exp42`)**, an author-defined suffix-recomputed recipe that uses Gaussian draw-mean centering, suffix recomputation from block $l+1$ onward, shared post-MLP block-output perturbations $\hat{g}_{\text{wr}}$ for both `attn.c_proj` and `mlp.c_proj` (which omits `attn.c_proj`'s path through the same block's MLP), an analytic local loss gradient on `lm_head`, and $\gamma = 0.98$ future-token accumulation on per-head attention-output alignment.

**Concurrent rStation Studies on Forward-Only and Delayed Transformer Credit Assignment.** Two concurrent rStation papers independently investigate delayed WM on `general_pretraining`:
- In `general_pretraining_18/v7`–`v8`, `cartan_transport` [5] independently develops token micro-groups (stream partitioning), column-isolated WM rewards, the replacement of sibling-unit interference by within-group token interference, exact symmetrized unordered-pair conditional covariance formulas, and SRAM-fused outgoing-column contractions (`Delayed-Trace-Co-WM-S`) combined with $P=64$ and $P=256$ suffix forward probes for attention on ClimbMix `d6` (`2.04412 ± 0.00014` BPB at $P=64$ in $633.6\text{s}/635.2\text{s}$, `2.08637 ± 0.01282` at $P=256$, and `2.14471 ± 0.00464` at $1,000$ steps), 2-seed (`s42` and `s137`) 200-step recency-weighted value-pooling diagnostics (`2.14681` and `2.13220` BPB at $\gamma = 1.00$, `2.13081` at $\gamma = 0.98$), and (in `v8`) $P$-probe local attention credit assignment with token-local query scoring ($\gamma = 0$) and discounted future-token key/value accumulation ($\gamma = 0.98$). Likewise, [8] (`general_pretraining_12/v2`) supplies the `Pre-Polar Muon` optimizer foundation. We explicitly credit [5] for the micro-group, symmetrized pair-covariance, SRAM-fused outgoing-column contraction, and concurrent local attention-probe foundations and [8] for `Pre-Polar Muon`. Our distinct contributions are: (i) reconciling the code-level $1/M_g$ normalization with 3-seed scale-normalized controls on both `d6` (`exp35`) and Native `50.3M` (`exp44`) and deriving the single-token cross-coordinate variance at non-separable `RMSNorm` interfaces (**Proposition 1(a–c)**); (ii) training `attn.c_v` via a **forward causal attention-filtered pre-synaptic trace** $\bar{x}_{\text{attn\_in}}^{(h)}$ (**Proposition 2**) without multi-probe value suffix sweeps; (iii) routing lexical credit across the diagonal skip surrogate $W_{\text{skip}} = \alpha_0 I_D$ (**Proposition 3**); (iv) isolating Stage-1 vs. Stage-2 post-installation drift across our 16-cell + 4-cell + compliant pure-`FP32` clock/optimizer factorials (**Proposition 4**, `exp32`–`exp35`, `exp42`); and (v) specializing register-tiled Triton kernels (`_triton_wm_reward_mg1_kernel` and `_triton_wm_reward_mg_grouped_kernel`), optimizer co-design, and **full 6-matrix decoupled $(W_Q, W_K)$ delayed WM with amortized $M \in \{1, 2\}$ local probing** to our 3-stage delayed pipeline (`2.0631 ± 0.0008` [$M=1$] and `2.0595 ± 0.0007` [$M=2$] DEV BPB at 360 steps on `d6` in `exp47`, and `1.8884 ± 0.0007` [$M=1$] and `1.8853 ± 0.0013` [$M=2$] DEV BPB at 100 steps on Native `50.3M` in `exp48`).
- In `general_pretraining_19/v4`–`v6`, `residual_cartographer` [6] analyzes short residual writer/reader shortcuts, two-world delayed WM (`1.98964` DEV BPB at $499.65\text{M}$ targets, `2.00286` at $134.2\text{M}$ targets on Native `50.3M` in `v4`), sequence-wide incoming-weight actions with previous-origin activity directions (`2.1184/2.0975` DEV BPB at $134.2\text{M}$ targets on Native `50.3M` in `v5`), and four-world role coding vs. two-world action persistence across depths 8 and 12 (`v6`). Because [5], [6], and [9] use different token budgets and schedules than our studies, raw BPB endpoints across [5], [6], and [9] are not direct head-to-head comparisons; rather, our work complements [6] by commuting $W_v^{(h)}$ with causal attention during the forward pass (**Proposition 2**), characterizing post-installation drift (**Proposition 4**), and testing `residual_cartographer`'s realized-component reader hypothesis in `exp36` Part B.

---

## 3. Mathematical Formulation and Estimator Analysis

### 3.1 Three-Stage Residual Pipeline

Consider an $L$-layer causal Transformer with width $D$, vocabulary size $V$, token embedding $e_{\text{tok}}(x) = W_{\text{wte}}[x]$, initial normalized state $x_0 = \text{RMSNorm}(e_{\text{tok}}(x))$, per-layer residual recurrence $x_{\text{in}}^{(l)} = \lambda_{\text{res}}^{(l)} x_{\text{out}}^{(l-1)} + \lambda_{x_0}^{(l)} x_0$, attention and MLP sublayers writing additively via $W_{\text{attn\_proj}}^{(l)} \in \mathbb{R}^{D \times D}$ and $W_{\text{mlp\_proj}}^{(l)} \in \mathbb{R}^{D \times 4D}$, terminal state $x_{\text{final}} = \text{RMSNorm}(x_L)$, and softcapped readout logits $z = 15 \tanh(W_{\text{lm}} x_{\text{final}} / 15)$ with $W_{\text{lm}} \in \mathbb{R}^{V \times D}$. Unrolling the linear residual scalings $\lambda_{\text{res}}^{(l)}, \lambda_{x_0}^{(l)}$ yields the exact terminal representation
$$
x_L = \alpha_0 x_0 + \sum_{l=0}^{L-1} \gamma_l \left(W_{\text{attn\_proj}}^{(l)} y_{\text{attn}}^{(l)} + W_{\text{mlp\_proj}}^{(l)} h_{\text{mlp}}^{(l)}\right), \qquad \gamma_l = \prod_{m=l+1}^{L-1} \lambda_{\text{res}}^{(m)}, \quad \alpha_0 = \prod_{m=0}^{L-1} \lambda_{\text{res}}^{(m)} + \sum_{l=0}^{L-1} \lambda_{x_0}^{(l)} \gamma_l.
$$
Exploiting these direct additive write paths into $x_L$, we partition all trainable parameters into three causal stages with maximum pipeline delay $2$ independent of depth $L$:
- **Stage 3 (`Delay = 0`, Terminal Readout):** $W_{\text{lm}} \in \mathbb{R}^{V \times D}$.
- **Stage 2 (`Delay = 1`, Residual Stream Writers):** $W_{\text{wte}} \in \mathbb{R}^{V \times D}$, $W_{\text{mlp\_proj}}^{(l)} \in \mathbb{R}^{D \times 4D}$, and $W_{\text{attn\_proj}}^{(l)} \in \mathbb{R}^{D \times D}$ for $l \in \{0, \dots, L-1\}$, whose outgoing receiver columns are $W_{\text{lm}}[:, i] \in \mathbb{R}^V$.
- **Stage 1 (`Delay = 2`, Internal Sublayer Writers):** $W_{\text{mlp\_fc}}^{(l)} \in \mathbb{R}^{4D \times D}$, $W_{v}^{(l)} \in \mathbb{R}^{D \times D}$, $W_{\text{ve}}^{(l)} \in \mathbb{R}^{V \times D}$, and (in our 5-matrix and 6-matrix configurations, `exp45`–`exp48`) $W_{q}^{(l)}, W_{k}^{(l)} \in \mathbb{R}^{D \times D}$, whose outgoing receiver columns are $W_{\text{mlp\_proj}}^{(l)}[:, p] \in \mathbb{R}^D$ and $W_{\text{attn\_proj}}^{(l)}[:, p] \in \mathbb{R}^D$.

### 3.2 Reconciled Code Contraction, Separable Pair Covariance, and `RMSNorm` Interface Moments (`Proposition 1`)

Let $N = BT$ denote the total number of tokens in a batch, and partition $\{1, \dots, N\} = \bigsqcup_{g=1}^G \mathcal{S}_g$ into $G$ disjoint streams of size $M_g = N / G$ tokens each. During the forward pass, unit $i$ at stage $m$ injects i.i.d. Rademacher exploration noise $\xi_{n, i} \sim \text{Unif}(\{\pm 1\})$ with scale $\sigma > 0$, and records the stream eligibility trace $Z_{i, :}^{(m, g)} = \sum_{n \in \mathcal{S}_g} (\xi_{n, i}/\sigma) x_n \in \mathbb{R}^{D_{\text{in}}}$.

In `compute_stream_wm_rewards_chunked` (`exp26`) and `compute_stream_wm_rewards_fast` (`exp32`–`exp35`), the stream reward divides by $M_g$ with an optional multiplier `scale_factor` (`mult = scale_factor / M_g`):
$$
R_i^{(m, g)}(\text{scale\_factor}) = \frac{\text{scale\_factor}}{M_g} \sum_{n' \in \mathcal{S}_g} u_{n'}^{(m+1), T} W_{:, i}^{(m+1)}(t_{\text{constr}}) \, \Delta y_{n', i}^{(m)},
$$
and the subsequent parameter update divides the tokenwise modulation–input product $\sum_{g=1}^G R_i^{(m, g)} Z_{i, :}^{(m, g)}$ by $N = BT$ (`-(mod_flat.t() @ x_flat) / BT`). Ignoring the descent sign, the **implemented contraction in code** is therefore
$$
\widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor}) := \frac{1}{N} \sum_{g=1}^G R_i^{(m, g)}(\text{scale\_factor}) \, Z_{i, :}^{(m, g)} = \frac{\text{scale\_factor}}{M_g} \widehat{G}_{i, :}^{\text{norm}}(G), \qquad \text{where} \qquad \widehat{G}_{i, :}^{\text{norm}}(G) := \frac{1}{G} \sum_{g=1}^G R_i^{(m, g)}(1) \, Z_{i, :}^{(m, g)}.
$$
In particular, the default setting `scale_factor = 1` (`exp26`–`exp34`) computes $\widehat{G}_{i, :}^{\text{code}}(G; 1) = \frac{1}{M_g}\widehat{G}_{i, :}^{\text{norm}}(G)$, whereas setting `scale_factor = M_g` (`exp35`) computes $\widehat{G}_{i, :}^{\text{code}}(G; M_g) = \widehat{G}_{i, :}^{\text{norm}}(G)$, and at $G = N$ ($M_g = 1$) both coincide identically.

Furthermore, we must distinguish **separable channels**—where coordinate $i$'s forward displacement depends only on $\xi_{n, i}$, as holds at **Stage 1** (`mlp.c_fc` with elementwise $\text{ReLU}^2$ and `attn.c_v` with linear head-value projection)—from **non-separable `RMSNorm` interfaces** at **Stage 2** (`x_final = RMSNorm(x_L)` and `x_0 = RMSNorm(e_tok)`), where $\text{rms}(z_n) = \sqrt{\frac{1}{D}\sum_{i'=1}^D z_{n, i'}^2}$ couples coordinate $i$ to $\xi_{n, i'}$ for $i' \neq i$.

**Proposition 1 (Conditional Moments of $\widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor})$ for Separable Channels and `RMSNorm` Interfaces).** *Conditioned on fixed pre-synaptic inputs $\{x_n\}_{n=1}^N \subset \mathbb{R}^{D_{\text{in}}}$ and downstream weights/modulations independent of the i.i.d. Rademacher draws $\xi_{n, i} \sim \text{Unif}(\{\pm 1\})$:*
1. **(a) Separable-Channel Conditional Mean (Stage 1: `mlp.c_fc, attn.c_v`):** *Suppose $\Delta y_{n, i}^{(m)} = \sigma c_{n, i} \xi_{n, i} + r_{n, i}(\sigma)$ with $c_{n, i}$ independent of $\xi_n$ and $r_{n, i}(\sigma) = O(\sigma^2)$, and let $a_{n, i} := c_{n, i} \sum_{j=1}^{d_{\text{next}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) u_{n, j}^{(m+1)}$. Then the first-order code estimator $\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})$ satisfies for every $G \in \{1, \dots, N\}$ ($M_g = N / G$):*
   $$
   \mathbb{E}_{\xi}\left[\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})\right] = \frac{\text{scale\_factor}}{M_g} \left(\frac{1}{N} \sum_{n=1}^N a_{n, i} x_n\right).
   $$
   *Specifically, under `scale_factor = 1`, the conditional mean scales as $\frac{1}{M_g}\left(\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n\right)$ (evaluating to $0.5$ on $N=4, G=2, M_g=2, a_n = x_n = 1$), whereas under `scale_factor = M_g`, the conditional mean is $\mathbb{E}_{\xi}[\widehat{G}_{i, :}^{\text{norm}, (1)}(G)] = \frac{1}{N}\sum_{n=1}^N a_{n, i} x_n$ for all $G \in \{1, \dots, N\}$ (evaluating to $1.0$ on $N=4, G=2, M_g=2, a_n = x_n = 1$).*
2. **(b) Separable-Channel Symmetrized Pair Covariance ([5], Section 2.3):** *Under the separable channel model of (a), the exact conditional covariance matrix of $\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})$ is*
   $$
   \text{Cov}_{\xi}\left(\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})\right) = \left(\frac{\text{scale\_factor}}{M_g}\right)^2 \frac{1}{N^2} \sum_{g=1}^G \sum_{\substack{n, n' \in \mathcal{S}_g \\ n < n'}} \left(a_{n', i} x_n + a_{n, i} x_{n'}\right)\left(a_{n', i} x_n + a_{n, i} x_{n'}\right)^T,
   $$
   *which sums over the $G\binom{M_g}{2} = \frac{N(M_g - 1)}{2}$ unordered within-stream token pairs and vanishes identically at $G = N$ ($M_g = 1$).*
3. **(c) Non-Separable `RMSNorm` Interface Moments at $M_g = 1$ (Stage 2: `x_final, x_0`):** *At an `RMSNorm` interface $x_n = \text{RMSNorm}(z_n) = z_n / \text{rms}(z_n)$ ($z_n \in \mathbb{R}^D$), tangent-projected or raw Rademacher perturbation yields the first-order forward displacement $\Delta x_n^{(1)} = \frac{\sigma}{\text{rms}(z_n)} P_{x_n} \xi_n$, whose $i$-th coordinate is $\Delta x_{n, i}^{(1)} = \frac{\sigma}{\text{rms}(z_n)}\left((1 - x_{n, i}^2/D)\xi_{n, i} - \frac{x_{n, i}}{D}\sum_{i' \neq i} x_{n, i'}\xi_{n, i'}\right)$. Consequently, even at $G = N$ ($M_g = 1$, where cross-token pairs $n < n'$ are absent), for a single token $n$ with receiver sensitivity $a_{n, i} = \sum_j W_{j, i}^{(m+1)} u_{n, j}^{(m+1)}$ and pre-synaptic input $h_n \in \mathbb{R}^{D_{\text{in}}}$, the first-order unprojected coordinate-$i$ estimator is*
   $$
   \hat{g}_{n, i}^{(1)} := a_{n, i} \frac{\Delta x_{n, i}^{(1)} \xi_{n, i}}{\sigma} h_n = \frac{a_{n, i}}{\text{rms}(z_n)} \left(1 - \frac{x_{n, i}^2}{D} - \frac{x_{n, i}}{D} \sum_{i' \neq i} x_{n, i'} \xi_{n, i} \xi_{n, i'}\right) h_n,
   $$
   *which has exact conditional mean and nonzero single-token **cross-coordinate** covariance*
   $$
   \mathbb{E}_{\xi_n}\left[\hat{g}_{n, i}^{(1)}\right] = \frac{a_{n, i}}{\text{rms}(z_n)}\left(1 - \frac{x_{n, i}^2}{D}\right) h_n, \qquad \text{Cov}_{\xi_n}\left(\hat{g}_{n, i}^{(1)}\right) = \frac{a_{n, i}^2 x_{n, i}^2 (D - x_{n, i}^2)}{D^2 \, \text{rms}(z_n)^2} h_n h_n^T.
   $$
   *For example, for a single token ($N=G=M_g=1$) at $D=2$ with $z_1 = (1, 2)$, $a_{1, 1} = 1$, and $h_1 = 1$, we have $\text{rms}(z_1) = \sqrt{2.5}$, $x_{1, 1}^2 = 0.4$, $x_{1, 1} x_{1, 2} = 0.8$, and $\hat{g}_{1, 1}^{(1)} = \frac{0.8 - 0.4 \xi_{1, 1}\xi_{1, 2}}{\sqrt{2.5}}$, with mean $0.8/\sqrt{2.5} \approx 0.5060$ and variance $\frac{0.4(2 - 0.4)}{4 \times 2.5} = 0.064$.*

*Proof.* For (a–b), expanding $\widehat{G}_{i, :}^{\text{norm}, (1)}(G) = \frac{1}{N}\sum_{g=1}^G (\sum_{n' \in \mathcal{S}_g} a_{n', i}\xi_{n', i})(\sum_{n \in \mathcal{S}_g} \xi_{n, i} x_n)$ using $\xi_{n, i}^2 \equiv 1$ gives $\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n + \frac{1}{N}\sum_{g=1}^G \sum_{n < n' \in \mathcal{S}_g} \eta_{\{n, n'\}, i} (a_{n', i} x_n + a_{n, i} x_{n'})$ where $\eta_{\{n, n'\}, i} = \xi_{n, i}\xi_{n', i}$ are pairwise uncorrelated with zero mean and unit variance; multiplying by $\text{scale\_factor} / M_g$ scales the mean by $\text{scale\_factor} / M_g$ and the covariance by $(\text{scale\_factor} / M_g)^2$. For (c), since $\xi_{n, i}^2 = 1$, multiplying $(P_{x_n}\xi_n)_i = (1 - x_{n, i}^2/D)\xi_{n, i} - \frac{x_{n, i}}{D}\sum_{i' \neq i} x_{n, i'}\xi_{n, i'}$ by $\xi_{n, i}$ yields the displayed expression for $\hat{g}_{n, i}^{(1)}$. Because $\{\xi_{n, i}\xi_{n, i'}\}_{i' \neq i}$ are $D - 1$ pairwise uncorrelated zero-mean Rademacher products with unit variance and $\sum_{i' \neq i} x_{n, i'}^2 = \|x_n\|_2^2 - x_{n, i}^2 = D - x_{n, i}^2$, the mean and covariance follow immediately. $\square$

### 3.3 Forward Causal Attention-Filtered Value Eligibility Trace (`Proposition 2`)

In causal multi-head self-attention with $H$ heads of dimension $D_h = D / H$, head $h$'s output at batch $b$, query token $\tau \in \{0, \dots, T-1\}$ is $y_{\text{attn}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} v^{(h)}(b, s)$, where $v^{(h)}(b, s) = W_v^{(h)} x_{\text{attn\_in}}(b, s)$ ($W_v^{(h)} \in \mathbb{R}^{D_h \times D}$) and $A_{b, h, \tau, s}$ is the causal attention weight ($\sum_{s \le \tau} A_{b, h, \tau, s} = 1$, $A_{b, h, \tau, s} = 0$ for $s > \tau$).

**Proposition 2 (Exact Commutation of $W_v^{(h)}$ with Causal Attention).** *Because $W_v^{(h)} \in \mathbb{R}^{D_h \times D}$ is shared across sequence positions $s \in \{0, \dots, T-1\}$, the post-attention value aggregation at position $\tau$ satisfies the exact linear identity*
$$
y_{\text{attn}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} \left(W_v^{(h)} x_{\text{attn\_in}}(b, s)\right) = W_v^{(h)} \underbrace{\left(\sum_{s \le \tau} A_{b, h, \tau, s} x_{\text{attn\_in}}(b, s)\right)}_{\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau) \in \mathbb{R}^D}.
$$
*Consequently, injecting exploration noise $\psi_v^{(h)}(b, \tau) \sim \text{Unif}(\{\pm 1\}^{D_h})$ at the post-attention value output $y_{\text{attn}}^{(h)}(b, \tau)$ and recording the **forward causal attention-filtered pre-synaptic trace** $Z_{v}^{(h, g)}(t-2) = \sum_{(b, \tau) \in \mathcal{S}_g} (\psi_v^{(h)}(b, \tau)/\sigma_1) \, \bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)^T$ yields an exact single-position linear channel $y_{\text{attn}}^{(h)}(b, \tau) = W_v^{(h)} \bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)$ at each output token $\tau$, eliminating causal cross-position perturbation mixing with zero reverse-time pooling.*

### 3.4 Diagonal Identity-Skip Surrogate & Post-Installation Credit Decomposition (`Propositions 3 & 4`)

**Proposition 3 (Algebraic Diagonal Skip-Routing Surrogate for `wte`).** *In the unrolled residual equation $x_L = \alpha_0 x_0 + \dots$, the direct path from $x_0 = \text{RMSNorm}(W_{\text{wte}}[x])$ to $x_L$ is the fixed diagonal scaling $W_{\text{skip}} = \alpha_0 I_D$. Treating $W_{\text{skip}} = \alpha_0 I_D$ as an algebraic surrogate receiver matrix with post-synaptic modulation $s_{\text{bus}}(n) = \text{mod\_bus\_base}(n) \in \mathbb{R}^D$ and pre-synaptic forward displacement $\Delta x_0(n) = \text{RMSNorm}(e_{\text{tok}}(n) + \sigma \tilde{\xi}_{\text{wte}}(n)) - x_0(n)$, the column-norm contraction reduces to the $O(BT \cdot D)$ elementwise contraction*
$$
R_{\text{wte}, i}^{\text{raw}, (g)}(\text{scale\_factor}) = \sum_{j=1}^D (\alpha_0 \delta_{j, i}) \left(\frac{\text{scale\_factor}}{M_g}\sum_{n \in \mathcal{S}_g} s_{\text{bus}, n, j} \, \Delta x_{0, n, i}\right) = \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} \left(\alpha_0 s_{\text{bus}, n, i}\right) \Delta x_{0, n, i},
$$
*(followed in `compute_identity_skip_wm_rewards_fast` by stream-mean centering $R_{\text{wte}, i}^{(g)} = R_{\text{wte}, i}^{\text{raw}, (g)} - \frac{1}{G}\sum_{g'=1}^G R_{\text{wte}, i}^{\text{raw}, (g')}$), eliminating the second $O(G V D)$ `lm_head` contraction.*

Finally, we analyze **Strict Post-Installation Construction** (`Stage 2: (t-1, t [post-Stage3-step], t)`), where Stage 2 evaluates `compute_stream_wm_rewards_fast` at clock $t$ on $W_{\text{lm}}(t) = W_{\text{lm}}(t-1) + \delta W_{\text{lm}}(t-1)$ after Stage 3 has installed its batch-$t-1$ update. In our implementation (`custom_entrywise_adamw_step` and `row_isotropic_adamw_step`), at optimizer step $t_{\text{step}}$, the installed Stage-3 displacement is
$$
\delta W_{\text{lm}, j, i}(t-1) = -\eta_3 \lambda_{\text{wd}} W_{\text{lm}, j, i}(t-1) - \frac{\eta_3}{1 - \beta_1^{t_{\text{step}}}} \cdot d_{j, i} \left(\beta_1 m_{j, i}^{(t-2)} + (1 - \beta_1) g_{j, i}^{(t-1)}\right), \qquad d_{j, i} := \frac{1}{\sqrt{\hat{v}_{j, i}^{(t-1)}} + \epsilon},
$$
where $g_{j, i}^{(t-1)} = -\frac{1}{BT}\sum_{n'=1}^{BT} s_{\text{lm}, n', j} x_{\text{final}, n', i}$, $\hat{v}_{j, i}^{(t-1)} = v_{j, i}^{(t-1)} / (1 - \beta_2^{t_{\text{step}}})$ under `entrywise` moments (or row-averaged $d_{j, i} \equiv d_j = (\sqrt{\hat{v}_j^{(t-1)}} + \epsilon)^{-1}$ with $\hat{v}_j^{(t-1)} = \frac{1}{D}\sum_{i=1}^D \hat{v}_{j, i}^{(t-1)}$ under `row_iso`), and $\epsilon > 0$ is added *outside* the square root (`denom = sqrt(v_hat) + eps`). At $G = BT$ ($M_g = 1$), let $u_{\text{bus}, i}^{\text{pre}}(n) = R_{\text{bus}, i}^{\text{pre}}(n) (\xi_{n, i}/\sigma)$ and $u_{\text{bus}, i}^{\text{post}}(n) = R_{\text{bus}, i}^{\text{post}}(n) (\xi_{n, i}/\sigma)$ denote the unprojected Stage-2 credit before and after installing $\delta W_{\text{lm}}(t-1)$.

**Proposition 4 (Decomposition of Installed-Update Terminal Credit Shift, Data-Dependent Fourth-Moment Bound, and Preconditioning Regimes).** *Under Strict Post-Installation Construction at $G = BT$ ($M_g = 1$), the post-minus-pre unprojected credit shift $\delta u_{\text{bus}}(n) := u_{\text{bus}}^{\text{post}}(n) - u_{\text{bus}}^{\text{pre}}(n)$ decomposes into three additive components corresponding to decoupled weight decay, first-moment history $m^{(t-2)}$, and current-batch gradient $g^{(t-1)}$:*
$$
\delta u_{\text{bus}, i}(n) = -\eta_3 \lambda_{\text{wd}} u_{\text{bus}, i}^{\text{pre}}(n) + \delta u_{\text{mom}, i}(n) + \delta u_{\text{cur}, i}^{(n' = n)}(n) + \delta u_{\text{cur}, i}^{(n' \neq n)}(n),
$$
*where:*
1. **Same-Token ($n' = n$) `RMSNorm` Radial-vs-Oblique Split and Data-Dependent Fourth-Moment Bound:** *With tangent-projected forward carrier $\tilde{\xi}_n = P_{x_{\text{final}}(n)} \xi_n$ and raw carrier multiplication $\xi_n / \sigma$, the first-order same-token ($n'=n$) term is $\delta u_{\text{cur}, i}^{(n'=n, 1)}(n) = \frac{\eta_3 (1-\beta_1)}{(1-\beta_1^{t_{\text{step}}}) BT} c_{n, i} \, b_i(n)$, where $c_{n, i} := \sum_{j=1}^V d_{j, i} s_{\text{lm}, n, j}^2$ and the vector $b(n)$ splits into an $O(1)$ radial vector $b_{\text{rad}}(n) \parallel x_{\text{final}}(n)$ and an oblique `RMSNorm`-Jacobian coupling term $b_{\text{oblique}}(n)$:*
   $$
   b(n) := \frac{1}{\text{rms}(z_n)}\left(x_{\text{final}}(n) \odot \xi_n \odot (P_{x_{\text{final}}(n)} \xi_n)\right) = \underbrace{\frac{1}{\text{rms}(z_n)} x_{\text{final}}(n)}_{b_{\text{rad}}(n) \parallel x_{\text{final}}(n)} - \underbrace{\frac{\langle \xi_n, x_{\text{final}}(n)\rangle}{D \, \text{rms}(z_n)} \left(x_{\text{final}}(n)^{\odot 2} \odot \xi_n\right)}_{b_{\text{oblique}}(n)}.
   $$
   *Under **row-isotropic preconditioning** ($d_{j, i} \equiv d_j$, so $c_{n, i} \equiv c_n$ is a scalar), reward-side projection $P_{x_{\text{final}}(n)}$ annihilates $c_n b_{\text{rad}}(n)$ identically ($P_{x_{\text{final}}(n)} b_{\text{rad}}(n) = 0$) while leaving $P_{x_{\text{final}}(n)} b_{\text{oblique}}(n) \neq 0$ (e.g., at $D=2, z_n=(1, 2), \xi_n=(1, 1)$, $b(n)=(0.16, -0.16)$, $b_{\text{rad}}(n)=(0.4, 0.8)$, and $P_{x_{\text{final}}(n)} b(n) = (0.192, -0.096)$). Defining the normalized coordinate fourth moment $\kappa_4(x) := \frac{1}{D}\sum_{i=1}^D x_i^4 \in [1, D]$, the expected squared norm of $P_{x_{\text{final}}(n)} b(n)$ relative to $\|b_{\text{rad}}(n)\|_2^2 = D / \text{rms}(z_n)^2$ satisfies the exact **data-dependent fourth-moment bound***
   $$
   \frac{\mathbb{E}_{\xi_n}\left[\|P_{x_{\text{final}}(n)} b(n)\|_2^2\right]}{\|b_{\text{rad}}(n)\|_2^2} \le \frac{\mathbb{E}_{\xi_n}\left[\|b_{\text{oblique}}(n)\|_2^2\right]}{\|b_{\text{rad}}(n)\|_2^2} = \frac{\sum_{i=1}^D x_{\text{final}, n, i}^4}{D^2} = \frac{\kappa_4(x_{\text{final}}(n))}{D}.
   $$
   *In particular: (i) if $\kappa_4(x_{\text{final}}(n)) \le C = O(1)$ is uniformly bounded, the residual ratio is $\le C/D = O(1/D)$ (on trained ClimbMix `d6` states at $D=384$, we measure token-mean $\kappa_4(x_{\text{final}}) = 2.9398$, giving bound $\kappa_4/D = 0.007656$ and empirical ratio $0.007458$); (ii) without a fourth-moment bound, $\|x\|_2^2 = D$ alone does **not** imply a vanishing residual—for the 2-sparse family $x = \sqrt{D/5}(1, 2, 0, \dots, 0)$ ($z = x$), $\kappa_4(x)/D = 0.68$ and $\frac{\mathbb{E}_{\xi}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} = 0.0576$ for every $D \ge 2$; and (iii) under **entrywise preconditioning**, coordinate variation in $c_{n, i} = \sum_j d_{j, i} s_{\text{lm}, n, j}^2$ breaks exact radial collinearity ($P_{x_{\text{final}}(n)}(c_n \odot b_{\text{rad}}(n)) \neq 0$) unless $\epsilon \gg \sqrt{\hat{v}_{j, i}^{(t-1)}}$, so increasing $\epsilon: 10^{-8} \to 10^{-4}$ is required to damp entrywise coordinate spread.*
2. **Cross-Token ($n' \neq n$) Current-Gradient Drift Bound:** *Since $d_{j, i} = (\sqrt{\hat{v}_{j, i}^{(t-1)}} + \epsilon)^{-1} \le \epsilon^{-1}$, the cross-token ($n' \neq n$) current-gradient contribution satisfies*
   $$
   \left|\delta u_{\text{cur}, i}^{(n' \neq n)}(n)\right| \le \frac{\eta_3 (1 - \beta_1)}{(1 - \beta_1^{t_{\text{step}}}) BT \, \epsilon} \sum_{n' \neq n} \left(\sum_{j=1}^V |s_{\text{lm}, n, j}| \, |s_{\text{lm}, n', j}|\right) |x_{\text{final}, n', i}| \, \left|\frac{\Delta x_{\text{final}, n, i}}{\sigma}\right|.
   $$

### 3.5 Fused Register-Tiled Contraction, Optimizer Co-Design, and Node-Perturbation Controls

1. **Eliminating 3D Outer-Product Materialization via Register-Tiled Contraction (`_triton_wm_reward_mg1_kernel`):** Building on the SRAM-fused outgoing-column contraction concept introduced in [5] (version 5), notice that in the stream-partitioned WM reward (Section 3.2),
   $$
   R_i^{(m, g)}(\text{scale\_factor}) = \frac{\text{scale\_factor}}{M_g} \sum_{j=1}^{D_{\text{out}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) \sum_{n \in \mathcal{S}_g} u_{n, j}^{(m+1)} \, \Delta y_{n, i}^{(m)} = \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} \Delta y_{n, i}^{(m)} \left(\sum_{j=1}^{D_{\text{out}}} u_{n, j}^{(m+1)} W_{j, i}^{(m+1)}(t_{\text{constr}})\right),
   $$
   the 3D plasticity tensor $\Delta \tilde{W}^{(m+1, g)} \in \mathbb{R}^{G \times D_{\text{out}} \times D_{\text{in}}}$ never needs to be materialized in GPU HBM. In our fused Triton kernel (`_triton_wm_reward_mg1_kernel` with tile sizes `BLOCK_G=4, BLOCK_R=64, BLOCK_C=64` for $M_g=1$, and `_triton_wm_reward_mg_grouped_kernel` for $M_g > 1$), each thread block loads a `(BLOCK_R, BLOCK_C)` tile of $W_{\text{rec}}$ from L2 cache once for `BLOCK_G` streams and accumulates `R[g, c]` directly in registers with **`0.0 MB` temporary HBM allocation**, cutting the 13-matrix `d6` WM kernel suite **$38.4\times$ from `134.31 ms` to `3.50 ms/step`** at $G=2,048$ (and **$4.4\times$ from `4.51 ms` to `1.03 ms/step`** at $G=64$).
2. **Joint Co-Design of the Optimizer and Delayed WM Credit:** Adapting the `Pre-Polar Muon` optimizer foundation of [8] to delayed WM, we note that the downstream optimizer step installs $W^{(m+1)}(t_{\text{constr}})$—which acts as Stage $m$'s **credit reader**:
   - **Update Orthogonalization and Receiver Stable Rank (Descriptive Association):** On our Seed-46 Block-0 diagnostic audit (`run_geo_and_attn_steering_audit.py`), under `AdamW` (`adamw_hidden`), Stage-2 receiver matrices $W^{(2)}$ exhibit low stable rank (`mlp.c_proj` $\operatorname{srank}(W) := \|W\|_F^2 / \|W\|_2^2 = 2.75$, `attn.c_proj` $\operatorname{srank}(W) = 1.11$). `Pre-Polar Muon` [8] orthogonalizes each step's momentum update direction (which does not constrain the accumulated weight matrix $W$ to a Stiefel manifold); descriptively, this update orthogonalization is associated with a higher accumulated receiver stable rank on Block-0 `mlp.c_proj` ($127.24$ under `paper20_prepolar` and $141.33$ under `codesigned_prepolar$, $51.4\times$ higher than `adamw_hidden`) and `attn.c_proj` ($54.85$ and $52.72$, $47.5\times$ higher than `adamw_hidden`).
   - **Scale-Matched Sparse Embedding Velocity & Post-Installation Drift Annealing:** Because `wte` is initialized at $\sigma_{\text{init}} = 1.0$ before `RMSNorm(e_tok)` ($\|W_{\text{wte}}[v]\|_2 \approx \sqrt{D} = 19.6$), small embedding learning rates (`lr_wte = 3e-3` to `4e-3`) rotate normalized token directions slowly. Matching sparse embedding step size on `d6` ($BT = 2,048$) to $\sigma_{\text{init}} = 1.0$ (`lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)`) and linearly annealing learning rates (`lrm: 1.0 -> 0.2`) increases held-out validation attention-map shift $\|\mathbf{A}^{(l)}(360) - \mathbf{A}^{(l)}(0)\|_F / \|\mathbf{A}^{(l)}(0)\|_F$ (Seed 46) by **$9.82\times$ in Layer 0 (`0.0838 -> 0.8233`)** and **$1.42\times$ averaged across all 6 layers (`0.6046 -> 0.8587`)**.
3. **Author-Defined Node-Perturbation Comparators (`SR-LNP` and `SR-NP`):** We compare Delayed WM against two specified node-perturbation recipes on the matched 4-matrix parameter subset (neither comparator is a zero-same-step-relay delayed pipeline, and neither is claimed as a replica of published Dust [9]):
   - **Surrogate-Routed Local Node Perturbation (`SR-LNP`, `train_dust_d6_fast` in `run_exp37_38_wm_vs_dust.py`):** Uses the same terminal-skip residual factors ($\gamma_l$ for blocks and $\alpha_0$ for `wte`) as Delayed WM, but differs in credit timing (it constructs $\hat{g}_{\text{bus}}$ via $K \in \{4, 16, 64\}$ perturbation sub-passes and immediately uses $\hat{g}_{\text{bus}}$ on the same step for embedding and hidden-layer estimates, whereas Delayed WM uses 3-stage delayed packets), perturbation location, and attention-credit treatment (scoring value perturbations locally with $\gamma = 0$ rather than via the forward attention-filtered trace). Thus, `SR-LNP` vs. Delayed WM compares two complete forward-only recipes rather than isolating sibling variance alone.
   - **Suffix-Recomputed Node Perturbation (`SR-NP`, `train_faithful_dust_d6` in `run_exp42_compliant_fp32_and_faithful_dust.py`):** An author-defined suffix-recomputed comparator on the matched 4-matrix `d6` subset ($K_{\text{site}} = 4 \implies 76$ suffix-recomputed draws/step $+ 24$ attention-internal draws/step) implementing Gaussian perturbations $a_t \sim \mathcal{N}(0, I)$, draw-mean centering $c_s^{(i)} = \tilde{\ell}_s - \ell_s^{(i)}$, suffix recomputation from block $l+1$ onward (`_run_suffix_from_block`), per-head local attention-output alignment $c_{s, h}^{(i)} = -\langle \hat{g}_{\text{attn\_out}, s, h}, \Delta o_{s, h}^{(i)} \rangle$, and future-token reward accumulation $r_{t, h}^{(i)} = \sum_{s \ge t} \gamma^{s - t} c_{s, h}^{(i)}$ with $\gamma = 0.98$ (whose discounted accumulation is verified by the `1.47` unit test). In `train_faithful_dust_d6`, both projection writers (`attn.c_proj` and `mlp.c_proj`) use $\hat{g}_{\text{wr}}$ obtained by perturbing the block-output residual after the MLP (which for `attn.c_proj` omits its path through the same block's MLP, a path not restored by suffix recomputation from block $l+1$ onward), and `lm_head` receives an analytic local loss gradient rather than vocabulary-slab perturbations.

---

## 4. Experimental Results and Mechanistic Verification

![Six-panel summary of genuinely backward-free 3-stage delayed eligibility-trace Weight Maximization, fused Triton kernels, co-designed optimizer, and 6-matrix attention routing across exp26–exp48](supplement/figure-1.png)

### 4.1 Experimental Setup, Arm-Classification Ledger, Attention Backend, and Timing Provenance

All experiments (`128` multi-seed experimental arms = `383` total runs across seeds $\{46, 47, 48\}$ in `exp18, exp21, exp26`–`exp48`: **`91` three-seed Delayed-WM arms [`273` runs]**, **`13` three-seed node-perturbation arms [`11` `SR-LNP` + `2` `SR-NP` = `39` runs]**, **`4` head/lexical-only backward-free anchors [`11` runs: three 3-seed + one 2-seed]**, **`7` three-seed local-pullback diagnostic controls in `exp36` [`21` runs, `Rev. Matmuls = 1`]**, and **`13` three-seed exact-backpropagation reference arms [`39` runs: `4` on `d6` in Table 3, `7` across `d6` and widths $D \in \{256, 384, 512, 768\}$ in Tables 5–6, and `2` on Native `50.3M` in Table 7]**) use the supplied `rustbpe` tokenizer (`vocab_size = 8,192`) and ClimbMix data stream on two Transformer regimes:
1. **ClimbMix `d6` ($L=6, D \in \{256, 384, 512, 768\}, H=D/64, D_h=64, V=8,192, B=8, T=256, BT=2,048$ tokens/step, $360$–$720$ steps $= 0.737\text{M}$–$1.475\text{M}$ tokens):** At our primary width $D=384$, total parameters $= 26,346,060$, with `active_params = 24,576,000` ($24.58\text{M}$) in the 4-matrix configuration (`mlp.c_fc, mlp.c_proj, attn.c_v, attn.c_proj` active per block, `attn.c_q, attn.c_k` frozen; $25.46\text{M}$ = `96.6%` when `attn.c_q` or `attn.c_k` is also active in `exp30`–`exp31` and `exp45`; and $26.35\text{M}$ = `100%` of matrix parameters when all 6 hidden matrices are active in `exp45` and `exp47`). In Section 4.7 (`exp39`), we also scale width across $D \in \{256, 384, 512, 768\}$ (`active_params` $\in \{14.55\text{M}, 24.58\text{M}, 36.70\text{M}, 67.11\text{M}\}$). Disjoint evaluation splits (`evaluate_bpb_disjoint`) score `dev_tokens = 261,889` (`dev_bytes = 1,037,477`) and `valb_tokens = 262,009` (`valb_bytes = 1,050,500`), where $\text{CE (nats)} = \text{BPB} \cdot \ln(2) \cdot (\text{bytes}/\text{tokens})$ and $\text{PPL} = \exp(\text{CE})$.
2. **Native `50,332,176` ($L=8, D=512, H=4, D_h=128, V=8,192, B_{\text{eff}}=64, T=2,048, BT=131,072$ tokens/step, $100$ steps $= 13,107,200$ training tokens):** Total parameters $= 50,332,176$, with `active_params = 46,137,344` ($46.14\text{M}$, `91.7%`) in the 4-matrix configuration (`c_q, c_k` $= 4,194,304$ and $528$ gate/scalar parameters frozen), `active_params = 48,234,496` ($48.23\text{M}$, `95.8%`) in the 5-matrix Spherical Active-$W_Q$ configuration (`exp46`, where `c_k` $= 2,097,152$ and $528$ scalars are frozen), and `active_params = 50,331,648` ($50.33\text{M}$, `99.999%` of total parameters and `100%` of matrix parameters) in the 6-matrix Decoupled Active-$(W_Q, W_K)$ configuration (`exp48`, where only the $528$ scalar parameters are frozen). To fit stream-partitioned eligibility-trace tensors in GPU memory, the backward-free Native `50.3M` runs accumulate `grad_accum_steps = 16` physical microbatches of `micro_batch_size = 4` at $G_{\text{micro}} \in \{256, 1024, 8192\}$ (`G_total` $\in \{4096, 16384, 131072\}$ streams/step, corresponding to $M_g \in \{32, 8, 1\}$ tokens/stream), departing from the supplied task's single-microbatch $B=64$, $499.65\text{M}$-token backpropagation recipe. Disjoint evaluation splits score `dev_tokens = 523,826` (`dev_bytes = 2,126,272`, $\text{bytes}/\text{token} = 4.059119$) and `valb_tokens = 523,715` (`valb_bytes = 2,129,527`). Because both `DEV` and `VAL-B` are evaluated at checkpoints across the ablation campaign, `VAL-B` serves as a second disjoint validation stream rather than an unopened holdout.
3. **Attention Backend and Timing Provenance (`model_and_data.py`, `benchmark_uncontended_timings.json`):** In `model_and_data.py`, the `fa3` handle wraps PyTorch's native fused `F.scaled_dot_product_attention(..., is_causal=True)`. In Tables 4–6, `Run GPU` reports the **measured 3-seed training-region GPU time (`total_train_gpu_s_mean`)** summed over forward and learning-rule CUDA intervals across all training steps (excluding evaluation checkpoints), while `Est.` reports the **30-step steady-state benchmark estimate (`est_360step_s` in `benchmark_uncontended_timings.json`)**, which measures 35 steps on an uncontended RTX 4090, discards 5 warmup steps, and excludes evaluation, dataloader, and host-transfer overhead. In Tables 3 and 7, `Fwd / Rule / Wall (ms)` reports per-step forward GPU time, learning-rule GPU time, and either total step GPU time (`Fwd + Rule` in `exp40`–`exp48`) or complete elapsed wall-clock step time including evaluation checkpoints (`t_wall_step_ms_including_eval` in `exp26`–`exp36`).

### 4.2 Verification of Propositions 1 and 2: Stream Partitioning, Normalization Controls, and Filtered Traces (`exp26`, `exp28`, `exp35`)

Table 1 reports the single-step cosine similarity at initialization between our backward-free eligibility-trace pseudo-gradient estimator and the **language-model cross-entropy loss gradient** $\nabla_W \mathcal{L}_{\text{CE}}$ (computed via a separate diagnostic reference call `loss_ref.backward()`) on `d6` ($BT = 2,048$) across stream counts $G = BT / M_g$ (`exp26_results.json`, `exp28_results.json`). We emphasize that Table 1 is a single-step directional diagnostic rather than a claim of maintained gradient alignment throughout training.

| Stream Count $G$ ($M_g = BT / G$) | Stage 2 `wte` Cos | Stage 2 `mlp.c_proj` Cos | Stage 2 `attn.c_proj` Cos | Stage 1 `mlp.c_fc` Cos | Stage 1 `attn.c_v` (Unfiltered $x_{\text{in}}$) | Stage 1 `attn.c_v` (Filtered $\bar{x}_{\text{in}}$, Prop. 2) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| $G = 1$ ($M_g = 2,048$) | $+0.0313$ | $+0.0187$ | $+0.0170$ | $+0.0037$ | $+0.0036$ | $+0.0295$ |
| $G = 64$ ($M_g = 32$) | $+0.2034$ | $+0.2978$ | $+0.2469$ | $+0.0305$ | $+0.0222$ | $+0.2361$ |
| $G = 256$ ($M_g = 8$) | $+0.3941$ | $+0.5341$ | $+0.4570$ | $+0.1430$ | $+0.0416$ | $+0.4519$ |
| **$G = 2,048$ ($M_g = 1$)** | **$+0.9139$** | **$+0.9364$** | **$+0.9068$** | **$+0.9089$** | $+0.1104$ | **$+0.9069$ ($8.2\times$)** |

**Table 1:** Single-step cosine alignment with the language-model cross-entropy loss gradient $\nabla_W \mathcal{L}_{\text{CE}}$ on ClimbMix `d6` ($BT = 2,048$) across stream counts $G \in \{1, 64, 256, 2048\}$ (`exp26`, `exp28`).

**Stream Scale-Normalization Controls (`exp35` Part A):** To test whether the stream scaling effect in Table 1 survives when update magnitude is held invariant to $M_g$ (Proposition 1), we ran a controlled 3-seed comparison on `d6` setting `scale_factor = float(M_g)` so that $\widehat{G}_{\text{norm}}(G) = M_g \widehat{G}_{\text{code}}(G)$ has the identical conditional mean $\frac{1}{N}\sum_n a_{n, i} x_n$ across all stream sizes:
- At $G = 1$ ($M_g = 2,048$): `scale_factor = 2048.0` yields **`2.6097 ± 0.0111` DEV BPB** (`2.5719 ± 0.0106` VAL-B), compared to `scale_factor = 1.0` (`2.5939 ± 0.0109` DEV BPB).
- At $G = 64$ ($M_g = 32$): `scale_factor = 32.0` yields **`2.3361 ± 0.0084` DEV BPB** (`2.2995 ± 0.0086` VAL-B), compared to `scale_factor = 1.0` (`2.3364 ± 0.0051` DEV BPB).
- At $G = 256$ ($M_g = 8$): `scale_factor = 8.0` yields **`2.3008 ± 0.0074` DEV BPB** (`2.2635 ± 0.0065` VAL-B), compared to `scale_factor = 1.0` (`2.2966 ± 0.0115` DEV BPB).
- On the canonical `Pre-Polar Muon` trainer (`strict_post_install`, `entrywise_b09_eps1e4`, $G=64$, `exp35` Part A2): `scale_factor = 1.0` reaches **`2.2541 ± 0.0013` DEV BPB** and `scale_factor = 32.0` reaches **`2.2539 ± 0.0013` DEV BPB** (agreeing to within $0.0002$ BPB).
These controls establish that the learning gains from increasing $G$ ($M_g \to 1$) are driven by the elimination of within-stream cross-token pair interference (Proposition 1(b)), not by artificial update rescaling.

### 4.3 Verification of Propositions 3 and 4: 20-Step Post-Installation Credit Drift, Kurtosis Audit, and Batch-Size Occupancy (`exp31`, `exp34`–`exp36`)

To test how receiver weight installation affects Stage-1 (`c_proj` receiver) and Stage-2 (`lm_head` receiver) rewards, we measured the 20-step mean cosine similarity, relative L2 norm drift $\|u_{\text{post}} - u_{\text{pre}}\|_2 / \|u_{\text{pre}}\|_2$, activation fourth moments, and batch-size occupancy on `d6` (`exp31`, `exp34`–`exp36` Part 0, Table 2):

1. **Stage-1 Receiver Drift (`exp31` Part 0, `exp35` Part 0):** When Stage 2 (`mlp.c_proj, attn.c_proj`) is updated with `AdamW`, one receiver step $\delta W^{(2)}$ induces a **`23.56%` relative L2 drift** in Stage-1 rewards ($\cos = 0.976474$). Replacing `AdamW` on Stage 2 with Stiefel-orthogonalized `Pre-Polar Muon` reduces Stage-1 relative L2 reward drift to **`2.85%` ($\cos = 0.999597$)** in `exp31` and **`4.04%` ($\cos = 0.999238$)** in `exp35` Part 0.
2. **Activation Kurtosis Expectation Bound vs. Realized Token Ratios (`exp35` Part 0):** Across the 20-step diagnostic trajectory (`run_part0_kurtosis_and_rmsnorm_variance_audit`, which updates `lm_head` and block-0 `mlp.c_proj` over 20 steps on `d6`), the normalized fourth moment $\kappa_4(x_{\text{final}}) = \frac{1}{D}\sum_{i=1}^D x_{\text{final}, i}^4$ has token mean **`2.9398`** (token max `4.2932`). Proposition 4 bounds the **expected** first-order same-token residual ratio $\frac{\mathbb{E}_{\xi_n}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} \le \frac{\kappa_4(x_{\text{final}})}{D}$; on these states, the mean theoretical expectation bound is **`0.007656`** and the empirical mean ratio across all $40,960$ token evaluations is **`0.007458`**, while individual realized draws $\xi_n$ can fluctuate higher (token maximum **`0.124439`**).
3. **Stage-2 Strict Post-Install Terminal Drift Across the $2 \times 2 \times 2$ Optimizer Cube (`exp34` Part 0, Table 2) and Batch-Size Occupancy (`exp36` Part 0A):** Across all 8 cells of `{entrywise, row_iso} x {beta1=0.5, 0.9} x {eps=1e-8, 1e-4}`, combining $(\beta_1=0.9, \epsilon=10^{-4})$ cuts current-gradient drift by **$47.8\%$ (`0.3511 -> 0.1832`)** and total post-install drift from `39.41%` to **`26.05%` ($\cos = 0.964738$)**. Furthermore, holding the `d6` architecture and sequence length $T=256$ fixed while sweeping batch size $BT \in \{512, 1024, 2048, 4096, 8192\}$ (`exp36` Part 0A) shows that singleton target-token occupancy decreases monotonically (`0.3957 -> 0.3516 -> 0.2958 -> 0.2265 -> 0.1620`) and post-installation relative L2 credit drift decreases monotonically (**`44.88% -> 41.95% -> 39.41% -> 37.46% -> 35.68%`**), confirming that larger effective token batches dilute single-token self-update collisions in $\delta W_{\text{lm}}(t-1)$.

| `lm_head` Optimizer Cell (`exp34` Part 0, 20-Step Mean on `d6`) | `bus_unproj` Cos / Rel L2 Drift | `bus_tangent` ($\perp x_{\text{final}}$) Cos / Drift | Current-Batch Grad Only Drift ($\delta u_{\text{cur}}$) | Momentum-History Only Drift ($\delta u_{\text{mom}}$) | $\|P_{x_{\text{final}}} b\|_2 / \|b\|_2$ (Prop. 4) |
| :--- | :---: | :---: | :---: | :---: | :---: |
| `entrywise_b05_eps1e8` ($\beta_1=0.5, \epsilon=10^{-8}$) | $0.940066$ / $0.465747$ | $0.948719$ / $0.394051$ | $0.351118$ | $0.080159$ | $0.087114$ |
| `entrywise_b09_eps1e8` ($\beta_1=0.9, \epsilon=10^{-8}$) | $0.951336$ / $0.392834$ | $0.954834$ / $0.353878$ | $0.269559$ | $0.112059$ | $0.087114$ |
| `entrywise_b05_eps1e4` ($\beta_1=0.5, \epsilon=10^{-4}$) | $0.943013$ / $0.391255$ | $0.956556$ / $0.307328$ | $0.270305$ | $0.085703$ | $0.087114$ |
| **`entrywise_b09_eps1e4` ($\beta_1=0.9, \epsilon=10^{-4}$)** | **$0.958144$ / $0.304684$** | **$0.964738$ / $0.260504$** | **$0.183231$** | $0.113457$ | $0.087114$ |
| `row_iso_b05_eps1e8` ($\beta_1=0.5, \epsilon=10^{-8}$) | $0.941012$ / $0.471413$ | $0.952909$ / $0.367834$ | $0.317646$ | $0.076778$ | $0.087114$ |
| `row_iso_b09_eps1e8` ($\beta_1=0.9, \epsilon=10^{-8}$) | $0.951433$ / $0.400026$ | $0.956413$ / $0.342871$ | $0.252057$ | $0.108569$ | $0.087114$ |
| `row_iso_b05_eps1e4` ($\beta_1=0.5, \epsilon=10^{-4}$) | $0.942453$ / $0.403128$ | $0.959755$ / $0.292793$ | $0.251836$ | $0.081251$ | $0.087114$ |
| **`row_iso_b09_eps1e4` ($\beta_1=0.9, \epsilon=10^{-4}$)** | **$0.957127$ / $0.317284$** | **$0.966045$ / $0.255660$** | **$0.174360$** | $0.110404$ | $0.087114$ |

**Table 2:** 20-step Pre-Install vs. Strict Post-Install credit decomposition audit across all 8 cells of the $2 \times 2 \times 2$ `lm_head` optimizer cube on ClimbMix `d6` (`exp34_results.json` Part 0).

### 4.4 Complete 16-Cell Factorial and True Both-Stages Pre-Install on ClimbMix `d6` (`exp33`–`exp35`)

Table 3 presents the 3-seed ClimbMix `d6` results across stream counts, the complete 16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Pooling $\times$ Momentum $\times$ Damping cube (`exp34` + `exp35` Part B1), and the 4-cell True Both-Stages Pre-Install comparison (`exp35` Part B2). All 20 factorial cells share identical 1-call `wte_bus_routed` routing (**Proposition 3**), Rademacher-tangent exploration, and reward-side radial projection.

| Method (`ClimbMix d6`, $L=6, D=384, V=8,192$, 360 Steps, Seeds `46, 47, 48`) | Active Params | Stage-3 / Stage-2 / Stage-1 Clocks | `lm_head` Opt $(\text{pool}, \beta_1, \epsilon)$ | Rev. Matmuls / Bwd | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Fwd / Rule / Wall (`ms`) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| `supplied_muon_adamw_backprop` (Full 6-Matrix BP) | $26.35\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | `entry, 0.5, 1e-8` | $-1 / 1$ | $2.1112 / 2.1119 / 2.1105$ | $2.1112 \pm 0.0007$ | $2.0750 \pm 0.0007$ | $331.49$ | $4.6 / 18.0 / 46.2$ |
| `backprop_prepolar_muon_damped_lm_frozen_qk_control` (`exp33`) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | `row_iso, 0.9, 1e-4` | $-1 / 1$ | $2.1155 / 2.1147 / 2.1153$ | $2.1152 \pm 0.0004$ | $2.0780 \pm 0.0002$ | $333.01$ | $4.5 / 19.3 / 47.3$ |
| `backprop_prepolar_muon_frozen_qk_control` (`exp32`) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | `entry, 0.5, 1e-8` | $-1 / 1$ | $2.1207 / 2.1203 / 2.1161$ | $2.1190 \pm 0.0025$ | $2.0825 \pm 0.0027$ | $336.53$ | $4.6 / 19.0 / 47.2$ |
| `backprop_adamw_frozen_qk_control` (`exp31`) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | `entry, 0.5, 1e-8` | $-1 / 1$ | $2.1628 / 2.1579 / 2.1654$ | $2.1621 \pm 0.0038$ | $2.1242 \pm 0.0039$ | $378.74$ | $4.4 / 8.5 / 36.8$ |
| `frozen_backbone_lm_head_only` (`lm_head` only) | $3.15\text{M}$ | $(t,t,t)/\text{frozen}/\text{frozen}$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.2636 / 2.2646 / 2.2660$ | $2.2647 \pm 0.0012$ | $2.2266 \pm 0.0008$ | $491.36$ | $3.5 / 1.3 / 28.4$ |
| `pure_lexical_x0_skip_only_true_zero` (`wte + lm_head`) | $6.29\text{M}$ | $(t,t,t)/(t,t,t)/\text{frozen}$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.2442 / 2.2449 / 2.2472$ | $2.2454 \pm 0.0016$ | $2.2078 \pm 0.0019$ | $469.19$ | $3.8 / 81.4 / 108.9$ |
| `causal_trace_wm_G2048_mismatched_origin_control` (`exp28`) | $24.58\text{M}$ | $(t,t,t)/(t-1\neq t)/(t-2\neq t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.7672 / 2.7582 / 2.7570$ | $2.7608 \pm 0.0056$ | $2.7229 \pm 0.0057$ | $1960.79$ | $5.6 / 396.4 / 425.8$ |
| `causal_trace_wm_G1_centered` ($G=1$, `exp26`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.5831 / 2.6048 / 2.5938$ | $2.5939 \pm 0.0109$ | $2.5564 \pm 0.0103$ | $1240.17$ | $5.5 / 19.4 / 48.6$ |
| `causal_trace_wm_G1_centered_scaleMg` ($G=1$, `exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.6093 / 2.6209 / 2.5988$ | $2.6097 \pm 0.0111$ | $2.5719 \pm 0.0106$ | $1294.94$ | $14.3 / 26.3 / 93.9$ |
| `causal_trace_wm_G64_centered` ($G=64$, `exp26`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.3396 / 2.3305 / 2.3391$ | $2.3364 \pm 0.0051$ | $2.2994 \pm 0.0052$ | $611.31$ | $5.5 / 38.4 / 67.5$ |
| `causal_trace_wm_G64_centered_scaleMg` ($G=64$, `exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.3458 / 2.3316 / 2.3310$ | $2.3361 \pm 0.0084$ | $2.2995 \pm 0.0086$ | $610.94$ | $14.2 / 59.9 / 127.5$ |
| `causal_trace_wm_G256_centered` ($G=256$, `exp26`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.2889 / 2.2911 / 2.3098$ | $2.2966 \pm 0.0115$ | $2.2602 \pm 0.0117$ | $548.11$ | $5.6 / 98.4 / 127.6$ |
| `causal_trace_wm_G256_centered_scaleMg` ($G=256$, `exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.3018 / 2.3077 / 2.2930$ | $2.3008 \pm 0.0074$ | $2.2635 \pm 0.0065$ | $554.47$ | $14.2 / 176.7 / 243.7$ |
| `causal_trace_wm_G2048_wtrace1_tangent_explore` (`AdamW`, `exp27`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.2182 / 2.1924 / 2.1964$ | $2.2023 \pm 0.0139$ | $2.1661 \pm 0.0118$ | $423.21$ | $5.6 / 395.7 / 425.0$ |
| **Complete 16-Cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer Cube (`exp34` + `exp35` Part B1, Stage 1 = `(t-2, t [post-Stage2], t)`):** | | | | | | | | | |
| `d6_pre_install_..._entrywise_b05_eps1e8` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1380 / 2.1364 / 2.1335$ | $2.1360 \pm 0.0023$ | $2.0993 \pm 0.0024$ | $352.56$ | $6.6 / 160.3 / 190.6$ |
| `d6_pre_install_..._entrywise_b09_eps1e8` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `entry, 0.9, 1e-8` | $0 / 0$ | $2.1454 / 2.1415 / 2.1424$ | $2.1431 \pm 0.0020$ | $2.1062 \pm 0.0017$ | $359.53$ | $6.5 / 149.3 / 179.9$ |
| **`d6_pre_install_..._entrywise_b05_eps1e4` (`exp34`)** | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `entry, 0.5, 1e-4` | $0 / 0$ | $2.1217 / 2.1223 / 2.1211$ | **$2.1217 \pm 0.0006$** | **$2.0849 \pm 0.0006$** | **$338.98$** | $6.5 / 149.1 / 179.6$ |
| `d6_pre_install_..._entrywise_b09_eps1e4` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `entry, 0.9, 1e-4` | $0 / 0$ | $2.1255 / 2.1255 / 2.1248$ | $2.1253 \pm 0.0004$ | $2.0881 \pm 0.0002$ | $342.34$ | $6.5 / 149.1 / 179.7$ |
| `d6_pre_install_..._row_iso_b05_eps1e8` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `row_iso, 0.5, 1e-8` | $0 / 0$ | $2.1578 / 2.1574 / 2.1578$ | $2.1577 \pm 0.0002$ | $2.1210 \pm 0.0002$ | $374.18$ | $6.5 / 155.1 / 185.6$ |
| `d6_pre_install_..._row_iso_b09_eps1e8` (`exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `row_iso, 0.9, 1e-8` | $0 / 0$ | $2.1663 / 2.1660 / 2.1663$ | $2.1662 \pm 0.0002$ | $2.1289 \pm 0.0001$ | $383.06$ | $11.3 / 283.0 / 331.7$ |
| `d6_pre_install_..._row_iso_b05_eps1e4` (`exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `row_iso, 0.5, 1e-4` | $0 / 0$ | $2.1298 / 2.1293 / 2.1296$ | $2.1296 \pm 0.0003$ | $2.0928 \pm 0.0001$ | $346.40$ | $11.2 / 283.6 / 333.9$ |
| `d6_pre_install_..._row_iso_b09_eps1e4` (`exp33`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | `row_iso, 0.9, 1e-4` | $0 / 0$ | $2.1335 / 2.1332 / 2.1338$ | $2.1335 \pm 0.0003$ | $2.0961 \pm 0.0002$ | $350.18$ | $6.5 / 149.0 / 179.5$ |
| `d6_strict_post_install_..._entrywise_b05_eps1e8` (`exp33`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1580 / 2.1582 / 2.1574$ | $2.1579 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.39$ | $6.4 / 149.2 / 179.8$ |
| `d6_strict_post_install_..._entrywise_b09_eps1e8` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.9, 1e-8` | $0 / 0$ | $2.1504 / 2.1499 / 2.1477$ | $2.1494 \pm 0.0014$ | $2.1122 \pm 0.0015$ | $365.74$ | $6.5 / 154.0 / 184.5$ |
| **`d6_strict_post_install_..._entrywise_b05_eps1e4` (`exp34`)** | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.5, 1e-4` | $0 / 0$ | $2.1301 / 2.1302 / 2.1297$ | **$2.1300 \pm 0.0003$** | **$2.0930 \pm 0.0003$** | $346.83$ | $6.5 / 154.2 / 184.7$ |
| **`d6_strict_post_install_..._entrywise_b09_eps1e4` (`exp34`)** | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.9, 1e-4` | $0 / 0$ | $2.1275 / 2.1273 / 2.1271$ | **$2.1273 \pm 0.0002$** | **$2.0900 \pm 0.0001$** | **$344.25$** | $6.5 / 154.1 / 184.6$ |
| `d6_strict_post_install_..._row_iso_b05_eps1e8` (`exp33`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `row_iso, 0.5, 1e-8` | $0 / 0$ | $2.1581 / 2.1578 / 2.1580$ | $2.1580 \pm 0.0001$ | $2.1214 \pm 0.0004$ | $374.50$ | $6.4 / 149.0 / 179.7$ |
| `d6_strict_post_install_..._row_iso_b09_eps1e8` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `row_iso, 0.9, 1e-8` | $0 / 0$ | $2.1661 / 2.1655 / 2.1660$ | $2.1659 \pm 0.0003$ | $2.1286 \pm 0.0001$ | $382.72$ | $6.5 / 149.1 / 179.7$ |
| `d6_strict_post_install_..._row_iso_b05_eps1e4` (`exp34`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `row_iso, 0.5, 1e-4` | $0 / 0$ | $2.1301 / 2.1297 / 2.1297$ | $2.1298 \pm 0.0002$ | $2.0932 \pm 0.0002$ | $346.64$ | $6.5 / 149.0 / 179.5$ |
| `d6_strict_post_install_..._row_iso_b09_eps1e4` (`exp33`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `row_iso, 0.9, 1e-4` | $0 / 0$ | $2.1333 / 2.1332 / 2.1337$ | $2.1334 \pm 0.0003$ | $2.0960 \pm 0.0003$ | $350.06$ | $6.5 / 148.9 / 179.3$ |
| **True Both-Stages Pre-Install (`both_stages_pre_install`, `exp35` Part B2, Stage 1 = `(t-2, t-1 [pre-Stage2], t)`):** | | | | | | | | | |
| `d6_both_stages_pre_install_entrywise_b05_eps1e8` (`exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1359 / 2.1347 / 2.1318$ | $2.1341 \pm 0.0021$ | $2.0975 \pm 0.0024$ | $350.78$ | $9.5 / 244.1 / 284.4$ |
| `d6_both_stages_pre_install_entrywise_b09_eps1e8` (`exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.9, 1e-8` | $0 / 0$ | $2.1441 / 2.1404 / 2.1409$ | $2.1418 \pm 0.0020$ | $2.1047 \pm 0.0019$ | $358.24$ | $9.2 / 216.7 / 255.6$ |
| **`d6_both_stages_pre_install_entrywise_b05_eps1e4` (`exp35`)** | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.5, 1e-4` | $0 / 0$ | $2.1195 / 2.1200 / 2.1190$ | **$2.1195 \pm 0.0005$** | **$2.0827 \pm 0.0006$** | **$336.96$** | $10.1 / 181.2 / 223.5$ |
| `d6_both_stages_pre_install_entrywise_b09_eps1e4` (`exp35`) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | `entry, 0.9, 1e-4` | $0 / 0$ | $2.1231 / 2.1231 / 2.1224$ | $2.1229 \pm 0.0004$ | $2.0857 \pm 0.0002$ | $340.09$ | $9.8 / 184.8 / 228.6$ |
| **Compliant Zero-Pullback Pure-`FP32` Trace-Cache Clock Controls (`exp42` Part A, `_triton_wm_reward_mg1_kernel`):** | | | | | | | | | |
| `d6_pre_install_fp32_cache_triton_b05_eps1e8` (`exp42` A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1380 / 2.1362 / 2.1336$ | $2.1359 \pm 0.0022$ | $2.0993 \pm 0.0023$ | $352.53$ | $5.2 / 21.7 / 26.9$ |
| `d6_strict_post_install_fp32_cache_triton_b05_eps1e8` (`exp42` A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1580 / 2.1582 / 2.1574$ | $2.1579 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.36$ | $5.1 / 21.3 / 26.5$ |
| **`d6_strict_post_install_fp32_cache_triton_b09_eps1e4` (`exp42` A)** | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.9, 1e-4` | $0 / 0$ | $2.1275 / 2.1275 / 2.1271$ | **$2.1274 \pm 0.0002$** | **$2.0901 \pm 0.0001$** | **$344.34$** | $4.9 / 20.5 / 25.4$ |
| **Local-Pullback `s_lm @ W_lm` Clock Controls & Realized-Component Reader Controls (`exp36` Parts A & B):** | | | | | | | | | |
| `d6_pre_install_fp32_cache_entrywise_b05_eps1e8` (`exp36` A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $1 / 0$ | $2.1380 / 2.1365 / 2.1336$ | $2.1360 \pm 0.0022$ | $2.0993 \pm 0.0023$ | $352.54$ | $6.3 / 92.1 / 122.1$ |
| `d6_strict_post_install_fp32_cache_entrywise_b05_eps1e8` (`exp36` A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $1 / 0$ | $2.1581 / 2.1581 / 2.1573$ | $2.1578 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.37$ | $6.3 / 95.3 / 125.1$ |
| `d6_strict_post_install_fp32_cache_entrywise_b09_eps1e4` (`exp36` A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | `entry, 0.9, 1e-4` | $1 / 0$ | $2.1274 / 2.1273 / 2.1271$ | $2.1273 \pm 0.0001$ | $2.0900 \pm 0.0002$ | $344.23$ | $6.3 / 95.3 / 125.3$ |
| `d6_strict_post_install_internal_muon_colnorm_reader_b05_eps1e8` (`exp36` B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.1616 / 2.1615 / 2.1607$ | $2.1613 \pm 0.0005$ | $2.1247 \pm 0.0006$ | $377.93$ | $6.3 / 100.3 / 130.5$ |
| `d6_strict_post_install_internal_muon_colnorm_reader_b09_eps1e4` (`exp36` B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.9, 1e-4` | $0 / 0$ | $2.1341 / 2.1337 / 2.1335$ | $2.1338 \pm 0.0003$ | $2.0965 \pm 0.0004$ | $350.42$ | $6.3 / 100.2 / 130.2$ |
| `d6_strict_post_install_head_adamw_frobnorm_reader_b05_eps1e8` (`exp36` B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | `entry, 0.5, 1e-8` | $0 / 0$ | $2.2143 / 2.2121 / 2.2121$ | $2.2128 \pm 0.0013$ | $2.1757 \pm 0.0011$ | $435.32$ | $6.3 / 118.2 / 148.6$ |

**Table 3:** Controlled 3-seed ClimbMix `d6` progression, complete 16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer cube (`exp34` + `exp35` Part B1), 4-cell True Both-Stages Pre-Install comparison (`exp35` Part B2), compliant zero-pullback pure-`FP32` trace-cache clock controls (`exp42` Part A), local-pullback `s_lm @ W_lm` clock controls (`exp36` Part A, `Rev. Matmuls = 1`), and realized-component reader controls (`exp36` Part B).

**Key Mechanistic Findings:**
1. **Stage-1 vs. Stage-2 Post-Installation Drift Separation and Compliant Zero-Pullback Pure-`FP32` Clock Controls (`exp35` Part B2, `exp42` Part A):** Comparing True Both-Stages Pre-Install `(t-1, t-1, t)/(t-2, t-1, t)` with Stage-2 Pre-Install `(t-1, t-1, t)/(t-2, t, t)` shows that delaying Stage-1 reward construction until step $t$ shifts DEV BPB by only **`+0.0013` to `+0.0024` BPB** across all 4 `entrywise` cells (`2.1341 -> 2.1360` at `b05, 1e-8`; `2.1195 -> 2.1217` at `b05, 1e-4`), whereas delaying Stage-2 reward construction until step $t$ (`strict_post_install`) shifts DEV BPB by **`+0.0219 ± 0.0020` BPB** at `(entrywise, b05, 1e-8)` (`2.1360 -> 2.1579`). Because `exp34` cached `mod_bus_base` in `BF16` on the pre-install path and cached `x_final_clean` in `BF16` on the post-install path (`zero_dw_bf16_cache_rel_l2_diff_mean = 0.001687`), and because `exp36` Part A used a local pullback `s_lm @ W_lm` (`Rev. Matmuls = 1`), `exp42` Part A (`exp42_fp32_triton_results.json`) executes a strictly zero-pullback (`reverse_error_matmuls_per_step = 0`) pure-`FP32` trace-cache control via `_triton_wm_reward_mg1_kernel` (`zero_dw_fp32_cache_rel_l2_diff_mean = 0.0`). Across 3 seeds (`46, 47, 48`), the compliant zero-pullback pure-`FP32` clock contrast is **`2.1359 ± 0.0022` vs. `2.1579 ± 0.0004` DEV BPB (`+0.0219 ± 0.0019` BPB)** at `b05, 1e-8` and **`2.1274 ± 0.0002` DEV BPB (`2.0901 ± 0.0001` VAL-B BPB)** at `b09, 1e-4`, confirming that the clock shift is a genuine receiver-installation effect.
2. **Row-Isotropic vs. Entrywise Preconditioning and Realized-Component Readers (`Proposition 4`, `exp34`–`exp36`):** Under `row_iso` preconditioning, Pre-Install and Strict Post-Install agree to within $\le 0.0003$ BPB across all 4 cells (`2.1577` vs. `2.1580`; `2.1662` vs. `2.1659`; `2.1296` vs. `2.1298`; `2.1335` vs. `2.1334`), consistent with Proposition 4's radial cancellation under scalar $c_n$. Under `entrywise` preconditioning, increasing $\epsilon: 10^{-8} \to 10^{-4}$ improves Strict Post-Install by **`-0.0279 ± 0.0001` BPB** (`2.1579 -> 2.1300`), and combining $(\beta_1=0.9, \epsilon=10^{-4})$ reaches **`2.1273 ± 0.0002` DEV BPB** (`2.1274 ± 0.0002` in pure `FP32`). Conversely, folding the internal `Pre-Polar Muon` operator (`2.1613` vs. `2.1579` at `b05, 1e-8`; `2.1338` vs. `2.1273` at `b09, 1e-4`, `exp36` Part B) or the `lm_head` `AdamW` preconditioner (`2.2128` vs. `2.1579`) directly into the WM reader degrades learning because forward perturbations propagate through the unpreconditioned forward weights $W_{\text{live}}$.

### 4.5 Fused Register-Tiled Triton WM Speedup and Node-Perturbation Comparator Timing (`exp37`, `exp42`)

Profiling `compute_stream_wm_rewards_fast` in `exp34` revealed that materializing the 3D intermediate outer-product tensor $\Delta \tilde{W}^{(m+1, g)} \in \mathbb{R}^{G \times D_{\text{out}} \times D_{\text{in}}}$ in GPU HBM (`3,498.9 MB/step` across 13 matrices on `d6`) consumed **`134.31 ms/step`** (`86.3%` of total step time). Table 4 compares our fused register-tiled Triton kernels (`_triton_wm_reward_mg1_kernel` for $M_g=1$ and `_triton_wm_reward_mg_grouped_kernel` for $M_g > 1$), batched `Pre-Polar Muon` (`prepolar_muon_orthogonalize_batched`: `10.35 -> 1.37 ms/step`), **Surrogate-Routed Local Node Perturbation (`SR-LNP`)** (`train_dust_d6_fast`, using `@torch.compile` fused TensorCore chunk kernels `fused_bus_ce_chunk`, `fused_wte_dust_chunk`, `fused_mlp_dust_chunk`), and **Suffix-Recomputed Node Perturbation (`SR-NP`)** (`train_faithful_dust_d6` in `run_exp42_compliant_fp32_and_faithful_dust.py`, which executes exact suffix recomputation from block $l$ onward across $K_{\text{tot}} = 76$ block-level draws plus $24$ local attention draws per step with $\gamma = 0.98$) on an uncontended RTX 4090 (`benchmark_uncontended_timings.json`, `exp37_39_master_results.json`, `exp42_faithful_dust_results.json`). Both 30-step steady-state benchmark estimates (`est_360step_s`) and actual 3-seed training GPU times (`total_train_gpu_s_mean`) are reported.

| Implementation / Kernel Suite (`ClimbMix d6`, $L=6, D=384, BT=2,048$) | Perturbed Passes $K$ | Stream Count $G$ ($M_g$) | 13-Matrix Credit Kernel (`ms`) | Temp HBM Alloc (`MB`) | Uncontended Step Time (`ms`) | Steady-State 360-Step Est. (`s`) | Measured 3-Seed Run GPU (`s`) | Numerical Match / Fidelity |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| PyTorch `compute_stream_wm_rewards_fast` (`exp34`, Paper #20 v1) | $1$ | $2,048$ ($1$) | $134.31$ | $3,498.9$ | $155.60$ | $56.02$ | $66.46$ | Reference (`1.0 / 0.0`) |
| **Fused Register-Tiled Triton WM (`_triton_wm_reward_mg1_kernel`)** | **$1$** | **$2,048$ ($1$)** | **$3.50$ ($38.4\times$)** | **$0.0$** | **$18.68 \pm 0.21$ ($8.33\times$)** | **$6.72$** | **$6.74$** | **$1.0000000 / 2.26\text{e-}8$** |
| **Fused Grouped Triton WM (`_triton_wm_reward_mg_grouped_kernel`)** | **$1$** | **$64$ ($32$)** | **$1.03$ ($4.4\times$)** | **$0.0$** | **$15.82 \pm 0.17$** | **$5.69$** | **$5.79$** | **$0.9999973 / 2.31\text{e-}3$** |
| Compiled `SR-LNP` (`train_dust_d6_fast`, $K=4$) | $4$ | Tokenwise | $7.31$ | $0.0$ | $15.23 \pm 0.25$ | $5.48$ | $6.32$ | Local Surrogate ($\gamma=0$) |
| Compiled `SR-LNP` (`train_dust_d6_fast`, $K=16$) | $16$ | Tokenwise | $17.71$ | $0.0$ | $25.63 \pm 0.28$ | $9.23$ | $9.43$ | Local Surrogate ($\gamma=0$) |
| Compiled `SR-LNP` (`train_dust_d6_fast`, $K=64$) | $64$ | Tokenwise | $56.59$ | $0.0$ | $64.51 \pm 0.25$ | $23.22$ | $23.66$ | Local Surrogate ($\gamma=0$) |
| Suffix-Recomputed `SR-NP` (`exp42`, $K_{\text{site}}=4, K_{\text{tot}}=76$, `Muon`) | $76\text{ suffix}$ | Tokenwise | $167.2$ | $0.0$ | $181.08$ | $65.19$ | $65.19$ | Suffix + $\gamma=0.98$ |
| Suffix-Recomputed `SR-NP` (`exp42`, $K_{\text{site}}=4, K_{\text{tot}}=76$, `AdamW`) | $76\text{ suffix}$ | Tokenwise | $220.5$ | $0.0$ | $234.39$ | $84.38$ | $84.38$ | Suffix + $\gamma=0.98$ |
| Exact Backpropagation (`backprop_d6_codesigned_opt`, Matched Subset) | $0$ (`1 bwd`) | --- | --- | --- | $11.49 \pm 0.23$ | $4.14$ | $6.56$ | Exact Autograd Reference |

**Table 4:** Uncontended RTX 4090 kernel timings, 30-step steady-state 360-step estimates (`benchmark_uncontended_timings.json`), and actual 3-seed training GPU times (`exp37_39_master_results.json`, `exp42_faithful_dust_results.json`) across PyTorch WM, Fused Register-Tiled Triton WM, Compiled `SR-LNP`, Suffix-Recomputed `SR-NP`, and Exact Backpropagation on ClimbMix `d6`.

### 4.6 Joint Co-Design of the Optimizer, Three-Tier Attention Hierarchy, and Full 6-Matrix Active-$(W_Q, W_K)$ Routing (`exp38`, `exp45`, `exp47`)

Table 5 reports our controlled 3-seed optimizer co-design and 5/6-matrix attention-routing ablations on ClimbMix `d6` ($360$ steps, Strict Post-Installation Construction `(t-1, t [post-Stage3], t)/(t-2, t [post-Stage2], t)`, `exp37_39_master_results.json`, `exp45_d6_attn_routing_results.json`, `exp47_d6_6mat_joint_qk_results.json`) alongside the Seed-46 Block-0 geometric diagnostic (`run_geo_and_attn_steering_audit.py`, `geo_and_attn_steering_audit.json`):
1. **Orthogonalized Update Directions and Receiver Stable Rank (`51.4x` Descriptive Association on Seed 46 Block 0):** Measuring Block 0 at step 360 on Seed 46 (`run_geo_and_attn_steering_audit.py`), under `AdamW` on hidden matrices (`adamw_hidden`), Stage-2 receiver columns collapse onto a low-rank subspace (`mlp.c_proj` stable rank $\operatorname{srank}(W) = 2.75$, off-diagonal column coherence $0.3829$; `attn.c_proj` $\operatorname{srank}(W) = 1.11$, coherence $0.6024$). Under `Pre-Polar Muon`, orthogonalizing each step's momentum update direction is descriptively associated with a higher Block-0 weight stable rank on `mlp.c_proj` (**`127.24` under `paper20_prepolar`, coherence $0.0419$; and `141.33` under `codesigned_prepolar`, coherence $0.0411$, $51.4\times$ higher than `adamw_hidden`**) and `attn.c_proj` (**`54.85` under `paper20_prepolar` and `52.72` under `codesigned_prepolar`, $47.5\times$ higher than `adamw_hidden`**).
2. **Scale-Matched Sparse Embedding Velocity (`9.82x` Layer-0 / `1.42x` 6-Layer Mean Attention Shift) & Drift Annealing:** Matching sparse embedding angular velocity (`lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)`) to $\sigma_{\text{init}}(\text{wte}) = 1.0$ before `RMSNorm(e_tok)` (`-0.0194` BPB gain alone: `2.1273 -> 2.1079`) and linearly annealing learning rates (`lrm: 1.0 -> 0.5`: `-0.0296` BPB to `2.0783`; `lrm: 1.0 -> 0.2`: `-0.0101` BPB to `2.0682`) improves 3-seed 360-step `d6` validation BPB from `2.1273 ± 0.0001` to **`2.0682 ± 0.0006` DEV BPB (`2.0315 ± 0.0004` VAL-B BPB, `292.66` PPL)**—within **`0.0150` BPB** of matched-subset Exact Backpropagation (`2.0532 ± 0.0008`). On the first held-out validation batch (`split="val"`, `B=8, T=256`, Seed 46), the co-designed schedule increases causal attention-map relative Frobenius shift $\|\mathbf{A}^{(l)}(360) - \mathbf{A}^{(l)}(0)\|_F / \|\mathbf{A}^{(l)}(0)\|_F$ by **$9.82\times$ in Layer 0 (`0.0838 -> 0.8233`)** and **$1.42\times$ averaged across all 6 layers (`0.6046 -> 0.8587`)** (`geo_and_attn_steering_audit.json`: per-layer shifts `[0.0838, 0.5327, 0.6784, 0.7494, 0.7733, 0.8100] -> [0.8233, 0.8472, 0.8682, 0.8639, 0.8690, 0.8807]`) even with $W_Q, W_K$ frozen. Notably, matched-subset Exact Backpropagation also improves by **`-0.0548` BPB (`2.1080 ± 0.0006 -> 2.0532 ± 0.0008`)** under the same optimizer package, whereas applying the same package to `SR-LNP (K=16)` (`dust_K16_matched_codesigned_opt`) yields **`2.2602 ± 0.0010` vs. `2.2603 ± 0.0018` DEV BPB** (`-0.0001` BPB), establishing an empirical interaction between the optimizer package and the credit estimator in that `SR-LNP` configuration.
3. **Three-Tier Attention Credit-Assignment Hierarchy ($W_V$ vs. $W_Q$ vs. $W_K$), Key Truncation vs. Future-Query Accumulation, and 5-Matrix Spherical Active-$W_Q$ (`exp45`, `exp47` Part 1):** Building on the per-head attention-output reward, token-local query credit ($\gamma = 0$), and discounted future-token key credit ($\gamma = 0.98$) formulations of Dust [9] (`Section 2.3`) and concurrent `general_pretraining_18/v8` [5], extending our backward-free Stage-1 estimator to `attn.c_q` ($W_Q$) and `attn.c_k` ($W_K$) reveals a three-tier hierarchy in single-draw ($K=1$) credit assignment at $M_g = 1$, governed by the effective coupled subspace dimension $m_{\text{eff}}$. In our diagnostic suite (`run_attention_subspace_cosine_diagnostic`), pre-attention activations are held fixed and downstream gradient $\nabla_{y_{\text{attn}}}\mathcal{L}$ is supplied as an oracle downstream vector (`gy_attn` from `loss.backward()` at initialized `seed = 46`, Layer 0) specifically to isolate single-block attention transport mechanics from multi-layer delayed credit variance:
   - **Tier 1 (`attn.c_v`, $m_{\text{eff}} = 1$, Coordinate-Isolated & Forward-Factorizable via $\bar{x}_{\text{attn\_in}}^{(h)}$):** By Proposition 2, perturbing $y_{\text{attn}}^{(h)}(b, \tau)$ coordinate-by-coordinate and contracting against $\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)$ isolates every head coordinate without cross-coordinate or cross-token coupling, satisfying an exact conditional single-block reconstruction identity **`cos = 1.0000`** given oracle downstream credit at $K=1$ (distinct from its `+0.9069` end-to-end delayed backward-free cosine in Table 1).
   - **Tier 2 (`attn.c_q`, $m_{\text{eff}} = d_h = 64$, Token-Local in Time $\tau$ / Coupled Across $d_h$ Head Channels):** Because query $q_\tau^{(h)}$ enters only row $\tau$ of the causal attention matrix $\mathbf{A}_{b, h, \tau, :}$, perturbing $q_\tau^{(h)} \mapsto q_\tau^{(h)} + \sigma \tilde{\xi}_{q, \tau}^{(h)}$ alters only output token $\tau$'s attention output $\Delta y_{\text{attn}}^{(h)}(b, \tau)$, so **zero cross-time interference arises at $M_g = 1$**. However, all $d_h = 64$ coordinates of $q_\tau^{(h)}$ mix into the scalar dot product $\langle q_\tau^{(h)}, k_s^{(h)}\rangle / \sqrt{d_h}$, sharing a single per-head reward $R_{q, \tau}^{(h)} = \sum_{c=1}^{d_h} R_{\text{attn\_proj}, \tau, (h, c)}$. For this specific single-draw per-head estimator at the initialized diagnostic state (`attention_subspace_cosine_diagnostic`), empirical cosine alignment is **`cos = 0.1357` at $K=1$** (comparable to $1 / \sqrt{d_h} = 0.1250$), scaling with draw count $K$ to **`0.2657` at $K=4$, `0.4748` at $K=16$, and `0.7385` at $K=64$**.
   - **Tier 3 (`attn.c_k`, Non-Local Across Future Queries $\tau \ge s$, Truncation Bias vs. Discounted Future-Query Credit):** Key $k_s^{(h)}$ enters the attention distribution of *every future query* $\tau \in \{s, \dots, T-1\}$ via the query-dependent softmax Jacobian $\operatorname{diag}(A_\tau) - A_\tau A_\tau^T$, which does not factorize into a query-independent forward trace. Conditioned on fixed pre-attention activations and downstream sensitivity vectors $\{g_\tau\}$, the discounted future-query key estimator satisfies the first-order expectation identity $\mathbb{E}[\widehat{g}_{k_s}^{(\gamma_K)}] = \sum_{\tau \ge s} \gamma_K^{\tau - s} J_{\tau s}^\top g_\tau + O(\sigma)$. Multiplying `r_head` at position $s$ by $\xi_{k,s}$ at the *same* position $s$ ($\gamma_K = 0$) captures only the diagonal self-attention sensitivity $J_{ss}^\top g_s$ while omitting all future-query credit $\sum_{\tau > s} J_{\tau s}^\top g_\tau$ (`cos = 0.0076` at $K=1$, `0.0534` at $K=64$). In our pure-Python 2-token finite-difference Rademacher enumeration (`eps_fd = 1e-5`, verified to tolerance `1e-9` in `verify_paper5.py`), same-position key correlation yields $(0.0, 0.0)$ across all 16 perturbation patterns when loss is supported at token 2, whereas accumulating future query rewards $\widetilde{R}_{k,s}^{(\gamma_K)} = \sum_{\tau \ge s, \tau - s \le W_l} \gamma_K^{\tau - s} R_{\text{head},\tau}$ at $\gamma_K = 1$ restores all future terms and matches the finite-difference key derivative $[0.317182, 0.422909]$. Across an 8-point discount horizon sweep $\gamma_K \in \{0.00, 0.50, 0.75, 0.85, 0.90, 0.95, 0.98, 1.00\}$ on `d6` Layer 0 (`exp47` Part 1), $K=64$ cosine alignment rises from **`0.0534` ($\gamma_K=0.00$) $\to$ `0.0887` ($0.50$) $\to$ `0.1171` ($0.75$) $\to$ `0.1329` ($0.85$) $\to$ `0.1423` ($0.90$) $\to$ `0.1529` ($0.95$) $\to$ `0.1565` ($0.98$)** before dipping to **`0.1340` ($\gamma_K=1.00$)**: $\gamma_K = 1$ removes truncation bias completely, while $\gamma_K \in [0.85, 0.98]$ trades a small exponential discount bias for lower cross-time key superposition variance across the $T - s$ future query terms.
   - **5-Matrix Spherical Pre-Polar Muon for `attn.c_q` (`2.0682 -> 2.0641 ± 0.0006` DEV BPB) & Factorized Controls (`exp45`):** Because `q_r = norm(apply_rotary_emb(q, cos, sin))` is $0$-homogeneous in $\|W_Q^{(l)}\|_F$, standard unconstrained `Pre-Polar Muon` updates (`wm_G2048_codesigned_active_q_snr_matched_triton`, $\eta_Q = 0.001$: `2.0681 ± 0.0006`) or properly bound per-head scalar temperatures (`wm_G2048_codesigned_per_head_temp_triton`, where `block.attn.head_log_temp` is bound to the module for evaluation parity: `2.0681 ± 0.0006`) leave DEV BPB virtually unchanged (`-0.0001` BPB). By combining (i) unit-RMS tangent query perturbation, (ii) spherical Frobenius retraction $\|W_Q^{(l)}\|_F \equiv \|W_{Q, 0}^{(l)}\|_F$, (iii) high momentum $\beta_Q = 0.95$, and (iv) $\sqrt{d_h}$-scaled angular step size $\eta_Q = \sqrt{d_h}\,\eta_V = 0.048$, `wm_G2048_codesigned_spherical_active_q_triton` reaches **`2.0641 / 2.0636 / 2.0647` $\to$ `2.0641 ± 0.0006` DEV BPB (`2.0273 ± 0.0005` VAL-B BPB, `289.43` PPL)**—improving upon the 4-matrix baseline by **`-0.0041` BPB (`-3.23` PPL)** and capturing **`45.6%`** of the exact-BP 4-to-5-matrix $W_Q$-unfreezing gain (`2.0532 -> 2.0442`, `-0.0090` BPB). To factorize the geometric vs. hyperparameter components cleanly, we evaluate two matched-hyperparameter ($\beta_Q = 0.95, \eta_Q = 0.048$) Euclidean controls in `exp45`: (a) `wm_G2048_codesigned_euclidean_active_q_matched_hp_triton` (`spherical_qk = False, unit_rms_qk_pert = False, tangent_proj_qk = True`), which removes spherical Frobenius retraction and unit-RMS perturbation scaling while retaining tangent projection (`2.0648 / 2.0643 / 2.0655` $\to$ **`2.0649 ± 0.0006` DEV BPB, `290.02` PPL**); and (b) `wm_G2048_codesigned_unprojected_active_q_matched_hp_triton` (`tangent_proj_qk = False, spherical_qk = False, unit_rms_qk_pert = False`), which additionally removes query tangent projection (`2.0648 / 2.0643 / 2.0655` $\to$ **`2.0649 ± 0.0006` DEV BPB, `290.03` PPL**). Both controls attribute `-0.0033` BPB of the `-0.0041` BPB gain to step-size/momentum matching ($\eta_Q=0.048, \beta_Q=0.95$) and `-0.0008` BPB specifically to spherical Frobenius retraction + unit-RMS perturbation scaling, while confirming that explicit query tangent projection is redundant up to $O(\sigma^2)$ because `norm(q)` projects onto the tangent space automatically.
4. **Full 6-Matrix (`100%` Active Parameters) Joint $(W_Q, W_K)$ Routing via Decoupled Forward Probes, Discounted Future-Query Key Credit, and Amortized Multi-Probe ($M=2$) Probing (`exp47_d6_6mat_joint_qk_results.json`):**
   In `exp45`, naively activating both $W_Q$ and $W_K$ (`wm_G2048_codesigned_spherical_active_qk_triton`: `2.0669 ± 0.0005` DEV BPB) degraded performance relative to 5-matrix Active-$W_Q$ (`2.0641 ± 0.0006`). In `exp47` Part 2, a controlled 8-arm 3-seed study on ClimbMix `d6` ($26.35\text{M} / 26.35\text{M} = 100\%$ active matrix parameters) diagnoses and resolves all three causes of that degradation:
   - **Cause 1 — Cross-Probe Perturbation Contamination in Joint SDPA (`-0.0014` BPB Main Effect from Decoupling):** Perturbing both queries and keys simultaneously in a single FlashAttention call `sdpa(q_r_p, k_r_p, v)` produces $\Delta y_{qk} = \Delta y_q + \Delta y_k + O(\sigma^2)$, so the query reward $R_q \propto \langle w_{\text{head}}, \Delta y_{qk}\rangle$ is directly contaminated by the independent key perturbation $\Delta y_k(\xi_k)$ (and vice versa), doubling local probe variance. Evaluating **decoupled local attention probes** $\Delta y_q = \text{sdpa}(q_{\text{pert}}, k_{\text{clean}}, v_{\text{clean}}) - y_{\text{clean}}$ and $\Delta y_k = \text{sdpa}(q_{\text{clean}}, k_{\text{pert}}, v_{\text{clean}}) - y_{\text{clean}}$ at consumption time ($t-2$) eliminates cross-probe contamination: at $\gamma_K = 0.0, \eta_K = 0.012$, moving from coupled same-position (`wm_d6_6mat_coupled_qk_samepos_g000_lrk012_triton`: `2.0660 ± 0.0005` DEV BPB) to decoupled same-position (`wm_d6_6mat_decoupled_qk_samepos_g000_lrk012_triton`: `2.0647 ± 0.0005` DEV BPB) improves 6-matrix learning by **`-0.0013` BPB**, and at $\gamma_K = 0.85, \eta_K = 0.012$, moving from coupled future-accumulated (`wm_d6_6mat_coupled_qk_future_g085_lrk012_triton`: `2.0646 ± 0.0001` DEV BPB) to decoupled future-accumulated (`wm_d6_6mat_decoupled_qk_future_g085_lrk012_triton`: **`2.0631 ± 0.0008` DEV BPB, `2.0264 ± 0.0011` VAL-B BPB, `288.64` PPL**) improves learning by **`-0.0015` BPB** (or `-0.0018` BPB over step-multiplexed alternating $W_Q/W_K$ steps `wm_d6_6mat_multiplexed_qk_future_g098_lrk012_triton`: `2.0649 ± 0.0007`).
   - **Cause 2 — Key Truncation vs. Discounted Future-Query Accumulation & SNR-Matched Key Step Size (`-0.0015` BPB Main Effect):** Combining SNR-matched key learning rate $\eta_K = 0.012, \beta_K = 0.98$ with discounted future-query key accumulation improves decoupled 6-matrix learning from `2.0647 ± 0.0005` ($\gamma_K = 0.00$) to **`2.0632 ± 0.0007` DEV BPB (`288.73` PPL)** at $\gamma_K = 0.98$ (`wm_d6_6mat_decoupled_qk_future_g098_lrk012_triton`) and **`2.0631 ± 0.0008` DEV BPB (`288.64` PPL)** at $\gamma_K = 0.85$ (`wm_d6_6mat_decoupled_qk_future_g085_lrk012_triton`), surpassing 5-matrix Active-$W_Q$ (`2.0641 ± 0.0006`) by **`-0.0010` BPB** at $M=1$.
   - **Amortized Multi-Probe ($M=2$) Local $(W_Q, W_K)$ Probing (`2.0631 -> 2.0595 ± 0.0007` DEV BPB, `-0.0036` BPB Gain):** Because the clean pre-synaptic input $x_{\text{attn\_in}}(t-2)$ and outgoing receiver columns $W_{\text{attn\_proj}}^{(l)}(t)$ are shared across local attention probes on the same clean forward trunk, averaging $M = 2$ independent Rademacher query/key probes inside the post-synaptic modulation vector $\bar{u}^{(1)} = \frac{1}{M}\sum_{r=1}^M R^{(1, r)} (\xi^{(1, r)}/\sigma)$ *before* the single Stage-3 `bmm` (`-(bar_u.t() @ x_attn_in) / BT`) and single `Pre-Polar Muon` orthogonalization step halves local Rademacher probe variance with **zero additional Stage-3 outer-product matmuls or Muon steps** (`8.22s -> 9.34s` 360-step run GPU time). Across Seeds `46, 47, 48`, $M=2$ decoupled 6-matrix Delayed WM reaches **`2.0596 / 2.0588 / 2.0604` $\to$ `2.0596 ± 0.0008` DEV BPB (`2.0228 ± 0.0006` VAL-B BPB, `285.87` PPL)** (`wm_d6_6mat_decoupled_qk_future_g085_m2_lrk012_triton`), and **`2.0595 / 2.0588 / 2.0602` $\to$ `2.0595 ± 0.0007` DEV BPB (`2.0227 ± 0.0008` VAL-B BPB, `285.81` PPL)** (`wm_d6_6mat_decoupled_qk_future_g085_m2_rng_matched_lrk012_triton`, which draws probe 2 from an isolated CUDA RNG generator so the main model/data RNG stream is bit-for-bit identical to $M=1$). This represents a **`-0.0087` BPB (`-6.85` PPL)** improvement over 4-matrix Delayed WM (`2.0682 ± 0.0006`), a **`-0.0046` BPB (`-3.62` PPL)** improvement over 5-matrix Active-$W_Q$ (`2.0641 ± 0.0006`), and narrows the gap to 4-matrix Exact Backpropagation (`2.0532 ± 0.0008`) to **`+0.0063` BPB** (while capturing **`45.0%`** of the full 4-to-6-matrix exact-BP gain `2.0532 -> 2.0316`).

| Method (`ClimbMix d6`, Strict Post-Install, 360 Steps, Seeds `46, 47, 48`) | Active Params (Matrices) | Sparse Embed Opt (`lr_wte / lr_ve`) | Drift Decay `lrm_min` | Block-0 `mlp.c_proj` srank | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Run GPU / Est. (`s`) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| `wm_G2048_paper20_opt_triton` (Paper #20 v1 Opt, Fused Triton) | $24.58\text{M}$ ($4$) | `0.004 / 0.004` | $1.0$ | $127.24$ | $2.1274 / 2.1274 / 2.1272$ | $2.1273 \pm 0.0001$ | $2.0900 \pm 0.0001$ | $344.26$ | $6.95 / 6.72$ |
| `wm_G2048_scale_embed_const_lr` (+ Scale-Matched Sparse Embeds) | $24.58\text{M}$ ($4$) | `0.120 / 0.060` | $1.0$ | --- | $2.1077 / 2.1077 / 2.1084$ | $2.1079 \pm 0.0004$ | $2.0716 \pm 0.0003$ | $326.41$ | $6.95 / 6.72$ |
| `wm_G2048_scale_embed_lrm05` (+ Moderate Drift Decay `lrm -> 0.5`) | $24.58\text{M}$ ($4$) | `0.120 / 0.060` | $0.5$ | --- | $2.0780 / 2.0780 / 2.0788$ | $2.0783 \pm 0.0005$ | $2.0419 \pm 0.0003$ | $300.89$ | $6.76 / 6.72$ |
| **`wm_G2048_codesigned_opt_triton` (4-Matrix Co-Designed Delayed WM)** | **$24.58\text{M}$ ($4$)** | **`0.120 / 0.060`** | **$0.2$** | **$141.33$** | **$2.0681 / 2.0676 / 2.0688$** | **$2.0682 \pm 0.0006$** | **$2.0315 \pm 0.0004$** | **$292.66$** | **$6.74 / 6.72$** |
| `wm_G2048_codesigned_per_head_temp_triton` (`exp45`, Attached Per-Head Temp $\tau_{l,h}$) | $24.58\text{M}$ ($4+\tau$) | `0.120 / 0.060` | $0.2$ | --- | $2.0681 / 2.0676 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.64$ | $7.22 / \text{---}$ |
| `wm_G2048_codesigned_active_q_snr_matched_triton` (`exp45`, Unconstrained $\eta_Q=10^{-3}$) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0680 / 2.0675 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0314 \pm 0.0004$ | $292.56$ | $12.20 / \text{---}$ |
| `wm_G2048_codesigned_active_qk_snr_matched_triton` (`exp45`, Unconstrained $\eta_{QK}=10^{-3}$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0680 / 2.0675 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.60$ | $12.48 / \text{---}$ |
| `wm_G2048_codesigned_euclidean_active_q_matched_hp_triton` (`exp45`, Euclidean Control: Tangent Proj Only) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0648 / 2.0643 / 2.0655$ | $2.0649 \pm 0.0006$ | $2.0281 \pm 0.0005$ | $290.02$ | $7.41 / \text{---}$ |
| `wm_G2048_codesigned_unprojected_active_q_matched_hp_triton` (`exp45`, Unprojected Euclidean Control) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0648 / 2.0643 / 2.0655$ | $2.0649 \pm 0.0006$ | $2.0281 \pm 0.0005$ | $290.03$ | $7.08 / \text{---}$ |
| **`wm_G2048_codesigned_spherical_active_q_triton` (`exp45`, Spherical Active-$W_Q$, $\eta_Q=0.048, \beta_Q=0.95$)** | **$25.46\text{M}$ ($5$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0641 / 2.0636 / 2.0647$** | **$2.0641 \pm 0.0006$** | **$2.0273 \pm 0.0005$** | **$289.43$** | **$11.49 / \text{---}$** |
| `wm_G2048_codesigned_spherical_active_k_triton` (`exp45`, Same-Pos Truncated Active-$W_K$) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0683 / 2.0680 / 2.0689$ | $2.0684 \pm 0.0005$ | $2.0317 \pm 0.0004$ | $292.85$ | $11.67 / \text{---}$ |
| `wm_G2048_codesigned_spherical_active_k_future_g098_triton` (`exp45`, Future-Accum Active-$W_K, \gamma_K=0.98$) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0671 / 2.0667 / 2.0678$ | $2.0672 \pm 0.0006$ | $2.0305 \pm 0.0006$ | $291.90$ | $7.47 / \text{---}$ |
| `wm_G2048_codesigned_spherical_active_qk_triton` (`exp45`, Coupled Same-Pos Active-$W_Q+W_K, \eta_K=0.048$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0670 / 2.0664 / 2.0674$ | $2.0669 \pm 0.0005$ | $2.0303 \pm 0.0004$ | $291.66$ | $12.82 / \text{---}$ |
| **Full 6-Matrix (`100%` Active Matrix Params) Joint $(W_Q, W_K)$ Routing & Amortized Multi-Probe Ablation (`exp47`, $\eta_Q=0.048, \eta_K=0.012, \beta_K=0.98$):** | | | | | | | | | |
| `wm_d6_6mat_coupled_qk_samepos_g000_lrk012_triton` (`exp47`, Coupled Probes, $\gamma_K=0.00, M=1$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0659 / 2.0655 / 2.0665$ | $2.0660 \pm 0.0005$ | $2.0293 \pm 0.0003$ | $290.91$ | $8.28 / \text{---}$ |
| `wm_d6_6mat_coupled_qk_future_g085_lrk012_triton` (`exp47`, Coupled Probes, $\gamma_K=0.85, M=1$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0645 / 2.0645 / 2.0647$ | $2.0646 \pm 0.0001$ | $2.0279 \pm 0.0001$ | $289.84$ | $8.11 / \text{---}$ |
| `wm_d6_6mat_decoupled_qk_samepos_g000_lrk012_triton` (`exp47`, Decoupled Probes, $\gamma_K=0.00, M=1$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0645 / 2.0643 / 2.0652$ | $2.0647 \pm 0.0005$ | $2.0279 \pm 0.0005$ | $289.85$ | $8.26 / \text{---}$ |
| `wm_d6_6mat_multiplexed_qk_future_g098_lrk012_triton` (`exp47`, Step-Multiplexed $W_Q/W_K, \gamma_K=0.98$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0649 / 2.0642 / 2.0655$ | $2.0649 \pm 0.0007$ | $2.0282 \pm 0.0006$ | $290.08$ | $7.46 / \text{---}$ |
| **`wm_d6_6mat_decoupled_qk_future_g098_lrk012_triton` (`exp47`, Decoupled Probes, $\gamma_K=0.98, M=1$)** | **$26.35\text{M}$ ($6$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0631 / 2.0626 / 2.0640$** | **$2.0632 \pm 0.0007$** | **$2.0265 \pm 0.0010$** | **$288.73$** | **$8.35 / \text{---}$** |
| **`wm_d6_6mat_decoupled_qk_future_g085_lrk012_triton` (`exp47`, Decoupled Probes, $\gamma_K=0.85, M=1$)** | **$26.35\text{M}$ ($6$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0628 / 2.0625 / 2.0640$** | **$2.0631 \pm 0.0008$** | **$2.0264 \pm 0.0011$** | **$288.64$** | **$8.22 / \text{---}$** |
| **`wm_d6_6mat_decoupled_qk_future_g085_m2_lrk012_triton` (`exp47`, Decoupled, $\gamma_K=0.85, M=2$)** | **$26.35\text{M}$ ($6$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0596 / 2.0588 / 2.0604$** | **$2.0596 \pm 0.0008$** | **$2.0228 \pm 0.0006$** | **$285.87$** | **$9.34 / \text{---}$** |
| **`wm_d6_6mat_decoupled_qk_future_g085_m2_rng_matched_lrk012_triton` (`exp47`, Decoupled, $\gamma_K=0.85, M=2$ Isolated RNG)** | **$26.35\text{M}$ ($6$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0595 / 2.0588 / 2.0602$** | **$2.0595 \pm 0.0007$** | **$2.0227 \pm 0.0008$** | **$285.81$** | **$9.35 / \text{---}$** |
| `dust_K16_tuned_fast` (`SR-LNP` $K=16$) | $24.58\text{M}$ ($4$) | `0.008 / 0.008` | $0.1$ | --- | $2.2583 / 2.2615 / 2.2612$ | $2.2603 \pm 0.0018$ | $2.2236 \pm 0.0014$ | $496.07$ | $9.43 / 9.23$ |
| `dust_K16_matched_codesigned_opt` (`SR-LNP` $K=16$ + Co-Designed Opt) | $24.58\text{M}$ ($4$) | `0.120 / 0.060` | $0.2$ | --- | $2.2591 / 2.2603 / 2.2611$ | $2.2602 \pm 0.0010$ | $2.2234 \pm 0.0003$ | $495.83$ | $9.71 / 9.23$ |
| `backprop_d6_paper20_opt` (Exact BP, Paper #20 v1 Opt, 4-Matrix Subset) | $24.58\text{M}$ ($4$) | `0.004 / 0.004` | $1.0$ | --- | $2.1084 / 2.1073 / 2.1084$ | $2.1080 \pm 0.0006$ | $2.0708 \pm 0.0006$ | $326.54$ | $6.38 / 4.14$ |
| **`backprop_d6_codesigned_opt` (Exact BP, Co-Designed Opt, 4-Matrix Subset)** | **$24.58\text{M}$ ($4$)** | **`0.120 / 0.060`** | **$0.2$** | --- | **$2.0523 / 2.0536 / 2.0536$** | **$2.0532 \pm 0.0008$** | **$2.0160 \pm 0.0007$** | **$280.86$** | **$6.56 / 4.14$** |
| `backprop_d6_5mat_active_q_codesigned` (`exp45`, Exact BP, 5-Matrix Active-$W_Q$) | $25.46\text{M}$ ($5$) | `0.120 / 0.060` | $0.2$ | --- | $2.0428 / 2.0450 / 2.0448$ | $2.0442 \pm 0.0012$ | $2.0071 \pm 0.0012$ | $274.01$ | $8.90 / \text{---}$ |
| `backprop_d6_6mat_active_qk_codesigned` (`exp45`, Exact BP, Full 6-Matrix Active-$W_Q+W_K$) | $26.35\text{M}$ ($6$) | `0.120 / 0.060` | $0.2$ | --- | $2.0306 / 2.0319 / 2.0322$ | $2.0316 \pm 0.0009$ | $1.9944 \pm 0.0006$ | $264.68$ | $8.56 / \text{---}$ |

**Table 5:** 3-seed optimizer co-design, factorized component ablations, 5-matrix Active-$W_Q/W_K$ routing (`exp45_d6_attn_routing_results.json`), and full 6-matrix (`100%` active parameters) joint $(W_Q, W_K)$ routing + $M=2$ amortized multi-probe ablations (`exp47_d6_6mat_joint_qk_results.json`) on ClimbMix `d6` (`exp37_39_master_results.json`), reporting both measured 3-seed training GPU time (`Run GPU`) and 30-step steady-state benchmark estimates (`Est.`).

### 4.7 Comparison with Surrogate-Routed (`SR-LNP`) and Suffix-Recomputed (`SR-NP`) Node Perturbation Across Tokens, Compute, and Width $D \in \{256, 384, 512, 768\}$ (`exp37`, `exp39`, `exp42`)

Table 6 reports our 3-seed comparison between **Fused Triton Delayed WM**, **Surrogate-Routed Local Node Perturbation (`SR-LNP`, $K \in \{4, 16, 64\}$)**, and **Suffix-Recomputed Node Perturbation (`SR-NP`, $K_{\text{site}}=4, K_{\text{tot}}=76$ suffix-recomputed block draws $+ 24$ local attention draws/step, $\gamma = 0.98$, `exp42`)** across equal token budgets ($360$ steps $= 0.737\text{M}$ tokens), equal wall-clock compute ($500$ steps $= 9.41\text{s}$ run GPU matching `SR-LNP K=16`'s $9.43\text{s}$, and $720$ steps $= 13.57\text{s}$ run GPU vs. `SR-LNP K=64`'s $23.66\text{s}$), and model widths $D \in \{256, 384, 512, 768\}$ (`exp37_39_master_results.json`, `exp42_faithful_dust_results.json`). We make no claim of superiority over published Dust [9]:
1. **Equal-Token Comparison on `d6` ($360$ Steps, $D=384$):** `wm_G2048_codesigned_opt_triton` (`2.0682 ± 0.0006` DEV BPB in `6.74s` run GPU / `6.72s` est.) outperforms `SR-LNP (K=4)` (`2.3071 ± 0.0016` in `6.32s` / `5.48s` est.) by **`-0.2389` BPB (`-48.1%` PPL)**, `SR-LNP (K=16)` (`2.2603 ± 0.0018` in `9.43s` / `9.23s` est.) by **`-0.1921` BPB (`-41.0%` PPL) in `1.40x` less run GPU time**, `SR-LNP (K=64)` (`2.2041 ± 0.0013` in `23.66s` / `23.22s` est.) by **`-0.1359` BPB (`-31.2%` PPL) in `3.51x` less run GPU time**, and Suffix-Recomputed `SR-NP (K_tot=76)` under `Adam` (`2.4264 ± 0.0088` in `84.38s`) and `Pre-Polar Muon` (`2.2415 ± 0.0015` in `65.19s`) by **`-0.3582` BPB** and **`-0.1733` BPB in `9.67x` less GPU time**. Even at $G=64$ (`5.79s` run GPU / `5.70s` est.), `wm_G64_codesigned_opt_triton` (`2.2415 ± 0.0011`) matches `SR-NP (K_tot=76, Muon)` in $11.3\times$ less GPU time.
2. **Equal-Compute Comparison (`9.41s` and `13.57s` Run GPU on `d6`, $D=384$):** At equal wall-clock time to `SR-LNP (K=16)` (`9.41s` vs. `9.43s` run GPU; `9.34s` vs. `9.23s` est., `500` steps), `wm_G2048_codesigned_500steps_match_dustK16_time` reaches **`2.0322 ± 0.0007` DEV BPB (`1.9975 ± 0.0006` VAL-B BPB, `265.13` PPL)**—outperforming `SR-LNP (K=16)` by **`-0.2281` BPB (`-46.6%` PPL)**. At `720` steps (`13.57s` run GPU / `13.45s` est., `1.74x` faster than `SR-LNP K=64` and `4.80x` faster than `SR-NP K_tot=76`), `wm_G2048_codesigned_720steps_half_dustK64_time` breaks the `2.00` DEV BPB barrier backward-free at **`1.9989 ± 0.0008` DEV BPB (`1.9646 ± 0.0008` VAL-B BPB, `241.96` PPL)**—outperforming `SR-LNP (K=64)` by **`-0.2052` BPB (`-43.1%` PPL)**.
3. **Model Width Scaling ($D \in \{256, 384, 512, 768\}$):** As width $D$ grows from `256 -> 384 -> 512 -> 768` (`14.55M -> 67.11M` active parameters), `Delayed WM`'s gap to matched-subset Exact Backpropagation **shrinks from `+0.0144` BPB to `+0.0125` BPB** (`2.0933` vs. `2.0789` at $D=256$; `2.0558 ± 0.0002` vs. `2.0433 ± 0.0006` at $D=768$), while `Delayed WM` at the smallest width $D=256$ (`2.0933 ± 0.0004` in `5.03s` run GPU / `4.83s` est.) outperforms `SR-LNP (K=64)` at the largest width $D=768$ (`2.1729 ± 0.0004` in `95.74s` run GPU / `57.26s` est.) by **`-0.0796` BPB**.

| Width $D$ (`Active Params`) | Method (`exp37`, `exp39`, `exp42`, Seeds `46, 47, 48`) | Steps (Tokens) | Perturb Passes $K$ ($G$) | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Run GPU / Est. (`s`) | Gap vs. Exact BP / Comparator |
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| $384$ ($24.58\text{M}$) | `dust_K4_tuned_fast` (`SR-LNP` $K=4$) | $360$ ($0.74\text{M}$) | $4$ (Token) | $2.3054 / 2.3074 / 2.3085$ | $2.3071 \pm 0.0016$ | $2.2705 \pm 0.0018$ | $564.03$ | $6.32 / 5.48$ | $+0.2539$ vs. BP |
| $384$ ($24.58\text{M}$) | `dust_K16_tuned_fast` (`SR-LNP` $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2583 / 2.2615 / 2.2612$ | $2.2603 \pm 0.0018$ | $2.2236 \pm 0.0014$ | $496.07$ | $9.43 / 9.23$ | $+0.2071$ vs. BP |
| $384$ ($24.58\text{M}$) | `dust_K64_tuned_fast` (`SR-LNP` $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.2026 / 2.2050 / 2.2047$ | $2.2041 \pm 0.0013$ | $2.1669 \pm 0.0006$ | $425.09$ | $23.66 / 23.22$ | $+0.1509$ vs. BP |
| $384$ ($24.58\text{M}$) | `faithful_dust_K76_adam` (`SR-NP` $K_{\text{tot}}=76$, Adam) | $360$ ($0.74\text{M}$) | $76$ (Suffix) | $2.4167 / 2.4340 / 2.4284$ | $2.4264 \pm 0.0088$ | $2.3865 \pm 0.0092$ | $782.77$ | $84.38 / 84.38$ | $+0.3732$ vs. BP |
| $384$ ($24.58\text{M}$) | `faithful_dust_K76_prepolar_muon` (`SR-NP` $K_{\text{tot}}=76$, Muon) | $360$ ($0.74\text{M}$) | $76$ (Suffix) | $2.2398 / 2.2424 / 2.2424$ | $2.2415 \pm 0.0015$ | $2.2048 \pm 0.0012$ | $471.11$ | $65.19 / 65.19$ | $+0.1883$ vs. BP |
| $384$ ($24.58\text{M}$) | `wm_G64_codesigned_opt_triton` | $360$ ($0.74\text{M}$) | $1$ ($64$) | $2.2406 / 2.2413 / 2.2427$ | $2.2415 \pm 0.0011$ | $2.2043 \pm 0.0013$ | $471.15$ | $5.79 / 5.69$ | $-0.0188$ vs. `SR-LNP` $K=16$ |
| **$384$ ($24.58\text{M}$)** | **`wm_G2048_codesigned_opt_triton` (Equal Tokens)** | **$360$ ($0.74\text{M}$)** | **$1$ ($2,048$)** | **$2.0681 / 2.0676 / 2.0688$** | **$2.0682 \pm 0.0006$** | **$2.0315 \pm 0.0004$** | **$292.66$** | **$6.74 / 6.72$** | **$-0.1733$ vs. `SR-NP` ($9.7\times$ faster)** |
| **$384$ ($24.58\text{M}$)** | **`wm_G2048_codesigned_500steps_match_dustK16_time`** | **$500$ ($1.02\text{M}$)** | **$1$ ($2,048$)** | **$2.0321 / 2.0315 / 2.0329$** | **$2.0322 \pm 0.0007$** | **$1.9975 \pm 0.0006$** | **$265.13$** | **$9.41 / 9.34$** | **$-0.2281$ at $=$Time (`SR-LNP` $K=16$)** |
| **$384$ ($24.58\text{M}$)** | **`wm_G2048_codesigned_720steps_half_dustK64_time`** | **$720$ ($1.47\text{M}$)** | **$1$ ($2,048$)** | **$1.9990 / 1.9981 / 1.9996$** | **$1.9989 \pm 0.0008$** | **$1.9646 \pm 0.0008$** | **$241.96$** | **$13.57 / 13.45$** | **$-0.2052$ vs. `SR-LNP` $K=64$** |
| $256$ ($14.55\text{M}$) | `width256_dust_K16_tuned` (`SR-LNP` $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2978 / 2.2984 / 2.2982$ | $2.2981 \pm 0.0003$ | $2.2614 \pm 0.0003$ | $550.35$ | $7.19 / 6.53$ | $+0.2192$ vs. BP ($D=256$) |
| $256$ ($14.55\text{M}$) | `width256_dust_K64_tuned` (`SR-LNP` $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.2369 / 2.2374 / 2.2380$ | $2.2374 \pm 0.0006$ | $2.1999 \pm 0.0004$ | $465.84$ | $28.60 / 14.88$ | $+0.1585$ vs. BP ($D=256$) |
| **$256$ ($14.55\text{M}$)** | **`width256_wm_G2048_codesigned`** | **$360$ ($0.74\text{M}$)** | **$1$ ($2,048$)** | **$2.0935 / 2.0929 / 2.0936$** | **$2.0933 \pm 0.0004$** | **$2.0566 \pm 0.0003$** | **$313.60$** | **$5.03 / 4.83$** | **$+0.0144$ vs. BP / $-0.1441$ vs. $K=64$** |
| $256$ ($14.55\text{M}$) | `width256_backprop_codesigned` | $360$ ($0.74\text{M}$) | $0$ (`1 bwd`) | $2.0789 / 2.0778 / 2.0800$ | $2.0789 \pm 0.0011$ | $2.0415 \pm 0.0009$ | $301.41$ | $7.62 / 3.12$ | Exact BP ($D=256$) |
| $512$ ($36.70\text{M}$) | `width512_dust_K16_tuned` (`SR-LNP` $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2359 / 2.2359 / 2.2369$ | $2.2362 \pm 0.0006$ | $2.2006 \pm 0.0002$ | $464.32$ | $13.95 / 12.62$ | $+0.1920$ vs. BP ($D=512$) |
| $512$ ($36.70\text{M}$) | `width512_dust_K64_tuned` (`SR-LNP` $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.1878 / 2.1892 / 2.1883$ | $2.1884 \pm 0.0007$ | $2.1521 \pm 0.0009$ | $407.16$ | $33.88 / 32.90$ | $+0.1442$ vs. BP ($D=512$) |
| **$512$ ($36.70\text{M}$)** | **`width512_wm_G2048_codesigned`** | **$360$ ($0.74\text{M}$)** | **$1$ ($2,048$)** | **$2.0582 / 2.0590 / 2.0583$** | **$2.0585 \pm 0.0004$** | **$2.0219 \pm 0.0003$** | **$285.02$** | **$9.59 / 9.28$** | **$+0.0143$ vs. BP / $-0.1299$ vs. $K=64$** |
| $512$ ($36.70\text{M}$) | `width512_backprop_codesigned` | $360$ ($0.74\text{M}$) | $0$ (`1 bwd`) | $2.0440 / 2.0448 / 2.0439$ | $2.0442 \pm 0.0005$ | $2.0072 \pm 0.0003$ | $274.06$ | $7.38 / 5.40$ | Exact BP ($D=512$) |
| $768$ ($67.11\text{M}$) | `width768_dust_K16_tuned` (`SR-LNP` $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2113 / 2.2114 / 2.2112$ | $2.2113 \pm 0.0001$ | $2.1754 \pm 0.0003$ | $433.55$ | $23.50 / 22.11$ | $+0.1680$ vs. BP ($D=768$) |
| $768$ ($67.11\text{M}$) | `width768_dust_K64_tuned` (`SR-LNP` $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.1733 / 2.1730 / 2.1725$ | $2.1729 \pm 0.0004$ | $2.1362 \pm 0.0002$ | $390.22$ | $95.74 / 57.26$ | $+0.1296$ vs. BP ($D=768$) |
| **$768$ ($67.11\text{M}$)** | **`width768_wm_G2048_codesigned`** | **$360$ ($0.74\text{M}$)** | **$1$ ($2,048$)** | **$2.0555 / 2.0559 / 2.0559$** | **$2.0558 \pm 0.0002$** | **$2.0189 \pm 0.0003$** | **$282.87$** | **$17.85 / 17.71$** | **$+0.0125$ vs. BP / $-0.1171$ vs. $K=64$** |
| $768$ ($67.11\text{M}$) | `width768_backprop_codesigned` | $360$ ($0.74\text{M}$) | $0$ (`1 bwd`) | $2.0426 / 2.0438 / 2.0434$ | $2.0433 \pm 0.0006$ | $2.0062 \pm 0.0007$ | $273.31$ | $21.47 / 9.54$ | Exact BP ($D=768$) |

**Table 6:** 3-seed comparison between Fused Triton Delayed WM, Surrogate-Routed Local Node Perturbation (`SR-LNP`, $K \in \{4, 16, 64\}$), Suffix-Recomputed Node Perturbation (`SR-NP`, $K_{\text{tot}}=76$), and Exact Backpropagation across equal tokens, equal wall-clock compute, and model width $D \in \{256, 384, 512, 768\}$ (`exp37_39_master_results.json`, `exp42_faithful_dust_results.json`, `benchmark_uncontended_timings.json`). Both measured 3-seed training GPU time (`Run GPU`) and 30-step steady-state benchmark estimates (`Est.`) are reported.

### 4.8 Long-Context Native `50,332,176` Scale Transfer, Isolated Kernel Substitution vs. Combined Package Speedup, and Full 6-Matrix Active-$(W_Q, W_K)$ Pretraining (`exp29`, `exp30`, `exp32`, `exp40`, `exp41`, `exp43`, `exp44`, `exp46`, `exp48`)

Table 7 reports the 3-seed scale-transfer results on the 8-layer, 512-width Native `50,332,176` benchmark ($B_{\text{eff}} = 64, T = 2,048$, $100$ steps $= 13.1\text{M}$ tokens, Seeds `46, 47, 48`, `dev_tokens = 523,826, dev_bytes = 2,126,272`).

| Method (`Native 50,332,176`, $L=8, D=512, V=8,192, T=2,048$, 100 Steps, Seeds `46, 47, 48`) | Active Params | MicroB $\times$ Accum ($M_g$) | Stage-3 / Stage-2 / Stage-1 Clocks | Rev. Matmuls / Bwd | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Fwd / Rule / Wall (`ms`) | Peak VRAM (`GB`) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| `supplied_muon_adamw_backprop` (Full 6-Matrix BP) | $50.33\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/(t,t,t)$ | $-1 / 1$ | $1.6994 / 1.6989 / 1.6984$ | $1.6989 \pm 0.0005$ | $1.6828 \pm 0.0005$ | $119.04$ | $461.2 / 1142.5 / 1884.0$ | $13.84$ |
| `unrestricted_adamw_backprop` (Full 6-Matrix `AdamW` BP) | $50.33\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/(t,t,t)$ | $-1 / 1$ | $1.9894 / 1.9647 / 1.9660$ | $1.9734 \pm 0.0139$ | $1.9580 \pm 0.0139$ | $258.55$ | $462.6 / 1106.7 / 1849.0$ | $14.12$ |
| `frozen_backbone_lm_head_only` (`exp18`, 2 seeds) | $4.19\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/\text{frozen}/\text{frozen}$ | $0 / 0$ | $2.1617 / 2.1602 / \text{---}$ | $2.1610 \pm 0.0011$ | $2.1094 \pm 0.0001$ | $416.69$ | $510.4 / 213.4 / 812.5$ | $5.38$ |
| `native50m_dust_K16_tuned_fast` (`exp41`, `SR-LNP` $K=16$) | $46.14\text{M}$ | $4 \times 16$ (Token) | Same-Step Local Surrogate ($\gamma=0$) | $0 / 0$ | $2.1322 / 2.1327 / 2.1330$ | $2.1326 \pm 0.0004$ | $2.1095 \pm 0.0004$ | $403.57$ | $1027.5 / 2340.1 / 3367.6$ | $9.18$ |
| `pure_lexical_x0_skip_only_true_zero` (`exp21`) | $8.39\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/\text{frozen}$ | $0 / 0$ | $2.0096 / 2.0086 / 2.0094$ | $2.0092 \pm 0.0005$ | $1.9586 \pm 0.0005$ | $272.81$ | $73.6 / 344.4 / 537.4$ | $7.26$ |
| `native50m_true_delayed_construction_M8_adamw` (`exp30`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $2.0742 / 2.0887 / 2.0870$ | $2.0833 \pm 0.0079$ | $2.0627 \pm 0.0059$ | $351.29$ | $1526.4 / 6910.1 / 8650.2$ | $10.34$ |
| `native50m_wm_M8_bt_scaled_triton` (`exp41`, Skip `wte`, `lrm->0.2, lr_wte=0.015`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $2.0251 / 2.0249 / 2.0255$ | $2.0252 \pm 0.0003$ | $2.0054 \pm 0.0003$ | $298.25$ | $1350.0 / 1591.2 / 2941.2$ | $9.12$ |
| `native50m_wm_M8_codesigned_triton` (`exp40`, Skip `wte`, `lrm->0.2, lr_wte=0.10`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9848 / 1.9842 / 1.9846$ | $1.9845 \pm 0.0003$ | $1.9666 \pm 0.0002$ | $266.03$ | $1352.8 / 1591.2 / 2944.0$ | $9.12$ |
| `native50m_causal_trace_wm_M32_filtered_v_tangent_prepolar_muon` (`exp29`) | $46.14\text{M}$ | $4 \times 16$ ($32$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9672 / 1.9670 / 1.9673$ | $1.9672 \pm 0.0002$ | $1.9502 \pm 0.0003$ | $253.34$ | $1420.8 / 1205.1 / 2825.0$ | $9.49$ |
| `native50m_causal_trace_wm_M8_unprojected_prepolar_muon` (`exp29`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9519 / 1.9518 / 1.9514$ | $1.9517 \pm 0.0003$ | $1.9359 \pm 0.0006$ | $242.55$ | $909.0 / 3173.8 / 4283.0$ | $8.92$ |
| `native50m_true_delayed_construction_M8_unprojected_prepolar_muon` (`exp30`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $1.9518 / 1.9522 / 1.9518$ | $1.9520 \pm 0.0002$ | $1.9361 \pm 0.0007$ | $242.73$ | $909.9 / 4776.3 / 5888.0$ | $8.84$ |
| `native50m_causal_trace_wm_M8_filtered_v_tangent_prepolar_muon` (`exp29`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9342 / 1.9334 / 1.9341$ | $1.9339 \pm 0.0004$ | $1.9187 \pm 0.0001$ | $230.71$ | $1421.6 / 3240.8 / 4864.0$ | $9.50$ |
| `native50m_true_delayed_construction_M8_filtered_v_tangent_prepolar_muon` (`exp30`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $1.9339 / 1.9332 / 1.9339$ | $1.9337 \pm 0.0004$ | $1.9186 \pm 0.0001$ | $230.58$ | $1420.7 / 5195.8 / 6819.0$ | $9.41$ |
| `native50m_strict_post_install_M8_unprojected_prepolar_muon` (`exp32`) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9823 / 1.9837 / 1.9839$ | $1.9833 \pm 0.0009$ | $1.9653 \pm 0.0013$ | $265.12$ | $916.8 / 6006.2 / 7128.6$ | $8.54$ |
| **`native50m_strict_post_install_M8_filtered_v_tangent_prepolar_muon` (`exp32`, CPU-Offload PyTorch `lr_embed=3e-3`)** | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9362 / 1.9357 / 1.9364$ | **$1.9361 \pm 0.0004$** | **$1.9207 \pm 0.0003$** | **$232.14$** | $1425.0 / 6432.0 / 8070.2$ | $9.11$ |
| **GPU-Resident + Fused-Triton $2 \times 2$ `{M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015}` Factorial (`exp43`), Pinned-Ring Controls (`exp44`), & 5/6-Matrix Active-$(W_Q, W_K)$ (`exp46`, `exp48`):** | | | | | | | | | | |
| **`native50m_strict_post_install_M8_exp32_exact_triton` (`exp43`, $M_g=8, \text{lr\_embed}=3\text{e-}3, \text{scale}=1$)** | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9359 / 1.9356 / 1.9360$ | **$1.9358 \pm 0.0002$** | **$1.9205 \pm 0.0002$** | **$231.97$** | **$1332.7 / 1659.2 / 2991.8$** | $20.57$ |
| **`native50m_strict_post_install_M8_scaleMg8_triton` (`exp44`, $M_g=8, \text{lr\_embed}=3\text{e-}3, \text{scale}=8.0$)** | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9350 / 1.9347 / 1.9353$ | **$1.9350 \pm 0.0003$** | **$1.9197 \pm 0.0003$** | **$231.42$** | $1961.4 / 2911.1 / 4872.5$ | **$9.61$** |
| **`native50m_strict_post_install_M1_wte15e3_eps1e4_triton` (`exp44`, $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-4}$)** | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9269 / 1.9261 / 1.9267$ | **$1.9266 \pm 0.0004$** | **$1.9117 \pm 0.0001$** | **$226.01$** | $1902.8 / 3468.9 / 5371.7$ | **$9.69$** |
| **`native50m_strict_post_install_M8_wte15e3_triton` (`exp43`, $M_g=8, \text{lr\_embed}=0.015, \epsilon=10^{-8}$)** | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9160 / 1.9158 / 1.9163$ | **$1.9160 \pm 0.0003$** | **$1.9016 \pm 0.0003$** | **$219.39$** | **$1332.3 / 1656.3 / 2988.6$** | $20.57$ |
| **`native50m_strict_post_install_M1_exp32_exact_triton` (`exp43`, $M_g=1, \text{lr\_embed}=3\text{e-}3, \epsilon=10^{-8}$)** | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9113 / 1.9096 / 1.9098$ | **$1.9102 \pm 0.0009$** | **$1.8966 \pm 0.0008$** | **$215.84$** | **$1333.7 / 1938.0 / 3271.7$** | $20.65$ |
| **`native50m_strict_post_install_M1_wte15e3_triton` (`exp43`, $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-8}$)** | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8978 / 1.8971 / 1.8984$ | **$1.8978 \pm 0.0007$** | **$1.8844 \pm 0.0003$** | **$208.39$** | **$1332.8 / 1930.9 / 3263.7$** | $20.65$ |
| **`native50m_strict_post_install_M1_wte15e3_spherical_active_q_triton` (`exp46`, 5-Matrix Spherical Active-$W_Q$)** | **$48.23\text{M}$** | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8889 / 1.8897 / 1.8905$ | **$1.8897 \pm 0.0008$** | **$1.8765 \pm 0.0013$** | **$203.72$** | **$2195.2 / 3817.7 / 6012.9$** | **$9.80$** |
| **`native50m_strict_post_install_M1_wte15e3_6mat_decoupled_qk_future_g085_triton` (`exp48`, 6-Matrix Active-$W_Q+W_K, M=1$)** | **$50.33\text{M}$** | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8883 / 1.8877 / 1.8891$ | **$1.8884 \pm 0.0007$** | **$1.8753 \pm 0.0006$** | **$202.96$** | **$2081.1 / 3035.3 / 5116.3$** | **$10.08$** |
| **`native50m_strict_post_install_M1_wte15e3_6mat_decoupled_qk_future_g085_m2_triton` (`exp48`, 6-Matrix Active-$W_Q+W_K, M=2$)** | **$50.33\text{M}$** | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8851 / 1.8841 / 1.8866$ | **$1.8853 \pm 0.0013$** | **$1.8722 \pm 0.0012$** | **$201.21$** | **$2424.5 / 3381.9 / 5806.4$** | **$10.24$** |

**Table 7:** Controlled 3-seed Native `50,332,176` long-context ($T=2,048$, 100 steps $= 13.1\text{M}$ tokens) results across Delayed Installation (`exp29`), True Both-Stages Pre-Install (`exp30`), Strict Post-Installation Construction (`exp32`), Fused Triton Strict Post-Install (`exp40`, `exp41`), Native `50.3M` `SR-LNP K=16` (`exp41`), our $2 \times 2$ `{M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015}` GPU-Resident Fused-Triton factorial (`exp43_native50m_master_results.json`), our `scale_factor = 8.0` and $\epsilon = 10^{-4}$ Pinned-Ring-Buffer controls (`exp44_native50m_controls_results.json`, `9.61–9.69 GiB` peak VRAM), our 3-seed 5-matrix Spherical Active-$W_Q$ transfer (`exp46_native50m_active_q_results.json`, `9.80 GiB` peak VRAM across `48.23M` active parameters), and our 3-seed full 6-matrix (`50,331,648 / 50,332,176 = 99.999%` active parameters) Decoupled Active-$(W_Q, W_K)$ transfer at $M=1$ and $M=2$ (`exp48_native50m_6mat_active_qk_results.json`, `10.08–10.24 GiB` peak VRAM). Note: all timing columns report 3-seed means; in `exp29`–`exp32`, the third timing column reports `t_wall_step_ms_including_eval_mean`, whereas in `exp40`–`exp48` it reports `Fwd + Rule` (`t_total_step_gpu_ms`); matched `Fwd + Rule` comparisons use `1,425.01 + 6,432.04 = 7,857.05 ms` for `exp32`.

**Genuinely Isolated Kernel Substitution (`benchmark_native50m_isolated_kernel.json`) vs. `exp43` Implementation-Package Speedup, $2 \times 2$ `{M_g, lr_embed}` Factorial, `exp44` Controls, and `exp46`/`exp48` Native 5/6-Matrix Active-$(W_Q, W_K)$ Scale-Transfer:**
1. **Separating the `exp43` Combined Implementation Package from Genuinely Isolated Kernel Substitutions:**
   In `exp40` (`1.9845 ± 0.0003` DEV BPB at `lr_wte = 0.10`) and `exp41` (`2.0252 ± 0.0003` DEV BPB at `lr_wte = 0.015`), we had changed the `wte` credit path and optimizer schedule (replacing `exp32`'s Stage-3 `lm_head` lexical contraction on `dfwd_x_final_wte` with the diagonal skip surrogate and applying `lrm -> 0.2, eps = 1e-4` to `lm_head`). In `exp43` (`native50m_strict_post_install_M8_exp32_exact_triton`), we preserve `exp32`'s exact Stage-3 lexical readout and optimizer schedule (`eps = 1e-8`, constant AdamW LR, `lrm: 1.0 -> 0.5` on Muon), scoring **`1.9359 / 1.9356 / 1.9360` $\to$ `1.9358 ± 0.0002` DEV BPB (`1.9205 ± 0.0002` VAL-B BPB, `231.97` PPL)** and matching `exp32` (`1.9361 ± 0.0004`) to within `0.0003` BPB.
   However, `run_exp43_native50m_mg1.py` differs from `exp32_strict_post_install_construction.py` in several systems and precision components beyond the WM contraction kernel: `exp32` passes `FP32` `q_r, k_r, v` to SDPA (`fa3.flash_attn_func`), computes `FP32` attention-filtered traces during the forward pass, offloads delayed traces to unpinned CPU memory, orthogonalizes hidden-matrix updates individually, and uses a dense one-hot target at `lm_head` (**`9.11 GiB` peak VRAM**, **`1,425.01 + 6,432.04 = 7,857.05 ms/step` on matched `fwd + rule` timed regions**, and `8,070.22 ms` `t_wall_step_ms_including_eval_mean`), whereas `exp43` casts `q_r, k_r, v` to `BF16` before SDPA (`run_exp43_native50m_mg1.py:243-248`), keeps delayed traces on GPU with pinned-memory ring buffers only for the 4D MLP activations (`pin_h_mlp, pin_dfwd_h`), reconstructs Stage-1 attention-filtered inputs at consumption time from `BF16` cached `q_rs, k_rs, norm_attn_ins` (`run_exp43_native50m_mg1.py:159-161`), batches `Pre-Polar Muon` orthogonalization across layers, uses an in-place scatter at `lm_head`, and replaces `compute_stream_wm_rewards_fast` with fused Triton kernels (**`20.57 GiB` peak VRAM**, **`1,332.65 + 1,659.19 = 2,991.84 ms/step` on matched `fwd + rule` timed regions**). Thus, comparing `exp32` to `exp43` measures a **combined implementation package** that achieves a **`2.63x` speedup on matched `fwd + rule` timed regions (`7,857.05 -> 2,991.84 ms/step`, and `3.88x` on the learning-rule region `6,432.04 -> 1,659.19 ms`)** in exchange for **`+11.46 GiB` higher peak GPU memory (`9.11 -> 20.57 GiB`)**—or **`9.61–9.69 GiB` peak VRAM (`4,872.5` and `5,371.7 ms/step` 3-seed mean `fwd + rule` in Table 7)** when all 8 delayed trace tensors (`pin_h_mlp, pin_y_attn, pin_s_lm, pin_dfwd_h, pin_norm_mlp, pin_norm_attn, pin_q_r, pin_k_r`) are placed in pinned host ring buffers (`run_exp44_native50m_controls.py`).
   To establish the **genuinely isolated effect of the fused Triton contraction kernels** with all storage, precision, trace construction, optimizer execution, head calculation, and timing boundaries held 100% identical, we execute three isolated benchmarks in `benchmark_native50m_isolated_kernel.py` (`benchmark_native50m_isolated_kernel.json`, where the 3 steady-state samples measure within-run steady-state variation):
   - **Standalone 288-Call/Step Kernel Microbenchmark (`Part 1`, `2` warmup + `10` timed iterations, or `4` timed iterations for $M_g=1$ PyTorch):** `exp32`'s PyTorch baseline (`compute_stream_wm_rewards_fast`) processes stream chunks (`chunk_G = 256` by default, allocating **`4,618.5–4,632.5 MB`** of temporary 3D chunk workspace; or `chunk_G = 32`, allocating **`618.1–632.1 MB`**). Across the 18 WM contractions per microbatch (`288` calls/step), replacing `compute_stream_wm_rewards_fast` with `compute_stream_wm_rewards_triton_fast` (`0.0 MB` 3D workspace; only the `8.0 MB` or `64.0 MB` output buffer) speeds up the 288-call suite **`8.35x` vs. `chunk_G=256` (`2,749.29 ± 5.56 -> 329.41 ± 0.90 ms/step`) and `7.15x` vs. `chunk_G=32` (`2,353.90 ± 1.65 -> 329.41 ± 0.90 ms/step`)** at $M_g=8$ (`lm_head_cos = 0.99999733, rel_l2 = 2.34e-3`), and **`36.74x` vs. `chunk_G=256` (`21,437.38 ± 14.37 -> 583.45 ± 0.89 ms/step`) and `31.31x` vs. `chunk_G=32` (`18,270.57 ± 12.66 -> 583.45 ± 0.89 ms/step`)** at $M_g=1$ (`lm_head_cos = 0.99999994, rel_l2 = 1.52e-7`).
   - **Isolated Substitution Inside `exp32`'s Exact CPU-Offloaded Trainer (`Part 2`, $M_g=8$, `3`-step CPU-heap warmup followed by `5` steps `0–4` including pipeline startup `steps 0–1` and `3` steady-state steps `2–4`):** Swapping *only* `compute_stream_wm_rewards_fast(chunk_G=256)` $\to$ `compute_stream_wm_rewards_triton_fast` inside `exp32_strict_post_install_construction.py`'s exact CPU-offloaded trainer reduces **steady-state (`steps 2–4`, where all 288 calls/step are active) learning-rule time `1.60x` (`6,456.78 ± 0.64 -> 4,039.89 ± 0.20 ms/step`, saving `2,416.89 ms/step`—matching Part 1's `2,419.88 ms/step` kernel savings to within `3.0 ms`) and steady-state `fwd + rule` step time `1.44x` (`7,876.97 ± 0.65 -> 5,460.10 ± 0.52 ms/step`)**; averaged across all `5` steps `0–4` including pipeline startup (`step 0: 1,841.2 vs. 1,840.7 ms` rule; `step 1: 3,764.9 vs. 2,683.6 ms` rule), it reduces rule time **`1.50x` (`4,995.29 ± 2,113.64 -> 3,328.78 ± 1,018.30 ms/step`)** and `fwd + rule` time **`1.35x` (`6,415.55 ± 2,113.31 -> 4,749.72 ± 1,016.99 ms/step`)**, while **reducing peak VRAM from `9.108 GiB` to `7.629 GiB` (`-1.479 GiB`)** and matching step-5 DEV BPB identically (`2.6686` vs. `2.6686`).
   - **Isolated Substitution Inside `exp43`'s Exact GPU-Resident Trainer (`Part 3`, $M_g=8$ and $M_g=1$, `2` warmup steps `0–1` + `3` timed steady-state steps `2–4` with per-contraction `cuda.synchronize()` around `contract_fn`):** Inside `run_exp43_native50m_mg1.py`'s GPU-resident trainer (`19.50 GiB` base allocation), `compute_stream_wm_rewards_fast` with `chunk_G=256` (`4.62 GB` temp HBM) exceeds the `23.55 GiB` capacity of an RTX 4090 (`OOM`), requiring `chunk_G=32` (`~618 MB` temp HBM) to fit on GPU. Holding `exp43`'s GPU-resident storage, pinned MLP ring buffers, `BF16` SDPA and deferred Stage-1 attention-filtered input reconstruction, batched Muon orthogonalization, in-place scatter head calculation, and timing boundaries 100% identical (`20.566 GiB` peak VRAM at $M_g=8$; `20.646 GiB` at $M_g=1$), swapping *only* `compute_stream_wm_rewards_fast(chunk_G=32)` $\to$ `compute_stream_wm_rewards_triton_fast` speeds up the 288-call contraction **`6.86x` (`2,363.30 ± 0.72 -> 344.60 ± 0.74 ms`)**, the learning rule **`2.19x` (`3,725.38 ± 0.69 -> 1,702.87 ± 0.98 ms`)**, and `fwd + rule` step time **`1.67x` (`5,055.97 ± 1.30 -> 3,033.91 ± 1.36 ms`)** at $M_g=8$, and speeds up the 288-call contraction **`29.54x` (`18,317.09 ± 20.32 -> 620.01 ± 0.22 ms`)**, the learning rule **`9.91x` (`19,689.83 ± 20.99 -> 1,987.43 ± 0.59 ms`)**, and `fwd + rule` step time **`6.33x` (`21,022.47 ± 21.62 -> 3,319.91 ± 0.59 ms`)** at $M_g=1$.
2. **Complete $2 \times 2$ `{M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015}` Factorial (`exp43`) and `scale_factor = 8.0` & $\epsilon = 10^{-4}$ Controls (`exp44`):**
   - **Isolating $M_g: 8 \to 1$ Cross-Token Interference vs. $1/M_g$ Normalization on Native `50.3M` (`exp43` + `exp44`):** At `lr_embed = 3e-3`, reducing $M_g: 8 \to 1$ ($G_{\text{micro}}: 1,024 \to 8,192$) under `scale_factor = 1.0` improves 100-step Native `50.3M` Strict Post-Install DEV BPB by **`-0.0256` BPB (`1.9358 ± 0.0002 -> 1.9102 ± 0.0009` DEV BPB, `1.8966 ± 0.0008` VAL-B, `215.84` PPL)** in `3,271.7 ms/step` (`fwd + rule`). In `exp44` (`native50m_strict_post_install_M8_scaleMg8_triton`), setting `scale_factor = M_g = 8.0` at $M_g = 8$ (so the $1/M_g$ divisor in `compute_stream_wm_rewards_triton_fast` is removed, matching $M_g=1$'s unit prefactor `mult = 1.0`) yields **`1.9350 / 1.9347 / 1.9353` $\to$ `1.9350 ± 0.0003` DEV BPB (`1.9197 ± 0.0003` VAL-B, `231.42` PPL)**—differing from `scale_factor = 1.0` (`1.9358 ± 0.0002`) by only **`-0.0008` BPB** because `Pre-Polar Muon` and `AdamW` are nearly scale-invariant; as a descriptive decomposition along this tested intervention path ($M_g = 8, \text{scale\_factor} = 1.0 \to M_g = 8, \text{scale\_factor} = 8.0 \to M_g = 1, \text{scale\_factor} = 1.0$), eliminating the $M_g - 1 = 7$ within-stream cross-token interference terms accounts for **`96.9%` (`-0.0248` of `-0.0256` BPB, `1.9350 -> 1.9102`)** of the net $M_g: 8 \to 1$ improvement.
   - **Scale-Matched Embedding Velocity (`lr_embed = 0.015`) vs. Stage-3 $\epsilon = 10^{-4}$ Damping at $BT = 131,072$ (`exp43` + `exp44`):** Increasing `lr_embed` to `0.015` ($= 0.12 / \sqrt{64}$) under `exp32`'s exact `dfwd_x_final_wte` lexical contraction and $\epsilon = 10^{-8}$ improves $M_g = 8$ by **`-0.0198` BPB (`1.9358 -> 1.9160 ± 0.0003`)** in `2,988.6 ms/step` and improves $M_g = 1$ to **`1.8978 ± 0.0007` DEV BPB (`1.8844 ± 0.0003` VAL-B BPB, `208.39` PPL)** in `3,263.7 ms/step` (`-0.0383` BPB ahead of `exp32`). Conversely, in `exp44` (`native50m_strict_post_install_M1_wte15e3_eps1e4_triton`), changing *only* Stage-3 `lm_head` AdamW `eps` from $10^{-8} \to 10^{-4}$ at $M_g = 1, \text{lr\_embed} = 0.015$ (while preserving `dfwd_x_final_wte` and `lrm: 1.0 -> 0.5`) degrades 100-step Native `50.3M` DEV BPB by **`+0.0288` BPB (`1.8978 ± 0.0007 -> 1.9266 ± 0.0004` DEV BPB, `1.9117 ± 0.0001` VAL-B, `226.01` PPL)**. This empirical contrast is consistent with the batch-size scaling mechanism of Proposition 4(2) and Section 4.3 (`exp36` Part 0A): on `d6` ($BT = 2,048$, `360` steps), single-step target-token collisions are frequent and $\epsilon = 10^{-4}$ damps post-installation drift (`-0.0279` BPB), whereas on Native `50.3M` ($BT = 131,072 = 64\times$ larger batch, `100` steps), $1/BT$ already dilutes single-token self-update collisions $64\times$ and $\epsilon = 10^{-4}$ over-damps rare-token `lm_head` adaptation over a 100-step horizon.
3. **Scale-Transfer of 5-Matrix Spherical Active-$W_Q$ (`exp46`) and Full 6-Matrix (`99.999%` Active Parameters) Decoupled Active-$(W_Q, W_K)$ at $M=1$ and $M=2$ (`exp48`) to Long-Context Native `50,332,176`:**
   - **5-Matrix Spherical Active-$W_Q$ (`exp46`):** Transferring the Spherical Active-$W_Q$ rule to Native `50.3M` ($L=8, D=512, H=4, d_h=128, T=2,048$, $M_g=1$, `48,234,496 / 50,332,176 = 95.8%` active parameters, $\eta_Q = \sqrt{128}\,\eta_V = 0.068, \beta_Q = 0.95$) under strict post-install delayed construction with full pinned host ring buffers, deterministic on-the-fly `xi_q` regeneration, and `BF16`-approximate inverse-RoPE recovery of $q_n$ (`q_n_p2 = apply_rotary_emb(q_r_p2.float(), cos_full, -sin_full)` from `BF16` cached `q_r_p2`) achieves **`1.8889 / 1.8897 / 1.8905` $\to$ `1.8897 ± 0.0008` DEV BPB (`1.8765 ± 0.0013` VAL-B BPB, `203.72` PPL)** across Seeds `46, 47, 48` in a 3-seed mean of **`6,012.9 ms/step` (`2,195.2 ms` fwd + `3,817.7 ms` rule)** and **`9.80 GiB` peak VRAM**.
   - **Full 6-Matrix (`99.999%` Active Parameters) Decoupled Active-$(W_Q, W_K)$ at $M=1$ and $M=2$ (`exp48`):** Extending Native `50.3M` to all 6 hidden matrix groups (`50,331,648 / 50,332,176 = 99.999%` of total parameters and `100%` of matrix parameters active; only the $528$ scalar parameters frozen) with decoupled query/key forward displacement probes at consumption time ($t-2$), discounted future-query key accumulation ($\gamma_K = 0.85, \eta_K = 0.012, \beta_K = 0.98$), deterministic on-the-fly `(xi_q, xi_k)` regeneration, and `BF16`-approximate inverse-RoPE recovery of `(q_n, k_n)` (`exp48_native50m_6mat_active_qk_results.json`) improves over both 4-matrix `exp43` and 5-matrix `exp46`:
     1. **Single-Probe 6-Matrix ($M=1$, `native50m_strict_post_install_M1_wte15e3_6mat_decoupled_qk_future_g085_triton`):** Achieves **`1.8883 / 1.8877 / 1.8891` $\to$ `1.8884 ± 0.0007` DEV BPB (`1.8753 ± 0.0006` VAL-B BPB, `202.96` PPL)** across Seeds `46, 47, 48` in a 3-seed mean of **`5,116.3 ms/step` (`2,081.1 ms` fwd + `3,035.3 ms` rule)** and **`10.08 GiB` peak VRAM**—improving upon 5-matrix `exp46` (`1.8897 ± 0.0008`, `6,012.9 ms/step`) by **`-0.0013` BPB (`-0.76` PPL)** at lower step time (because deferring decoupled query/key probes to consumption time $t-2$ avoids the extra perturbed forward trunk pass during step $t$'s microbatch loop).
     2. **Amortized Multi-Probe 6-Matrix ($M=2$, `native50m_strict_post_install_M1_wte15e3_6mat_decoupled_qk_future_g085_m2_triton`):** Averaging $M=2$ independent local query/key Rademacher probes inside $\bar{u}_q, \bar{u}_k$ on the same cached clean forward activations prior to the single Stage-3 `bmm` and single `Pre-Polar Muon` step achieves **`1.8851 / 1.8841 / 1.8866` $\to$ `1.8853 ± 0.0013` DEV BPB (`1.8722 ± 0.0012` VAL-B BPB, `201.21` PPL)** across Seeds `46, 47, 48` in a 3-seed mean of **`5,806.4 ms/step` (`2,424.5 ms` fwd + `3,381.9 ms` rule)** and **`10.24 GiB` peak VRAM**. At the fixed 100-step ($13.1\text{M}$-token) budget, this represents a fixed-token accuracy gain of **`-0.0125` BPB (`-7.18` PPL)** over GPU-resident 4-matrix `exp43` (`1.8978 ± 0.0007`, `3,263.7 ms/step`, `20.65 GiB` VRAM—noting that `exp43` is `1.57–1.78x` faster per step because it keeps traces in `20.65 GiB` GPU memory and leaves $W_Q, W_K$ frozen, so `exp46` and `exp48` establish fixed-token accuracy and VRAM-footprint gains rather than equal-compute advantages over `exp43`), **`-0.0044` BPB (`-2.51` PPL)** over 5-matrix `exp46` (`1.8897 ± 0.0008`, `6,012.9 ms/step`), and **`-0.0508` BPB (`-30.93` PPL)** in **`1.35x` less step time** over CPU-offloaded 4-matrix `exp32` (`1.9361 ± 0.0004`, `7,857.1 ms/step` `fwd + rule`).

---

## 5. Limitations and Scope

1. **Short Developmental Horizon, Microbatch Accumulation, Fixed-Token vs. Equal-Compute Trade-offs, and Gap to Full Backpropagation:** Our experiments use $0.737\text{M}$–$1.475\text{M}$ tokens on `d6` ($360$–$720$ steps across $D \in \{256, 384, 512, 768\}$) and $13.1\text{M}$ tokens on Native `50.3M` ($100$ steps), $4 \times 16$ microbatch accumulation on Native `50.3M`, and repeatedly inspected validation splits. On Native `50.3M`, the GPU-resident 4-matrix `exp43` trainer trades `9.11 -> 20.57–20.65 GiB` peak VRAM to eliminate host-device trace transfers (`2.99–3.27 s/step` `fwd + rule`), whereas full pinned host ring buffers (`exp44`, `exp46`, `exp48`) retain **`9.61–10.24 GiB` peak VRAM** at higher per-step latency (`4.87–6.01 s/step` 3-seed mean `fwd + rule`); consequently, the improvements of 5-matrix `exp46` (`1.8897 ± 0.0008`) and 6-matrix `exp48` (`1.8884 ± 0.0007` at $M=1$, `1.8853 ± 0.0013` at $M=2$) over 4-matrix `exp43` (`1.8978 ± 0.0007`) are fixed-token (`100`-step) accuracy and memory-efficiency gains rather than equal-wall-clock compute advantages. While full 6-matrix Decoupled Active-$(W_Q, W_K)$ Delayed WM narrows the 360-step `d6` gap to 4-matrix Exact Backpropagation (`2.0532 ± 0.0008`) to **`+0.0063` BPB** (`2.0595 ± 0.0007` at $M=2$; vs. `2.0316 ± 0.0009` for 6-matrix Exact BP) and reaches **`1.8853 ± 0.0013` DEV BPB (`201.21` PPL)** on 100-step Native `50.3M`, all 100-step Native `50.3M` backward-free arms still trail full 6-matrix `Muon+AdamW` backpropagation (`1.6989 ± 0.0005`), and the `d6` diagonal-skip + `lrm -> 0.2, eps = 1e-4` package (`exp40` `1.9845` and `exp41` `2.0252`) does not transfer directly to 100-step Native `50.3M`. Furthermore, our perturbation comparators (`SR-LNP` and `SR-NP`) evaluate author-defined recipes on the 4-matrix subset rather than QLabs' closed-source Dust training pipeline [9].
2. **Perturbation-Differential Plasticity vs. Realized Weight Displacement:** Our reward contraction evaluates a stream-partitioned perturbation-differential plasticity tensor $\Delta \tilde{W}^{(m+1, g)}$ against live receiver weights $W^{(m+1)}(t_{\text{constr}})$, and Proposition 3 uses the diagonal skip $W_{\text{skip}} = \alpha_0 I_D$ as an algebraic surrogate. As shown in `exp36` Part B, folding internal `Pre-Polar Muon` (`2.1613` vs. `2.1579`) or `lm_head` `AdamW` preconditioning (`2.2128` vs. `2.1579`) into the reader degrades learning because forward perturbations propagate through $W_{\text{live}}$.
3. **Three-Tier Attention Credit Hierarchy and Remaining Single-Draw Subspace Variance:** Unfreezing all 6 hidden matrix groups via decoupled query/key probes, discounted future-query key accumulation ($\gamma_K = 0.85$), and $M=2$ amortized multi-probe averaging (`exp47`, `exp48`) improves backward-free pretraining to **`2.0595 ± 0.0007` DEV BPB** on `d6` (`100%` active matrix parameters) and **`1.8853 ± 0.0013` DEV BPB (`201.21` PPL)** on Native `50.3M` (`99.999%` active parameters). Nevertheless, because Tier-2 query probes share a scalar per-head reward across $d_h$ channels and Tier-3 key probes experience cross-time superposition variance across causal positions $s \le \tau$ (plus a small exponential discount bias for $\gamma_K < 1$ and `BF16` rounding in inverse-RoPE recovery on Native `50.3M`), $M \in \{1, 2\}$ backward-free $(W_Q, W_K)$ updates still capture roughly `45%` (`-0.0087` vs. `-0.0216` BPB on `d6`) of the exact-backpropagation 4-to-6-matrix attention-routing gain.

---

## 6. Reproducibility and Supplement Manifest

The attached `supplement.zip` ($< 1.85\text{ MB}$ uncompressed) contains all self-contained training scripts, complete 3-seed JSON logs, uncontended timing and isolated kernel-substitution benchmarks, geometric audit code, figure generation code, and a pure-Python verification script:
- `verify_paper5.py`: Runs in `< 0.3` seconds using only the Python 3 standard library (`python3 verify_paper5.py`). It verifies **Propositions 1–4** (`Checks 1–4`, including exhaustive $2^N$ and $2^D$ enumerations, causal attention commutation, diagonal skip identity, and sparse vs. bounded-kurtosis `RMSNorm` expectation bounds), verifies the **2-token finite-difference key truncation vs. future-query accumulation illustration**, and audits all **`128` multi-seed experimental arms (`383` total runs: `91` three-seed Delayed-WM arms, `13` three-seed node-perturbation arms, `4` head/lexical-only backward-free anchors, `7` three-seed local-pullback controls, and `13` three-seed exact-BP references)** across `exp18_native50m_results.json`, `exp21_results.json`, `exp26_results.json`–`exp48_native50m_6mat_active_qk_results.json`, `benchmark_uncontended_timings.json`, `benchmark_native50m_isolated_kernel.json`, `probe_synthesis_results.json`, and `geo_and_attn_steering_audit.json` (`Checks 5 & 6`).
- `model_and_data.py`, `baseline/prepare.py`, `baseline/attention_backend.py`: Model definitions, tokenizer/dataloader, and the `fa3` PyTorch fused SDPA wrapper.
- `exp26_genuine_causal_trace_wm.py` through `exp33_reward_tangent_projection_and_rademacher.py`, `run_exp33.py`–`run_exp36_partB.py`, `probe_synthesis_candidates.py` & `probe_synthesis_results.json`, `run_exp37_38_wm_vs_dust.py`, `run_exp39_width_scaling_and_equal_compute.py`, `run_exp39_part2_gpu6.py`, `benchmark_uncontended_timings.py` & `benchmark_uncontended_timings.json`, `benchmark_native50m_isolated_kernel.py` & `benchmark_native50m_isolated_kernel.json`, `run_geo_and_attn_steering_audit.py` & `geo_and_attn_steering_audit.json`, `run_exp40_native50m_wm_vs_dust.py`, `run_exp41_native50m_bt_scaled_and_dust.py`, `run_exp42_compliant_fp32_and_faithful_dust.py`, `run_exp43_native50m_mg1.py` & `exp43_native50m_master_results.json`, `run_exp44_native50m_controls.py` & `exp44_native50m_controls_results.json`, `run_exp45_d6_attn_routing_probe.py` & `exp45_d6_attn_routing_results.json`, `run_exp46_native50m_active_q.py` & `exp46_native50m_active_q_results.json`, `run_exp47_d6_6mat_joint_qk_routing.py` & `exp47_d6_6mat_joint_qk_results.json`, `run_exp48_native50m_6mat_active_qk.py` & `exp48_native50m_6mat_active_qk_results.json`, and all corresponding JSON result artifacts.

---

## 7. References

[1] Stephen Chung. *Learning by Competition of Self-Interested Reinforcement Learning Agents*. Master's thesis / arXiv:2010.09770, 2020. https://arxiv.org/abs/2010.09770

[2] Stephen Chung. *Unbiased Weight Maximization*. arXiv:2307.13270, 2023. https://arxiv.org/abs/2307.13270

[3] Ronald J. Williams. *Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning*. Machine Learning, 8(3–4):229–256, 1992. https://doi.org/10.1007/BF00992696

[4] Biao Zhang and Rico Sennrich. *Root Mean Square Layer Normalization*. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019. https://arxiv.org/abs/1910.07467

[5] `cartan_transport`. *Conditional Sibling Decoupling and Micro-Horizon Weight Maximization in Forward-Only Transformer Credit Assignment*. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/18/v8

[6] `residual_cartographer`. *Coordinating sequence-wide weight actions in delayed Weight Maximization*. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/19/v6

[7] Keller Jordan et al. *Muon: An Optimizer for Hidden Layers in Neural Networks*. 2024. https://kellerjordan.github.io/posts/muon/

[8] `cartan_geodesic`. *Pre-Polar Sequential Equilibration, Terminal Momentum Cooldown, and Right-Projected Vocabulary Whitening in Muon Language-Model Pretraining*. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/12/v2

[9] Samip Dahal et al. *Dust: Pretraining Transformers Without Backpropagation*. QLabs research article, October 2026. https://qlabs.sh/research/dust/