1. Problem, Operational Scope, and Contributions

Training multi-layer causal Transformers without a global backward pass is a core problem in local credit assignment [1, 2, 3, 9]. A foundational methodological distinction separates backward-free local credit assignment from approximate or manual backpropagation: disabling automatic differentiation (torch.no_grad()) or replacing local analytical derivatives with finite-difference probes does not remove the backward phase if current-step downstream error vectors are multiplied by layer weight matrices in reverse order ($\mathbf{e}_{\text{in}} = \mathbf{e}_{\text{out}} W$) or relayed from downstream blocks to upstream blocks within the same training step.

In Chung's Weight Maximization (WM) with eligibility traces [1], each unit $i$ in stage $m$ injects a local forward exploration perturbation $\xi_{n, i}^{(m)}(t)$ during the forward pass of batch $t$, records a local pre-synaptic eligibility trace $Z_{i, :}^{(m)}(t)$, and updates at a subsequent step using the column weight-norm change $\sum_{j} W_{j, i}^{(m+1)} \Delta W_{j, i}^{(m+1)}$ of its outgoing receiver synapses $W_{:, i}^{(m+1)}$.

1.1 Operational Definition of Delayed Perturbation-Differential Column-Norm Credit and Pipeline Clocks

To adapt delayed WM to multi-token Transformer minibatches where receiver layers are updated by adaptive or spectrally orthogonalized optimizers (AdamW or Pre-Polar Muon [7, 8]), we state precisely what feedback our algorithm constructs, how its normalization is parameterized in code, and how pipeline clocks are ordered:

1. Installed Receiver Displacement vs. Perturbation-Differential Plasticity Tensor: At step $t$, each receiver layer $m+1$ updates its live weights $W^{(m+1)}(t) = W^{(m+1)}(t-1) + \delta W^{(m+1)}(t-1)$ via its optimizer from the clean batch gradient (or delayed WM pseudo-gradient). To evaluate the directional effect of stage $m$'s forward perturbation $\xi^{(m)}(t_{\text{orig}})$ on receiver $m+1$'s synaptic plasticity without confounding from other stages' noise or optimizer state history, our reward routine (compute_stream_wm_rewards_chunked in exp26; compute_stream_wm_rewards_fast in exp32–exp35, parameterized by scale_factor) constructs for each token stream $\mathcal{S}_g$ ($g \in \{1, \dots, G\}$, $|\mathcal{S}_g| = M_g = BT / G$) a perturbation-differential plasticity tensor$$ \Delta \tilde{W}_{j, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor}) := \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} u_{n, j}^{(m+1)}(t_{\text{orig}}) \, \Delta y_{n, i}^{(m)}(t_{\text{orig}}), \qquad \Delta y_{n, i}^{(m)} := y_{n, i}^{\text{pert}}(\sigma \xi^{(m)}) - y_{n, i}^{\text{clean}}(0), $$where $u_{n, :}^{(m+1)}(t_{\text{orig}})$ is the receiver's local post-synaptic modulation vector and $\Delta y_{n, :}^{(m)}(t_{\text{orig}})$ is the perturbed-minus-clean forward activation displacement at stage $m$'s output. At reward construction clock $t_{\text{constr}}$, unit $i$ contracts $\Delta \tilde{W}_{:, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor})$ with its live outgoing receiver column $W_{:, i}^{(m+1)}(t_{\text{constr}})$:$$ R_i^{(m, g)}(t_{\text{orig}}, t_{\text{constr}}; \text{scale\_factor}) = \sum_{j=1}^{d_{\text{next}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) \, \Delta \tilde{W}_{j, i}^{(m+1, g)}(t_{\text{orig}}; \text{scale\_factor}). $$The parameter update then divides the stream-summed modulation–input product $\sum_{g=1}^G R_i^{(m, g)} Z_{i, :}^{(m, g)}$ by $N = BT$ (-(mod_flat.t() @ x_flat) / BT). Thus, $\Delta \tilde{W}^{(m+1, g)}$ is a local perturbation-differential Hebbian plasticity tensor rather than a decomposition of the installed optimizer displacement $\delta W^{(m+1)}$, and reading $W^{(m+1)}(t_{\text{constr}})$ at construction clock $t_{\text{constr}}$ makes the reward directly sensitive to any receiver weight update $\delta W^{(m+1)}(t-1) = W^{(m+1)}(t) - W^{(m+1)}(t-1)$ installed between origin $t_{\text{orig}}$ and construction $t_{\text{constr}}$.
2. Three-Clock Pipeline Schedules: For each stage $m \in \{1, 2, 3\}$, we track $(t_{\text{orig}}, t_{\text{constr}}, t_{\text{inst}})$ relative to installation step $t_{\text{inst}} = t$. Across exp26–exp35, three distinct clock schedules are evaluated:
   • True Both-Stages Pre-Install (both_stages_pre_install, exp26–exp30 and exp35 Part B2): Stage 3: $(t, t, t)$; Stage 2: $(t-1, t-1 \text{ [pre-Stage3-step]}, t)$ (reward constructed at $t-1$ on $W_{\text{lm}}(t-1)$ prior to Stage 3's batch-$t-1$ update, installed at $t$); Stage 1: $(t-2, t-1 \text{ [pre-Stage2-step]}, t)$ (reward constructed and cached at $t-1$ on $W_{\text{proj}}(t-1)$ prior to Stage 2's installation of its batch-$t-2$ update, then installed into Stage 1 at $t$).
   • Stage-2 Pre-Install / Stage-1 Post-Install (stage2_pre_install, run_exp33_partB.py, run_exp34.py, exp35 Part B1): Stage 3: $(t, t, t)$; Stage 2: $(t-1, t-1 \text{ [pre-Stage3-step]}, t)$ (constructed at $t-1$ prior to Stage 3's update); Stage 1: $(t-2, t \text{ [post-Stage2-step]}, t)$ (constructed at $t$ on live $W_{\text{proj}}(t)$ after Stage 2 installed its batch-$t-2$ update at the end of $t-1$).
   • Strict Post-Installation Construction (strict_post_install, exp32–exp35): Stage 3: $(t, t, t)$; Stage 2: $(t-1, t \text{ [post-Stage3-step]}, t)$ (constructed at $t$ on $W_{\text{lm}}(t) = W_{\text{lm}}(t-1) + \delta W_{\text{lm}}(t-1)$ strictly after Stage 3 installed its batch-$t-1$ update); Stage 1: $(t-2, t \text{ [post-Stage2-step]}, t)$ (constructed at $t$ on $W_{\text{proj}}(t)$ strictly after Stage 2 installed its batch-$t-2$ update). (Note: The CausalAuditLedger fields in our JSON logs record declared trainer configuration metadata summarizing the code path of each arm rather than runtime hardware profiler counters; in exp33–exp35, all stage2_pre_install ledgers are reconciled to record (t-2, t [post-Stage2-step], t)).

1.2 Summary of Contributions

Across 117 multi-seed experimental arms (350 total runs across Seeds 46, 47, 48: 80 three-seed Delayed-WM arms [240 runs], 13 three-seed node-perturbation arms [11 SR-LNP + 2 SR-NP = 39 runs], 4 head/lexical-only backward-free anchors [11 runs], 7 three-seed local-pullback diagnostic controls [21 runs], and 13 three-seed exact-backpropagation reference arms [39 runs] across exp18, exp21, exp26–exp46) on ClimbMix d6 ($L=6, D \in \{256, 384, 512, 768\}, V=8,192$, $360$–$720$ steps $= 0.737\text{M}$–$1.475\text{M}$ tokens) and Native 50,332,176 ($L=8, D=512, V=8,192, T=2,048$, $100$ steps $= 13.1\text{M}$ tokens), where every Delayed-WM run executes inside with torch.no_grad(): (assert not torch.is_grad_enabled()) with zero reverse matrix pullbacks, zero VJP helpers, and zero same-step relays, we establish seven results within the short-horizon developmental regime tested:

• Reconciled Stream-Partitioned Estimator Analysis & Scale-Normalized Controls (Proposition 1, exp26, exp35, exp44): We reconcile the exact implemented contraction $\widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor}) = \frac{1}{N}\sum_{g=1}^G R_i^{(m, g)}(\text{scale\_factor}) Z_{i, :}^{(m, g)} = \frac{\text{scale\_factor}}{M_g} \widehat{G}_{i, :}^{\text{norm}}(G)$ across both the default code convention scale_factor = 1 (whose conditional mean scales as $1/M_g$ and covariance as $1/M_g^2$, giving mean $0.5$ on $N=4, G=2, M_g=2, a_n=x_n=1$) and the scale-normalized convention scale_factor = M_g (whose conditional mean $\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n$ is invariant to $M_g$, giving $1.0$). We further distinguish separable channels (Stage 1: mlp.c_fc, attn.c_v, Proposition 1(a–b), where first-order variance vanishes at $M_g = 1$) from non-separable RMSNorm interfaces (Stage 2: x_final, x_0, Proposition 1(c)), deriving the exact single-token ($M_g=1$) cross-coordinate variance $\frac{a_{n, i}^2 x_{n, i}^2 (D - x_{n, i}^2)}{D^2 \text{rms}(z_n)^2} \|h_n\|_2^2$ (giving $0.064$ at $D=2, z=(1, 2)$). Controlled 3-seed scale_factor = M_g runs (exp35 on d6 and exp44 on Native 50.3M) confirm that holding the conditional mean invariant to $M_g$ leaves learning curves virtually unchanged (2.6097 -> 2.3361 -> 2.3008 -> 2.2338 for scale_factor = M_g vs. 2.5939 -> 2.3364 -> 2.2966 -> 2.2338 for scale_factor = 1 across $G \in \{1, 64, 256, 2048\}$ on d6; 2.2539 ± 0.0013 vs. 2.2541 ± 0.0013 at $G=64$ under Pre-Polar Muon; and 1.9350 ± 0.0003 for scale_factor = 8.0 vs. 1.9358 ± 0.0002 for scale_factor = 1.0 at $M_g=8, \text{lr\_embed}=3\text{e-}3$ on Native 50.3M, so along this tested intervention path 96.9% (-0.0248 of -0.0256 BPB) of the $M_g: 8 \to 1$ gain (1.9350 -> 1.9102) comes from eliminating $7/8$ within-stream cross-token interference terms), establishing that variance reduction rather than raw step-scale changes drives the stream-partitioning gain.
• Forward Causal Attention-Filtered Value Eligibility Trace (Proposition 2, exp28): Because $v(b, s) = W_v x_{\text{attn\_in}}(b, s)$ commutes with causal attention over $s \le \tau$, computing the causal attention-filtered pre-synaptic trace $\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} x_{\text{attn\_in}}(b, s)$ during the forward pass and perturbing post-attention value outputs lifts single-step backward-free attn.c_v cosine alignment with the language-model cross-entropy loss gradient $8.2\times$ from +0.1104 to +0.9069. Pairing $Z^{(2)}(t-1)$ with independent current-batch rewards $R^{(2)}(t)$ (mismatched_origin_control, exp28) degrades d6 from 2.2338 ± 0.0102 to 2.7608 ± 0.0056 DEV BPB (-0.5270 BPB contrast).
• Diagonal Skip-Routing Surrogate, Data-Dependent Kurtosis Expectation Bound, & Compliant Pure-FP32 Clock/Optimizer Factorials (Propositions 3 & 4, exp31–exp36, exp42): Routing lexical credit to wte via the diagonal skip surrogate $W_{\text{skip}} = \alpha_0 I_D$ (Proposition 3) cuts PyTorch d6 rule time by $27.4\%$ (205.5 -> 149.2 ms/step) with identical accuracy (2.1579 ± 0.0004 vs. 2.1577 ± 0.0004). For Strict Post-Installation Construction, Proposition 4 decomposes $\delta W_{\text{lm}}(t-1)$ into weight decay, momentum history $\beta_1 m^{(t-2)}$, and current-batch gradient $(1-\beta_1) g^{(t-1)}$. Under row-isotropic preconditioning, reward-side projection $P_{x_{\text{final}}(n)}$ annihilates the $O(1)$ radial self-update vector $b_{\text{rad}}(n)$ identically, leaving an oblique RMSNorm-Jacobian coupling residual whose expected squared norm relative to $\|b_{\text{rad}}(n)\|_2^2$ is bounded by the data-dependent coordinate fourth moment $\frac{\mathbb{E}_{\xi_n}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} \le \frac{\kappa_4(x_{\text{final}}(n))}{D}$ where $\kappa_4(x) := \frac{1}{D}\sum_{i=1}^D x_i^4$. We show both that unrestricted sparse vectors $x = \sqrt{D/5}(1, 2, 0, \dots, 0)$ have non-vanishing expectation ratio $0.0576$ ($\kappa_4(x)/D = 0.68$) for all $D \ge 2$, and that on a 20-step head/block-0 diagnostic trajectory on ClimbMix d6 (exp35 Part 0), $\kappa_4(x_{\text{final}})$ has token mean $2.9398$, yielding mean theoretical expectation bound $\kappa_4(x_{\text{final}})/D = 0.007656$ and empirical mean ratio $0.007458$ (with individual token maximum $0.124439$ across $40,960$ token evaluations). Across our 16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer factorial (exp34 + exp35 Part B1), 4-cell True Both-Stages Pre-Install comparison (exp35 Part B2), and compliant zero-pullback pure-FP32 trace-cache Triton WM clock controls (exp42 Part A):
  1. Isolating Stage-1 vs. Stage-2 Clock Transitions & Compliant Pure-FP32 Cache Controls (exp35 Part B2, exp42 Part A): Moving Stage 1 from (t-2, t-1 [pre-Stage2], t) (both_stages_pre_install) to (t-2, t [post-Stage2], t) (stage2_pre_install) shifts d6 DEV BPB by only +0.0013 to +0.0024 BPB across all 4 entrywise cells (2.1341 -> 2.1360 at b05, 1e-8; 2.1195 ± 0.0005 -> 2.1217 ± 0.0006 at b05, 1e-4), consistent with update-orthogonalized Pre-Polar Muon inducing only a 4.04% (cos = 0.999238) Stage-1 reward drift. Moving Stage 2 from (t-1, t-1, t) to (t-1, t [post-Stage3], t) at (entrywise, b05, 1e-8) shifts DEV BPB by +0.0219 ± 0.0020 BPB (2.1360 -> 2.1579). Eliminating BF16 trace caching in a compliant zero-pullback Triton WM trainer (reverse_error_matmuls_per_step = 0, exp42_fp32_triton_results.json) reproduces the exact same 3-seed clock contrast (2.1359 ± 0.0022 vs. 2.1579 ± 0.0004 DEV BPB at b05, 1e-8, and 2.1274 ± 0.0002 DEV BPB at b09, 1e-4), confirming that the +0.0219 BPB shift is a genuine receiver-installation effect rather than a BF16 cache-rounding artifact (while exp36 provides matching local-pullback diagnostic controls).
  2. Row-Isotropic vs. Entrywise Preconditioning (Proposition 4, exp34–exp35): Across all 4 row_iso cells of the 16-cell cube, Stage-2 Pre-Install and Strict Post-Install agree to within 0.0003 BPB (2.1577 vs. 2.1580; 2.1662 vs. 2.1659; 2.1296 vs. 2.1298; 2.1335 vs. 2.1334). Under entrywise moments, increasing $\epsilon: 10^{-8} \to 10^{-4}$ damps coordinate spread in $d_{j, i} = (\sqrt{\hat{v}_{j, i}} + \epsilon)^{-1}$ and improves Strict Post-Install by -0.0279 ± 0.0001 BPB (2.1579 -> 2.1300), while $(\beta_1=0.9, \epsilon=10^{-4})$ reaches 2.1273 ± 0.0002 DEV BPB (2.0900 ± 0.0001 VAL-B).
• Fused Register-Tiled Triton WM Kernels (38.4x Kernel Speedup & 8.33x Steady-State Step Speedup on d6; Isolated 1.44x–6.33x Steady-State Step Speedup & 2.63x Implementation-Package Speedup on Native 50.3M, exp37, exp43, exp44): Building on the SRAM-fused outgoing-column contraction concept introduced in [5] (version 5), we specialize fused register-tiled Triton kernels (_triton_wm_reward_mg1_kernel for $M_g=1$ and _triton_wm_reward_mg_grouped_kernel for $M_g > 1$) for our 3-stage delayed perturbation-differential WM pipeline. Eliminating the 3D outer-product workspace (3,498.9 MB on d6 lm_head at $G=2,048$) cuts the 13-matrix d6 WM kernel suite 38.4x from 134.31 ms to 3.50 ms/step at $G=2,048$ (cos = 1.000000, rel_l2 = 2.26e-8 on mlp.c_proj) and 4.4x from 4.51 ms to 1.03 ms/step at $G=64$ (cos = 0.9999973, rel_l2 = 2.31e-3). Combined with batched Pre-Polar Muon, steady-state d6 fwd + rule step time at $G=2,048$ drops 8.33x from 155.6 ms to 18.68 ± 0.21 ms/step (6.72s steady-state estimate for 360 steps; 6.74s–6.95s measured 3-seed training GPU time). On Native 50.3M ($B_{\text{eff}}=64, T=2,048$), exp32's PyTorch baseline (compute_stream_wm_rewards_fast) processes stream chunks (chunk_G=256, allocating 4.62 GB temporary chunk workspace; or chunk_G=32, allocating 0.62 GB). In genuinely isolated kernel substitutions (benchmark_native50m_isolated_kernel.json, measuring within-run steady-state variation across steps 2–4) holding all storage, precision, trace construction, optimizer execution, and timing boundaries fixed: (i) inside exp32's CPU-offloaded trainer ($M_g=8$, after a 3-step CPU-heap warmup), swapping only compute_stream_wm_rewards_fast(chunk_G=256) $\to$ compute_stream_wm_rewards_triton_fast speeds up the steady-state (steps 2–4) learning rule 1.60x (6,456.78 ± 0.64 -> 4,039.89 ± 0.20 ms/step) and steady-state fwd + rule 1.44x (7,876.97 ± 0.65 -> 5,460.10 ± 0.52 ms/step, or 1.50x rule and 1.35x fwd + rule across all 5 steps 0–4 including pipeline startup) while reducing peak VRAM from 9.108 GiB to 7.629 GiB; and (ii) inside exp43's GPU-resident trainer (20.57–20.65 GiB peak VRAM, 2 warmup + 3 timed steady-state steps 2–4, where chunk_G=256 OOMs), swapping only compute_stream_wm_rewards_fast(chunk_G=32) $\to$ compute_stream_wm_rewards_triton_fast speeds up the 288-call contraction 6.86x (2,363.30 ± 0.72 -> 344.60 ± 0.74 ms) and fwd + rule 1.67x (5,055.97 ± 1.30 -> 3,033.91 ± 1.36 ms) at $M_g=8$, and speeds up the 288-call contraction 29.54x (18,317.09 ± 20.32 -> 620.01 ± 0.22 ms), the learning rule 9.91x (19,689.83 ± 20.99 -> 1,987.43 ± 0.59 ms), and fwd + rule 6.33x (21,022.47 ± 21.62 -> 3,319.91 ± 0.59 ms) at $M_g=1$. Across 100-step 3-seed training runs (exp43–exp44), our combined implementation package (combining the Triton kernels with GPU-resident traces + pinned MLP ring buffers, BF16 SDPA and deferred Stage-1 attention-filtered input reconstruction from BF16 cached inputs, batched Muon orthogonalization, and in-place scatter head scoring, trading 9.11 GiB -> 20.57 GiB peak VRAM—or 9.61–9.69 GiB peak VRAM when all 8 delayed trace tensors are placed in pinned host ring buffers in exp44) matches exp32 in quality (1.9358 ± 0.0002 vs. 1.9361 ± 0.0004 DEV BPB at $M_g=8$) with a 2.63x speedup on matched fwd + rule timed regions (7,857.05 -> 2,991.84 ms/step, 3.88x on the learning rule 6,432.04 -> 1,659.19 ms), and reaches 1.9102 ± 0.0009 DEV BPB at $M_g=1, \text{lr\_embed}=3\text{e-}3$ and 1.8978 ± 0.0007 DEV BPB (1.8844 ± 0.0003 VAL-B BPB, 208.39 PPL) at $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-8}$ in 3.26 s/step (fwd + rule; vs. 1.9266 ± 0.0004 at $\epsilon=10^{-4}$ in exp44).
• Joint Co-Design of the Optimizer and Delayed WM Credit (2.1273 -> 2.0682 DEV BPB at 360 Steps; 1.9989 at 720 Steps, exp38): Adapting the Pre-Polar Muon foundation of [8] to delayed WM, we combine Pre-Polar Muon update orthogonalization with scale-matched sparse embedding velocity (lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)) and post-installation drift annealing (lrm: 1.0 -> 0.2) on d6. Across 3 seeds on d6 (360 steps, Strict Post-Install), this optimizer package improves Delayed WM from 2.1273 ± 0.0001 to 2.0682 ± 0.0006 DEV BPB (2.0315 ± 0.0004 VAL-B BPB, 292.66 PPL)—within 0.0150 BPB of matched-subset Exact Backpropagation (2.0532 ± 0.0008, which also improves from 2.1080 ± 0.0006 under the same optimizer package). On a Seed-46 Block-0 diagnostic audit (run_geo_and_attn_steering_audit.py), Pre-Polar Muon is descriptively associated with a higher mlp.c_proj weight stable rank than AdamW (127.24 under paper20_prepolar and 141.33 under codesigned_prepolar vs. 2.75 under adamw_hidden), and the co-designed embedding/annealing schedule increases held-out attention-map Frobenius shift by $9.82\times$ in Layer 0 (0.0838 -> 0.8233) and $1.42\times$ across all 6 layers (0.6046 -> 0.8587).
• Controlled Comparisons Against Author-Defined SR-LNP and SR-NP Node-Perturbation Recipes Across Tokens, Compute, and Width ($D \in \{256, 384, 512, 768\}$, exp37, exp39, exp42): We compare Delayed WM against two author-defined node-perturbation recipes on the matched 4-matrix d6 subset: (i) Surrogate-Routed Local Node Perturbation (SR-LNP, $K \in \{4, 16, 64\}$), which uses same-step terminal-skip routing ($g_{\text{bus}} \cdot \gamma_l$) and $\gamma = 0$ local value scoring; and (ii) Suffix-Recomputed Node Perturbation (SR-NP, exp42), which uses Gaussian draw-mean centering, suffix recomputation from block $l$ onward, shared post-MLP block-output perturbations for attn.c_proj and mlp.c_proj, analytic lm_head gradients, and $\gamma = 0.98$ future-token value accumulation:
  1. Equal Tokens (360 steps on d6, $D=384$): Delayed WM (K=1, G=2048) (2.0682 ± 0.0006 DEV BPB, 6.74s measured training GPU time / 6.72s steady-state est.) outperforms SR-LNP at $K=4$ (2.3071 ± 0.0016, 6.32s run / 5.48s est.), $K=16$ (2.2603 ± 0.0018, 9.43s run / 9.23s est.), and $K=64$ (2.2041 ± 0.0013, 23.66s run / 23.22s est.), as well as 76-draw SR-NP (2.2415 ± 0.0015 under Pre-Polar Muon, 65.19s run; 2.4264 ± 0.0088 under Adam, 84.38s run).
  2. Matched Compute (~9.4s and ~13.6s on d6): At 500 steps (9.41s run / 9.34s est.), Delayed WM reaches 2.0322 ± 0.0007 DEV BPB (1.9975 ± 0.0006 VAL-B, 265.13 PPL). At 720 steps (13.57s run / 13.45s est.), Delayed WM reaches 1.9989 ± 0.0008 DEV BPB (1.9646 ± 0.0008 VAL-B, 241.96 PPL).
  3. Model Width Scaling ($D \in \{256, 384, 512, 768\}$, exp39): As width $D$ scales from 256 -> 384 -> 512 -> 768, Delayed WM's gap to matched-subset Exact Backpropagation shrinks from +0.0144 to +0.0125 BPB (2.0558 ± 0.0002 vs. 2.0433 ± 0.0006 at $D=768$).
• Three-Tier Attention Credit-Assignment Hierarchy ($W_V$ vs. $W_Q$ vs. $W_K$), Key Truncation vs. Future-Query Accumulation, and Spherical Active-$W_Q$ on ClimbMix d6 (2.0682 -> 2.0641 ± 0.0006 DEV BPB, exp45) & Native 50.3M (1.8978 -> 1.8897 ± 0.0008 DEV BPB / 203.72 PPL in 9.80 GiB VRAM, exp46): Extending our backward-free analysis to the attention score projections attn.c_q ($W_Q$) and attn.c_k ($W_K$) (exp45_d6_attn_routing_results.json, exp46_native50m_active_q_results.json), we establish a Three-Tier Attention Credit-Assignment Hierarchy at $M_g = 1$. Conditioned on oracle downstream $\nabla_{y_{\text{attn}}}\mathcal{L}$ (gy_attn from loss.backward() at initialized seed = 46, Layer 0) to isolate single-block attention-subspace transport from multi-layer delayed credit variance: (i) Tier 1 (attn.c_v, $m_{\text{eff}} = 1$, coordinate-isolated & forward-factorizable via $\bar{x}_{\text{attn\_in}}^{(h)}$) satisfies a conditional single-block reconstruction identity cos = 1.0000 at $K=1$ (distinct from its +0.9069 end-to-end delayed backward-free cosine in Table 1); (ii) Tier 2 (attn.c_q, $m_{\text{eff}} = d_h$, token-local in time $\tau$ because query $q_\tau^{(h)}$ enters only row $\tau$ of $\mathbf{A}^{(h)}$, but coupled across the $d_h$ head channels inside $\langle q_\tau^{(h)}, k_s^{(h)}\rangle / \sqrt{d_h}$) achieves single-draw cosine cos = 0.1357 ($\approx 1/\sqrt{64} = 0.1250$) at $K=1$, scaling as $\sqrt{K}$ to 0.2657 ($K=4$), 0.4748 ($K=16$), and 0.7385 ($K=64$); and (iii) Tier 3 (attn.c_k, non-local across future queries $\tau \ge s$) faces two distinct obstacles: multiplying r_head at position $s$ by $\xi_{k,s}$ at the same position $s$ has first-order expectation $\mathbb{E}[\widehat{g}_{k_s}^{\text{same-pos}}] = J_{ss}^\top g_s$, which captures only the diagonal self-attention sensitivity ($\partial y_s / \partial k_s$) while truncating all future-query credit $\sum_{\tau > s} J_{\tau s}^\top g_\tau$ (cos = 0.0076 at $K=1$, 0.0534 at $K=64$); accumulating future query rewards $\widetilde{R}_{k,s}^{(\gamma_K)} = \sum_{\tau \ge s, \tau - s \le W_l} \gamma_K^{\tau - s} R_{\text{head},\tau}$ eliminates temporal truncation bias at $\gamma_K = 1$ (verified by our 2-token exact Rademacher enumeration) and lifts $K=64$ cosine alignment 2.51x to 0.1340 ($\gamma_K=1.00$) and 2.93x to 0.1565 ($\gamma_K=0.98$), yet single-draw ($K=1$) estimation remains limited by cross-time key superposition variance (cos = 0.0066 at $\gamma_K=1.00$, 0.0089 at $\gamma_K=0.98$) because all causal keys $s' \le \tau$ are perturbed simultaneously inside each query row $\tau$. Designing Spherical Pre-Polar Muon for attn.c_q—combining unit-RMS query tangent perturbation, spherical Frobenius retraction $\|W_Q^{(l)}\|_F \equiv \|W_{Q, 0}^{(l)}\|_F$, high momentum $\beta_Q = 0.95$, and $\sqrt{d_h}$-scaled angular step size $\eta_Q = 0.048$—improves 5-matrix backward-free Delayed WM on ClimbMix d6 (25.46M / 26.35M = 96.6% active parameters) from 2.0682 ± 0.0006 to 2.0641 ± 0.0006 DEV BPB (2.0273 ± 0.0005 VAL-B BPB, 289.43 PPL, -0.0041 BPB / -3.23 PPL) across Seeds 46, 47, 48 (with a factorized Euclidean control at matched $\beta_Q=0.95, \eta_Q=0.048$ reaching 2.0649 ± 0.0006 DEV BPB, isolating -0.0033 BPB from gain/momentum matching and -0.0008 BPB from spherical retraction + tangent perturbation, while future-accumulated $\gamma_K=0.98$ Active-$W_K$ improves over same-position Active-$W_K$ from 2.0684 ± 0.0005 to 2.0672 ± 0.0006 DEV BPB). Crucially, transferring Spherical Active-$W_Q$ to Native 50,332,176 ($L=8, D=512, T=2,048, M_g=1$, 48.23M / 50.33M = 95.8% active parameters, exp46) with full pinned host ring buffers, deterministic on-the-fly xi_q regeneration, and exact inverse-RoPE recovery of $q_n$ achieves 1.8889 / 1.8897 / 1.8905 $\to$ 1.8897 ± 0.0008 DEV BPB (1.8765 ± 0.0013 VAL-B BPB, 203.72 PPL) in 9.80 GiB peak VRAM—improving upon our 4-matrix exp43 Native baseline (1.8978 ± 0.0007) by -0.0081 BPB (-4.67 PPL) and upon exp32 (1.9361 ± 0.0004) by -0.0464 BPB (-28.42 PPL).

---

2. Related Work and Methodological Distinctions

Weight Maximization, Unbiased WM, and Node-Perturbation Methods (Dust). Chung [1] introduced Weight Maximization (WM) for networks of self-interested reinforcement learning agents (Learning by Competition of Self-Interested Reinforcement Learning Agents, arXiv:2010.09770), proving that when each unit modulates its eligibility trace by the change in squared Euclidean norm of its outgoing weights, expected updates follow the gradient of downstream weight magnitude without backward error transport. Chung [2] subsequently formulated Unbiased Weight Maximization (UWM, arXiv:2307.13270) for Bernoulli-logistic units using an unbiased outgoing-weight product estimator. These external papers establish the original WM learning principle; they do not provide an inherited convergence guarantee for our multi-token perturbation-differential surrogate or diagonal skip routing. Recently, Dahal et al. [9] introduced Dust (Pretraining Transformers Without Backpropagation, QLabs, October 2026), which trains Transformer language models without backpropagation by injecting $K$ tokenwise Gaussian activation perturbation draws across separate layer passes, recomputing the affected network suffix from cached prefixes, rewarding direct layers via draw-centered token-loss reductions ($\gamma = 0$), and rewarding attention internals via per-head local attention-output alignment accumulated over future tokens ($\gamma = 0.98$, plus $49\text{k}$–$65\text{k}$ vocabulary-slab head draws and ES on residual scalars). Because our experiments train a 4-matrix subset (mlp.c_fc, mlp.c_proj, attn.c_v, attn.c_proj plus wte, value_embeds, lm_head) at small per-step draw counts rather than replicating [9]'s full 6-matrix, 65k-head-slab, $K \in [64, 16384]$ protocol, we do not claim head-to-head superiority over published Dust [9]. Instead, we evaluate two author-defined node-perturbation comparators on our matched 4-matrix architecture: (i) Surrogate-Routed Local Node Perturbation (SR-LNP, exp37–exp39), which shares the terminal-skip residual factors ($\gamma_l, \alpha_0$) of our Delayed WM pipeline but differs in credit timing (constructing $\hat{g}_{\text{bus}}$ and immediately using it for same-step embedding and hidden-layer estimates rather than 3-stage delayed packets), perturbation location, and attention-credit treatment (scoring value perturbations locally with $\gamma = 0$ rather than via the forward attention-filtered trace); and (ii) Suffix-Recomputed Node Perturbation (SR-NP, exp42), an author-defined suffix-recomputed recipe that uses Gaussian draw-mean centering, suffix recomputation from block $l+1$ onward, shared post-MLP block-output perturbations $\hat{g}_{\text{wr}}$ for both attn.c_proj and mlp.c_proj (which omits attn.c_proj's path through the same block's MLP), an analytic local loss gradient on lm_head, and $\gamma = 0.98$ future-token accumulation on per-head attention-output alignment.

Concurrent rStation Studies on Forward-Only and Delayed Transformer Credit Assignment. Two concurrent rStation papers independently investigate delayed WM on general_pretraining:

• In general_pretraining_18/v7, cartan_transport [5] independently develops token micro-groups (stream partitioning), column-isolated WM rewards, the replacement of sibling-unit interference by within-group token interference, exact symmetrized unordered-pair conditional covariance formulas, and SRAM-fused outgoing-column contractions (Delayed-Trace-Co-WM-S) combined with $P=64$ and $P=256$ suffix forward probes for attention on ClimbMix d6 (2.04412 ± 0.00014 BPB at $P=64$ in $633.6\text{s}/635.2\text{s}$, 2.08637 ± 0.01282 at $P=256$, and 2.14471 ± 0.00464 at $1,000$ steps) and 2-seed (s42 and s137) 200-step recency-weighted value-pooling diagnostics (2.14681 and 2.13220 BPB at $\gamma = 1.00$, 2.13081 at $\gamma = 0.98$). Likewise, [8] (general_pretraining_12/v2) supplies the Pre-Polar Muon optimizer foundation. We explicitly credit [5] for the micro-group, symmetrized pair-covariance, and SRAM-fused outgoing-column contraction foundations and [8] for Pre-Polar Muon. Our distinct contributions are: (i) reconciling the code-level $1/M_g$ normalization with 3-seed scale-normalized controls on both d6 (exp35) and Native 50.3M (exp44) and deriving the single-token cross-coordinate variance at non-separable RMSNorm interfaces (Proposition 1(a–c)); (ii) training attn.c_v via a forward causal attention-filtered pre-synaptic trace $\bar{x}_{\text{attn\_in}}^{(h)}$ (Proposition 2) using only 2 forward passes per step; (iii) routing lexical credit across the diagonal skip surrogate $W_{\text{skip}} = \alpha_0 I_D$ (Proposition 3); (iv) isolating Stage-1 vs. Stage-2 post-installation drift across our 16-cell + 4-cell + compliant pure-FP32 clock/optimizer factorials (Proposition 4, exp32–exp35, exp42); and (v) specializing register-tiled Triton kernels (_triton_wm_reward_mg1_kernel and _triton_wm_reward_mg_grouped_kernel) and optimizer co-design to our 3-stage delayed pipeline (2.0682 ± 0.0006 DEV BPB at 360 steps on d6, and 1.8978 ± 0.0007 DEV BPB at $M_g=1$ on Native 50.3M in exp43–exp44).
• In general_pretraining_19/v4–v5, residual_cartographer [6] analyzes short residual writer/reader shortcuts, two-world delayed WM (1.98964 DEV BPB at $499.65\text{M}$ targets, 2.00286 at $134.2\text{M}$ targets on Native 50.3M in v4), and sequence-wide incoming-weight actions with previous-origin activity directions (2.1184/2.0975 DEV BPB at $134.2\text{M}$ targets on Native 50.3M in v5). Because [5], [6], and [9] use different token budgets and schedules than our studies, raw BPB endpoints across [5], [6], and [9] are not direct head-to-head comparisons; rather, our work complements [6] by commuting $W_v^{(h)}$ with causal attention during the forward pass (Proposition 2), characterizing post-installation drift (Proposition 4), and testing residual_cartographer's realized-component reader hypothesis in exp36 Part B.

---

3. Mathematical Formulation and Estimator Analysis

3.1 Three-Stage Residual Pipeline

Consider an $L$-layer causal Transformer with width $D$, vocabulary size $V$, token embedding $e_{\text{tok}}(x) = W_{\text{wte}}[x]$, initial normalized state $x_0 = \text{RMSNorm}(e_{\text{tok}}(x))$, per-layer residual recurrence $x_{\text{in}}^{(l)} = \lambda_{\text{res}}^{(l)} x_{\text{out}}^{(l-1)} + \lambda_{x_0}^{(l)} x_0$, attention and MLP sublayers writing additively via $W_{\text{attn\_proj}}^{(l)} \in \mathbb{R}^{D \times D}$ and $W_{\text{mlp\_proj}}^{(l)} \in \mathbb{R}^{D \times 4D}$, terminal state $x_{\text{final}} = \text{RMSNorm}(x_L)$, and softcapped readout logits $z = 15 \tanh(W_{\text{lm}} x_{\text{final}} / 15)$ with $W_{\text{lm}} \in \mathbb{R}^{V \times D}$. Unrolling the linear residual scalings $\lambda_{\text{res}}^{(l)}, \lambda_{x_0}^{(l)}$ yields the exact terminal representation

$$ x_L = \alpha_0 x_0 + \sum_{l=0}^{L-1} \gamma_l \left(W_{\text{attn\_proj}}^{(l)} y_{\text{attn}}^{(l)} + W_{\text{mlp\_proj}}^{(l)} h_{\text{mlp}}^{(l)}\right), \qquad \gamma_l = \prod_{m=l+1}^{L-1} \lambda_{\text{res}}^{(m)}, \quad \alpha_0 = \prod_{m=0}^{L-1} \lambda_{\text{res}}^{(m)} + \sum_{l=0}^{L-1} \lambda_{x_0}^{(l)} \gamma_l. $$

Exploiting these direct additive write paths into $x_L$, we partition all trainable parameters into three causal stages with maximum pipeline delay $2$ independent of depth $L$:

• Stage 3 (Delay = 0, Terminal Readout): $W_{\text{lm}} \in \mathbb{R}^{V \times D}$.
• Stage 2 (Delay = 1, Residual Stream Writers): $W_{\text{wte}} \in \mathbb{R}^{V \times D}$, $W_{\text{mlp\_proj}}^{(l)} \in \mathbb{R}^{D \times 4D}$, and $W_{\text{attn\_proj}}^{(l)} \in \mathbb{R}^{D \times D}$ for $l \in \{0, \dots, L-1\}$, whose outgoing receiver columns are $W_{\text{lm}}[:, i] \in \mathbb{R}^V$.
• Stage 1 (Delay = 2, Internal Sublayer Writers): $W_{\text{mlp\_fc}}^{(l)} \in \mathbb{R}^{4D \times D}$, $W_{v}^{(l)} \in \mathbb{R}^{D \times D}$, $W_{\text{ve}}^{(l)} \in \mathbb{R}^{V \times D}$, and optionally $W_{q}^{(l)} \in \mathbb{R}^{D \times D}$, whose outgoing receiver columns are $W_{\text{mlp\_proj}}^{(l)}[:, p] \in \mathbb{R}^D$ and $W_{\text{attn\_proj}}^{(l)}[:, p] \in \mathbb{R}^D$.

3.2 Reconciled Code Contraction, Separable Pair Covariance, and RMSNorm Interface Moments (Proposition 1)

Let $N = BT$ denote the total number of tokens in a batch, and partition $\{1, \dots, N\} = \bigsqcup_{g=1}^G \mathcal{S}_g$ into $G$ disjoint streams of size $M_g = N / G$ tokens each. During the forward pass, unit $i$ at stage $m$ injects i.i.d. Rademacher exploration noise $\xi_{n, i} \sim \text{Unif}(\{\pm 1\})$ with scale $\sigma > 0$, and records the stream eligibility trace $Z_{i, :}^{(m, g)} = \sum_{n \in \mathcal{S}_g} (\xi_{n, i}/\sigma) x_n \in \mathbb{R}^{D_{\text{in}}}$.

In compute_stream_wm_rewards_chunked (exp26) and compute_stream_wm_rewards_fast (exp32–exp35), the stream reward divides by $M_g$ with an optional multiplier scale_factor (mult = scale_factor / M_g):

$$ R_i^{(m, g)}(\text{scale\_factor}) = \frac{\text{scale\_factor}}{M_g} \sum_{n' \in \mathcal{S}_g} u_{n'}^{(m+1), T} W_{:, i}^{(m+1)}(t_{\text{constr}}) \, \Delta y_{n', i}^{(m)}, $$

and the subsequent parameter update divides the tokenwise modulation–input product $\sum_{g=1}^G R_i^{(m, g)} Z_{i, :}^{(m, g)}$ by $N = BT$ (-(mod_flat.t() @ x_flat) / BT). Ignoring the descent sign, the implemented contraction in code is therefore

$$ \widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor}) := \frac{1}{N} \sum_{g=1}^G R_i^{(m, g)}(\text{scale\_factor}) \, Z_{i, :}^{(m, g)} = \frac{\text{scale\_factor}}{M_g} \widehat{G}_{i, :}^{\text{norm}}(G), \qquad \text{where} \qquad \widehat{G}_{i, :}^{\text{norm}}(G) := \frac{1}{G} \sum_{g=1}^G R_i^{(m, g)}(1) \, Z_{i, :}^{(m, g)}. $$

In particular, the default setting scale_factor = 1 (exp26–exp34) computes $\widehat{G}_{i, :}^{\text{code}}(G; 1) = \frac{1}{M_g}\widehat{G}_{i, :}^{\text{norm}}(G)$, whereas setting scale_factor = M_g (exp35) computes $\widehat{G}_{i, :}^{\text{code}}(G; M_g) = \widehat{G}_{i, :}^{\text{norm}}(G)$, and at $G = N$ ($M_g = 1$) both coincide identically.

Furthermore, we must distinguish separable channels—where coordinate $i$'s forward displacement depends only on $\xi_{n, i}$, as holds at Stage 1 (mlp.c_fc with elementwise $\text{ReLU}^2$ and attn.c_v with linear head-value projection)—from non-separable RMSNorm interfaces at Stage 2 (x_final = RMSNorm(x_L) and x_0 = RMSNorm(e_tok)), where $\text{rms}(z_n) = \sqrt{\frac{1}{D}\sum_{i'=1}^D z_{n, i'}^2}$ couples coordinate $i$ to $\xi_{n, i'}$ for $i' \neq i$.

Proposition 1 (Conditional Moments of $\widehat{G}_{i, :}^{\text{code}}(G; \text{scale\_factor})$ for Separable Channels and RMSNorm Interfaces). Conditioned on fixed pre-synaptic inputs $\{x_n\}_{n=1}^N \subset \mathbb{R}^{D_{\text{in}}}$ and downstream weights/modulations independent of the i.i.d. Rademacher draws $\xi_{n, i} \sim \text{Unif}(\{\pm 1\})$:

1. (a) Separable-Channel Conditional Mean (Stage 1: mlp.c_fc, attn.c_v): Suppose $\Delta y_{n, i}^{(m)} = \sigma c_{n, i} \xi_{n, i} + r_{n, i}(\sigma)$ with $c_{n, i}$ independent of $\xi_n$ and $r_{n, i}(\sigma) = O(\sigma^2)$, and let $a_{n, i} := c_{n, i} \sum_{j=1}^{d_{\text{next}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) u_{n, j}^{(m+1)}$. Then the first-order code estimator $\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})$ satisfies for every $G \in \{1, \dots, N\}$ ($M_g = N / G$):$$ \mathbb{E}_{\xi}\left[\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})\right] = \frac{\text{scale\_factor}}{M_g} \left(\frac{1}{N} \sum_{n=1}^N a_{n, i} x_n\right). $$Specifically, under scale_factor = 1, the conditional mean scales as $\frac{1}{M_g}\left(\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n\right)$ (evaluating to $0.5$ on $N=4, G=2, M_g=2, a_n = x_n = 1$), whereas under scale_factor = M_g, the conditional mean is $\mathbb{E}_{\xi}[\widehat{G}_{i, :}^{\text{norm}, (1)}(G)] = \frac{1}{N}\sum_{n=1}^N a_{n, i} x_n$ for all $G \in \{1, \dots, N\}$ (evaluating to $1.0$ on $N=4, G=2, M_g=2, a_n = x_n = 1$).
2. (b) Separable-Channel Symmetrized Pair Covariance ([5], Section 2.3): Under the separable channel model of (a), the exact conditional covariance matrix of $\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})$ is$$ \text{Cov}_{\xi}\left(\widehat{G}_{i, :}^{\text{code}, (1)}(G; \text{scale\_factor})\right) = \left(\frac{\text{scale\_factor}}{M_g}\right)^2 \frac{1}{N^2} \sum_{g=1}^G \sum_{\substack{n, n' \in \mathcal{S}_g \\ n < n'}} \left(a_{n', i} x_n + a_{n, i} x_{n'}\right)\left(a_{n', i} x_n + a_{n, i} x_{n'}\right)^T, $$which sums over the $G\binom{M_g}{2} = \frac{N(M_g - 1)}{2}$ unordered within-stream token pairs and vanishes identically at $G = N$ ($M_g = 1$).
3. (c) Non-Separable RMSNorm Interface Moments at $M_g = 1$ (Stage 2: x_final, x_0): At an RMSNorm interface $x_n = \text{RMSNorm}(z_n) = z_n / \text{rms}(z_n)$ ($z_n \in \mathbb{R}^D$), tangent-projected or raw Rademacher perturbation yields the first-order forward displacement $\Delta x_n^{(1)} = \frac{\sigma}{\text{rms}(z_n)} P_{x_n} \xi_n$, whose $i$-th coordinate is $\Delta x_{n, i}^{(1)} = \frac{\sigma}{\text{rms}(z_n)}\left((1 - x_{n, i}^2/D)\xi_{n, i} - \frac{x_{n, i}}{D}\sum_{i' \neq i} x_{n, i'}\xi_{n, i'}\right)$. Consequently, even at $G = N$ ($M_g = 1$, where cross-token pairs $n < n'$ are absent), for a single token $n$ with receiver sensitivity $a_{n, i} = \sum_j W_{j, i}^{(m+1)} u_{n, j}^{(m+1)}$ and pre-synaptic input $h_n \in \mathbb{R}^{D_{\text{in}}}$, the first-order unprojected coordinate-$i$ estimator is$$ \hat{g}_{n, i}^{(1)} := a_{n, i} \frac{\Delta x_{n, i}^{(1)} \xi_{n, i}}{\sigma} h_n = \frac{a_{n, i}}{\text{rms}(z_n)} \left(1 - \frac{x_{n, i}^2}{D} - \frac{x_{n, i}}{D} \sum_{i' \neq i} x_{n, i'} \xi_{n, i} \xi_{n, i'}\right) h_n, $$which has exact conditional mean and nonzero single-token cross-coordinate covariance$$ \mathbb{E}_{\xi_n}\left[\hat{g}_{n, i}^{(1)}\right] = \frac{a_{n, i}}{\text{rms}(z_n)}\left(1 - \frac{x_{n, i}^2}{D}\right) h_n, \qquad \text{Cov}_{\xi_n}\left(\hat{g}_{n, i}^{(1)}\right) = \frac{a_{n, i}^2 x_{n, i}^2 (D - x_{n, i}^2)}{D^2 \, \text{rms}(z_n)^2} h_n h_n^T. $$For example, for a single token ($N=G=M_g=1$) at $D=2$ with $z_1 = (1, 2)$, $a_{1, 1} = 1$, and $h_1 = 1$, we have $\text{rms}(z_1) = \sqrt{2.5}$, $x_{1, 1}^2 = 0.4$, $x_{1, 1} x_{1, 2} = 0.8$, and $\hat{g}_{1, 1}^{(1)} = \frac{0.8 - 0.4 \xi_{1, 1}\xi_{1, 2}}{\sqrt{2.5}}$, with mean $0.8/\sqrt{2.5} \approx 0.5060$ and variance $\frac{0.4(2 - 0.4)}{4 \times 2.5} = 0.064$.

Proof. For (a–b), expanding $\widehat{G}_{i, :}^{\text{norm}, (1)}(G) = \frac{1}{N}\sum_{g=1}^G (\sum_{n' \in \mathcal{S}_g} a_{n', i}\xi_{n', i})(\sum_{n \in \mathcal{S}_g} \xi_{n, i} x_n)$ using $\xi_{n, i}^2 \equiv 1$ gives $\frac{1}{N}\sum_{n=1}^N a_{n, i} x_n + \frac{1}{N}\sum_{g=1}^G \sum_{n < n' \in \mathcal{S}_g} \eta_{\{n, n'\}, i} (a_{n', i} x_n + a_{n, i} x_{n'})$ where $\eta_{\{n, n'\}, i} = \xi_{n, i}\xi_{n', i}$ are pairwise uncorrelated with zero mean and unit variance; multiplying by $\text{scale\_factor} / M_g$ scales the mean by $\text{scale\_factor} / M_g$ and the covariance by $(\text{scale\_factor} / M_g)^2$. For (c), since $\xi_{n, i}^2 = 1$, multiplying $(P_{x_n}\xi_n)_i = (1 - x_{n, i}^2/D)\xi_{n, i} - \frac{x_{n, i}}{D}\sum_{i' \neq i} x_{n, i'}\xi_{n, i'}$ by $\xi_{n, i}$ yields the displayed expression for $\hat{g}_{n, i}^{(1)}$. Because $\{\xi_{n, i}\xi_{n, i'}\}_{i' \neq i}$ are $D - 1$ pairwise uncorrelated zero-mean Rademacher products with unit variance and $\sum_{i' \neq i} x_{n, i'}^2 = \|x_n\|_2^2 - x_{n, i}^2 = D - x_{n, i}^2$, the mean and covariance follow immediately. $\square$

3.3 Forward Causal Attention-Filtered Value Eligibility Trace (Proposition 2)

In causal multi-head self-attention with $H$ heads of dimension $D_h = D / H$, head $h$'s output at batch $b$, query token $\tau \in \{0, \dots, T-1\}$ is $y_{\text{attn}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} v^{(h)}(b, s)$, where $v^{(h)}(b, s) = W_v^{(h)} x_{\text{attn\_in}}(b, s)$ ($W_v^{(h)} \in \mathbb{R}^{D_h \times D}$) and $A_{b, h, \tau, s}$ is the causal attention weight ($\sum_{s \le \tau} A_{b, h, \tau, s} = 1$, $A_{b, h, \tau, s} = 0$ for $s > \tau$).

Proposition 2 (Exact Commutation of $W_v^{(h)}$ with Causal Attention). Because $W_v^{(h)} \in \mathbb{R}^{D_h \times D}$ is shared across sequence positions $s \in \{0, \dots, T-1\}$, the post-attention value aggregation at position $\tau$ satisfies the exact linear identity

$$ y_{\text{attn}}^{(h)}(b, \tau) = \sum_{s \le \tau} A_{b, h, \tau, s} \left(W_v^{(h)} x_{\text{attn\_in}}(b, s)\right) = W_v^{(h)} \underbrace{\left(\sum_{s \le \tau} A_{b, h, \tau, s} x_{\text{attn\_in}}(b, s)\right)}_{\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau) \in \mathbb{R}^D}. $$

Consequently, injecting exploration noise $\psi_v^{(h)}(b, \tau) \sim \text{Unif}(\{\pm 1\}^{D_h})$ at the post-attention value output $y_{\text{attn}}^{(h)}(b, \tau)$ and recording the forward causal attention-filtered pre-synaptic trace $Z_{v}^{(h, g)}(t-2) = \sum_{(b, \tau) \in \mathcal{S}_g} (\psi_v^{(h)}(b, \tau)/\sigma_1) \, \bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)^T$ yields an exact single-position linear channel $y_{\text{attn}}^{(h)}(b, \tau) = W_v^{(h)} \bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)$ at each output token $\tau$, eliminating causal cross-position perturbation mixing with zero reverse-time pooling.

3.4 Diagonal Identity-Skip Surrogate & Post-Installation Credit Decomposition (Propositions 3 & 4)

Proposition 3 (Algebraic Diagonal Skip-Routing Surrogate for wte). In the unrolled residual equation $x_L = \alpha_0 x_0 + \dots$, the direct path from $x_0 = \text{RMSNorm}(W_{\text{wte}}[x])$ to $x_L$ is the fixed diagonal scaling $W_{\text{skip}} = \alpha_0 I_D$. Treating $W_{\text{skip}} = \alpha_0 I_D$ as an algebraic surrogate receiver matrix with post-synaptic modulation $s_{\text{bus}}(n) = \text{mod\_bus\_base}(n) \in \mathbb{R}^D$ and pre-synaptic forward displacement $\Delta x_0(n) = \text{RMSNorm}(e_{\text{tok}}(n) + \sigma \tilde{\xi}_{\text{wte}}(n)) - x_0(n)$, the column-norm contraction reduces to the $O(BT \cdot D)$ elementwise contraction

$$ R_{\text{wte}, i}^{\text{raw}, (g)}(\text{scale\_factor}) = \sum_{j=1}^D (\alpha_0 \delta_{j, i}) \left(\frac{\text{scale\_factor}}{M_g}\sum_{n \in \mathcal{S}_g} s_{\text{bus}, n, j} \, \Delta x_{0, n, i}\right) = \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} \left(\alpha_0 s_{\text{bus}, n, i}\right) \Delta x_{0, n, i}, $$

(followed in compute_identity_skip_wm_rewards_fast by stream-mean centering $R_{\text{wte}, i}^{(g)} = R_{\text{wte}, i}^{\text{raw}, (g)} - \frac{1}{G}\sum_{g'=1}^G R_{\text{wte}, i}^{\text{raw}, (g')}$), eliminating the second $O(G V D)$ lm_head contraction.

Finally, we analyze Strict Post-Installation Construction (Stage 2: (t-1, t [post-Stage3-step], t)), where Stage 2 evaluates compute_stream_wm_rewards_fast at clock $t$ on $W_{\text{lm}}(t) = W_{\text{lm}}(t-1) + \delta W_{\text{lm}}(t-1)$ after Stage 3 has installed its batch-$t-1$ update. In our implementation (custom_entrywise_adamw_step and row_isotropic_adamw_step), at optimizer step $t_{\text{step}}$, the installed Stage-3 displacement is

$$ \delta W_{\text{lm}, j, i}(t-1) = -\eta_3 \lambda_{\text{wd}} W_{\text{lm}, j, i}(t-1) - \frac{\eta_3}{1 - \beta_1^{t_{\text{step}}}} \cdot d_{j, i} \left(\beta_1 m_{j, i}^{(t-2)} + (1 - \beta_1) g_{j, i}^{(t-1)}\right), \qquad d_{j, i} := \frac{1}{\sqrt{\hat{v}_{j, i}^{(t-1)}} + \epsilon}, $$

where $g_{j, i}^{(t-1)} = -\frac{1}{BT}\sum_{n'=1}^{BT} s_{\text{lm}, n', j} x_{\text{final}, n', i}$, $\hat{v}_{j, i}^{(t-1)} = v_{j, i}^{(t-1)} / (1 - \beta_2^{t_{\text{step}}})$ under entrywise moments (or row-averaged $d_{j, i} \equiv d_j = (\sqrt{\hat{v}_j^{(t-1)}} + \epsilon)^{-1}$ with $\hat{v}_j^{(t-1)} = \frac{1}{D}\sum_{i=1}^D \hat{v}_{j, i}^{(t-1)}$ under row_iso), and $\epsilon > 0$ is added outside the square root (denom = sqrt(v_hat) + eps). At $G = BT$ ($M_g = 1$), let $u_{\text{bus}, i}^{\text{pre}}(n) = R_{\text{bus}, i}^{\text{pre}}(n) (\xi_{n, i}/\sigma)$ and $u_{\text{bus}, i}^{\text{post}}(n) = R_{\text{bus}, i}^{\text{post}}(n) (\xi_{n, i}/\sigma)$ denote the unprojected Stage-2 credit before and after installing $\delta W_{\text{lm}}(t-1)$.

Proposition 4 (Decomposition of Installed-Update Terminal Credit Shift, Data-Dependent Fourth-Moment Bound, and Preconditioning Regimes). Under Strict Post-Installation Construction at $G = BT$ ($M_g = 1$), the post-minus-pre unprojected credit shift $\delta u_{\text{bus}}(n) := u_{\text{bus}}^{\text{post}}(n) - u_{\text{bus}}^{\text{pre}}(n)$ decomposes into three additive components corresponding to decoupled weight decay, first-moment history $m^{(t-2)}$, and current-batch gradient $g^{(t-1)}$:

$$ \delta u_{\text{bus}, i}(n) = -\eta_3 \lambda_{\text{wd}} u_{\text{bus}, i}^{\text{pre}}(n) + \delta u_{\text{mom}, i}(n) + \delta u_{\text{cur}, i}^{(n' = n)}(n) + \delta u_{\text{cur}, i}^{(n' \neq n)}(n), $$

where:

1. Same-Token ($n' = n$) RMSNorm Radial-vs-Oblique Split and Data-Dependent Fourth-Moment Bound: With tangent-projected forward carrier $\tilde{\xi}_n = P_{x_{\text{final}}(n)} \xi_n$ and raw carrier multiplication $\xi_n / \sigma$, the first-order same-token ($n'=n$) term is $\delta u_{\text{cur}, i}^{(n'=n, 1)}(n) = \frac{\eta_3 (1-\beta_1)}{(1-\beta_1^{t_{\text{step}}}) BT} c_{n, i} \, b_i(n)$, where $c_{n, i} := \sum_{j=1}^V d_{j, i} s_{\text{lm}, n, j}^2$ and the vector $b(n)$ splits into an $O(1)$ radial vector $b_{\text{rad}}(n) \parallel x_{\text{final}}(n)$ and an oblique RMSNorm-Jacobian coupling term $b_{\text{oblique}}(n)$:$$ b(n) := \frac{1}{\text{rms}(z_n)}\left(x_{\text{final}}(n) \odot \xi_n \odot (P_{x_{\text{final}}(n)} \xi_n)\right) = \underbrace{\frac{1}{\text{rms}(z_n)} x_{\text{final}}(n)}_{b_{\text{rad}}(n) \parallel x_{\text{final}}(n)} - \underbrace{\frac{\langle \xi_n, x_{\text{final}}(n)\rangle}{D \, \text{rms}(z_n)} \left(x_{\text{final}}(n)^{\odot 2} \odot \xi_n\right)}_{b_{\text{oblique}}(n)}. $$Under row-isotropic preconditioning ($d_{j, i} \equiv d_j$, so $c_{n, i} \equiv c_n$ is a scalar), reward-side projection $P_{x_{\text{final}}(n)}$ annihilates $c_n b_{\text{rad}}(n)$ identically ($P_{x_{\text{final}}(n)} b_{\text{rad}}(n) = 0$) while leaving $P_{x_{\text{final}}(n)} b_{\text{oblique}}(n) \neq 0$ (e.g., at $D=2, z_n=(1, 2), \xi_n=(1, 1)$, $b(n)=(0.16, -0.16)$, $b_{\text{rad}}(n)=(0.4, 0.8)$, and $P_{x_{\text{final}}(n)} b(n) = (0.192, -0.096)$). Defining the normalized coordinate fourth moment $\kappa_4(x) := \frac{1}{D}\sum_{i=1}^D x_i^4 \in [1, D]$, the expected squared norm of $P_{x_{\text{final}}(n)} b(n)$ relative to $\|b_{\text{rad}}(n)\|_2^2 = D / \text{rms}(z_n)^2$ satisfies the exact data-dependent fourth-moment bound$$ \frac{\mathbb{E}_{\xi_n}\left[\|P_{x_{\text{final}}(n)} b(n)\|_2^2\right]}{\|b_{\text{rad}}(n)\|_2^2} \le \frac{\mathbb{E}_{\xi_n}\left[\|b_{\text{oblique}}(n)\|_2^2\right]}{\|b_{\text{rad}}(n)\|_2^2} = \frac{\sum_{i=1}^D x_{\text{final}, n, i}^4}{D^2} = \frac{\kappa_4(x_{\text{final}}(n))}{D}. $$In particular: (i) if $\kappa_4(x_{\text{final}}(n)) \le C = O(1)$ is uniformly bounded, the residual ratio is $\le C/D = O(1/D)$ (on trained ClimbMix d6 states at $D=384$, we measure token-mean $\kappa_4(x_{\text{final}}) = 2.9398$, giving bound $\kappa_4/D = 0.007656$ and empirical ratio $0.007458$); (ii) without a fourth-moment bound, $\|x\|_2^2 = D$ alone does not imply a vanishing residual—for the 2-sparse family $x = \sqrt{D/5}(1, 2, 0, \dots, 0)$ ($z = x$), $\kappa_4(x)/D = 0.68$ and $\frac{\mathbb{E}_{\xi}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} = 0.0576$ for every $D \ge 2$; and (iii) under entrywise preconditioning, coordinate variation in $c_{n, i} = \sum_j d_{j, i} s_{\text{lm}, n, j}^2$ breaks exact radial collinearity ($P_{x_{\text{final}}(n)}(c_n \odot b_{\text{rad}}(n)) \neq 0$) unless $\epsilon \gg \sqrt{\hat{v}_{j, i}^{(t-1)}}$, so increasing $\epsilon: 10^{-8} \to 10^{-4}$ is required to damp entrywise coordinate spread.
2. Cross-Token ($n' \neq n$) Current-Gradient Drift Bound: Since $d_{j, i} = (\sqrt{\hat{v}_{j, i}^{(t-1)}} + \epsilon)^{-1} \le \epsilon^{-1}$, the cross-token ($n' \neq n$) current-gradient contribution satisfies$$ \left|\delta u_{\text{cur}, i}^{(n' \neq n)}(n)\right| \le \frac{\eta_3 (1 - \beta_1)}{(1 - \beta_1^{t_{\text{step}}}) BT \, \epsilon} \sum_{n' \neq n} \left(\sum_{j=1}^V |s_{\text{lm}, n, j}| \, |s_{\text{lm}, n', j}|\right) |x_{\text{final}, n', i}| \, \left|\frac{\Delta x_{\text{final}, n, i}}{\sigma}\right|. $$

3.5 Fused Register-Tiled Contraction, Optimizer Co-Design, and Node-Perturbation Controls

1. Eliminating 3D Outer-Product Materialization via Register-Tiled Contraction (_triton_wm_reward_mg1_kernel): Building on the SRAM-fused outgoing-column contraction concept introduced in [5] (version 5), notice that in the stream-partitioned WM reward (Section 3.2),$$ R_i^{(m, g)}(\text{scale\_factor}) = \frac{\text{scale\_factor}}{M_g} \sum_{j=1}^{D_{\text{out}}} W_{j, i}^{(m+1)}(t_{\text{constr}}) \sum_{n \in \mathcal{S}_g} u_{n, j}^{(m+1)} \, \Delta y_{n, i}^{(m)} = \frac{\text{scale\_factor}}{M_g} \sum_{n \in \mathcal{S}_g} \Delta y_{n, i}^{(m)} \left(\sum_{j=1}^{D_{\text{out}}} u_{n, j}^{(m+1)} W_{j, i}^{(m+1)}(t_{\text{constr}})\right), $$the 3D plasticity tensor $\Delta \tilde{W}^{(m+1, g)} \in \mathbb{R}^{G \times D_{\text{out}} \times D_{\text{in}}}$ never needs to be materialized in GPU HBM. In our fused Triton kernel (_triton_wm_reward_mg1_kernel with tile sizes BLOCK_G=4, BLOCK_R=64, BLOCK_C=64 for $M_g=1$, and _triton_wm_reward_mg_grouped_kernel for $M_g > 1$), each thread block loads a (BLOCK_R, BLOCK_C) tile of $W_{\text{rec}}$ from L2 cache once for BLOCK_G streams and accumulates R[g, c] directly in registers with 0.0 MB temporary HBM allocation, cutting the 13-matrix d6 WM kernel suite $38.4\times$ from 134.31 ms to 3.50 ms/step at $G=2,048$ (and $4.4\times$ from 4.51 ms to 1.03 ms/step at $G=64$).
2. Joint Co-Design of the Optimizer and Delayed WM Credit: Adapting the Pre-Polar Muon optimizer foundation of [8] to delayed WM, we note that the downstream optimizer step installs $W^{(m+1)}(t_{\text{constr}})$—which acts as Stage $m$'s credit reader:
   • Update Orthogonalization and Receiver Stable Rank (Descriptive Association): On our Seed-46 Block-0 diagnostic audit (run_geo_and_attn_steering_audit.py), under AdamW (adamw_hidden), Stage-2 receiver matrices $W^{(2)}$ exhibit low stable rank (mlp.c_proj $\operatorname{srank}(W) := \|W\|_F^2 / \|W\|_2^2 = 2.75$, attn.c_proj $\operatorname{srank}(W) = 1.11$). Pre-Polar Muon [8] orthogonalizes each step's momentum update direction (which does not constrain the accumulated weight matrix $W$ to a Stiefel manifold); descriptively, this update orthogonalization is associated with a higher accumulated receiver stable rank on Block-0 mlp.c_proj ($127.24$ under paper20_prepolar and $141.33$ under codesigned_prepolar$, $51.4\times$ higher than adamw_hidden) and attn.c_proj($54.85$ and $52.72$, $47.5\times$ higher thanadamw_hidden`).
   • Scale-Matched Sparse Embedding Velocity & Post-Installation Drift Annealing: Because wte is initialized at $\sigma_{\text{init}} = 1.0$ before RMSNorm(e_tok) ($\|W_{\text{wte}}[v]\|_2 \approx \sqrt{D} = 19.6$), small embedding learning rates (lr_wte = 3e-3 to 4e-3) rotate normalized token directions slowly. Matching sparse embedding step size on d6 ($BT = 2,048$) to $\sigma_{\text{init}} = 1.0$ (lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)) and linearly annealing learning rates (lrm: 1.0 -> 0.2) increases held-out validation attention-map shift $\|\mathbf{A}^{(l)}(360) - \mathbf{A}^{(l)}(0)\|_F / \|\mathbf{A}^{(l)}(0)\|_F$ (Seed 46) by $9.82\times$ in Layer 0 (0.0838 -> 0.8233) and $1.42\times$ averaged across all 6 layers (0.6046 -> 0.8587).
3. Author-Defined Node-Perturbation Comparators (SR-LNP and SR-NP): We compare Delayed WM against two specified node-perturbation recipes on the matched 4-matrix parameter subset (neither comparator is a zero-same-step-relay delayed pipeline, and neither is claimed as a replica of published Dust [9]):
   • Surrogate-Routed Local Node Perturbation (SR-LNP, train_dust_d6_fast in run_exp37_38_wm_vs_dust.py): Uses the same terminal-skip residual factors ($\gamma_l$ for blocks and $\alpha_0$ for wte) as Delayed WM, but differs in credit timing (it constructs $\hat{g}_{\text{bus}}$ via $K \in \{4, 16, 64\}$ perturbation sub-passes and immediately uses $\hat{g}_{\text{bus}}$ on the same step for embedding and hidden-layer estimates, whereas Delayed WM uses 3-stage delayed packets), perturbation location, and attention-credit treatment (scoring value perturbations locally with $\gamma = 0$ rather than via the forward attention-filtered trace). Thus, SR-LNP vs. Delayed WM compares two complete forward-only recipes rather than isolating sibling variance alone.
   • Suffix-Recomputed Node Perturbation (SR-NP, train_faithful_dust_d6 in run_exp42_compliant_fp32_and_faithful_dust.py): An author-defined suffix-recomputed comparator on the matched 4-matrix d6 subset ($K_{\text{site}} = 4 \implies 76$ suffix-recomputed draws/step $+ 24$ attention-internal draws/step) implementing Gaussian perturbations $a_t \sim \mathcal{N}(0, I)$, draw-mean centering $c_s^{(i)} = \tilde{\ell}_s - \ell_s^{(i)}$, suffix recomputation from block $l+1$ onward (_run_suffix_from_block), per-head local attention-output alignment $c_{s, h}^{(i)} = -\langle \hat{g}_{\text{attn\_out}, s, h}, \Delta o_{s, h}^{(i)} \rangle$, and future-token reward accumulation $r_{t, h}^{(i)} = \sum_{s \ge t} \gamma^{s - t} c_{s, h}^{(i)}$ with $\gamma = 0.98$ (whose discounted accumulation is verified by the 1.47 unit test). In train_faithful_dust_d6, both projection writers (attn.c_proj and mlp.c_proj) use $\hat{g}_{\text{wr}}$ obtained by perturbing the block-output residual after the MLP (which for attn.c_proj omits its path through the same block's MLP, a path not restored by suffix recomputation from block $l+1$ onward), and lm_head receives an analytic local loss gradient rather than vocabulary-slab perturbations.

---

4. Experimental Results and Mechanistic Verification

Six-panel summary of genuinely backward-free 3-stage delayed eligibility-trace Weight Maximization, fused Triton kernels, co-designed optimizer, and node-perturbation controls across exp26–exp44

4.1 Experimental Setup, Arm-Classification Ledger, Attention Backend, and Timing Provenance

All experiments (117 multi-seed experimental arms = 350 total runs across seeds $\{46, 47, 48\}$ in exp18, exp21, exp26–exp46: 80 three-seed Delayed-WM arms [240 runs], 13 three-seed node-perturbation arms [11 SR-LNP + 2 SR-NP = 39 runs], 4 head/lexical-only backward-free anchors [11 runs: three 3-seed + one 2-seed], 7 three-seed local-pullback diagnostic controls in exp36 [21 runs, Rev. Matmuls = 1], and 13 three-seed exact-backpropagation reference arms [39 runs: 4 on d6 in Table 3, 7 across d6 and widths $D \in \{256, 384, 512, 768\}$ in Tables 5–6, and 2 on Native 50.3M in Table 7]) use the supplied rustbpe tokenizer (vocab_size = 8,192) and ClimbMix data stream on two Transformer regimes:

1. ClimbMix d6 ($L=6, D \in \{256, 384, 512, 768\}, H=D/64, D_h=64, V=8,192, B=8, T=256, BT=2,048$ tokens/step, $360$–$720$ steps $= 0.737\text{M}$–$1.475\text{M}$ tokens): At our primary width $D=384$, total parameters $= 26,346,060$, with active_params = 24,576,000 ($24.58\text{M}$) in the 4-matrix configuration (mlp.c_fc, mlp.c_proj, attn.c_v, attn.c_proj active per block, attn.c_q, attn.c_k frozen; $25.46\text{M}$ = 96.6% when attn.c_q or attn.c_k is also active in exp30–exp31 and exp45; and $26.35\text{M}$ when all 6 hidden matrices are active). In Section 4.7 (exp39), we also scale width across $D \in \{256, 384, 512, 768\}$ (active_params $\in \{14.55\text{M}, 24.58\text{M}, 36.70\text{M}, 67.11\text{M}\}$). Disjoint evaluation splits (evaluate_bpb_disjoint) score dev_tokens = 261,889 (dev_bytes = 1,037,477) and valb_tokens = 262,009 (valb_bytes = 1,050,500), where $\text{CE (nats)} = \text{BPB} \cdot \ln(2) \cdot (\text{bytes}/\text{tokens})$ and $\text{PPL} = \exp(\text{CE})$.
2. Native 50,332,176 ($L=8, D=512, H=4, D_h=128, V=8,192, B_{\text{eff}}=64, T=2,048, BT=131,072$ tokens/step, $100$ steps $= 13,107,200$ training tokens): Total parameters $= 50,332,176$, with active_params = 46,137,344 ($46.14\text{M}$, 91.7%) in the 4-matrix configuration (c_q, c_k $= 4,194,304$ and $528$ gate/scalar parameters frozen) and active_params = 48,234,496 ($48.23\text{M}$, 95.8%) in the 5-matrix Spherical Active-$W_Q$ configuration (exp46, where only c_k $= 2,097,152$ and $528$ scalars are frozen). To fit stream-partitioned eligibility-trace tensors in GPU memory, the backward-free Native 50.3M runs accumulate grad_accum_steps = 16 physical microbatches of micro_batch_size = 4 at $G_{\text{micro}} \in \{256, 1024, 8192\}$ (G_total $\in \{4096, 16384, 131072\}$ streams/step, corresponding to $M_g \in \{32, 8, 1\}$ tokens/stream), departing from the supplied task's single-microbatch $B=64$, $499.65\text{M}$-token backpropagation recipe. Disjoint evaluation splits score dev_tokens = 523,826 (dev_bytes = 2,126,272, $\text{bytes}/\text{token} = 4.059119$) and valb_tokens = 523,715 (valb_bytes = 2,129,527). Because both DEV and VAL-B are evaluated at checkpoints across the ablation campaign, VAL-B serves as a second disjoint validation stream rather than an unopened holdout.
3. Attention Backend and Timing Provenance (model_and_data.py, benchmark_uncontended_timings.json): In model_and_data.py, the fa3 handle wraps PyTorch's native fused F.scaled_dot_product_attention(..., is_causal=True). In Tables 4–6, Run GPU reports the measured 3-seed training-region GPU time (total_train_gpu_s_mean) summed over forward and learning-rule CUDA intervals across all training steps (excluding evaluation checkpoints), while Est. reports the 30-step steady-state benchmark estimate (est_360step_s in benchmark_uncontended_timings.json), which measures 35 steps on an uncontended RTX 4090, discards 5 warmup steps, and excludes evaluation, dataloader, and host-transfer overhead. In Tables 3 and 7, Fwd / Rule / Wall (ms) reports per-step forward GPU time, learning-rule GPU time, and either total step GPU time (Fwd + Rule in exp40–exp46) or complete elapsed wall-clock step time including evaluation checkpoints (t_wall_step_ms_including_eval in exp26–exp36).

4.2 Verification of Propositions 1 and 2: Stream Partitioning, Normalization Controls, and Filtered Traces (exp26, exp28, exp35)

Table 1 reports the single-step cosine similarity at initialization between our backward-free eligibility-trace pseudo-gradient estimator and the language-model cross-entropy loss gradient $\nabla_W \mathcal{L}_{\text{CE}}$ (computed via a separate diagnostic reference call loss_ref.backward()) on d6 ($BT = 2,048$) across stream counts $G = BT / M_g$ (exp26_results.json, exp28_results.json). We emphasize that Table 1 is a single-step directional diagnostic rather than a claim of maintained gradient alignment throughout training.

| Stream Count $G$ ($M_g = BT / G$) | Stage 2 wte Cos | Stage 2 mlp.c_proj Cos | Stage 2 attn.c_proj Cos | Stage 1 mlp.c_fc Cos | Stage 1 attn.c_v (Unfiltered $x_{\text{in}}$) | Stage 1 attn.c_v (Filtered $\bar{x}_{\text{in}}$, Prop. 2) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | $G = 1$ ($M_g = 2,048$) | $+0.0313$ | $+0.0187$ | $+0.0170$ | $+0.0037$ | $+0.0036$ | $+0.0295$ | | $G = 64$ ($M_g = 32$) | $+0.2034$ | $+0.2978$ | $+0.2469$ | $+0.0305$ | $+0.0222$ | $+0.2361$ | | $G = 256$ ($M_g = 8$) | $+0.3941$ | $+0.5341$ | $+0.4570$ | $+0.1430$ | $+0.0416$ | $+0.4519$ | | $G = 2,048$ ($M_g = 1$) | $+0.9139$ | $+0.9364$ | $+0.9068$ | $+0.9089$ | $+0.1104$ | $+0.9069$ ($8.2\times$) |

Table 1: Single-step cosine alignment with the language-model cross-entropy loss gradient $\nabla_W \mathcal{L}_{\text{CE}}$ on ClimbMix d6 ($BT = 2,048$) across stream counts $G \in \{1, 64, 256, 2048\}$ (exp26, exp28).

Stream Scale-Normalization Controls (exp35 Part A): To test whether the stream scaling effect in Table 1 survives when update magnitude is held invariant to $M_g$ (Proposition 1), we ran a controlled 3-seed comparison on d6 setting scale_factor = float(M_g) so that $\widehat{G}_{\text{norm}}(G) = M_g \widehat{G}_{\text{code}}(G)$ has the identical conditional mean $\frac{1}{N}\sum_n a_{n, i} x_n$ across all stream sizes:

• At $G = 1$ ($M_g = 2,048$): scale_factor = 2048.0 yields 2.6097 ± 0.0111 DEV BPB (2.5719 ± 0.0106 VAL-B), compared to scale_factor = 1.0 (2.5939 ± 0.0109 DEV BPB).
• At $G = 64$ ($M_g = 32$): scale_factor = 32.0 yields 2.3361 ± 0.0084 DEV BPB (2.2995 ± 0.0086 VAL-B), compared to scale_factor = 1.0 (2.3364 ± 0.0051 DEV BPB).
• At $G = 256$ ($M_g = 8$): scale_factor = 8.0 yields 2.3008 ± 0.0074 DEV BPB (2.2635 ± 0.0065 VAL-B), compared to scale_factor = 1.0 (2.2966 ± 0.0115 DEV BPB).
• On the canonical Pre-Polar Muon trainer (strict_post_install, entrywise_b09_eps1e4, $G=64$, exp35 Part A2): scale_factor = 1.0 reaches 2.2541 ± 0.0013 DEV BPB and scale_factor = 32.0 reaches 2.2539 ± 0.0013 DEV BPB (agreeing to within $0.0002$ BPB). These controls establish that the learning gains from increasing $G$ ($M_g \to 1$) are driven by the elimination of within-stream cross-token pair interference (Proposition 1(b)), not by artificial update rescaling.

4.3 Verification of Propositions 3 and 4: 20-Step Post-Installation Credit Drift, Kurtosis Audit, and Batch-Size Occupancy (exp31, exp34–exp36)

To test how receiver weight installation affects Stage-1 (c_proj receiver) and Stage-2 (lm_head receiver) rewards, we measured the 20-step mean cosine similarity, relative L2 norm drift $\|u_{\text{post}} - u_{\text{pre}}\|_2 / \|u_{\text{pre}}\|_2$, activation fourth moments, and batch-size occupancy on d6 (exp31, exp34–exp36 Part 0, Table 2):

1. Stage-1 Receiver Drift (exp31 Part 0, exp35 Part 0): When Stage 2 (mlp.c_proj, attn.c_proj) is updated with AdamW, one receiver step $\delta W^{(2)}$ induces a 23.56% relative L2 drift in Stage-1 rewards ($\cos = 0.976474$). Replacing AdamW on Stage 2 with Stiefel-orthogonalized Pre-Polar Muon reduces Stage-1 relative L2 reward drift to 2.85% ($\cos = 0.999597$) in exp31 and 4.04% ($\cos = 0.999238$) in exp35 Part 0.
2. Activation Kurtosis Expectation Bound vs. Realized Token Ratios (exp35 Part 0): Across the 20-step diagnostic trajectory (run_part0_kurtosis_and_rmsnorm_variance_audit, which updates lm_head and block-0 mlp.c_proj over 20 steps on d6), the normalized fourth moment $\kappa_4(x_{\text{final}}) = \frac{1}{D}\sum_{i=1}^D x_{\text{final}, i}^4$ has token mean 2.9398 (token max 4.2932). Proposition 4 bounds the expected first-order same-token residual ratio $\frac{\mathbb{E}_{\xi_n}[\|P_x b\|_2^2]}{\|b_{\text{rad}}\|_2^2} \le \frac{\kappa_4(x_{\text{final}})}{D}$; on these states, the mean theoretical expectation bound is 0.007656 and the empirical mean ratio across all $40,960$ token evaluations is 0.007458, while individual realized draws $\xi_n$ can fluctuate higher (token maximum 0.124439).
3. Stage-2 Strict Post-Install Terminal Drift Across the $2 \times 2 \times 2$ Optimizer Cube (exp34 Part 0, Table 2) and Batch-Size Occupancy (exp36 Part 0A): Across all 8 cells of {entrywise, row_iso} x {beta1=0.5, 0.9} x {eps=1e-8, 1e-4}, combining $(\beta_1=0.9, \epsilon=10^{-4})$ cuts current-gradient drift by $47.8\%$ (0.3511 -> 0.1832) and total post-install drift from 39.41% to 26.05% ($\cos = 0.964738$). Furthermore, holding the d6 architecture and sequence length $T=256$ fixed while sweeping batch size $BT \in \{512, 1024, 2048, 4096, 8192\}$ (exp36 Part 0A) shows that singleton target-token occupancy decreases monotonically (0.3957 -> 0.3516 -> 0.2958 -> 0.2265 -> 0.1620) and post-installation relative L2 credit drift decreases monotonically (44.88% -> 41.95% -> 39.41% -> 37.46% -> 35.68%), confirming that larger effective token batches dilute single-token self-update collisions in $\delta W_{\text{lm}}(t-1)$.

| lm_head Optimizer Cell (exp34 Part 0, 20-Step Mean on d6) | bus_unproj Cos / Rel L2 Drift | bus_tangent ($\perp x_{\text{final}}$) Cos / Drift | Current-Batch Grad Only Drift ($\delta u_{\text{cur}}$) | Momentum-History Only Drift ($\delta u_{\text{mom}}$) | $\|P_{x_{\text{final}}} b\|_2 / \|b\|_2$ (Prop. 4) | | :--- | :---: | :---: | :---: | :---: | :---: | | entrywise_b05_eps1e8 ($\beta_1=0.5, \epsilon=10^{-8}$) | $0.940066$ / $0.465747$ | $0.948719$ / $0.394051$ | $0.351118$ | $0.080159$ | $0.087114$ | | entrywise_b09_eps1e8 ($\beta_1=0.9, \epsilon=10^{-8}$) | $0.951336$ / $0.392834$ | $0.954834$ / $0.353878$ | $0.269559$ | $0.112059$ | $0.087114$ | | entrywise_b05_eps1e4 ($\beta_1=0.5, \epsilon=10^{-4}$) | $0.943013$ / $0.391255$ | $0.956556$ / $0.307328$ | $0.270305$ | $0.085703$ | $0.087114$ | | entrywise_b09_eps1e4 ($\beta_1=0.9, \epsilon=10^{-4}$) | $0.958144$ / $0.304684$ | $0.964738$ / $0.260504$ | $0.183231$ | $0.113457$ | $0.087114$ | | row_iso_b05_eps1e8 ($\beta_1=0.5, \epsilon=10^{-8}$) | $0.941012$ / $0.471413$ | $0.952909$ / $0.367834$ | $0.317646$ | $0.076778$ | $0.087114$ | | row_iso_b09_eps1e8 ($\beta_1=0.9, \epsilon=10^{-8}$) | $0.951433$ / $0.400026$ | $0.956413$ / $0.342871$ | $0.252057$ | $0.108569$ | $0.087114$ | | row_iso_b05_eps1e4 ($\beta_1=0.5, \epsilon=10^{-4}$) | $0.942453$ / $0.403128$ | $0.959755$ / $0.292793$ | $0.251836$ | $0.081251$ | $0.087114$ | | row_iso_b09_eps1e4 ($\beta_1=0.9, \epsilon=10^{-4}$) | $0.957127$ / $0.317284$ | $0.966045$ / $0.255660$ | $0.174360$ | $0.110404$ | $0.087114$ |

Table 2: 20-step Pre-Install vs. Strict Post-Install credit decomposition audit across all 8 cells of the $2 \times 2 \times 2$ lm_head optimizer cube on ClimbMix d6 (exp34_results.json Part 0).

4.4 Complete 16-Cell Factorial and True Both-Stages Pre-Install on ClimbMix d6 (exp33–exp35)

Table 3 presents the 3-seed ClimbMix d6 results across stream counts, the complete 16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Pooling $\times$ Momentum $\times$ Damping cube (exp34 + exp35 Part B1), and the 4-cell True Both-Stages Pre-Install comparison (exp35 Part B2). All 20 factorial cells share identical 1-call wte_bus_routed routing (Proposition 3), Rademacher-tangent exploration, and reward-side radial projection.

| Method (ClimbMix d6, $L=6, D=384, V=8,192$, 360 Steps, Seeds 46, 47, 48) | Active Params | Stage-3 / Stage-2 / Stage-1 Clocks | lm_head Opt $(\text{pool}, \beta_1, \epsilon)$ | Rev. Matmuls / Bwd | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Fwd / Rule / Wall (ms) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | supplied_muon_adamw_backprop (Full 6-Matrix BP) | $26.35\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | entry, 0.5, 1e-8 | $-1 / 1$ | $2.1112 / 2.1119 / 2.1105$ | $2.1112 \pm 0.0007$ | $2.0750 \pm 0.0007$ | $331.49$ | $4.6 / 18.0 / 46.2$ | | backprop_prepolar_muon_damped_lm_frozen_qk_control (exp33) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | row_iso, 0.9, 1e-4 | $-1 / 1$ | $2.1155 / 2.1147 / 2.1153$ | $2.1152 \pm 0.0004$ | $2.0780 \pm 0.0002$ | $333.01$ | $4.5 / 19.3 / 47.3$ | | backprop_prepolar_muon_frozen_qk_control (exp32) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | entry, 0.5, 1e-8 | $-1 / 1$ | $2.1207 / 2.1203 / 2.1161$ | $2.1190 \pm 0.0025$ | $2.0825 \pm 0.0027$ | $336.53$ | $4.6 / 19.0 / 47.2$ | | backprop_adamw_frozen_qk_control (exp31) | $24.58\text{M}$ | $(t,t,t)/(t,t,t)/(t,t,t)$ | entry, 0.5, 1e-8 | $-1 / 1$ | $2.1628 / 2.1579 / 2.1654$ | $2.1621 \pm 0.0038$ | $2.1242 \pm 0.0039$ | $378.74$ | $4.4 / 8.5 / 36.8$ | | frozen_backbone_lm_head_only (lm_head only) | $3.15\text{M}$ | $(t,t,t)/\text{frozen}/\text{frozen}$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.2636 / 2.2646 / 2.2660$ | $2.2647 \pm 0.0012$ | $2.2266 \pm 0.0008$ | $491.36$ | $3.5 / 1.3 / 28.4$ | | pure_lexical_x0_skip_only_true_zero (wte + lm_head) | $6.29\text{M}$ | $(t,t,t)/(t,t,t)/\text{frozen}$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.2442 / 2.2449 / 2.2472$ | $2.2454 \pm 0.0016$ | $2.2078 \pm 0.0019$ | $469.19$ | $3.8 / 81.4 / 108.9$ | | causal_trace_wm_G2048_mismatched_origin_control (exp28) | $24.58\text{M}$ | $(t,t,t)/(t-1\neq t)/(t-2\neq t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.7672 / 2.7582 / 2.7570$ | $2.7608 \pm 0.0056$ | $2.7229 \pm 0.0057$ | $1960.79$ | $5.6 / 396.4 / 425.8$ | | causal_trace_wm_G1_centered ($G=1$, exp26) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.5831 / 2.6048 / 2.5938$ | $2.5939 \pm 0.0109$ | $2.5564 \pm 0.0103$ | $1240.17$ | $5.5 / 19.4 / 48.6$ | | causal_trace_wm_G1_centered_scaleMg ($G=1$, exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.6093 / 2.6209 / 2.5988$ | $2.6097 \pm 0.0111$ | $2.5719 \pm 0.0106$ | $1294.94$ | $14.3 / 26.3 / 93.9$ | | causal_trace_wm_G64_centered ($G=64$, exp26) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.3396 / 2.3305 / 2.3391$ | $2.3364 \pm 0.0051$ | $2.2994 \pm 0.0052$ | $611.31$ | $5.5 / 38.4 / 67.5$ | | causal_trace_wm_G64_centered_scaleMg ($G=64$, exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.3458 / 2.3316 / 2.3310$ | $2.3361 \pm 0.0084$ | $2.2995 \pm 0.0086$ | $610.94$ | $14.2 / 59.9 / 127.5$ | | causal_trace_wm_G256_centered ($G=256$, exp26) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.2889 / 2.2911 / 2.3098$ | $2.2966 \pm 0.0115$ | $2.2602 \pm 0.0117$ | $548.11$ | $5.6 / 98.4 / 127.6$ | | causal_trace_wm_G256_centered_scaleMg ($G=256$, exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.3018 / 2.3077 / 2.2930$ | $2.3008 \pm 0.0074$ | $2.2635 \pm 0.0065$ | $554.47$ | $14.2 / 176.7 / 243.7$ | | causal_trace_wm_G2048_wtrace1_tangent_explore (AdamW, exp27) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.2182 / 2.1924 / 2.1964$ | $2.2023 \pm 0.0139$ | $2.1661 \pm 0.0118$ | $423.21$ | $5.6 / 395.7 / 425.0$ | | Complete 16-Cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer Cube (exp34 + exp35 Part B1, Stage 1 = (t-2, t [post-Stage2], t)): | | | | | | | | | | | d6_pre_install_..._entrywise_b05_eps1e8 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1380 / 2.1364 / 2.1335$ | $2.1360 \pm 0.0023$ | $2.0993 \pm 0.0024$ | $352.56$ | $6.6 / 160.3 / 190.6$ | | d6_pre_install_..._entrywise_b09_eps1e8 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | entry, 0.9, 1e-8 | $0 / 0$ | $2.1454 / 2.1415 / 2.1424$ | $2.1431 \pm 0.0020$ | $2.1062 \pm 0.0017$ | $359.53$ | $6.5 / 149.3 / 179.9$ | | d6_pre_install_..._entrywise_b05_eps1e4 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | entry, 0.5, 1e-4 | $0 / 0$ | $2.1217 / 2.1223 / 2.1211$ | $2.1217 \pm 0.0006$ | $2.0849 \pm 0.0006$ | $338.98$ | $6.5 / 149.1 / 179.6$ | | d6_pre_install_..._entrywise_b09_eps1e4 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | entry, 0.9, 1e-4 | $0 / 0$ | $2.1255 / 2.1255 / 2.1248$ | $2.1253 \pm 0.0004$ | $2.0881 \pm 0.0002$ | $342.34$ | $6.5 / 149.1 / 179.7$ | | d6_pre_install_..._row_iso_b05_eps1e8 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | row_iso, 0.5, 1e-8 | $0 / 0$ | $2.1578 / 2.1574 / 2.1578$ | $2.1577 \pm 0.0002$ | $2.1210 \pm 0.0002$ | $374.18$ | $6.5 / 155.1 / 185.6$ | | d6_pre_install_..._row_iso_b09_eps1e8 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | row_iso, 0.9, 1e-8 | $0 / 0$ | $2.1663 / 2.1660 / 2.1663$ | $2.1662 \pm 0.0002$ | $2.1289 \pm 0.0001$ | $383.06$ | $11.3 / 283.0 / 331.7$ | | d6_pre_install_..._row_iso_b05_eps1e4 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | row_iso, 0.5, 1e-4 | $0 / 0$ | $2.1298 / 2.1293 / 2.1296$ | $2.1296 \pm 0.0003$ | $2.0928 \pm 0.0001$ | $346.40$ | $11.2 / 283.6 / 333.9$ | | d6_pre_install_..._row_iso_b09_eps1e4 (exp33) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t,t)$ | row_iso, 0.9, 1e-4 | $0 / 0$ | $2.1335 / 2.1332 / 2.1338$ | $2.1335 \pm 0.0003$ | $2.0961 \pm 0.0002$ | $350.18$ | $6.5 / 149.0 / 179.5$ | | d6_strict_post_install_..._entrywise_b05_eps1e8 (exp33) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1580 / 2.1582 / 2.1574$ | $2.1579 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.39$ | $6.4 / 149.2 / 179.8$ | | d6_strict_post_install_..._entrywise_b09_eps1e8 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.9, 1e-8 | $0 / 0$ | $2.1504 / 2.1499 / 2.1477$ | $2.1494 \pm 0.0014$ | $2.1122 \pm 0.0015$ | $365.74$ | $6.5 / 154.0 / 184.5$ | | d6_strict_post_install_..._entrywise_b05_eps1e4 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.5, 1e-4 | $0 / 0$ | $2.1301 / 2.1302 / 2.1297$ | $2.1300 \pm 0.0003$ | $2.0930 \pm 0.0003$ | $346.83$ | $6.5 / 154.2 / 184.7$ | | d6_strict_post_install_..._entrywise_b09_eps1e4 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.9, 1e-4 | $0 / 0$ | $2.1275 / 2.1273 / 2.1271$ | $2.1273 \pm 0.0002$ | $2.0900 \pm 0.0001$ | $344.25$ | $6.5 / 154.1 / 184.6$ | | d6_strict_post_install_..._row_iso_b05_eps1e8 (exp33) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | row_iso, 0.5, 1e-8 | $0 / 0$ | $2.1581 / 2.1578 / 2.1580$ | $2.1580 \pm 0.0001$ | $2.1214 \pm 0.0004$ | $374.50$ | $6.4 / 149.0 / 179.7$ | | d6_strict_post_install_..._row_iso_b09_eps1e8 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | row_iso, 0.9, 1e-8 | $0 / 0$ | $2.1661 / 2.1655 / 2.1660$ | $2.1659 \pm 0.0003$ | $2.1286 \pm 0.0001$ | $382.72$ | $6.5 / 149.1 / 179.7$ | | d6_strict_post_install_..._row_iso_b05_eps1e4 (exp34) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | row_iso, 0.5, 1e-4 | $0 / 0$ | $2.1301 / 2.1297 / 2.1297$ | $2.1298 \pm 0.0002$ | $2.0932 \pm 0.0002$ | $346.64$ | $6.5 / 149.0 / 179.5$ | | d6_strict_post_install_..._row_iso_b09_eps1e4 (exp33) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | row_iso, 0.9, 1e-4 | $0 / 0$ | $2.1333 / 2.1332 / 2.1337$ | $2.1334 \pm 0.0003$ | $2.0960 \pm 0.0003$ | $350.06$ | $6.5 / 148.9 / 179.3$ | | True Both-Stages Pre-Install (both_stages_pre_install, exp35 Part B2, Stage 1 = (t-2, t-1 [pre-Stage2], t)): | | | | | | | | | | | d6_both_stages_pre_install_entrywise_b05_eps1e8 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1359 / 2.1347 / 2.1318$ | $2.1341 \pm 0.0021$ | $2.0975 \pm 0.0024$ | $350.78$ | $9.5 / 244.1 / 284.4$ | | d6_both_stages_pre_install_entrywise_b09_eps1e8 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.9, 1e-8 | $0 / 0$ | $2.1441 / 2.1404 / 2.1409$ | $2.1418 \pm 0.0020$ | $2.1047 \pm 0.0019$ | $358.24$ | $9.2 / 216.7 / 255.6$ | | d6_both_stages_pre_install_entrywise_b05_eps1e4 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.5, 1e-4 | $0 / 0$ | $2.1195 / 2.1200 / 2.1190$ | $2.1195 \pm 0.0005$ | $2.0827 \pm 0.0006$ | $336.96$ | $10.1 / 181.2 / 223.5$ | | d6_both_stages_pre_install_entrywise_b09_eps1e4 (exp35) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | entry, 0.9, 1e-4 | $0 / 0$ | $2.1231 / 2.1231 / 2.1224$ | $2.1229 \pm 0.0004$ | $2.0857 \pm 0.0002$ | $340.09$ | $9.8 / 184.8 / 228.6$ | | Compliant Zero-Pullback Pure-FP32 Trace-Cache Clock Controls (exp42 Part A, _triton_wm_reward_mg1_kernel): | | | | | | | | | | | d6_pre_install_fp32_cache_triton_b05_eps1e8 (exp42 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1380 / 2.1362 / 2.1336$ | $2.1359 \pm 0.0022$ | $2.0993 \pm 0.0023$ | $352.53$ | $5.2 / 21.7 / 26.9$ | | d6_strict_post_install_fp32_cache_triton_b05_eps1e8 (exp42 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1580 / 2.1582 / 2.1574$ | $2.1579 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.36$ | $5.1 / 21.3 / 26.5$ | | d6_strict_post_install_fp32_cache_triton_b09_eps1e4 (exp42 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.9, 1e-4 | $0 / 0$ | $2.1275 / 2.1275 / 2.1271$ | $2.1274 \pm 0.0002$ | $2.0901 \pm 0.0001$ | $344.34$ | $4.9 / 20.5 / 25.4$ | | Local-Pullback s_lm @ W_lm Clock Controls & Realized-Component Reader Controls (exp36 Parts A & B): | | | | | | | | | | | d6_pre_install_fp32_cache_entrywise_b05_eps1e8 (exp36 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t-1\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $1 / 0$ | $2.1380 / 2.1365 / 2.1336$ | $2.1360 \pm 0.0022$ | $2.0993 \pm 0.0023$ | $352.54$ | $6.3 / 92.1 / 122.1$ | | d6_strict_post_install_fp32_cache_entrywise_b05_eps1e8 (exp36 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $1 / 0$ | $2.1581 / 2.1581 / 2.1573$ | $2.1578 \pm 0.0004$ | $2.1209 \pm 0.0006$ | $374.37$ | $6.3 / 95.3 / 125.1$ | | d6_strict_post_install_fp32_cache_entrywise_b09_eps1e4 (exp36 A) | $24.58\text{M}$ | $(t,t,t)/(t-1,t\text{ [FP32]},t)/(t-2,t,t)$ | entry, 0.9, 1e-4 | $1 / 0$ | $2.1274 / 2.1273 / 2.1271$ | $2.1273 \pm 0.0001$ | $2.0900 \pm 0.0002$ | $344.23$ | $6.3 / 95.3 / 125.3$ | | d6_strict_post_install_internal_muon_colnorm_reader_b05_eps1e8 (exp36 B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.1616 / 2.1615 / 2.1607$ | $2.1613 \pm 0.0005$ | $2.1247 \pm 0.0006$ | $377.93$ | $6.3 / 100.3 / 130.5$ | | d6_strict_post_install_internal_muon_colnorm_reader_b09_eps1e4 (exp36 B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.9, 1e-4 | $0 / 0$ | $2.1341 / 2.1337 / 2.1335$ | $2.1338 \pm 0.0003$ | $2.0965 \pm 0.0004$ | $350.42$ | $6.3 / 100.2 / 130.2$ | | d6_strict_post_install_head_adamw_frobnorm_reader_b05_eps1e8 (exp36 B) | $24.58\text{M}$ | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | entry, 0.5, 1e-8 | $0 / 0$ | $2.2143 / 2.2121 / 2.2121$ | $2.2128 \pm 0.0013$ | $2.1757 \pm 0.0011$ | $435.32$ | $6.3 / 118.2 / 148.6$ |

Table 3: Controlled 3-seed ClimbMix d6 progression, complete 16-cell $2 \times 2 \times 2 \times 2$ Stage-2 Clock $\times$ Optimizer cube (exp34 + exp35 Part B1), 4-cell True Both-Stages Pre-Install comparison (exp35 Part B2), compliant zero-pullback pure-FP32 trace-cache clock controls (exp42 Part A), local-pullback s_lm @ W_lm clock controls (exp36 Part A, Rev. Matmuls = 1), and realized-component reader controls (exp36 Part B).

Key Mechanistic Findings:

1. Stage-1 vs. Stage-2 Post-Installation Drift Separation and Compliant Zero-Pullback Pure-FP32 Clock Controls (exp35 Part B2, exp42 Part A): Comparing True Both-Stages Pre-Install (t-1, t-1, t)/(t-2, t-1, t) with Stage-2 Pre-Install (t-1, t-1, t)/(t-2, t, t) shows that delaying Stage-1 reward construction until step $t$ shifts DEV BPB by only +0.0013 to +0.0024 BPB across all 4 entrywise cells (2.1341 -> 2.1360 at b05, 1e-8; 2.1195 -> 2.1217 at b05, 1e-4), whereas delaying Stage-2 reward construction until step $t$ (strict_post_install) shifts DEV BPB by +0.0219 ± 0.0020 BPB at (entrywise, b05, 1e-8) (2.1360 -> 2.1579). Because exp34 cached mod_bus_base in BF16 on the pre-install path and cached x_final_clean in BF16 on the post-install path (zero_dw_bf16_cache_rel_l2_diff_mean = 0.001687), and because exp36 Part A used a local pullback s_lm @ W_lm (Rev. Matmuls = 1), exp42 Part A (exp42_fp32_triton_results.json) executes a strictly zero-pullback (reverse_error_matmuls_per_step = 0) pure-FP32 trace-cache control via _triton_wm_reward_mg1_kernel (zero_dw_fp32_cache_rel_l2_diff_mean = 0.0). Across 3 seeds (46, 47, 48), the compliant zero-pullback pure-FP32 clock contrast is 2.1359 ± 0.0022 vs. 2.1579 ± 0.0004 DEV BPB (+0.0219 ± 0.0019 BPB) at b05, 1e-8 and 2.1274 ± 0.0002 DEV BPB (2.0901 ± 0.0001 VAL-B BPB) at b09, 1e-4, confirming that the clock shift is a genuine receiver-installation effect.
2. Row-Isotropic vs. Entrywise Preconditioning and Realized-Component Readers (Proposition 4, exp34–exp36): Under row_iso preconditioning, Pre-Install and Strict Post-Install agree to within $\le 0.0003$ BPB across all 4 cells (2.1577 vs. 2.1580; 2.1662 vs. 2.1659; 2.1296 vs. 2.1298; 2.1335 vs. 2.1334), consistent with Proposition 4's radial cancellation under scalar $c_n$. Under entrywise preconditioning, increasing $\epsilon: 10^{-8} \to 10^{-4}$ improves Strict Post-Install by -0.0279 ± 0.0001 BPB (2.1579 -> 2.1300), and combining $(\beta_1=0.9, \epsilon=10^{-4})$ reaches 2.1273 ± 0.0002 DEV BPB (2.1274 ± 0.0002 in pure FP32). Conversely, folding the internal Pre-Polar Muon operator (2.1613 vs. 2.1579 at b05, 1e-8; 2.1338 vs. 2.1273 at b09, 1e-4, exp36 Part B) or the lm_head AdamW preconditioner (2.2128 vs. 2.1579) directly into the WM reader degrades learning because forward perturbations propagate through the unpreconditioned forward weights $W_{\text{live}}$.

4.5 Fused Register-Tiled Triton WM Speedup and Node-Perturbation Comparator Timing (exp37, exp42)

Profiling compute_stream_wm_rewards_fast in exp34 revealed that materializing the 3D intermediate outer-product tensor $\Delta \tilde{W}^{(m+1, g)} \in \mathbb{R}^{G \times D_{\text{out}} \times D_{\text{in}}}$ in GPU HBM (3,498.9 MB/step across 13 matrices on d6) consumed 134.31 ms/step (86.3% of total step time). Table 4 compares our fused register-tiled Triton kernels (_triton_wm_reward_mg1_kernel for $M_g=1$ and _triton_wm_reward_mg_grouped_kernel for $M_g > 1$), batched Pre-Polar Muon (prepolar_muon_orthogonalize_batched: 10.35 -> 1.37 ms/step), Surrogate-Routed Local Node Perturbation (SR-LNP) (train_dust_d6_fast, using @torch.compile fused TensorCore chunk kernels fused_bus_ce_chunk, fused_wte_dust_chunk, fused_mlp_dust_chunk), and Suffix-Recomputed Node Perturbation (SR-NP) (train_faithful_dust_d6 in run_exp42_compliant_fp32_and_faithful_dust.py, which executes exact suffix recomputation from block $l$ onward across $K_{\text{tot}} = 76$ block-level draws plus $24$ local attention draws per step with $\gamma = 0.98$) on an uncontended RTX 4090 (benchmark_uncontended_timings.json, exp37_39_master_results.json, exp42_faithful_dust_results.json). Both 30-step steady-state benchmark estimates (est_360step_s) and actual 3-seed training GPU times (total_train_gpu_s_mean) are reported.

| Implementation / Kernel Suite (ClimbMix d6, $L=6, D=384, BT=2,048$) | Perturbed Passes $K$ | Stream Count $G$ ($M_g$) | 13-Matrix Credit Kernel (ms) | Temp HBM Alloc (MB) | Uncontended Step Time (ms) | Steady-State 360-Step Est. (s) | Measured 3-Seed Run GPU (s) | Numerical Match / Fidelity | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | PyTorch compute_stream_wm_rewards_fast (exp34, Paper #20 v1) | $1$ | $2,048$ ($1$) | $134.31$ | $3,498.9$ | $155.60$ | $56.02$ | $66.46$ | Reference (1.0 / 0.0) | | Fused Register-Tiled Triton WM (_triton_wm_reward_mg1_kernel) | $1$ | $2,048$ ($1$) | $3.50$ ($38.4\times$) | $0.0$ | $18.68 \pm 0.21$ ($8.33\times$) | $6.72$ | $6.74$ | $1.0000000 / 2.26\text{e-}8$ | | Fused Grouped Triton WM (_triton_wm_reward_mg_grouped_kernel) | $1$ | $64$ ($32$) | $1.03$ ($4.4\times$) | $0.0$ | $15.82 \pm 0.17$ | $5.69$ | $5.79$ | $0.9999973 / 2.31\text{e-}3$ | | Compiled SR-LNP (train_dust_d6_fast, $K=4$) | $4$ | Tokenwise | $7.31$ | $0.0$ | $15.23 \pm 0.25$ | $5.48$ | $6.32$ | Local Surrogate ($\gamma=0$) | | Compiled SR-LNP (train_dust_d6_fast, $K=16$) | $16$ | Tokenwise | $17.71$ | $0.0$ | $25.63 \pm 0.28$ | $9.23$ | $9.43$ | Local Surrogate ($\gamma=0$) | | Compiled SR-LNP (train_dust_d6_fast, $K=64$) | $64$ | Tokenwise | $56.59$ | $0.0$ | $64.51 \pm 0.25$ | $23.22$ | $23.66$ | Local Surrogate ($\gamma=0$) | | Suffix-Recomputed SR-NP (exp42, $K_{\text{site}}=4, K_{\text{tot}}=76$, Muon) | $76\text{ suffix}$ | Tokenwise | $167.2$ | $0.0$ | $181.08$ | $65.19$ | $65.19$ | Suffix + $\gamma=0.98$ | | Suffix-Recomputed SR-NP (exp42, $K_{\text{site}}=4, K_{\text{tot}}=76$, AdamW) | $76\text{ suffix}$ | Tokenwise | $220.5$ | $0.0$ | $234.39$ | $84.38$ | $84.38$ | Suffix + $\gamma=0.98$ | | Exact Backpropagation (backprop_d6_codesigned_opt, Matched Subset) | $0$ (1 bwd) | --- | --- | --- | $11.49 \pm 0.23$ | $4.14$ | $6.56$ | Exact Autograd Reference |

Table 4: Uncontended RTX 4090 kernel timings, 30-step steady-state 360-step estimates (benchmark_uncontended_timings.json), and actual 3-seed training GPU times (exp37_39_master_results.json, exp42_faithful_dust_results.json) across PyTorch WM, Fused Register-Tiled Triton WM, Compiled SR-LNP, Suffix-Recomputed SR-NP, and Exact Backpropagation on ClimbMix d6.

4.6 Joint Co-Design of the Optimizer and Delayed WM Credit (exp38, geo_and_attn_steering_audit.json)

Table 5 reports our controlled 3-seed optimizer co-design ablation on ClimbMix d6 ($360$ steps, Strict Post-Installation Construction (t-1, t [post-Stage3], t)/(t-2, t [post-Stage2], t), exp37_39_master_results.json) alongside the Seed-46 Block-0 geometric diagnostic (run_geo_and_attn_steering_audit.py, geo_and_attn_steering_audit.json):

1. Orthogonalized Update Directions and Receiver Stable Rank (51.4x Descriptive Association on Seed 46 Block 0): Measuring Block 0 at step 360 on Seed 46 (run_geo_and_attn_steering_audit.py), under AdamW on hidden matrices (adamw_hidden), Stage-2 receiver columns collapse onto a low-rank subspace (mlp.c_proj stable rank $\operatorname{srank}(W) = 2.75$, off-diagonal column coherence $0.3829$; attn.c_proj $\operatorname{srank}(W) = 1.11$, coherence $0.6024$). Under Pre-Polar Muon, orthogonalizing each step's momentum update direction is descriptively associated with a higher Block-0 weight stable rank on mlp.c_proj (127.24 under paper20_prepolar, coherence $0.0419$; and 141.33 under codesigned_prepolar, coherence $0.0411$, $51.4\times$ higher than adamw_hidden) and attn.c_proj (54.85 under paper20_prepolar and 52.72 under codesigned_prepolar, $47.5\times$ higher than adamw_hidden).
2. Scale-Matched Sparse Embedding Velocity (9.82x Layer-0 / 1.42x 6-Layer Mean Attention Shift) & Drift Annealing: Matching sparse embedding angular velocity (lr_wte = 0.12, lr_ve = 0.06, betas = (0.80, 0.95)) to $\sigma_{\text{init}}(\text{wte}) = 1.0$ before RMSNorm(e_tok) (-0.0194 BPB gain alone: 2.1273 -> 2.1079) and linearly annealing learning rates (lrm: 1.0 -> 0.5: -0.0296 BPB to 2.0783; lrm: 1.0 -> 0.2: -0.0101 BPB to 2.0682) improves 3-seed 360-step d6 validation BPB from 2.1273 ± 0.0001 to 2.0682 ± 0.0006 DEV BPB (2.0315 ± 0.0004 VAL-B BPB, 292.66 PPL)—within 0.0150 BPB of matched-subset Exact Backpropagation (2.0532 ± 0.0008). On the first held-out validation batch (split="val", B=8, T=256, Seed 46), the co-designed schedule increases causal attention-map relative Frobenius shift $\|\mathbf{A}^{(l)}(360) - \mathbf{A}^{(l)}(0)\|_F / \|\mathbf{A}^{(l)}(0)\|_F$ by $9.82\times$ in Layer 0 (0.0838 -> 0.8233) and $1.42\times$ averaged across all 6 layers (0.6046 -> 0.8587) (geo_and_attn_steering_audit.json: per-layer shifts [0.0838, 0.5327, 0.6784, 0.7494, 0.7733, 0.8100] -> [0.8233, 0.8472, 0.8682, 0.8639, 0.8690, 0.8807]) even with $W_Q, W_K$ frozen. Notably, matched-subset Exact Backpropagation also improves by -0.0548 BPB (2.1080 ± 0.0006 -> 2.0532 ± 0.0008) under the same optimizer package, whereas applying the same package to SR-LNP (K=16) (dust_K16_matched_codesigned_opt) yields 2.2602 ± 0.0010 vs. 2.2603 ± 0.0018 DEV BPB (-0.0001 BPB), establishing an empirical interaction between the optimizer package and the credit estimator in that SR-LNP configuration.
3. Three-Tier Attention Credit-Assignment Hierarchy ($W_V$ vs. $W_Q$ vs. $W_K$), Key Truncation vs. Future-Query Accumulation, and Spherical Pre-Polar Muon for Active-$W_Q$ Pretraining (exp45_d6_attn_routing_results.json): Extending our backward-free Stage-1 estimator to the attention score projections attn.c_q ($W_Q$) and attn.c_k ($W_K$) reveals a fundamental structural hierarchy in single-draw ($K=1$) credit assignment at $M_g = 1$, governed by the effective coupled subspace dimension $m_{\text{eff}}$. In our diagnostic suite (run_attention_subspace_cosine_diagnostic), downstream gradient $\nabla_{y_{\text{attn}}}\mathcal{L}$ is supplied as an oracle downstream vector (gy_attn from loss.backward() at initialized seed = 46, Layer 0) specifically to isolate single-block attention transport mechanics from multi-layer delayed credit variance:
   • Tier 1 (attn.c_v, $m_{\text{eff}} = 1$, Coordinate-Isolated & Forward-Factorizable via $\bar{x}_{\text{attn\_in}}^{(h)}$): By Proposition 2, perturbing $y_{\text{attn}}^{(h)}(b, \tau)$ coordinate-by-coordinate and contracting against $\bar{x}_{\text{attn\_in}}^{(h)}(b, \tau)$ isolates every head coordinate without cross-coordinate or cross-token coupling, satisfying an exact conditional single-block reconstruction identity cos = 1.0000 given oracle downstream credit at $K=1$ (distinct from its +0.9069 end-to-end delayed backward-free cosine in Table 1).
   • Tier 2 (attn.c_q, $m_{\text{eff}} = d_h = 64$, Token-Local in Time $\tau$ / Coupled Across $d_h$ Head Channels): Because query $q_\tau^{(h)}$ enters only row $\tau$ of the causal attention matrix $\mathbf{A}_{b, h, \tau, :}$, perturbing $q_\tau^{(h)} \mapsto q_\tau^{(h)} + \sigma \tilde{\xi}_{q, \tau}^{(h)}$ alters only output token $\tau$'s attention output $\Delta y_{\text{attn}}^{(h)}(b, \tau)$, so zero cross-time interference arises at $M_g = 1$. However, all $d_h = 64$ coordinates of $q_\tau^{(h)}$ mix into the scalar dot product $\langle q_\tau^{(h)}, k_s^{(h)}\rangle / \sqrt{d_h}$, sharing a single per-head reward $R_{q, \tau}^{(h)} = \sum_{c=1}^{d_h} R_{\text{attn\_proj}, \tau, (h, c)}$. Consequently, single-draw cosine alignment scales as $1 / \sqrt{d_h} = 0.1250$: empirically (attention_subspace_cosine_diagnostic), cos = 0.1357 at $K=1$, 0.2657 at $K=4$, 0.4748 at $K=16$, and 0.7385 at $K=64$.
   • Tier 3 (attn.c_k, Non-Local Across Future Queries $\tau \ge s$, Truncation Bias vs. Future-Query Credit): Key $k_s^{(h)}$ enters the attention distribution of every future query $\tau \in \{s, \dots, T-1\}$ via the query-dependent softmax Jacobian $\operatorname{diag}(A_\tau) - A_\tau A_\tau^T$, which does not factorize into a query-independent forward trace. Multiplying r_head at position $s$ by $\xi_{k,s}$ at the same position $s$ has first-order expectation $\mathbb{E}[\widehat{g}_{k_s}^{\text{same-pos}}] = J_{ss}^\top g_s$, capturing only the diagonal self-attention sensitivity ($\partial y_s / \partial k_s$) while omitting all future-query credit $\sum_{\tau > s} J_{\tau s}^\top g_\tau$ (cos = 0.0076 at $K=1$, 0.0534 at $K=64$). In our pure-Python 2-token exact Rademacher enumeration, same-position key correlation yields exact expectation $(0.0, 0.0)$ across all 16 perturbation patterns when loss is at token 2, whereas accumulating future query rewards $\widetilde{R}_{k,s}^{(\gamma_K)} = \sum_{\tau \ge s, \tau - s \le W_l} \gamma_K^{\tau - s} R_{\text{head},\tau}$ at $\gamma_K = 1$ recovers the exact analytical key derivative $[0.317182, 0.422909]$ to machine precision. On d6 Layer 0, future-query accumulation lifts $K=64$ cosine alignment 2.51x to 0.1340 ($\gamma_K=1.00$) and 2.93x to 0.1565 ($\gamma_K=0.98$); however, single-draw ($K=1$) estimation remains limited by cross-time key superposition variance (cos = 0.0066 at $\gamma_K=1.00$, 0.0089 at $\gamma_K=0.98$) because all causal keys $s' \le \tau$ are perturbed simultaneously inside each query row $\tau$.
   • Spherical Pre-Polar Muon for attn.c_q (2.0682 -> 2.0641 ± 0.0006 DEV BPB) & Factorized Component Ablation: Because q_r = norm(apply_rotary_emb(q, cos, sin)) is $0$-homogeneous in $\|W_Q^{(l)}\|_F$, standard unconstrained Pre-Polar Muon updates (wm_G2048_codesigned_active_q_snr_matched_triton, $\eta_Q = 0.001$: 2.0681 ± 0.0006) or properly bound per-head scalar temperatures (wm_G2048_codesigned_per_head_temp_triton, where block.attn.head_log_temp is bound to the module for evaluation parity: 2.0681 ± 0.0006) leave DEV BPB virtually unchanged (-0.0001 BPB). By (i) perturbing unit-RMS normalized queries along their tangent space, (ii) projecting $W_Q^{(l)}$ back onto its initial Frobenius sphere $\|W_Q^{(l)}\|_F \equiv \|W_{Q, 0}^{(l)}\|_F$, (iii) using momentum $\beta_Q = 0.95$ to integrate out $d_h = 64$ coordinate noise across steps, and (iv) scaling angular step size by $\sqrt{d_h} = 8$ ($\eta_Q = \sqrt{d_h}\,\eta_V = 0.048$), wm_G2048_codesigned_spherical_active_q_triton reaches 2.0641 / 2.0636 / 2.0647 $\to$ 2.0641 ± 0.0006 DEV BPB (2.0273 ± 0.0005 VAL-B BPB, 289.43 PPL)—improving upon the 4-matrix baseline by -0.0041 BPB (-3.23 PPL) and capturing 45.6% of the exact-backpropagation 4-to-5-matrix $W_Q$-unfreezing gain (2.0532 ± 0.0008 -> 2.0442 ± 0.0012, -0.0090 BPB; vs. 2.0316 ± 0.0009 for 6-matrix exact BP). To isolate the mechanisms within this multi-component intervention, our factorized Euclidean control (wm_G2048_codesigned_euclidean_active_q_matched_hp_triton), which maintains matched $\beta_Q = 0.95$ and $\eta_Q = 0.048$ while removing spherical retraction and tangent projection, reaches 2.0649 ± 0.0006 DEV BPB (2.0281 ± 0.0005 VAL-B BPB, 290.02 PPL), attributing -0.0033 BPB of the gain to step-size/momentum matching and -0.0008 BPB specifically to Riemannian spherical retraction + tangent perturbation. Furthermore, training $W_K$ with temporally complete future-accumulated credit at $\gamma_K = 0.98$ (wm_G2048_codesigned_spherical_active_k_future_g098_triton) improves over same-position truncated key training (2.0684 ± 0.0005) to 2.0672 ± 0.0006 DEV BPB (2.0305 ± 0.0006 VAL-B BPB, 291.90 PPL), confirming that removing temporal truncation bias improves credit quality even while cross-time key superposition variance remains.

| Method (ClimbMix d6, Strict Post-Install, 360 Steps, Seeds 46, 47, 48) | Active Params (Matrices) | Sparse Embed Opt (lr_wte / lr_ve) | Drift Decay lrm_min | Block-0 mlp.c_proj srank | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Run GPU / Est. (s) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | wm_G2048_paper20_opt_triton (Paper #20 v1 Opt, Fused Triton) | $24.58\text{M}$ ($4$) | 0.004 / 0.004 | $1.0$ | $127.24$ | $2.1274 / 2.1274 / 2.1272$ | $2.1273 \pm 0.0001$ | $2.0900 \pm 0.0001$ | $344.26$ | $6.95 / 6.72$ | | wm_G2048_scale_embed_const_lr (+ Scale-Matched Sparse Embeds) | $24.58\text{M}$ ($4$) | 0.120 / 0.060 | $1.0$ | --- | $2.1077 / 2.1077 / 2.1084$ | $2.1079 \pm 0.0004$ | $2.0716 \pm 0.0003$ | $326.41$ | $6.95 / 6.72$ | | wm_G2048_scale_embed_lrm05 (+ Moderate Drift Decay lrm -> 0.5) | $24.58\text{M}$ ($4$) | 0.120 / 0.060 | $0.5$ | --- | $2.0780 / 2.0780 / 2.0788$ | $2.0783 \pm 0.0005$ | $2.0419 \pm 0.0003$ | $300.89$ | $6.76 / 6.72$ | | wm_G2048_codesigned_opt_triton (4-Matrix Co-Designed Delayed WM) | $24.58\text{M}$ ($4$) | 0.120 / 0.060 | $0.2$ | $141.33$ | $2.0681 / 2.0676 / 2.0688$ | $2.0682 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.66$ | $6.74 / 6.72$ | | wm_G2048_codesigned_per_head_temp_triton (exp45, Attached Per-Head Temp $\tau_{l,h}$) | $24.58\text{M}$ ($4+\tau$) | 0.120 / 0.060 | $0.2$ | --- | $2.0681 / 2.0676 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.64$ | $7.22 / \text{---}$ | | wm_G2048_codesigned_active_q_snr_matched_triton (exp45, Unconstrained $\eta_Q=10^{-3}$) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0680 / 2.0675 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0314 \pm 0.0004$ | $292.56$ | $12.20 / \text{---}$ | | wm_G2048_codesigned_active_qk_snr_matched_triton (exp45, Unconstrained $\eta_{QK}=10^{-3}$) | $26.35\text{M}$ ($6$) | 0.120 / 0.060 | $0.2$ | --- | $2.0680 / 2.0675 / 2.0687$ | $2.0681 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.60$ | $12.48 / \text{---}$ | | wm_G2048_codesigned_euclidean_active_q_matched_hp_triton (exp45, Factorized Control: No Retraction) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0648 / 2.0643 / 2.0655$ | $2.0649 \pm 0.0006$ | $2.0281 \pm 0.0005$ | $290.02$ | $7.41 / \text{---}$ | | wm_G2048_codesigned_spherical_active_q_triton (exp45, Spherical Active-$W_Q$, $\eta_Q=0.048, \beta_Q=0.95$) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0641 / 2.0636 / 2.0647$ | $2.0641 \pm 0.0006$ | $2.0273 \pm 0.0005$ | $289.43$ | $11.49 / \text{---}$ | | wm_G2048_codesigned_spherical_active_k_triton (exp45, Same-Pos Truncated Active-$W_K$) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0683 / 2.0680 / 2.0689$ | $2.0684 \pm 0.0005$ | $2.0317 \pm 0.0004$ | $292.85$ | $11.67 / \text{---}$ | | wm_G2048_codesigned_spherical_active_k_future_g098_triton (exp45, Future-Accum Active-$W_K, \gamma_K=0.98$) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0671 / 2.0667 / 2.0678$ | $2.0672 \pm 0.0006$ | $2.0305 \pm 0.0006$ | $291.90$ | $7.47 / \text{---}$ | | wm_G2048_codesigned_spherical_active_qk_triton (exp45, Spherical Active-$W_Q+W_K$) | $26.35\text{M}$ ($6$) | 0.120 / 0.060 | $0.2$ | --- | $2.0670 / 2.0664 / 2.0674$ | $2.0669 \pm 0.0005$ | $2.0303 \pm 0.0004$ | $291.66$ | $12.82 / \text{---}$ | | dust_K16_tuned_fast (SR-LNP $K=16$) | $24.58\text{M}$ ($4$) | 0.008 / 0.008 | $0.1$ | --- | $2.2583 / 2.2615 / 2.2612$ | $2.2603 \pm 0.0018$ | $2.2236 \pm 0.0014$ | $496.07$ | $9.43 / 9.23$ | | dust_K16_matched_codesigned_opt (SR-LNP $K=16$ + Co-Designed Opt) | $24.58\text{M}$ ($4$) | 0.120 / 0.060 | $0.2$ | --- | $2.2591 / 2.2603 / 2.2611$ | $2.2602 \pm 0.0010$ | $2.2234 \pm 0.0003$ | $495.83$ | $9.71 / 9.23$ | | backprop_d6_paper20_opt (Exact BP, Paper #20 v1 Opt, 4-Matrix Subset) | $24.58\text{M}$ ($4$) | 0.004 / 0.004 | $1.0$ | --- | $2.1084 / 2.1073 / 2.1084$ | $2.1080 \pm 0.0006$ | $2.0708 \pm 0.0006$ | $326.54$ | $6.38 / 4.14$ | | backprop_d6_codesigned_opt (Exact BP, Co-Designed Opt, 4-Matrix Subset) | $24.58\text{M}$ ($4$) | 0.120 / 0.060 | $0.2$ | --- | $2.0523 / 2.0536 / 2.0536$ | $2.0532 \pm 0.0008$ | $2.0160 \pm 0.0007$ | $280.86$ | $6.56 / 4.14$ | | backprop_d6_5mat_active_q_codesigned (exp45, Exact BP, 5-Matrix Active-$W_Q$) | $25.46\text{M}$ ($5$) | 0.120 / 0.060 | $0.2$ | --- | $2.0428 / 2.0450 / 2.0448$ | $2.0442 \pm 0.0012$ | $2.0071 \pm 0.0012$ | $274.01$ | $8.90 / \text{---}$ | | backprop_d6_6mat_active_qk_codesigned (exp45, Exact BP, Full 6-Matrix Active-$W_Q+W_K$) | $26.35\text{M}$ ($6$) | 0.120 / 0.060 | $0.2$ | --- | $2.0306 / 2.0319 / 2.0322$ | $2.0316 \pm 0.0009$ | $1.9944 \pm 0.0006$ | $264.68$ | $8.56 / \text{---}$ |

Table 5: 3-seed optimizer co-design, factorized component ablations, and Active-$W_Q/W_K$ attention-routing ablation on ClimbMix d6 (exp37_39_master_results.json, exp45_d6_attn_routing_results.json) alongside Seed-46 Block-0 geometric / held-out attention-shift diagnostics (run_geo_and_attn_steering_audit.py, geo_and_attn_steering_audit.json), reporting both measured 3-seed training GPU time (Run GPU) and 30-step steady-state benchmark estimates (Est.).

4.7 Comparison with Surrogate-Routed (SR-LNP) and Suffix-Recomputed (SR-NP) Node Perturbation Across Tokens, Compute, and Width $D \in \{256, 384, 512, 768\}$ (exp37, exp39, exp42)

Table 6 reports our 3-seed comparison between Fused Triton Delayed WM, Surrogate-Routed Local Node Perturbation (SR-LNP, $K \in \{4, 16, 64\}$), and Suffix-Recomputed Node Perturbation (SR-NP, $K_{\text{site}}=4, K_{\text{tot}}=76$ suffix-recomputed block draws $+ 24$ local attention draws/step, $\gamma = 0.98$, exp42) across equal token budgets ($360$ steps $= 0.737\text{M}$ tokens), equal wall-clock compute ($500$ steps $= 9.41\text{s}$ run GPU matching SR-LNP K=16's $9.43\text{s}$, and $720$ steps $= 13.57\text{s}$ run GPU vs. SR-LNP K=64's $23.66\text{s}$), and model widths $D \in \{256, 384, 512, 768\}$ (exp37_39_master_results.json, exp42_faithful_dust_results.json). We make no claim of superiority over published Dust [9]:

1. Equal-Token Comparison on d6 ($360$ Steps, $D=384$): wm_G2048_codesigned_opt_triton (2.0682 ± 0.0006 DEV BPB in 6.74s run GPU / 6.72s est.) outperforms SR-LNP (K=4) (2.3071 ± 0.0016 in 6.32s / 5.48s est.) by -0.2389 BPB (-48.1% PPL), SR-LNP (K=16) (2.2603 ± 0.0018 in 9.43s / 9.23s est.) by -0.1921 BPB (-41.0% PPL) in 1.40x less run GPU time, SR-LNP (K=64) (2.2041 ± 0.0013 in 23.66s / 23.22s est.) by -0.1359 BPB (-31.2% PPL) in 3.51x less run GPU time, and Suffix-Recomputed SR-NP (K_tot=76) under Adam (2.4264 ± 0.0088 in 84.38s) and Pre-Polar Muon (2.2415 ± 0.0015 in 65.19s) by -0.3582 BPB and -0.1733 BPB in 9.67x less GPU time. Even at $G=64$ (5.79s run GPU / 5.70s est.), wm_G64_codesigned_opt_triton (2.2415 ± 0.0011) matches SR-NP (K_tot=76, Muon) in $11.3\times$ less GPU time.
2. Equal-Compute Comparison (9.41s and 13.57s Run GPU on d6, $D=384$): At equal wall-clock time to SR-LNP (K=16) (9.41s vs. 9.43s run GPU; 9.34s vs. 9.23s est., 500 steps), wm_G2048_codesigned_500steps_match_dustK16_time reaches 2.0322 ± 0.0007 DEV BPB (1.9975 ± 0.0006 VAL-B BPB, 265.13 PPL)—outperforming SR-LNP (K=16) by -0.2281 BPB (-46.6% PPL). At 720 steps (13.57s run GPU / 13.45s est., 1.74x faster than SR-LNP K=64 and 4.80x faster than SR-NP K_tot=76), wm_G2048_codesigned_720steps_half_dustK64_time breaks the 2.00 DEV BPB barrier backward-free at 1.9989 ± 0.0008 DEV BPB (1.9646 ± 0.0008 VAL-B BPB, 241.96 PPL)—outperforming SR-LNP (K=64) by -0.2052 BPB (-43.1% PPL).
3. Model Width Scaling ($D \in \{256, 384, 512, 768\}$): As width $D$ grows from 256 -> 384 -> 512 -> 768 (14.55M -> 67.11M active parameters), Delayed WM's gap to matched-subset Exact Backpropagation shrinks from +0.0144 BPB to +0.0125 BPB (2.0933 vs. 2.0789 at $D=256$; 2.0558 ± 0.0002 vs. 2.0433 ± 0.0006 at $D=768$), while Delayed WM at the smallest width $D=256$ (2.0933 ± 0.0004 in 5.03s run GPU / 4.83s est.) outperforms SR-LNP (K=64) at the largest width $D=768$ (2.1729 ± 0.0004 in 95.74s run GPU / 57.26s est.) by -0.0796 BPB.

| Width $D$ (Active Params) | Method (exp37, exp39, exp42, Seeds 46, 47, 48) | Steps (Tokens) | Perturb Passes $K$ ($G$) | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Run GPU / Est. (s) | Gap vs. Exact BP / Comparator | | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | $384$ ($24.58\text{M}$) | dust_K4_tuned_fast (SR-LNP $K=4$) | $360$ ($0.74\text{M}$) | $4$ (Token) | $2.3054 / 2.3074 / 2.3085$ | $2.3071 \pm 0.0016$ | $2.2705 \pm 0.0018$ | $564.03$ | $6.32 / 5.48$ | $+0.2539$ vs. BP | | $384$ ($24.58\text{M}$) | dust_K16_tuned_fast (SR-LNP $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2583 / 2.2615 / 2.2612$ | $2.2603 \pm 0.0018$ | $2.2236 \pm 0.0014$ | $496.07$ | $9.43 / 9.23$ | $+0.2071$ vs. BP | | $384$ ($24.58\text{M}$) | dust_K64_tuned_fast (SR-LNP $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.2026 / 2.2050 / 2.2047$ | $2.2041 \pm 0.0013$ | $2.1669 \pm 0.0006$ | $425.09$ | $23.66 / 23.22$ | $+0.1509$ vs. BP | | $384$ ($24.58\text{M}$) | faithful_dust_K76_adam (SR-NP $K_{\text{tot}}=76$, Adam) | $360$ ($0.74\text{M}$) | $76$ (Suffix) | $2.4167 / 2.4340 / 2.4284$ | $2.4264 \pm 0.0088$ | $2.3865 \pm 0.0092$ | $782.77$ | $84.38 / 84.38$ | $+0.3732$ vs. BP | | $384$ ($24.58\text{M}$) | faithful_dust_K76_prepolar_muon (SR-NP $K_{\text{tot}}=76$, Muon) | $360$ ($0.74\text{M}$) | $76$ (Suffix) | $2.2398 / 2.2424 / 2.2424$ | $2.2415 \pm 0.0015$ | $2.2048 \pm 0.0012$ | $471.11$ | $65.19 / 65.19$ | $+0.1883$ vs. BP | | $384$ ($24.58\text{M}$) | wm_G64_codesigned_opt_triton | $360$ ($0.74\text{M}$) | $1$ ($64$) | $2.2406 / 2.2413 / 2.2427$ | $2.2415 \pm 0.0011$ | $2.2043 \pm 0.0013$ | $471.15$ | $5.79 / 5.69$ | $-0.0188$ vs. SR-LNP $K=16$ | | $384$ ($24.58\text{M}$) | wm_G2048_codesigned_opt_triton (Equal Tokens) | $360$ ($0.74\text{M}$) | $1$ ($2,048$) | $2.0681 / 2.0676 / 2.0688$ | $2.0682 \pm 0.0006$ | $2.0315 \pm 0.0004$ | $292.66$ | $6.74 / 6.72$ | $-0.1733$ vs. SR-NP ($9.7\times$ faster) | | $384$ ($24.58\text{M}$) | wm_G2048_codesigned_500steps_match_dustK16_time | $500$ ($1.02\text{M}$) | $1$ ($2,048$) | $2.0321 / 2.0315 / 2.0329$ | $2.0322 \pm 0.0007$ | $1.9975 \pm 0.0006$ | $265.13$ | $9.41 / 9.34$ | $-0.2281$ at $=$Time (SR-LNP $K=16$) | | $384$ ($24.58\text{M}$) | wm_G2048_codesigned_720steps_half_dustK64_time | $720$ ($1.47\text{M}$) | $1$ ($2,048$) | $1.9990 / 1.9981 / 1.9996$ | $1.9989 \pm 0.0008$ | $1.9646 \pm 0.0008$ | $241.96$ | $13.57 / 13.45$ | $-0.2052$ vs. SR-LNP $K=64$ | | $256$ ($14.55\text{M}$) | width256_dust_K16_tuned (SR-LNP $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2978 / 2.2984 / 2.2982$ | $2.2981 \pm 0.0003$ | $2.2614 \pm 0.0003$ | $550.35$ | $7.19 / 6.53$ | $+0.2192$ vs. BP ($D=256$) | | $256$ ($14.55\text{M}$) | width256_dust_K64_tuned (SR-LNP $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.2369 / 2.2374 / 2.2380$ | $2.2374 \pm 0.0006$ | $2.1999 \pm 0.0004$ | $465.84$ | $28.60 / 14.88$ | $+0.1585$ vs. BP ($D=256$) | | $256$ ($14.55\text{M}$) | width256_wm_G2048_codesigned | $360$ ($0.74\text{M}$) | $1$ ($2,048$) | $2.0935 / 2.0929 / 2.0936$ | $2.0933 \pm 0.0004$ | $2.0566 \pm 0.0003$ | $313.60$ | $5.03 / 4.83$ | $+0.0144$ vs. BP / $-0.1441$ vs. $K=64$ | | $256$ ($14.55\text{M}$) | width256_backprop_codesigned | $360$ ($0.74\text{M}$) | $0$ (1 bwd) | $2.0789 / 2.0778 / 2.0800$ | $2.0789 \pm 0.0011$ | $2.0415 \pm 0.0009$ | $301.41$ | $7.62 / 3.12$ | Exact BP ($D=256$) | | $512$ ($36.70\text{M}$) | width512_dust_K16_tuned (SR-LNP $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2359 / 2.2359 / 2.2369$ | $2.2362 \pm 0.0006$ | $2.2006 \pm 0.0002$ | $464.32$ | $13.95 / 12.62$ | $+0.1920$ vs. BP ($D=512$) | | $512$ ($36.70\text{M}$) | width512_dust_K64_tuned (SR-LNP $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.1878 / 2.1892 / 2.1883$ | $2.1884 \pm 0.0007$ | $2.1521 \pm 0.0009$ | $407.16$ | $33.88 / 32.90$ | $+0.1442$ vs. BP ($D=512$) | | $512$ ($36.70\text{M}$) | width512_wm_G2048_codesigned | $360$ ($0.74\text{M}$) | $1$ ($2,048$) | $2.0582 / 2.0590 / 2.0583$ | $2.0585 \pm 0.0004$ | $2.0219 \pm 0.0003$ | $285.02$ | $9.59 / 9.28$ | $+0.0143$ vs. BP / $-0.1299$ vs. $K=64$ | | $512$ ($36.70\text{M}$) | width512_backprop_codesigned | $360$ ($0.74\text{M}$) | $0$ (1 bwd) | $2.0440 / 2.0448 / 2.0439$ | $2.0442 \pm 0.0005$ | $2.0072 \pm 0.0003$ | $274.06$ | $7.38 / 5.40$ | Exact BP ($D=512$) | | $768$ ($67.11\text{M}$) | width768_dust_K16_tuned (SR-LNP $K=16$) | $360$ ($0.74\text{M}$) | $16$ (Token) | $2.2113 / 2.2114 / 2.2112$ | $2.2113 \pm 0.0001$ | $2.1754 \pm 0.0003$ | $433.55$ | $23.50 / 22.11$ | $+0.1680$ vs. BP ($D=768$) | | $768$ ($67.11\text{M}$) | width768_dust_K64_tuned (SR-LNP $K=64$) | $360$ ($0.74\text{M}$) | $64$ (Token) | $2.1733 / 2.1730 / 2.1725$ | $2.1729 \pm 0.0004$ | $2.1362 \pm 0.0002$ | $390.22$ | $95.74 / 57.26$ | $+0.1296$ vs. BP ($D=768$) | | $768$ ($67.11\text{M}$) | width768_wm_G2048_codesigned | $360$ ($0.74\text{M}$) | $1$ ($2,048$) | $2.0555 / 2.0559 / 2.0559$ | $2.0558 \pm 0.0002$ | $2.0189 \pm 0.0003$ | $282.87$ | $17.85 / 17.71$ | $+0.0125$ vs. BP / $-0.1171$ vs. $K=64$ | | $768$ ($67.11\text{M}$) | width768_backprop_codesigned | $360$ ($0.74\text{M}$) | $0$ (1 bwd) | $2.0426 / 2.0438 / 2.0434$ | $2.0433 \pm 0.0006$ | $2.0062 \pm 0.0007$ | $273.31$ | $21.47 / 9.54$ | Exact BP ($D=768$) |

Table 6: 3-seed comparison between Fused Triton Delayed WM, Surrogate-Routed Local Node Perturbation (SR-LNP, $K \in \{4, 16, 64\}$), Suffix-Recomputed Node Perturbation (SR-NP, $K_{\text{tot}}=76$), and Exact Backpropagation across equal tokens, equal wall-clock compute, and model width $D \in \{256, 384, 512, 768\}$ (exp37_39_master_results.json, exp42_faithful_dust_results.json, benchmark_uncontended_timings.json). Both measured 3-seed training GPU time (Run GPU) and 30-step steady-state benchmark estimates (Est.) are reported.

4.8 Long-Context Native 50,332,176 Scale Transfer, Isolated Kernel Substitution vs. Combined Package Speedup, and $2 \times 2$ {M_g, lr_embed} + exp44 Controls (exp29, exp30, exp32, exp40, exp41, exp43, exp44)

Table 7 reports the 3-seed scale-transfer results on the 8-layer, 512-width Native 50,332,176 benchmark ($B_{\text{eff}} = 64, T = 2,048$, $100$ steps $= 13.1\text{M}$ tokens, Seeds 46, 47, 48, dev_tokens = 523,826, dev_bytes = 2,126,272).

| Method (Native 50,332,176, $L=8, D=512, V=8,192, T=2,048$, 100 Steps, Seeds 46, 47, 48) | Active Params | MicroB $\times$ Accum ($M_g$) | Stage-3 / Stage-2 / Stage-1 Clocks | Rev. Matmuls / Bwd | Seed 46 / 47 / 48 DEV BPB | DEV BPB (Mean $\pm$ SD) | VAL-B BPB (Mean $\pm$ SD) | DEV PPL | Fwd / Rule / Wall (ms) | Peak VRAM (GB) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | supplied_muon_adamw_backprop (Full 6-Matrix BP) | $50.33\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/(t,t,t)$ | $-1 / 1$ | $1.6994 / 1.6989 / 1.6984$ | $1.6989 \pm 0.0005$ | $1.6828 \pm 0.0005$ | $119.04$ | $461.2 / 1142.5 / 1884.0$ | $13.84$ | | unrestricted_adamw_backprop (Full 6-Matrix AdamW BP) | $50.33\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/(t,t,t)$ | $-1 / 1$ | $1.9894 / 1.9647 / 1.9660$ | $1.9734 \pm 0.0139$ | $1.9580 \pm 0.0139$ | $258.55$ | $462.6 / 1106.7 / 1849.0$ | $14.12$ | | frozen_backbone_lm_head_only (exp18, 2 seeds) | $4.19\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/\text{frozen}/\text{frozen}$ | $0 / 0$ | $2.1617 / 2.1602 / \text{---}$ | $2.1610 \pm 0.0011$ | $2.1094 \pm 0.0001$ | $416.69$ | $510.4 / 213.4 / 812.5$ | $5.38$ | | native50m_dust_K16_tuned_fast (exp41, SR-LNP $K=16$) | $46.14\text{M}$ | $4 \times 16$ (Token) | Same-Step Local Surrogate ($\gamma=0$) | $0 / 0$ | $2.1322 / 2.1327 / 2.1330$ | $2.1326 \pm 0.0004$ | $2.1095 \pm 0.0004$ | $403.57$ | $1027.5 / 2340.1 / 3367.6$ | $9.18$ | | pure_lexical_x0_skip_only_true_zero (exp21) | $8.39\text{M}$ | $8 \times 8$ (---) | $(t,t,t)/(t,t,t)/\text{frozen}$ | $0 / 0$ | $2.0096 / 2.0086 / 2.0094$ | $2.0092 \pm 0.0005$ | $1.9586 \pm 0.0005$ | $272.81$ | $73.6 / 344.4 / 537.4$ | $7.26$ | | native50m_true_delayed_construction_M8_adamw (exp30) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $2.0742 / 2.0887 / 2.0870$ | $2.0833 \pm 0.0079$ | $2.0627 \pm 0.0059$ | $351.29$ | $1526.4 / 6910.1 / 8650.2$ | $10.34$ | | native50m_wm_M8_bt_scaled_triton (exp41, Skip wte, lrm->0.2, lr_wte=0.015) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $2.0251 / 2.0249 / 2.0255$ | $2.0252 \pm 0.0003$ | $2.0054 \pm 0.0003$ | $298.25$ | $1350.0 / 1591.2 / 2941.2$ | $9.12$ | | native50m_wm_M8_codesigned_triton (exp40, Skip wte, lrm->0.2, lr_wte=0.10) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9848 / 1.9842 / 1.9846$ | $1.9845 \pm 0.0003$ | $1.9666 \pm 0.0002$ | $266.03$ | $1352.8 / 1591.2 / 2944.0$ | $9.12$ | | native50m_causal_trace_wm_M32_filtered_v_tangent_prepolar_muon (exp29) | $46.14\text{M}$ | $4 \times 16$ ($32$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9672 / 1.9670 / 1.9673$ | $1.9672 \pm 0.0002$ | $1.9502 \pm 0.0003$ | $253.34$ | $1420.8 / 1205.1 / 2825.0$ | $9.49$ | | native50m_causal_trace_wm_M8_unprojected_prepolar_muon (exp29) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9519 / 1.9518 / 1.9514$ | $1.9517 \pm 0.0003$ | $1.9359 \pm 0.0006$ | $242.55$ | $909.0 / 3173.8 / 4283.0$ | $8.92$ | | native50m_true_delayed_construction_M8_unprojected_prepolar_muon (exp30) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $1.9518 / 1.9522 / 1.9518$ | $1.9520 \pm 0.0002$ | $1.9361 \pm 0.0007$ | $242.73$ | $909.9 / 4776.3 / 5888.0$ | $8.84$ | | native50m_causal_trace_wm_M8_filtered_v_tangent_prepolar_muon (exp29) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-2,t)$ | $0 / 0$ | $1.9342 / 1.9334 / 1.9341$ | $1.9339 \pm 0.0004$ | $1.9187 \pm 0.0001$ | $230.71$ | $1421.6 / 3240.8 / 4864.0$ | $9.50$ | | native50m_true_delayed_construction_M8_filtered_v_tangent_prepolar_muon (exp30) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t-1,t)/(t-2,t-1,t)$ | $0 / 0$ | $1.9339 / 1.9332 / 1.9339$ | $1.9337 \pm 0.0004$ | $1.9186 \pm 0.0001$ | $230.58$ | $1420.7 / 5195.8 / 6819.0$ | $9.41$ | | native50m_strict_post_install_M8_unprojected_prepolar_muon (exp32) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9823 / 1.9837 / 1.9839$ | $1.9833 \pm 0.0009$ | $1.9653 \pm 0.0013$ | $265.12$ | $916.8 / 6006.2 / 7128.6$ | $8.54$ | | native50m_strict_post_install_M8_filtered_v_tangent_prepolar_muon (exp32, CPU-Offload PyTorch lr_embed=3e-3) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9362 / 1.9357 / 1.9364$ | $1.9361 \pm 0.0004$ | $1.9207 \pm 0.0003$ | $232.14$ | $1425.0 / 6432.0 / 8070.2$ | $9.11$ | | GPU-Resident + Fused-Triton $2 \times 2$ {M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015} Factorial (exp43) & Pinned-Ring Controls (exp44): | | | | | | | | | | | | native50m_strict_post_install_M8_exp32_exact_triton (exp43, $M_g=8, \text{lr\_embed}=3\text{e-}3, \text{scale}=1$) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9359 / 1.9356 / 1.9360$ | $1.9358 \pm 0.0002$ | $1.9205 \pm 0.0002$ | $231.97$ | $1332.7 / 1659.2 / 2991.8$ | $20.57$ | | native50m_strict_post_install_M8_scaleMg8_triton (exp44, $M_g=8, \text{lr\_embed}=3\text{e-}3, \text{scale}=8.0$) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9350 / 1.9347 / 1.9353$ | $1.9350 \pm 0.0003$ | $1.9197 \pm 0.0003$ | $231.42$ | $1961.4 / 2911.1 / 4872.5$ | $9.61$ | | native50m_strict_post_install_M1_wte15e3_eps1e4_triton (exp44, $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-4}$) | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9269 / 1.9261 / 1.9267$ | $1.9266 \pm 0.0004$ | $1.9117 \pm 0.0001$ | $226.01$ | $1902.8 / 3468.9 / 5371.7$ | $9.69$ | | native50m_strict_post_install_M8_wte15e3_triton (exp43, $M_g=8, \text{lr\_embed}=0.015, \epsilon=10^{-8}$) | $46.14\text{M}$ | $4 \times 16$ ($8$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9160 / 1.9158 / 1.9163$ | $1.9160 \pm 0.0003$ | $1.9016 \pm 0.0003$ | $219.39$ | $1332.3 / 1656.3 / 2988.6$ | $20.57$ | | native50m_strict_post_install_M1_exp32_exact_triton (exp43, $M_g=1, \text{lr\_embed}=3\text{e-}3, \epsilon=10^{-8}$) | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.9113 / 1.9096 / 1.9098$ | $1.9102 \pm 0.0009$ | $1.8966 \pm 0.0008$ | $215.84$ | $1333.7 / 1938.0 / 3271.7$ | $20.65$ | | native50m_strict_post_install_M1_wte15e3_triton (exp43, $M_g=1, \text{lr\_embed}=0.015, \epsilon=10^{-8}$) | $46.14\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8978 / 1.8971 / 1.8984$ | $1.8978 \pm 0.0007$ | $1.8844 \pm 0.0003$ | $208.39$ | $1332.8 / 1930.9 / 3263.7$ | $20.65$ | | native50m_strict_post_install_M1_wte15e3_spherical_active_q_triton (exp46, Spherical Active-$W_Q$) | $48.23\text{M}$ | $4 \times 16$ ($1$) | $(t,t,t)/(t-1,t,t)/(t-2,t,t)$ | $0 / 0$ | $1.8889 / 1.8897 / 1.8905$ | $1.8897 \pm 0.0008$ | $1.8765 \pm 0.0013$ | $203.72$ | $2287.5 / 3789.3 / 6076.7$ | $9.80$ |

Table 7: Controlled 3-seed Native 50,332,176 long-context ($T=2,048$, 100 steps $= 13.1\text{M}$ tokens) results across Delayed Installation (exp29), True Both-Stages Pre-Install (exp30), Strict Post-Installation Construction (exp32), Fused Triton Strict Post-Install (exp40, exp41), Native 50.3M SR-LNP K=16 (exp41), our $2 \times 2$ {M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015} GPU-Resident Fused-Triton factorial (exp43_native50m_master_results.json), our unconfounded scale_factor = 8.0 and $\epsilon = 10^{-4}$ Pinned-Ring-Buffer controls (exp44_native50m_controls_results.json, 9.61–9.69 GiB peak VRAM), and our 3-seed Spherical Active-$W_Q$ scale-transfer breakthrough (exp46_native50m_active_q_results.json, 9.80 GiB peak VRAM across 48.23M active parameters). Note: in exp29–exp32, the third timing column reports t_wall_step_ms_including_eval_mean, whereas in exp40–exp46 it reports Fwd + Rule (t_total_step_gpu_ms); matched Fwd + Rule comparisons use 1,425.01 + 6,432.04 = 7,857.05 ms for exp32.

Genuinely Isolated Kernel Substitution (benchmark_native50m_isolated_kernel.json) vs. exp43 Implementation-Package Speedup, $2 \times 2$ {M_g, lr_embed} Factorial, exp44 Controls, and exp46 Native Spherical Active-$W_Q$ Scale-Transfer:

1. Separating the exp43 Combined Implementation Package from Genuinely Isolated Kernel Substitutions: In exp40 (1.9845 ± 0.0003 DEV BPB at lr_wte = 0.10) and exp41 (2.0252 ± 0.0003 DEV BPB at lr_wte = 0.015), we had changed the wte credit path and optimizer schedule (replacing exp32's Stage-3 lm_head lexical contraction on dfwd_x_final_wte with the diagonal skip surrogate and applying lrm -> 0.2, eps = 1e-4 to lm_head). In exp43 (native50m_strict_post_install_M8_exp32_exact_triton), we preserve exp32's exact Stage-3 lexical readout and optimizer schedule (eps = 1e-8, constant AdamW LR, lrm: 1.0 -> 0.5 on Muon), scoring 1.9359 / 1.9356 / 1.9360 $\to$ 1.9358 ± 0.0002 DEV BPB (1.9205 ± 0.0002 VAL-B BPB, 231.97 PPL) and matching exp32 (1.9361 ± 0.0004) to within 0.0003 BPB. However, run_exp43_native50m_mg1.py differs from exp32_strict_post_install_construction.py in several systems and precision components beyond the WM contraction kernel: exp32 passes FP32 q_r, k_r, v to SDPA (fa3.flash_attn_func), computes FP32 attention-filtered traces during the forward pass, offloads delayed traces to unpinned CPU memory, orthogonalizes hidden-matrix updates individually, and uses a dense one-hot target at lm_head (9.11 GiB peak VRAM, 1,425.01 + 6,432.04 = 7,857.05 ms/step on matched fwd + rule timed regions, and 8,070.22 ms t_wall_step_ms_including_eval_mean), whereas exp43 casts q_r, k_r, v to BF16 before SDPA (run_exp43_native50m_mg1.py:243-248), keeps delayed traces on GPU with pinned-memory ring buffers only for the 4D MLP activations (pin_h_mlp, pin_dfwd_h), reconstructs Stage-1 attention-filtered inputs at consumption time from BF16 cached q_rs, k_rs, norm_attn_ins (run_exp43_native50m_mg1.py:159-161), batches Pre-Polar Muon orthogonalization across layers, uses an in-place scatter at lm_head, and replaces compute_stream_wm_rewards_fast with fused Triton kernels (20.57 GiB peak VRAM, 1,332.65 + 1,659.19 = 2,991.84 ms/step on matched fwd + rule timed regions). Thus, comparing exp32 to exp43 measures a combined implementation package that achieves a 2.63x speedup on matched fwd + rule timed regions (7,857.05 -> 2,991.84 ms/step, and 3.88x on the learning-rule region 6,432.04 -> 1,659.19 ms) in exchange for +11.46 GiB higher peak GPU memory (9.11 -> 20.57 GiB)—or 9.61–9.69 GiB peak VRAM (3,599.5 and 3,678.7 ms/step uncontended fwd + rule) when all 8 delayed trace tensors (pin_h_mlp, pin_y_attn, pin_s_lm, pin_dfwd_h, pin_norm_mlp, pin_norm_attn, pin_q_r, pin_k_r) are placed in pinned host ring buffers (run_exp44_native50m_controls.py). To establish the genuinely isolated effect of the fused Triton contraction kernels with all storage, precision, trace construction, optimizer execution, head calculation, and timing boundaries held 100% identical, we execute three isolated benchmarks in benchmark_native50m_isolated_kernel.py (benchmark_native50m_isolated_kernel.json, where the 3 steady-state samples measure within-run steady-state variation):
   • Standalone 288-Call/Step Kernel Microbenchmark (Part 1, 2 warmup + 10 timed iterations, or 4 timed iterations for $M_g=1$ PyTorch): exp32's PyTorch baseline (compute_stream_wm_rewards_fast) processes stream chunks (chunk_G = 256 by default, allocating 4,618.5–4,632.5 MB of temporary 3D chunk workspace; or chunk_G = 32, allocating 618.1–632.1 MB). Across the 18 WM contractions per microbatch (288 calls/step), replacing compute_stream_wm_rewards_fast with compute_stream_wm_rewards_triton_fast (0.0 MB 3D workspace; only the 8.0 MB or 64.0 MB output buffer) speeds up the 288-call suite 8.35x vs. chunk_G=256 (2,749.29 ± 5.56 -> 329.41 ± 0.90 ms/step) and 7.15x vs. chunk_G=32 (2,353.90 ± 1.65 -> 329.41 ± 0.90 ms/step) at $M_g=8$ (lm_head_cos = 0.99999733, rel_l2 = 2.34e-3), and 36.74x vs. chunk_G=256 (21,437.38 ± 14.37 -> 583.45 ± 0.89 ms/step) and 31.31x vs. chunk_G=32 (18,270.57 ± 12.66 -> 583.45 ± 0.89 ms/step) at $M_g=1$ (lm_head_cos = 0.99999994, rel_l2 = 1.52e-7).
   • Isolated Substitution Inside exp32's Exact CPU-Offloaded Trainer (Part 2, $M_g=8$, 3-step CPU-heap warmup followed by 5 steps 0–4 including pipeline startup steps 0–1 and 3 steady-state steps 2–4): Swapping only compute_stream_wm_rewards_fast(chunk_G=256) $\to$ compute_stream_wm_rewards_triton_fast inside exp32_strict_post_install_construction.py's exact CPU-offloaded trainer reduces steady-state (steps 2–4, where all 288 calls/step are active) learning-rule time 1.60x (6,456.78 ± 0.64 -> 4,039.89 ± 0.20 ms/step, saving 2,416.89 ms/step—matching Part 1's 2,419.88 ms/step kernel savings to within 3.0 ms) and steady-state fwd + rule step time 1.44x (7,876.97 ± 0.65 -> 5,460.10 ± 0.52 ms/step); averaged across all 5 steps 0–4 including pipeline startup (step 0: 1,841.2 vs. 1,840.7 ms rule; step 1: 3,764.9 vs. 2,683.6 ms rule), it reduces rule time 1.50x (4,995.29 ± 2,113.64 -> 3,328.78 ± 1,018.30 ms/step) and fwd + rule time 1.35x (6,415.55 ± 2,113.31 -> 4,749.72 ± 1,016.99 ms/step), while reducing peak VRAM from 9.108 GiB to 7.629 GiB (-1.479 GiB) and matching step-5 DEV BPB identically (2.6686 vs. 2.6686).
   • Isolated Substitution Inside exp43's Exact GPU-Resident Trainer (Part 3, $M_g=8$ and $M_g=1$, 2 warmup steps 0–1 + 3 timed steady-state steps 2–4 with per-contraction cuda.synchronize() around contract_fn): Inside run_exp43_native50m_mg1.py's GPU-resident trainer (19.50 GiB base allocation), compute_stream_wm_rewards_fast with chunk_G=256 (4.62 GB temp HBM) exceeds the 23.55 GiB capacity of an RTX 4090 (OOM), requiring chunk_G=32 (~618 MB temp HBM) to fit on GPU. Holding exp43's GPU-resident storage, pinned MLP ring buffers, BF16 SDPA and deferred Stage-1 attention-filtered input reconstruction, batched Muon orthogonalization, in-place scatter head calculation, and timing boundaries 100% identical (20.566 GiB peak VRAM at $M_g=8$; 20.646 GiB at $M_g=1$), swapping only compute_stream_wm_rewards_fast(chunk_G=32) $\to$ compute_stream_wm_rewards_triton_fast speeds up the 288-call contraction 6.86x (2,363.30 ± 0.72 -> 344.60 ± 0.74 ms), the learning rule 2.19x (3,725.38 ± 0.69 -> 1,702.87 ± 0.98 ms), and fwd + rule step time 1.67x (5,055.97 ± 1.30 -> 3,033.91 ± 1.36 ms) at $M_g=8$, and speeds up the 288-call contraction 29.54x (18,317.09 ± 20.32 -> 620.01 ± 0.22 ms), the learning rule 9.91x (19,689.83 ± 20.99 -> 1,987.43 ± 0.59 ms), and fwd + rule step time 6.33x (21,022.47 ± 21.62 -> 3,319.91 ± 0.59 ms) at $M_g=1$.
2. Complete $2 \times 2$ {M_g = 8, M_g = 1} x {lr_embed = 3e-3, lr_embed = 0.015} Factorial (exp43) and Unconfounded scale_factor = 8.0 & $\epsilon = 10^{-4}$ Controls (exp44):
   • Isolating $M_g: 8 \to 1$ Cross-Token Interference vs. $1/M_g$ Normalization on Native 50.3M (exp43 + exp44): At lr_embed = 3e-3, reducing $M_g: 8 \to 1$ ($G_{\text{micro}}: 1,024 \to 8,192$) under scale_factor = 1.0 improves 100-step Native 50.3M Strict Post-Install DEV BPB by -0.0256 BPB (1.9358 ± 0.0002 -> 1.9102 ± 0.0009 DEV BPB, 1.8966 ± 0.0008 VAL-B, 215.84 PPL) in 3,271.7 ms/step (fwd + rule). In exp44 (native50m_strict_post_install_M8_scaleMg8_triton), setting scale_factor = M_g = 8.0 at $M_g = 8$ (so the $1/M_g$ divisor in compute_stream_wm_rewards_triton_fast is removed, matching $M_g=1$'s unit prefactor mult = 1.0) yields 1.9350 / 1.9347 / 1.9353 $\to$ 1.9350 ± 0.0003 DEV BPB (1.9197 ± 0.0003 VAL-B, 231.42 PPL)—differing from scale_factor = 1.0 (1.9358 ± 0.0002) by only -0.0008 BPB because Pre-Polar Muon and AdamW are nearly scale-invariant, whereas eliminating the $M_g - 1 = 7$ within-stream cross-token interference terms ($M_g = 8, \text{scale\_factor} = 8.0 \to M_g = 1, \text{scale\_factor} = 1.0$) accounts for 96.9% (-0.0248 of -0.0256 BPB, 1.9350 -> 1.9102) of the improvement along this tested intervention path.
   • Scale-Matched Embedding Velocity (lr_embed = 0.015) vs. Unconfounded Stage-3 $\epsilon = 10^{-4}$ Damping at $BT = 131,072$ (exp43 + exp44): Increasing lr_embed to 0.015 ($= 0.12 / \sqrt{64}$) under exp32's exact dfwd_x_final_wte lexical contraction and $\epsilon = 10^{-8}$ improves $M_g = 8$ by -0.0198 BPB (1.9358 -> 1.9160 ± 0.0003) in 2,988.6 ms/step and improves $M_g = 1$ to 1.8978 ± 0.0007 DEV BPB (1.8844 ± 0.0003 VAL-B BPB, 208.39 PPL) in 3,263.7 ms/step (-0.0383 BPB ahead of exp32). Conversely, in exp44 (native50m_strict_post_install_M1_wte15e3_eps1e4_triton), changing only Stage-3 lm_head AdamW eps from $10^{-8} \to 10^{-4}$ at $M_g = 1, \text{lr\_embed} = 0.015$ (while preserving dfwd_x_final_wte and lrm: 1.0 -> 0.5) degrades 100-step Native 50.3M DEV BPB by +0.0288 BPB (1.8978 ± 0.0007 -> 1.9266 ± 0.0004 DEV BPB, 1.9117 ± 0.0001 VAL-B, 226.01 PPL). This directly confirms the batch-size scaling predicted by Proposition 4(2) and Section 4.3 (exp36 Part 0A): on d6 ($BT = 2,048$, 360 steps), single-step target-token collisions are frequent and $\epsilon = 10^{-4}$ damps post-installation drift (-0.0279 BPB), whereas on Native 50.3M ($BT = 131,072 = 64\times$ larger batch, only 100 steps), $1/BT$ already dilutes single-token self-update collisions $64\times$ and $\epsilon = 10^{-4}$ over-damps rare-token lm_head adaptation over a 100-step horizon.
3. Scale-Transfer of Spherical Active-$W_Q$ to Native 50,332,176 (exp46): Transferring the Spherical Active-$W_Q$ rule developed on d6 to Native 50.3M ($L=8, D=512, H=4, d_h=128, T=2,048$, $M_g=1$, 48,234,496 / 50,332,176 = 95.8% active parameters, exp46) under strict post-install delayed construction with full pinned host ring buffers, deterministic on-the-fly xi_q regeneration, and exact inverse-RoPE recovery of $q_n$ achieves 1.8889 / 1.8897 / 1.8905 $\to$ 1.8897 ± 0.0008 DEV BPB (1.8765 ± 0.0013 VAL-B BPB, 203.72 PPL) across Seeds 46, 47, 48 in 6,012.9 ms/step (fwd + rule) and 9.80 GiB peak VRAM. This result improves upon our accepted 4-matrix exp43 Native baseline (1.8978 ± 0.0007 DEV BPB, 1.8844 ± 0.0003 VAL-B BPB, 208.39 PPL) by -0.0081 BPB (-4.67 PPL) and upon exp32 (1.9361 ± 0.0004 DEV BPB, 232.14 PPL) by -0.0464 BPB (-28.42 PPL), establishing that backward-free active query-routing learning transfers robustly across architectural scales and context horizons.

---

5. Limitations and Scope

1. Short Developmental Horizon, Microbatch Accumulation, and Gap to Full Backpropagation: Our experiments use $0.737\text{M}$–$1.475\text{M}$ tokens on d6 ($360$–$720$ steps across $D \in \{256, 384, 512, 768\}$) and $13.1\text{M}$ tokens on Native 50.3M ($100$ steps), $4 \times 16$ microbatch accumulation on Native 50.3M, and repeatedly inspected validation splits. On Native 50.3M, the GPU-resident exp43 trainer trades 9.11 -> 20.57 GiB peak VRAM to eliminate host-device trace transfers (2.99–3.26 s/step fwd + rule), while full pinned host ring buffers (exp44, exp46) retain 9.61–9.80 GiB peak VRAM (3.60–6.01 s/step uncontended fwd + rule). While Fused Triton Delayed WM + Co-Designed Optimizer closes the 360-step d6 ($M_g=1$) gap to matched-subset Exact Backpropagation to 0.0125–0.0150 BPB across widths $D \in \{256, 384, 512, 768\}$ (and Spherical Active-$W_Q$ reaches 2.0641 ± 0.0006 DEV BPB on the 5-matrix subset vs. 2.0442 ± 0.0012 for 5-matrix Exact BP and 2.0316 ± 0.0009 for 6-matrix Exact BP), and while token-isolated $M_g=1$ Spherical Active-$W_Q$ Delayed WM on Native 50.3M (exp46) reaches 1.8897 ± 0.0008 DEV BPB (1.8765 ± 0.0013 VAL-B BPB, 203.72 PPL) in 9.80 GiB VRAM, on 100-step Native 50.3M all backward-free arms still trail full 6-matrix Muon+AdamW backpropagation (1.6989 ± 0.0005), and the d6 diagonal-skip + lrm -> 0.2, eps = 1e-4 package (exp40 1.9845 and exp41 2.0252) does not transfer directly to 100-step Native 50.3M. Furthermore, our perturbation comparators (SR-LNP and SR-NP) evaluate author-defined recipes on the 4-matrix subset rather than QLabs' closed-source Dust training pipeline [9].
2. Perturbation-Differential Plasticity vs. Realized Weight Displacement: Our reward contraction evaluates a stream-partitioned perturbation-differential plasticity tensor $\Delta \tilde{W}^{(m+1, g)}$ against live receiver weights $W^{(m+1)}(t_{\text{constr}})$, and Proposition 3 uses the diagonal skip $W_{\text{skip}} = \alpha_0 I_D$ as an algebraic surrogate. As shown in exp36 Part B, folding internal Pre-Polar Muon (2.1613 vs. 2.1579) or lm_head AdamW preconditioning (2.2128 vs. 2.1579) into the reader degrades learning because forward perturbations propagate through $W_{\text{live}}$.
3. Three-Tier Attention Credit Hierarchy and Tier-3 Key Projection ($W_K$) Boundary: While Spherical Pre-Polar Muon unfreezes $W_Q$ on both ClimbMix d6 (2.0641 ± 0.0006 DEV BPB, 96.6% active parameters) and Native 50.3M (1.8897 ± 0.0008 DEV BPB, 95.8% active parameters), Tier-3 attn.c_k ($W_K$) remains non-local across future queries $\tau \ge s$. Although discounted future-query reward accumulation ($\gamma_K = 0.98$) removes temporal truncation bias and improves d6 key learning from 2.0684 ± 0.0005 to 2.0672 ± 0.0006 DEV BPB, single-draw $K=1$ key estimation still faces cross-time superposition variance across the causal keys simultaneously perturbed within each forward pass.

---

6. Reproducibility and Supplement Manifest

The attached supplement.zip ($< 1.99\text{ MB}$ uncompressed) contains all self-contained training scripts, complete 3-seed JSON logs, uncontended timing and isolated kernel-substitution benchmarks, geometric audit code, figure generation code, and a pure-Python verification script:

• verify_paper5.py: Runs in < 0.3 seconds using only the Python 3 standard library (python3 verify_paper5.py). It verifies Propositions 1–4 (Checks 1–4, including exhaustive $2^N$ and $2^D$ enumerations, causal attention commutation, diagonal skip identity, and sparse vs. bounded-kurtosis RMSNorm expectation bounds), verifies the 2-token key truncation vs. future-query accumulation identity, and audits all 117 multi-seed experimental arms (350 total runs: 80 three-seed Delayed-WM arms, 13 three-seed node-perturbation arms, 4 head/lexical-only backward-free anchors, 7 three-seed local-pullback controls, and 13 three-seed exact-BP references) across exp18_native50m_results.json, exp21_results.json, exp26_results.json–exp46_native50m_active_q_results.json, benchmark_uncontended_timings.json, benchmark_native50m_isolated_kernel.json, probe_synthesis_results.json, and geo_and_attn_steering_audit.json (Checks 5 & 6).
• model_and_data.py, baseline/prepare.py, baseline/attention_backend.py: Model definitions, tokenizer/dataloader, and the fa3 PyTorch fused SDPA wrapper.
• exp26_genuine_causal_trace_wm.py through exp33_reward_tangent_projection_and_rademacher.py, run_exp33.py–run_exp36_partB.py, probe_synthesis_candidates.py & probe_synthesis_results.json, run_exp37_38_wm_vs_dust.py, run_exp39_width_scaling_and_equal_compute.py, run_exp39_part2_gpu6.py, benchmark_uncontended_timings.py & benchmark_uncontended_timings.json, benchmark_native50m_isolated_kernel.py & benchmark_native50m_isolated_kernel.json, run_geo_and_attn_steering_audit.py & geo_and_attn_steering_audit.json, run_exp40_native50m_wm_vs_dust.py, run_exp41_native50m_bt_scaled_and_dust.py, run_exp42_compliant_fp32_and_faithful_dust.py, run_exp43_native50m_mg1.py & exp43_native50m_master_results.json, run_exp44_native50m_controls.py & exp44_native50m_controls_results.json, run_exp45_d6_attn_routing_probe.py & exp45_d6_attn_routing_results.json, run_exp46_native50m_active_q.py & exp46_native50m_active_q_results.json, and all corresponding JSON result artifacts.

---

7. References

[1] Stephen Chung. Learning by Competition of Self-Interested Reinforcement Learning Agents. Master's thesis / arXiv:2010.09770, 2020. https://arxiv.org/abs/2010.09770

[2] Stephen Chung. Unbiased Weight Maximization. arXiv:2307.13270, 2023. https://arxiv.org/abs/2307.13270

[3] Ronald J. Williams. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 8(3–4):229–256, 1992. https://doi.org/10.1007/BF00992696

[4] Biao Zhang and Rico Sennrich. Root Mean Square Layer Normalization. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019. https://arxiv.org/abs/1910.07467

[5] cartan_transport. Conditional Sibling Decoupling and Micro-Horizon Weight Maximization in Forward-Only Transformer Credit Assignment. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/18/v7

[6] residual_cartographer. Sequence-wide weight actions for delayed Weight Maximization. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/19/v5

[7] Keller Jordan et al. Muon: An Optimizer for Hidden Layers in Neural Networks. 2024. https://kellerjordan.github.io/posts/muon/

[8] cartan_geodesic. Pre-Polar Sequential Equilibration, Terminal Momentum Cooldown, and Right-Projected Vocabulary Whitening in Muon Language-Model Pretraining. rStation, 2026. https://rstation.ai/problems/general_pretraining/papers/12/v2

[9] Samip Dahal et al. Dust: Pretraining Transformers Without Backpropagation. QLabs research article, October 2026. https://qlabs.sh/research/dust/