Conditional Sibling Decoupling and Micro-Horizon Weight Maximization in Forward-Only Transformer Credit Assignment

1. Introduction and Architectural Ledger

1.1 The Width-Scaling Variance Bottleneck in Forward-Only Credit

Eliminating the global reverse-depth sweep of backpropagation is a central pursuit in neuromorphic computing and biologically plausible credit assignment [1, 2, 8]. While standard reverse-mode automatic differentiation computes exact gradients in $\mathcal{O}(1)$ depth sweeps, it enforces global backward locking: no upstream layer can update its weights until the entire network forward graph has executed and downstream layers have finished their reverse-depth passes.

Forward perturbation methods avoid backward depth locking by perturbing hidden states during forward execution and correlating the resulting loss shifts with the injected noise [5, 6, 8, 12]. However, conventional forward perturbation rules face a severe neuronal width-scaling bottleneck:

• Global Scalar-Reward Perturbation Control (fwd_global_scalar_K2): Simultaneous zero-mean perturbation across all parameters or activations correlated with a single global scalar loss shift $\Delta \mathcal{L}$ yields estimator variance that scales with the total parameter count $\mathcal{O}(|\Theta| / K)$. (Note: fwd_global_scalar_K2 is a simplified global-scalar control, not published DUST [5]; see Section 2.6 and Section 3.5 for our two-stage faithful DUST baseline.) In a 6-layer Transformer at $D=512$ ($M=2048$), relative variance of fwd_global_scalar_K2 at Layer 3 (midMLP) explodes to $3.26 \times 10^7$ (Table 3), stalling 200-step pretraining at $2.3898$ holdout BPB (Table 4) and 1000-step pretraining on d6 ($D=384$) at $2.44794 \pm 0.11598$ holdout BPB (Table 5).
• Token-Level Probing (fwd_token_uwm_K2): Evaluating per-token scalar rewards $\Delta \mathcal{L}_t \in \mathbb{R}$ isolates individual tokens, but shares a single scalar return across all $M = 4D$ sibling units in that sublayer. As Ren et al. [8] established, estimator variance scales as $\mathcal{O}(M / K)$, causing Layer 3 relative variance to grow from $26,968.01$ ($D=128$) to $103,931.10$ ($D=512$) (Table 3) and limiting 200-step holdout BPB to $2.1868$ at $D=512$ (Table 4) and 1000-step d6 holdout BPB to $2.09816 \pm 0.00521$ (Table 5).

1.2 Stephen Chung's Foundations and Sylvester-Hadamard Coordination

To overcome this width bottleneck without a global backward pass, we adapt Stephen Chung's credit principles:

1. Weight Maximization (arXiv:2010.09770v3) [2]: Hidden units do not require an activation-space transpose circuit $(V^\top \delta_t)_i$. Instead, each unit $i$ observes only the plasticity displacement $\Delta \mathbf{v}^{(i)}$ of its own outgoing synaptic column $\mathbf{v}^{(i)} = V_{:, i}$, reinforcing activity when its outgoing column norm expands ($G^{(i)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}^{(i)}$).
2. Unbiased Weight Maximization (arXiv:2307.13270) [3]: Centered and antithetic exploration cancels zero-order norm biases, and difference-quotient path averaging evaluates non-linear activation credit without analytical backward derivatives.
3. Structured Coordination [4] and Sylvester-Hadamard Exploration [11]: Multi-agent exploration noise coordinated via orthogonal Sylvester-Hadamard matrices eliminates cross-agent exploration interference. Following Choromanski et al. [11] (arXiv:1804.02395), who proved variance reduction for orthogonal and Hadamard perturbation directions, we adapt Sylvester-Hadamard exploration codebooks to the $S$-token micro-horizon of parameter-space column contractions. This is conceptually distinct from Chung's Structured Coordination [4] (arXiv:2307.13256), which investigated recurrent networks of stochastic Boltzmann units coordinated via multi-agent reinforcement learning.
4. MAP Propagation (arXiv:2010.07893v2) [1]: Layers iteratively settle post-activation states $H^1(n) = \mu^1 + e_1(n)$ to minimize output discrepancy, damping credit through maximum a posteriori (MAP) prior precision.

1.3 Strict No-Backward Credit vs. Global Reverse-Depth Removal with Local Reverse Credit Computations

We enforce a strict conceptual and experimental separation between strict no-backward credit assignment (local_reverse_derivative_calls = 0) and methods that remove the global reverse-depth sweep while retaining local reverse pullbacks:

1. Strict No-Backward Credit Assignment (Zero Reverse-Depth Sweeps AND Zero Local Reverse Pullbacks, local_reverse_derivative_calls = 0):
   • Published Faithful DUST (dust_faithful_sgd_P64, dust_faithful_muon_P64, dust_faithful_muon_P256) and Fairly Optimized DUST (dust_faithful_muon_P64_sc_sh, dust_faithful_muon_P256_sc_sh) [5]: Perturbs activation groups during forward execution (writers, hidden, hubs o@layer, wte, 256-column lm_head slabs) and correlates per-token centered loss rewards (and decayed future-token attention head rewards $\gamma=0.98$) directly with pre-synaptic states (dust_faithful.py, delayed_trace_cowm.py).
   • Strict No-Backward Delayed / Eligibility-Trace Coordinated Weight Maximization (Delayed-Trace-Co-WM-S, delayed_trace_cowm_*, hub_mode="dust_hub", local_reverse_derivative_calls = 0): Records local forward eligibility traces on mlp.c_fc during the clean forward pass using $S$-token Sylvester-Hadamard probes ($\xi_{g, s}^{(m)} = H_{s, c_m}$) and local forward activation evaluations $\phi(z \pm \sigma_{\text{act}})$, and modulates those pre-synaptic traces via 1-step cross-batch delayed (d1) or same-step (d0) outgoing-weight column contractions $c_g^{(i, m)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)}$ obtained from outgoing mlp.c_proj writer updates (_outgoing_col_dot_all8_kernel in delayed_trace_cowm.py). Critically, delayed_trace_cowm_* never reconstructs an activation-space error vector on mlp.c_fc and estimates attention hubs o@layer (targets[layer]) via DUST's own forward suffix perturbations (site_kind="hub", hub_mode="dust_hub"), requiring zero direct hidden (mlp.c_fc) forward passes and zero local reverse pullbacks (local_reverse_derivative_calls = 0). Reallocating 4 of the 27 saved hidden draws at $P=64$ to hubs (hubs = [2, 2, 2, 2, 2, 2], hub2) and 23 to writers (writers = [12, 10, 8, 7, 6, 6]) unlocks full-block simplex covariance centering (sc, draws >= 2) across all 6 attention hubs while intra-block sibling branch decoupling (bd025), target-separated Hadamard readout probing (sh), and forward scalar traces (st) remain 100% forward-only.
2. Global Reverse-Depth Removal with Local Reverse Credit Computations (Labelled Local-Reverse-Credit and Hybrid Controls):
   • Hybrid Delayed-Trace + Local Attention-Hub Pullback Controls (delayed_trace_full_*, delayed_trace_unified_*, hub_mode="co_wm_s", local_reverse_derivative_calls = L = 6): When hub_mode="co_wm_s" is enabled (contract_hub_outgoing_columns), micro-group $g$ forms $\Delta W_g = H^\top U_g$, contracts $C_g = \Delta W_g W_O$, and reconstructs the upstream activation-space tensor $\widehat{E}_g = \frac{1}{S} H C_g = U_g W_O \in \mathbb{R}^{B \times T \times D}$ at $K_{\text{wm}} = S = 8$ to supply targets[layer] to local_errors on the same step. Because $\widehat{E}_g = U_g W_O$ executes the 1-layer local linear reverse pullback of attn.c_proj ($Y = O W_O^\top$) in the Sylvester-Hadamard token basis, all hub_mode="co_wm_s" variants (delayed_trace_full_* and delayed_trace_unified_*) are explicitly classified and audited (local_reverse_derivative_calls = 6, is_pure_forward_only = False, has_local_backward_phase = True) as hybrid local-hub-pullback controls rather than strict no-backward methods.
   • Streaming Stagewise Local-Reverse-Credit Controls (fwd_streaming_cowm*, fwd_streaming_oracle*, fwd_lookahead_2b*): @torch.no_grad() is enforced (num_backward_calls = 0, num_depth_reverse_sweeps = 0) and no global backward graph is instantiated. However, within each detached block, these methods retain local reverse credit computations (d_relu2, _forward_rms_norm_directional_pull, inverse rotary pullback apply_rotary_emb(cos, -sin), and intra-block MLP-to-Attention residual pullback). Replacing analytical derivative formulas with central forward operator differences (OpDiff, *_opdiff_*) still evaluates local reverse sensitivity pullbacks and therefore does not remove backward credit computation. Accordingly, backpropagation (backprop, backprop_ckpt), block-detached autograd (block_*_bp), and all coupled local-error / analytic-surrogate / OpDiff / Lookahead-2B streaming variants are presented strictly as labelled local-reverse-credit controls.

We organize our study across three architectural paradigms:

• Paradigm A: Two-Level Coordinated Weight Maximization (fwd_two_level_cowm): Combines parameter-space Co-WM-S column contraction on lm_head over micro-horizons of $S=8$ tokens, $2 K_{\text{net}}$ end-to-end antithetic full-network passes ($x_0 \to x_L$) with either depth-Hadamard carrier modulation (net_carrier_mode="depth_hadamard") or independent per-probe carrier resampling (net_carrier_mode="iid_resample"), and intra-block parameter-space Co-WM-S on MLP projections ($W_1, V$) with head-decoupled forward FlashAttention probing on attention matrices ($W_Q, W_K, W_V, W_O$).
• Paradigm B: Streaming Stagewise Forward Co-WM-S and Local-Reverse-Credit Controls (fwd_streaming_cowm*, fwd_lookahead_2b*): Executes a single forward sweep through the network ($l = 0 \dots L-1$). At each block $l$:
  1. Forward output $x_l$ is evaluated and correlated with the language modeling target via an active sub-vocabulary projection ($V_{\text{sub}} = 256$ active tokens, or $V_{\text{sub}}=0$ for full vocabulary $V$);
  2. Block $l$'s parameters ($W_Q, W_K, W_V, W_O, W_1, V$) compute gradients blockwise into .grad (either uncoupled in fwd_streaming_cowm, where both sublayers receive $\delta_{\text{block}}$; intra-block coupled in fwd_streaming_cowm_coupled*, where the MLP update runs first and pulls upstream credit across $W_1$ via parameter-space Co-WM-S into the attention sublayer; or 2-block cooperative lookahead in fwd_lookahead_2b_orth_mv*, where a sliding 2-block window $(l, l+1)$ transports block $l+1$'s zero-variance MLP + Attention-Value input credit projected onto $\delta_{\text{local}, l}^\perp$ into block $l$);
  3. Block Deallocation: All intermediate activations of block $l$ ($x_{\text{prev}}, x_{\text{attn\_in}}, u_{\text{attn}}, x_{\text{mlp\_in}}, u_{\text{mlp}}$) are freed from GPU memory immediately before block $l+1$ (or $l+2$ under 2-block lookahead) begins execution, maintaining $\mathcal{O}(1)$ activation memory in depth $L$;
  4. The optimizer updates all parameters only after execute_pure_forward_step returns.
• Paradigm C: Strict No-Backward Credit Assignment (dust_faithful_*, delayed_trace_cowm_*) and Hybrid Local-Hub-Pullback Controls (delayed_trace_full_*, delayed_trace_unified_*): Detailed in Section 2.6 and evaluated in Section 3.5 across Stage 1 reference validation (dust_stage1_reference_validation.json), 5-component latency profiling across widths (contraction_and_trace_width_profile.json), pure hidden-neuron diagnostic scaling ($M \in \{256 \dots 4096\}$, dust_and_delayed_trace_width_diagnostic.json), Regime I 2-seed pure MLP-width pretraining scaling at fixed $D=256$ (regime1_fixed_D256_mlp_width_pretrain_200.json), Regime II full-model width pretraining scaling (dust_and_delayed_trace_width_sweep_200.json), 200-step component ablation screening (innovation_screen_200_d6.json), and 1000-step 2-seed d6 pretraining under matched-token and matched-wall-clock budgets (dust_and_delayed_trace_1000steps_d6.json).

To evaluate the incremental effects of successive training interventions in one specified ordering between end-to-end backpropagation and streaming Co-WM-S, we construct a sequential ablation ladder:

1. block_fixed_top_discrepancy_bp (Fixed Final-State Discrepancy + Detached Inter-Block Sublayers): Detaches residual stream inputs at each block boundary (bwd_calls = 6), evaluates the language modeling loss and lm_head update exclusively at the final state $x_{L-1}$ (w_stage = 1.0 on $L-1$, $0.0$ on $l < L-1$), and broadcasts the fixed final-state pre-norm discrepancy $\delta_{\text{final}} = \frac{\partial \mathcal{L}(x_{L-1})}{\partial x_{L-1}}$ directly into each detached block $l$ via the residual identity skip connection (x_l.backward(delta_final)). Comparing backprop to block_fixed_top_discrepancy_bp holds the downstream supervision discrepancy fixed at $\delta_{\text{final}}$ while replacing all intervening non-identity sublayer Jacobians ($l' > l$) with the identity residual path.
2. block_local_greedy_bp_tophead (Intermediate Local Supervision + Final-Only Readout Update): Retains the detached block boundaries (bwd_calls = 6), exact intra-block autograd, full vocabulary ($V$), and final-only lm_head update (w_stage = 1.0 on $L-1$, $0.0$ on $l < L-1$) of block_fixed_top_discrepancy_bp, but replaces the broadcast final-state discrepancy $\delta_{\text{final}}$ with each block's own intermediate local-loss discrepancy $\frac{\partial \mathcal{L}(x_l)}{\partial x_l}$.
3. block_local_greedy_bp (Mixed-Readout Block-Detached Local Autograd [8]): Identical to block_local_greedy_bp_tophead, except the shared lm_head accumulates $50\%$ of its gradient from the final block $L-1$ and $50\%$ uniformly across earlier blocks $l < L-1$ (w_stage = 0.5 / (L-1)), matching the readout update rule of fwd_streaming_*. (In Table 3, block_local_greedy_bp_vsub256 additionally restricts the readout pullback to $V_{\text{sub}}=256$.)
4. Incremental Intra-Block and 2-Block Local-Reverse-Credit Controls at Full Vocabulary ($V_{\text{sub}}=0$):
   • fwd_streaming_oracle_fullvocab (Uncoupled Baseline): Feeds $\delta_{\text{block}}$ into both MLP and Attention independently, with single-window CM-Attn ($K_v=1$) and pre-RoPE $K_{\text{qk}}=2$ attention probes.
   • fwd_streaming_oracle_mlp_pull_only_fullvocab (+MLP-Pullback Only): Adds only the upstream MLP-to-Attention residual pullback $\delta u_{\text{attn}} = \delta_{\text{block}} + \Delta_{\text{mlp}\to\text{attn}}$ (intra_block_coupling=True), keeping $K_v=1, K_{\text{qk}}=2$ and f32_hadamard_qk=False.
   • fwd_streaming_oracle_mlp_whv_fullvocab (+MLP-Pullback + CM-Attn-WH $K_v=8$): Additionally enables Two-Level Window-Hadamard value transport (K_v=8), keeping $K_{\text{qk}}=2$ and f32_hadamard_qk=False.
   • fwd_streaming_oracle_coupled_fullvocab (+Combined Hadamard-QK Package, $K_{\text{qk}}=8$): Jointly transitions Q/K probing from $K_{\text{qk}}=2$ pre-RoPE i.i.d. probes to the post-RoPE head-channel Sylvester-Hadamard Q/K package (K_qk=8, f32_hadamard_qk=True, $\sigma_{\text{qk}}=0.05$, float32 perturbation construction/reward accumulation with bfloat16 FlashAttention execution, and 16-lag scalar key lookahead), paired with fwd_streaming_cowm_coupled_fullvocab_K8 ($K=S=8$ parameter-space Co-WM-S).
   • fwd_streaming_oracle_coupled_qproj_unbiased_fullvocab (+Bilinear Query-Subspace Projected Key Probing Q-Proj-K): Holds $K_{\text{qk}}=8$, $\sigma_{\text{qk}}=0.05$, post-RoPE perturbation location, and float32/bfloat16 precision handling strictly identical to fwd_streaming_oracle_coupled_fullvocab, replacing only the 16-lag scalar key correlation with bilinear rank-1 query-subspace projection (qproj_k="unbiased"), paired with fwd_streaming_cowm_coupled_qproj_opdiff_fullvocab_K8 ($K=S=8$ parameter-space Co-WM-S + OpDiff).
   • fwd_lookahead_2b_orth_mv_a1.0_opdiff_qproj_fullvocab (+Adjacent 2-Block Orthogonal-Complement Cooperative Lookahead Lookahead-2B-OrthMV + OpDiff): Adds sliding 2-block orthogonal-complement lookahead transport (lookahead_2b="orth_mv", lookahead_alpha=1.0, opdiff=True).
5. Sub-Vocabulary Readout Truncation ($V_{\text{sub}}=256$): fwd_streaming_oracle, fwd_streaming_oracle_coupled, and their parameter-space Co-WM-S counterparts fwd_streaming_cowm_K8, fwd_streaming_cowm_coupled_K8, fwd_streaming_cowm_coupled_qproj_opdiff_K8, and fwd_lookahead_2b_orth_mv_cowm_opdiff_qproj_K8 restrict the readout pullback to $V_{\text{sub}}=256$ active columns.

Table 1 provides the complete arithmetic and pass-count ledger for all studied algorithms ($B=16, T=512$, $N_{\text{tok}}=8192$, micro-horizon $S=8$, $L=6$ blocks). Across widths $D \in \{128, 256, 384, 512\}$, exact trainable parameter counts are $6,422,732$, $15,204,748$, $26,346,060$, and $39,846,668$ for the pretraining vocabulary $V=8192$ (Tables 1, 4, 5, 8, 9), and $3,801,292$, $9,961,868$, $18,481,740$, and $29,360,908$ for the synthetic diagnostic vocabulary $V=4096$ (Table 3).

| Algorithm | Execution Paradigm | Full Forward Passes | Backward Calls | Depth Reverse Sweeps | Attn Forward Calls / Block | MLP Forward Calls / Block | Param-Space WM Contractions | Peak VRAM ($D=512$, $M=2048$) | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | backprop (Standard) | Autograd Forward-Backward | $1$ | $1$ | $1$ | $1$ clean | $1$ clean | $0$ | $3010.4\text{ MB}$ | | backprop_ckpt (Checkpointed) | Autograd + Block Recompute | $2$ | $1$ | $1$ | $2$ ($1$ fwd + $1$ recompute) | $2$ ($1$ fwd + $1$ recompute) | $0$ | $1731.4\text{ MB}$ | | block_fixed_top_discrepancy_bp | Block-Detached BP (Fixed Top $\delta_{\text{final}}$) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $3026.5\text{ MB}$ | | block_local_greedy_bp_tophead | Block-Detached BP (Local $\delta_l$, Top Head) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $2153.6\text{ MB}$ | | block_local_greedy_bp | Block-Detached BP (Mixed Head [8]) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $2169.6\text{ MB}$ | | fwd_global_scalar_K2 | Pure Forward (Global-Scalar Control) | $3$ | $0$ | $0$ | $3$ ($1$ clean + $2$ probe) | $3$ ($1$ clean + $2$ probe) | $0$ | $3864.8\text{ MB}$ | | fwd_token_uwm_K2 | Pure Forward (Token Probing) | $5$ | $0$ | $0$ | $5$ ($1$ clean + $4$ probe) | $5$ ($1$ clean + $4$ probe) | $0$ | $3864.8\text{ MB}$ | | dust_faithful_sgd_P64 | Strict No-Backward (Faithful DUST [5], SGD, rev=0) | $1 + 64$ suffix | $0$ | $0$ | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $3035.5\text{ MB}$ | | dust_faithful_muon_P64 | Strict No-Backward (Faithful DUST [5], Muon, rev=0) | $1 + 64$ suffix | $0$ | $0$ | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $3083.7\text{ MB}$ | | dust_faithful_muon_P64_sc_sh | Strict No-Backward (Fairly Opt. DUST, sc+sh, rev=0) | $1 + 64$ suffix | $0$ | $0$ | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $4631.9\text{ MB}$ | | delayed_trace_cowm_P64_fast_d1 | Strict No-Backward (Delayed-Trace-Co-WM-S, rev=0) | $1 + 37$ suffix | $0$ | $0$ | $23.7$ ($142$ block evals) | $23.7$ ($142$ block + $1$ local) | $49,152$ | $3733.9\text{ MB}$ | | delayed_trace_cowm_P64_realloc_d1 | Strict No-Backward (Delayed-Trace-Co-WM-S, rev=0) | $1 + 64$ suffix | $0$ | $0$ | $44.2$ ($265$ block evals) | $44.2$ ($265$ block + $1$ local) | $49,152$ | $3733.9\text{ MB}$ | | delayed_trace_cowm_P64_fast_sc_sh_d1 | Strict No-Backward (Delayed-Trace Fast + bd025+sc+sh, rev=0) | $1 + 41$ suffix | $0$ | $0$ | $26.0$ ($156$ block evals) | $26.0$ ($156$ block + $1$ local) | $49,152$ | $4977.9\text{ MB}$ | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 | Strict No-Backward (Delayed-Trace + hub2+bd025+sc+sh, rev=0) | $1 + 64$ suffix | $0$ | $0$ | $41.0$ ($246$ block evals) | $41.0$ ($246$ block + $1$ local) | $49,152$ | $4977.9\text{ MB}$ | | delayed_trace_cowm_P256_bd025_sc_sh_st_d1 | Strict No-Backward (Delayed-Trace $P=256$ + bd025+sc+sh+st, rev=0) | $1 + 146$ suffix | $0$ | $0$ | $92.3$ ($554$ block evals) | $92.3$ ($554$ block + $1$ local) | $49,152$ | $4476.8\text{ MB}^\dagger$ | | delayed_trace_full_P64_fast_d1 | Hybrid (Delayed-Trace + 1-Layer Hub Pullback, rev=6) | $1 + 29$ suffix | $0$ | $0$ | $18.2$ ($109$ block evals) | $18.2$ ($109$ block + $1$ local) | $61,440$ | $4706.2\text{ MB}$ | | delayed_trace_full_P64_realloc_d1 | Hybrid (Delayed-Trace + 1-Layer Hub Pullback, rev=6) | $1 + 64$ suffix | $0$ | $0$ | $41.3$ ($248$ block evals) | $41.3$ ($248$ block + $1$ local) | $61,440$ | $4706.2\text{ MB}$ | | delayed_trace_unified_P64_fast_sc_sh_d1 | Hybrid (Unified Fast + sc+sh, rev=6) | $1 + 29$ suffix | $0$ | $0$ | $18.2$ ($109$ block evals) | $18.2$ ($109$ block + $1$ local) | $61,440$ | $5956.4\text{ MB}$ | | delayed_trace_unified_P64_bd025_sc_sh_d1 | Hybrid (Unified Realloc + bd025+sc+sh, rev=6) | $1 + 64$ suffix | $0$ | $0$ | $41.3$ ($248$ block evals) | $41.3$ ($248$ block + $1$ local) | $61,440$ | $5956.4\text{ MB}$ | | delayed_trace_cowm_fixed16_d1 | Strict No-Backward (Delayed-Trace-Co-WM-S, All-Block 16w, rev=0) | $1 + 27$ suffix | $0$ | $0$ | $20.7$ ($124$ block evals) | $20.7$ ($124$ block + $1$ local) | $49,152$ | $3035.7\text{ MB}^\dagger$ | | dust_faithful_muon_P256 | Strict No-Backward (Faithful DUST [5], $P=256$, rev=0) | $1 + 256$ suffix | $0$ | $0$ | $180.0$ ($1080$ block evals) | $180.0$ ($1080$ block evals) | $0$ | $3369.7\text{ MB}^\dagger$ | | delayed_trace_cowm_P256_fast_d1 | Strict No-Backward (Delayed-Trace-Co-WM-S, $P=256$ Fast, rev=0) | $1 + 146$ suffix | $0$ | $0$ | $92.3$ ($554$ block evals) | $92.3$ ($554$ block + $1$ local) | $49,152$ | $3667.9\text{ MB}^\dagger$ | | delayed_trace_unified_P256_sc_sh_st_d1 | Hybrid (Unified $P=256$ + sc+sh+st, rev=6) | $1 + 146$ suffix | $0$ | $0$ | $95.7$ ($574$ block evals) | $95.7$ ($574$ block + $1$ local) | $61,440$ | $4828.2\text{ MB}^\dagger$ | | fwd_two_level_cowm_K4 | Pure Forward (End-to-End Co-WM) | $9$ | $0$ | $0$ | $19$ ($1$ clean + $18$ probe) | $10$ ($1$ clean + $9$ probe/recomp) | $106,496$ | $1850.3\text{ MB}$ | | fwd_streaming_cut_K4 | Pure Forward (Streaming Cut) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $6$ ($1$ clean + $5$ probe/recomp) | $24,576$ | $1820.3\text{ MB}$ | | fwd_streaming_wm_K1 ($K_{\text{qk}}=1$) | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $7$ ($1$ clean + $6$ probe) | $2$ ($1$ clean + $1$ local fwd) | $18,432$ | $1828.4\text{ MB}$ | | fwd_streaming_wm_K1_qk2 ($K_{\text{qk}}=2$) | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $18,432$ | $1820.3\text{ MB}$ | | fwd_streaming_wm_K4 | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $73,728$ | $1820.3\text{ MB}$ | | fwd_streaming_cowm_K4 | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $73,728$ | $1828.4\text{ MB}$ | | fwd_streaming_wm_K8 | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $147,456$ | $1820.3\text{ MB}$ | | fwd_streaming_cowm_K8 | Pure Forward (Streaming Dealloc) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $147,456$ | $1828.4\text{ MB}$ | | fwd_streaming_wm_map_N4| Pure Forward (Streaming MAP) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $6$ ($1$ clean + $5$ settle/recomp) | $73,728$ | $1828.4\text{ MB}$ | | fwd_streaming_oracle_fullvocab | Uncoupled Surrogate ($V=8192$) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $55,296$ | $2100.4\text{ MB}$ | | fwd_streaming_oracle_mlp_pull_only_fullvocab | +MLP-Pullback Only ($V=8192, K_v=1$) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ | | fwd_streaming_oracle_mlp_whv_fullvocab | +MLP-Pullback + CM-Attn-WH ($K_v=8$) | $1$ | $0$ | $0$ | $18$ ($1$ clean + $17$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ | | fwd_streaming_oracle | Uncoupled Surrogate ($V_{\text{sub}}=256$) | $1$ | $0$ | $0$ | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $55,296$ | $1820.3\text{ MB}$ | | fwd_streaming_cowm_coupled_K8 | Coupled Co-WM-S ($K=8, V_{\text{sub}}=256$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $1828.3\text{ MB}$ | | fwd_streaming_cowm_coupled_fullvocab_K8 | Coupled Co-WM-S ($K=8, V=8192$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2100.3\text{ MB}$ | | fwd_streaming_oracle_coupled | Coupled Surrogate ($V_{\text{sub}}=256$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $1828.3\text{ MB}$ | | fwd_streaming_oracle_coupled_fullvocab | Coupled Surrogate ($V=8192$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ | | fwd_streaming_cowm_coupled_qproj_opdiff_K8 | +Q-Proj-K + OpDiff Co-WM-S ($V_{\text{sub}}=256$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2084.3\text{ MB}$ | | fwd_streaming_cowm_coupled_qproj_opdiff_fullvocab_K8 | +Q-Proj-K + OpDiff Co-WM-S ($V=8192$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2442.2\text{ MB}^\dagger$ | | fwd_streaming_oracle_coupled_qproj_unbiased_fullvocab | +Q-Proj-K Surrogate ($V=8192$) | $1$ | $0$ | $0$ | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ | | fwd_lookahead_2b_orth_mv_a1.0_opdiff_qproj_fullvocab | +Lookahead-2B-OrthMV + OpDiff + Q-Proj-K ($V=8192$) | $1$ | $0$ | $0$ | $49.5$ ($297$ total) | $2.83$ ($17$ total) | $196,608$ | $2517.3\text{ MB}$ | | fwd_lookahead_2b_orth_mv_cowm_opdiff_qproj_K8 | +Lookahead-2B-OrthMV Co-WM-S ($V_{\text{sub}}=256$) | $1$ | $0$ | $0$ | $49.5$ ($297$ total) | $2.83$ ($17$ total) | $360,448$ | $2245.3\text{ MB}$ |

Table 1: Verified Computation and Pass-Count Ledger for 6-layer Transformer pretraining ($B=16, T=512$, $N_{\text{tok}}=8192$, micro-horizon $S=8$, $V=8192$; $^\dagger$measured on d6 $D=384$ in Table 5). All candidate forward-only methods enforce bwd_calls = 0 and depth_sweeps = 0. Uncoupled streaming contraction counts reflect $3$ linear sites per block (lm_head, mlp.c_proj, attn.c_proj) $\times 6$ blocks $\times K$ probes $\times (8192/8)$ groups ($147,456$ for $K=8$); coupled streaming adds the 4th parameter-space Co-WM-S site on mlp.c_fc ($W_1$) for the zero-pass MLP-to-Attention residual pullback ($196,608$ for $K=8$); 2-block lookahead adds $5$ inter-block lookahead evaluations of MLP ($V, W_1$) and Attention-Value ($W_O, W_V$, $9$ FlashAttention calls each).

---

2. Theoretical Analysis and Scaling Laws

2.1 Micro-Horizon Parameter-Space Weight Maximization (WM-S)

In Chung's Weight Maximization framework [2, Eq. 1–3], hidden unit $i \in \{1, \dots, M\}$ does not possess an activation-space transpose circuit $(V^\top \delta_t)_i$. Instead, each unit $i$ observes only the local plasticity changes $\Delta \mathbf{v}^{(i)}$ of its own outgoing weight column $\mathbf{v}^{(i)} = V_{:, i} \in \mathbb{R}^{D_{\text{out}}}$, reinforcing its activity when its outgoing synaptic column grows in squared $L^2$ norm.

To adapt this principle to sequence modeling without per-token activation transposes ($S = 1$) or full-batch temporal cross-talk ($S = N_{\text{tok}}$), we introduce Parameter-Space Micro-Horizon Weight Maximization (WM-S):

1. Partition the $N_{\text{tok}} = B \times T$ tokens into $G = N_{\text{tok}} / S$ disjoint micro-groups $\mathcal{S}_1, \dots, \mathcal{S}_G$ of size $S$ tokens ($S \in \{4, 8, 16, 64\}$).
2. For exploration draw $k \in \{1, \dots, K\}$, sample independent Rademacher perturbations $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ for each neuron $i \in \{1, \dots, M\}$.
3. Over each micro-group $\mathcal{S}_g$, the outgoing matrix $V \in \mathbb{R}^{D_{\text{out}} \times M}$ accumulates its parameter-space plasticity displacement: $$\Delta V_{\mathcal{S}_g}^{(k)} = \sum_{s \in \mathcal{S}_g} \delta_{g, s} \big(\xi_{g, s}^{(k)}\big)^\top \;\in\; \mathbb{R}^{D_{\text{out}} \times M}.$$
4. Neuron $i$ evaluates its parameter-space inner product from its own column $\mathbf{v}^{(i)} = V_{:, i}$ and column update $\Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)} = (\Delta V_{\mathcal{S}_g}^{(k)})_{:, i}$ [2, Eq. 2]: $$G_{\mathcal{S}_g}^{(i, k)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)} - \beta \left\|\Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)}\right\|_2^2 \;\in\; \mathbb{R}.$$
5. Neuron $i$ computes its input credit for token $t \in \mathcal{S}_g$ [2, Eq. 3]: $$\hat{g}_{g, t, i}^{\text{WM-S}} = \frac{1}{K} \sum_{k=1}^K G_{\mathcal{S}_g}^{(i, k)} \frac{\xi_{g, t, i}^{(k)}}{\sigma^2}, \qquad \nabla_{W_1} \mathcal{L} = \sum_{g=1}^G \sum_{t \in \mathcal{S}_g} \big(\hat{g}_{g, t}^{\text{WM-S}} \odot \phi'(a_{1, g, t})\big) x_{g, t}^\top.$$

Proposition 1 (Conditional Coordinate Variance and Sibling Decoupling of WM-S): Conditioned on fixed downstream linear targets $\delta_{g, s}$ independent of $\xi$, with independent Rademacher perturbations $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ (so $(\xi_{g, t, i}^{(k)})^2 / \sigma^2 \equiv 1$) and $\beta = 0$, expanding $\hat{g}_{g, t, i}^{\text{WM-S}}$ yields: $$\hat{g}_{g, t, i}^{\text{WM-S}} = \big(\mathbf{v}^{(i)} \cdot \delta_{g, t}\big) \underbrace{\frac{(\xi_{g, t, i}^{(k)})^2}{\sigma^2}}_{\equiv 1} + \frac{1}{K} \sum_{k=1}^K \sum_{s \in \mathcal{S}_g \setminus \{t\}} \big(\mathbf{v}^{(i)} \cdot \delta_{g, s}\big) \frac{\xi_{g, s, i}^{(k)} \xi_{g, t, i}^{(k)}}{\sigma^2}.$$ Because $\mathbb{E}[\xi_{g, s, i} \xi_{g, t, i}] = 0$ for $s \ne t$, the estimator is conditionally unbiased ($\mathbb{E}[\hat{g}_{g, t, i}^{\text{WM-S}} \mid \delta] = \mathbf{v}^{(i)} \cdot \delta_{g, t}$) with conditional variance: $$\text{Var}\!\left[\hat{g}_{g, t, i}^{\text{WM-S}} \;\middle|\; \delta\right] = \frac{1}{K} \sum_{s \in \mathcal{S}_g \setminus \{t\}} (\mathbf{v}^{(i)} \cdot \delta_{g, s})^2.$$

Decoupling Mechanism: In activation-space perturbation (Cut), the reward $R_t^{(k)}$ sums over all $M$ neurons at token $t$, yielding conditional variance $\frac{1}{K}\sum_{j \ne i} (\mathbf{v}^{(j)} \cdot \delta_t)^2$ that scales with $M-1$. In WM-S, parameter-space column isolation decouples all $M - 1$ sibling neurons $j \ne i$, replacing the sum over $M - 1$ neurons with a sum over $S - 1$ tokens in $\mathcal{S}_g$, which is conditionally independent of width $M$ for fixed per-unit targets.

2.2 Sylvester-Hadamard Temporal Coordination (Co-WM-S)

Because WM-S eliminates spatial sibling-neuron cross-talk under fixed $\delta$, the remaining conditional variance arises from the $S - 1$ cross-token terms within micro-horizon $\mathcal{S}_g$. In Co-WM-S, we adapt orthogonal Sylvester-Hadamard exploration [11] to token micro-horizons in sequence Transformers, structuring the $K \le S$ exploration probes across the $S$ tokens of micro-group $\mathcal{S}_g$ using $K$ distinct columns $\{c_1, \dots, c_K\}$ drawn uniformly without replacement from an $S \times S$ Sylvester-Hadamard matrix $H \in \{-1, +1\}^{S \times S}$ ($S$ a power of 2), modulated by an independent Rademacher carrier $r_{g, s, i} \in \{-1, +1\}$: $$\xi_{g, s, i}^{(k)} = \sigma \, H_{s, c_k} \, r_{g, s, i}.$$

Proposition 2 (Finite-Population Conditional Variance of Co-WM-S): Conditioned on fixed downstream targets $\delta_{g, s}$ and writing $c_s = \mathbf{v}^{(i)} \cdot \delta_{g, s}$, uniform selection of $K \in \{1, \dots, S\}$ distinct Hadamard columns without replacement at $\beta = 0$ yields $\mathbb{E}[\hat{g}_{g, t, i}^{\text{Co-WM-S}} \mid \delta] = c_t$ and exact conditional variance: $$\text{Var}\!\left[\hat{g}_{g, t, i}^{\text{Co-WM-S}} \;\middle|\; \delta\right] = \frac{S - K}{K (S - 1)} \sum_{s \in \mathcal{S}_g \setminus \{t\}} c_s^2.$$ Proof: For distinct tokens $s \ne t$, $p_{s, t}(c) = H_{s, c} H_{t, c}$ has population mean $\frac{1}{S}\sum_{c=1}^S H_{s, c} H_{t, c} = 0$ by Hadamard orthogonality and unit variance $p_{s, t}(c)^2 \equiv 1$. Sampling $K$ distinct columns without replacement yields the classical hypergeometric variance reduction factor $\frac{S - K}{K(S - 1)}$. At $K = S$, the conditional variance vanishes identically ($\text{Var} \equiv 0$) for every width $M$.

2.3 Exact Covariance Identity, Sufficient Conditions, and Multi-Checkpoint Trajectory Audit

For row $i$ of the incoming weight matrix $W_1$, let $c_{g, s, i} = \mathbf{v}^{(i)} \cdot \delta_{g, s} \in \mathbb{R}$ and $z_{g, t, i} = \phi'(a_{1, g, t, i}) x_{g, t} \in \mathbb{R}^D$, so the exact row gradient is $g_i = \sum_{g=1}^G \sum_{t \in \mathcal{S}_g} c_{g, t, i} z_{g, t, i} = \sum_{g=1}^G g_{g, i}$.

Proposition 3 (Exact Conditional Frobenius Weight-Gradient Covariance and Relative Variance Law): Under WM-S exploration with $K$ independent Rademacher draws $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ and $\beta = 0$, the estimator error decomposes over unordered token pairs $\{s, t\} \subset \mathcal{S}_g$ ($s < t$): $$\widehat{\nabla}_{W_{1, i}} - g_i = \frac{1}{K} \sum_{k=1}^K \sum_{g=1}^G \sum_{1 \le s < t \le S} \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big) \frac{\xi_{g, s, i}^{(k)} \xi_{g, t, i}^{(k)}}{\sigma^2}.$$ Because distinct pairs $\{s, t\} \ne \{s', t'\}$ are pairwise uncorrelated, the exact weight-gradient covariance is: $$\text{Cov}\!\left(\widehat{\nabla}_{W_{1, i}} \;\middle|\; c, z\right) = \frac{1}{K} \sum_{g=1}^G \sum_{1 \le s < t \le S} \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big) \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big)^\top,$$ and the exact conditional Frobenius variance summed across all $M$ neurons is: $$\mathbb{E}\!\left[\left\|\widehat{\nabla}_{W_1} - \nabla_{W_1}\right\|_F^2 \;\middle|\; c, z\right] = \kappa(S, K) \sum_{i=1}^M \sum_{g=1}^G \sum_{1 \le s < t \le S} \left\|c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\right\|_2^2,$$ where $\kappa(S, K) = 1/K$ for WM-S and $\kappa(S, K) = \frac{S-K}{K(S-1)}$ for Co-WM-S.

Algebraic Cross-Token Identity: By expanding the pair norm squared: $$\sum_{1 \le s < t \le S} \left\|c_s z_t + c_t z_s\right\|_2^2 = \left(\sum_{s=1}^S c_s^2\right) \left(\sum_{t=1}^S \|z_t\|_2^2\right) + \left\|g_{\mathcal{S}_g}\right\|_2^2 - 2 \sum_{t=1}^S c_t^2 \|z_t\|_2^2.$$ Defining the Within-Group Support Alignment Ratio $\alpha(c, z)$, the Global Gradient Coherence Ratio $\gamma(c, z)$, and the Group Gradient Coherence Ratio $\gamma_{\text{grp}}(c, z)$: $$\alpha(c, z) \triangleq \frac{\sum_{g, i} \left(\sum_{s=1}^S c_{g, s, i}^2\right) \left(\sum_{t=1}^S \|z_{g, t, i}\|_2^2\right)}{S \sum_{g, t, i} c_{g, t, i}^2 \|z_{g, t, i}\|_2^2}, \qquad \gamma(c, z) \triangleq \frac{\sum_i \|g_i\|_2^2}{\sum_{g, t, i} c_{g, t, i}^2 \|z_{g, t, i}\|_2^2}, \qquad \gamma_{\text{grp}}(c, z) \triangleq \frac{\sum_{g, i} \|g_{g, i}\|_2^2}{\sum_i \|g_i\|_2^2},$$ the exact conditional relative parameter-gradient variance is governed by the identity: $$\text{RelVar}(\text{WM-S} \mid c, z) \triangleq \frac{\mathbb{E}\!\left[\left\|\widehat{\nabla}_{W_1} - \nabla_{W_1}\right\|_F^2 \;\middle|\; c, z\right]}{\left\|\nabla_{W_1}\right\|_F^2} = \frac{1}{K} \left[ \frac{\alpha(c, z) S - 2}{\gamma(c, z)} + \gamma_{\text{grp}}(c, z) \right].$$

Analysis of Failure Modes and Counterexamples: Because $\alpha(c, z)$, $\gamma(c, z)$, and $\gamma_{\text{grp}}(c, z)$ depend on the empirical distribution of $c$ and $z$, Proposition 3 implies conditional width-invariance at fixed $(\alpha, \gamma, \gamma_{\text{grp}})$ rather than an unconditional guarantee across arbitrary network states:

1. Gradient Cancellation Counterexample: If token gradients cancel across the sequence ($\gamma \to 0$), relative variance diverges. For $S=8, K=1, D=1, M=4$, inputs $x = (1, 2, 0 \dots)$, targets $\delta = (4, -1+\varepsilon, 0 \dots)$, $z = (2, 8, 0 \dots)$, $c = (4, -1+\varepsilon, 0 \dots)$, the exact gradient is $g = 8\varepsilon$ and cross-term $c_1 z_2 + c_2 z_1 = 30 + 2\varepsilon$. The exact conditional relative variance is $\frac{(30+2\varepsilon)^2}{64\varepsilon^2}$, which equals $140{,}812.5625$ at $\varepsilon = 0.01$ (verified by exhaustive enumeration over all 256 Rademacher states).
2. Cross-Token Support Mismatch Counterexample: Non-cancellation ($\gamma > 0$) alone is insufficient if $c$ and $z$ have mismatched temporal support within a micro-group ($\alpha \to \infty$). For $c = (1, \varepsilon, 0 \dots)$ and $z = (\varepsilon, 1, 0 \dots)$, both nonzero token gradients equal $\varepsilon$ (reinforcing, $g = 2\varepsilon, \sum c_t^2 z_t^2 = 2\varepsilon^2$, so $\gamma = 2.0$), but $c_1 z_2 = 1 \gg \varepsilon^2$, giving $\alpha \approx \frac{1}{16\varepsilon^2}$. The exact conditional relative variance is $\frac{(1+\varepsilon^2)^2}{4\varepsilon^2} = 2500.500025$ at $\varepsilon = 0.01$ for $K=1$, $357.214289$ for $K=4$ Co-WM-S, and $0.0$ for $K=8$ Co-WM-S.

Empirical Trajectory Audit of Conditional Quantities (verify_learned_prop3_scaling.py): To bridge Proposition 3 to actual training behavior with full reproducibility from an empty output file (python beyond_backprop/verify_learned_prop3_scaling.py --regenerate-all, saving multiwidth_learned_prop3_diagnostic.json), we calculate the conditional quantities $\alpha(c, z), \gamma(c, z), \gamma_{\text{grp}}(c, z)$, and $\text{RelVar}(\cdot \mid c, z)$ from frozen Layer 3 checkpoint tensors ($S=8$) on batches of shape $B=16, T=512$ ($N_{\text{tok}}=8,192$ tokens/batch) across widths $D \in \{128, 256, 384, 512\}$ under three explicitly documented protocols:

1. Benchmark-Schedule Holdout Validation Trajectory (benchmark_holdout_trajectory_4batch):
   • Training Recipe: 200 steps of fwd_streaming_cowm ($K_{\text{wm}}=4, K_{\text{qk}}=2, S=8$) on train_batches[0:200] drawn from shard_00000.parquet ($B=16, T=512$) using the exact benchmark optimizer schedule from run_pure_forward_only_suite.py (linear LR warmdown over the final 45% of steps, Muon momentum warmup $0.85 \to 0.95$ over 200 steps, and linear Muon weight decay decay $0.2(1 - \text{progress})$).
   • Evaluation Split: Evaluated at 5 checkpoints ($t \in \{0, 50, 100, 150, 200\}$) across 4 disjoint holdout validation batches (holdout_val_batches[0:4], shape $B=16, T=512, N_{\text{tok}}=8,192$) drawn from shard_06542.parquet (row groups $40 \dots 82$).
   • Ranges of 4-Batch Means: Across all $4 \times 5 = 20$ width-checkpoint states, the 4-batch means span $\alpha \in [0.8572 \pm 0.0063, \, 1.0033 \pm 0.0012]$, $\gamma \in [2.3604 \pm 0.5815, \, 8.1838 \pm 5.2378]$, and $\gamma_{\text{grp}} \in [0.1779 \pm 0.1155, \, 0.4410 \pm 0.0956]$. Consequently, the 4-batch mean conditional relative variances stay bounded in $[1.0448 \pm 0.6991, \, 2.7498 \pm 0.6188]$ for WM-S ($K=1$) and $[0.1492 \pm 0.0999, \, 0.3929 \pm 0.0884]$ for Co-WM-S ($K=4$). At $t=200$ (4-batch mean $\pm$ SD):
     • $D=128$: $\alpha = 0.9439 \pm 0.0154$, $\gamma = 3.3446 \pm 1.9805$, WM-S ($K=1$) $= 2.5773 \pm 1.4312$, Co-WM-S ($K=4$) $= 0.3682 \pm 0.2045$, Cut-K4 $= 33.44 \pm 18.69$;
     • $D=256$: $\alpha = 0.9414 \pm 0.0070$, $\gamma = 3.5498 \pm 2.1871$, WM-S ($K=1$) $= 2.5365 \pm 1.5310$, Co-WM-S ($K=4$) $= 0.3624 \pm 0.2187$, Cut-K4 $= 51.37 \pm 31.06$;
     • $D=384$: $\alpha = 0.9320 \pm 0.0053$, $\gamma = 4.0057 \pm 2.7904$, WM-S ($K=1$) $= 2.5661 \pm 1.8502$, Co-WM-S ($K=4$) $= 0.3666 \pm 0.2643$, Cut-K4 $= 78.25 \pm 56.20$;
     • $D=512$: $\alpha = 0.9902 \pm 0.0105$, $\gamma = 2.8972 \pm 0.8824$, WM-S ($K=1$) $= 2.5779 \pm 0.8397$, Co-WM-S ($K=4$) $= 0.3683 \pm 0.1199$, Cut-K4 $= 123.91 \pm 39.04$.
2. Fixed-Optimizer Diagnostic Trajectory (streaming_trajectory_4batch):
   • Training & Evaluation Recipe: 200 steps of fwd_streaming_cowm ($K_{\text{wm}}=4, K_{\text{qk}}=2, S=8$) on train_batches[0:200] (shard_00000.parquet, $B=16, T=512$) with fixed default Muon/AdamW optimizer parameters (setup_optimizer(use_muon=True) without LR/momentum/WD scheduling), evaluated at $t \in \{0, 50, 100, 150, 200\}$ on 4 held-out training-stream batches train_batches[200:204] ($B=16, T=512, N_{\text{tok}}=8,192$). Across all 20 states, the 4-batch means span $\alpha \in [0.9062 \pm 0.0079, \, 1.0162 \pm 0.0080]$, $\gamma \in [2.2698 \pm 0.3875, \, 5.6916 \pm 0.0958]$, conditional WM-S ($K=1$) $\text{RelVar} \in [1.2203, \, 2.8887]$, and conditional Co-WM-S ($K=4$) $\text{RelVar} \in [0.1743, \, 0.4127]$.
3. Single-Batch Table 2 States (step_0_init, step_200_learned, step_200_streaming_learned):
   • Calculated from frozen tensors under fixed default Muon/AdamW parameters on shard_00000.parquet ($B=16, T=512$): step_0_init on train_batches[0], step_200_learned on train_batches[199] after 200 steps of Cut-scope detached_oracle, and step_200_streaming_learned on train_batches[199] after 200 steps of fwd_streaming_cowm ($K=4$). To isolate the local MLP estimation variance of Proposition 3 around a deterministic surrogate target on genuine learned network states, evaluate_layer3_streaming_prop3 conditions on the exact deterministic detached_oracle readout pullback at that step, decoupling internal parameter-contraction variance from downstream attention/readout sampling noise.

| Width $D$ ($M=4D$) | State Regime | $\alpha$ (Support Alignment) | $\gamma$ (Global Coherence) | $\gamma_{\text{grp}}$ (Group Coherence) | Cut-K4 Cond. RelVar | WM-S ($K=1$) Cond. RelVar | Co-WM-S ($K=4$) Cond. RelVar | Co-WM-S ($K=8$) Cond. RelVar | GS-MAP-N1 RelBias | GS-MAP-N2 RelBias | | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | $D=128$ ($M=512$) | step_0_init | $0.9990$ | $3.4755$ | $0.2885$ | $37.05$ | $2.0127$ | $0.2875$ | $0.0000$ | $0.00017$ | $0.05833$ | | $D=128$ ($M=512$) | step_200_learned | $1.0250$ | $1.4638$ | $0.6859$ | $55.34$ | $4.9214$ | $0.7031$ | $0.0000$ | $0.00022$ | $0.16049$ | | $D=128$ ($M=512$) | step_200_streaming | $0.9746$ | $2.1800$ | $0.4589$ | $41.17$ | $3.1179$ | $0.4454$ | $0.0000$ | $0.00020$ | $0.16297$ | | $D=256$ ($M=1024$) | step_0_init | $1.0013$ | $3.2002$ | $0.3134$ | $79.86$ | $2.1915$ | $0.3131$ | $0.0000$ | $0.00019$ | $0.05574$ | | $D=256$ ($M=1024$) | step_200_learned | $0.9275$ | $1.5571$ | $0.6404$ | $103.94$ | $4.1213$ | $0.5888$ | $0.0000$ | $0.00022$ | $0.13396$ | | $D=256$ ($M=1024$) | step_200_streaming | $0.9752$ | $2.0925$ | $0.4753$ | $66.99$ | $3.2478$ | $0.4640$ | $0.0000$ | $0.00018$ | $0.14256$ | | $D=384$ ($M=1536$) | step_0_init | $1.0003$ | $3.6960$ | $0.2714$ | $103.77$ | $1.8953$ | $0.2708$ | $0.0000$ | $0.00017$ | $0.05414$ | | $D=384$ ($M=1536$) | step_200_learned | $1.0329$ | $2.0612$ | $0.4860$ | $146.12$ | $3.5248$ | $0.5035$ | $0.0000$ | $0.00020$ | $0.12289$ | | $D=384$ ($M=1536$) | step_200_streaming | $0.9730$ | $1.9493$ | $0.5142$ | $123.91$ | $3.4815$ | $0.4974$ | $0.0000$ | $0.00019$ | $0.12925$ | | $D=512$ ($M=2048$) | step_0_init | $1.0002$ | $3.3703$ | $0.2976$ | $151.85$ | $2.0783$ | $0.2969$ | $0.0000$ | $0.00019$ | $0.05394$ | | $D=512$ ($M=2048$) | step_200_learned | $0.9729$ | $2.1721$ | $0.4605$ | $152.76$ | $3.1231$ | $0.4462$ | $0.0000$ | $0.00018$ | $0.13607$ | | $D=512$ ($M=2048$) | step_200_streaming | $0.9951$ | $2.4223$ | $0.4139$ | $150.28$ | $2.8746$ | $0.4107$ | $0.0000$ | $0.00018$ | $0.12879$ |

Table 2: Conditional Proposition 3 quantities calculated from frozen Layer 3 tensors on ClimbMix ($B=16, T=512$, multiwidth_learned_prop3_diagnostic.json). While activation-space Cut-K4 conditional relative variance grows with width ($41.17 \to 150.28$ along the streaming trajectory), conditional WM-S ($K=1$) and Co-WM-S ($K=4, 8$) relative variances remain bounded across tested widths ($2.87\text{--}3.48$ and $0.41\text{--}0.50$, respectively, and $0.00$ at $K=8$).

2.4 Whole-Network Probing: Sequence-Aware Variance, Sensitivity Thresholds, and Ablations

Propositions 1–3 establish conditional sibling-neuron decoupling given a fixed local output discrepancy $c$. Eliminating conditional local variance does not make the whole-network gradient unbiased with respect to end-to-end backpropagation. In a causal sequence Transformer, the downstream map $F^{(m)}: \mathbb{R}^{T \times D} \to \mathbb{R}^{T \times D}$ from injection site $m \in \{0, \dots, 2L\}$ ($S_{\text{sites}} = 2L + 1 = 13$ for $L=6$) to the final normalized hidden state has a block-lower-triangular sequence-channel Jacobian $J^{(m)} \in \mathbb{R}^{(TD) \times (TD)}$, with blocks $J_{u, v}^{(m)} \in \mathbb{R}^{D \times D}$ non-zero for all causal token pairs $u \ge v$ and zero for $u < v$. Given top sensitivity $c = (c_1, \dots, c_T) \in \mathbb{R}^{T \times D}$, the exact transpose sensitivity of site $m$ at token $t$ is $u_t^{(m)} = \sum_{u=t}^T (J_{u, t}^{(m)})^\top c_u \in \mathbb{R}^D$.

Sequence-Aware Within-Site vs. Between-Site Variance of Depth-Hadamard vs. Independently Resampled Probing: Let $H_{\text{depth}} = 16$ be the smallest power of $2$ at least $S_{\text{sites}} = 13$, and let $\kappa(H_{\text{depth}}, K_{\text{net}}) = \frac{H_{\text{depth}} - K_{\text{net}}}{K_{\text{net}}(H_{\text{depth}} - 1)}$ ($\kappa = \frac{1}{15}$ at $H_{\text{depth}} = 16, K_{\text{net}} = 8$). Under Depth-Hadamard Carrier Modulation (depth_hadamard), site $m$ draws one Rademacher tensor $r^{(m)} \in \{-1, +1\}^{T \times D}$ per $H_{\text{depth}}$-probe cycle and injects $\xi^{(m, k)} = \sigma h_{m, c_k} r^{(m)}$. Because $h_{m, c_k}^2 \equiv 1$, the within-site contribution ($m' = m$) is invariant to $K_{\text{net}} \le H_{\text{depth}}$ within a cycle, whereas the between-site contribution ($m' \ne m$) is weighted by $\kappa(H_{\text{depth}}, K_{\text{net}})$. Under Independently Resampled Carriers (iid_resample), fresh i.i.d. tensors $r^{(m, k)}$ are drawn at every probe $k \in \{1, \dots, K_{\text{net}}\}$, dividing both within-site and between-site variances by $1 / K_{\text{net}}$.

For each return-aggregation mode, the linear-limit expectation and exact sequence-aware activation-credit variance $\operatorname{Var}(\hat{g}^{(m)}) = A_{\text{mode}}^{(m)} + \kappa_{\text{mode}} B_{\text{mode}}^{(m)}$ ($\kappa_{\text{mode}} = \kappa(H_{\text{depth}}, K_{\text{net}})$ for depth_hadamard, or dividing $A_{\text{mode}}^{(m)} + B_{\text{mode}}^{(m)}$ by $K_{\text{net}}$ for iid_resample) are:

1. Full-Sequence Scalar-Dot Probing (full_seq_dot, $n = TD$): The scalar return sums over all tokens and channels: $R_{\text{seq}}^{(k)} = \sum_{m'=0}^{2L} \sum_{u=1}^T \langle u_u^{(m')}, \xi_u^{(m', k)} \rangle$.
   • Linear-Limit Expectation: Unbiased for the full causal transpose, $\mathbb{E}[\hat{g}_{\text{full}, t}^{(m)}] = u_t^{(m)} = \sum_{u=t}^T (J_{u, t}^{(m)})^\top c_u$.
   • Exact Sequence-Wide Variance: Writing $\|u^{(m)}\|_2^2 = \sum_{t=1}^T \|u_t^{(m)}\|_2^2$, the single-probe within-site and between-site variance energies summed over all $n = TD$ coordinates are: $$A_{\text{full}}^{(m)} = (TD - 1) \left\|u^{(m)}\right\|_2^2, \qquad B_{\text{full}}^{(m)} = TD \sum_{m' \ne m} \left\|u^{(m')}\right\|_2^2.$$
2. Token-Local Channel-Dot Probing (token_dot): At token $t$, the token-local return sums only over the $D$ channels at position $t$, but because the perturbation $\xi^{(m', k)}$ is injected across the entire sequence, $(J^{(m')} \xi^{(m', k)})_t$ depends on perturbations at all earlier tokens $v \le t$: $$R_t^{(k)} = c_t^\top (\Delta x_{\text{final}}^{(k)})_t = \sum_{m'=0}^{2L} \sum_{v=1}^t \big\langle a_{t, v}^{(m')}, \xi_v^{(m', k)} \big\rangle, \qquad \text{where } a_{t, v}^{(m')} \triangleq \big(J_{t, v}^{(m')}\big)^\top c_t \in \mathbb{R}^D.$$
   • Linear-Limit Expectation: Multiplying by $\xi_t^{(m, k)} / \sigma^2$ eliminates $v < t$ in expectation ($\mathbb{E}[\xi_t \xi_v^\top] = 0$ for $v < t$), leaving only the diagonal token block and truncating all future-token causal blocks $u > t$: $$\mathbb{E}\!\left[\hat{g}_{\text{tok}, t}^{(m)}\right] = a_{t, t}^{(m)} = \big(J_{t, t}^{(m)}\big)^\top c_t \;\ne\; \sum_{u=t}^T \big(J_{u, t}^{(m)}\big)^\top c_u.$$
   • Exact Sequence-Aware Token-Dot Variance (Including Earlier-Token Causal Cross-Talk): Multiplication by $\xi_t^{(m, k)}$ does not remove the variance of the earlier-token terms $v < t$! Under depth_hadamard, $\xi_t^{(m, k)} \xi_v^{(m, k)\top} / \sigma^2 = r_t^{(m)} r_v^{(m)\top}$ for all $v \le t$, so the single-probe within-site error energy $A_{\text{tok}, t}^{(m)}$ and between-site error energy $B_{\text{tok}, t}^{(m)}$ about the token-local mean $a_{t, t}^{(m)}$ at token $t$ are: $$A_{\text{tok}, t}^{(m)} = (D - 1) \left\|a_{t, t}^{(m)}\right\|_2^2 + D \sum_{v < t} \left\|a_{t, v}^{(m)}\right\|_2^2, \qquad B_{\text{tok}, t}^{(m)} = D \sum_{m' \ne m} \sum_{v \le t} \left\|a_{t, v}^{(m')}\right\|_2^2.$$ Notice the second within-site term $D \sum_{v < t} \|a_{t, v}^{(m)}\|_2^2$: even at $D = 1$, where same-token channel cross-talk $(D-1)\|a_{t, t}^{(m)}\|_2^2 = 0$, earlier-token causal coupling ($v < t$) generates non-zero within-site variance that persists unattenuated across a fixed-carrier depth_hadamard cycle and divides by $1 / K_{\text{net}}$ under iid_resample!
   • Explicit Causal Counterexample (Verified in Gate 2H): For $T=2, D=1$ (single site) with $J = \begin{pmatrix} 1 & 0 \\ 3 & 4 \end{pmatrix}$ and $c = \begin{pmatrix} 5 \\ 7 \end{pmatrix}$, at token $t = 2$ we have $a_{2, 2} = 28$ and $a_{2, 1} = 21$. The second-token token_dot estimator is $r_2(21 r_1 + 28 r_2) = 28 + 21 r_1 r_2$. Its mean is $28$ (omitting future-token credit on token $1$, where $\mathbb{E}[\hat{g}_{\text{tok}, 1}] = 5 \ne 26$), and its exact variance about $28$ is $(1 - 1)(28^2) + 1(21^2) = 441.0$—which equals $441.0$ at every $K_{\text{net}} \in \{1, 2, 4\}$ under fixed-carrier depth_hadamard, and $441.0, 220.5, 110.25$ ($= 441.0 / K_{\text{net}}$) at $K_{\text{net}} = 1, 2, 4$ under iid_resample (verified in Gate 2H).
3. Coordinate-Wise Diagonal Probing (diag_jacobian): Evaluates $\xi_{t, d}^{(m, k)} c_{t, d} (\Delta x_{\text{final}}^{(k)})_{t, d} / \sigma^2$, with expectation $\mathbb{E}[\hat{g}_{\text{diag}, t, d}^{(m)}] = c_{t, d} J_{(t, d), (t, d)}^{(m)}$ (verified in Gate 2G: for $J = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, c = \begin{pmatrix} 5 \\ 7 \end{pmatrix}$, $\mathbb{E}_{\text{diag}} = \begin{pmatrix} 5 \\ 28 \end{pmatrix} \ne J^\top c = \begin{pmatrix} 26 \\ 38 \end{pmatrix}$) and single-probe within-site and between-site variance energies: $$A_{\text{diag}}^{(m)} = \sum_{t, d} c_{t, d}^2 \left( \sum_{d' \ne d} \big(J_{(t, d), (t, d')}^{(m)}\big)^2 + \sum_{v < t} \sum_{d'=1}^D \big(J_{(t, d), (v, d')}^{(m)}\big)^2 \right), \quad B_{\text{diag}}^{(m)} = \sum_{m' \ne m} \sum_{t, d} c_{t, d}^2 \sum_{v \le t} \sum_{d'=1}^D \big(J_{(t, d), (v, d')}^{(m')}\big)^2.$$

Sensitivity-Energy Condition for Depth-Hadamard vs. Independent Resampling ($B^{(m)} > 15 A^{(m)}$): At $H_{\text{depth}} = 16, K_{\text{net}} = 8$, comparing $\operatorname{Var}_{\text{DH}} = A^{(m)} + \frac{1}{15} B^{(m)}$ against $\operatorname{Var}_{\text{iid}} = \frac{1}{8}(A^{(m)} + B^{(m)})$ yields the exact necessary and sufficient condition: $$A^{(m)} + \frac{1}{15} B^{(m)} < \frac{1}{8}\big(A^{(m)} + B^{(m)}\big) \quad \Longleftrightarrow \quad B^{(m)} > 15 A^{(m)}.$$ Having $S_{\text{sites}} = 13$ sites does not by itself guarantee $B^{(m)} > 15 A^{(m)}$: if all 13 sites had equal sensitivity energy $\|u^{(m)}\|_2^2 = 1$, then $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)} = \frac{12 n}{n - 1} < 15$ for all $n \ge 6$ (for example, at $n = 512$ and unit site energy, $A = 511$ and $B = 6144$, so $\operatorname{Var}_{\text{DH}} = 920.6 > \operatorname{Var}_{\text{iid}} = 831.875$, making independent resampling better). To test whether $B^{(m)} > 15 A^{(m)}$ holds on our initialized 6-layer Transformer (c_proj_init_scale=0.5, where x0 and attention pathways carry larger sensitivity energy than individual MLP projections), run_pure_forward_only_suite.py --mode diag (pure_forward_width_diagnostic.json, site_sensitivity_audit) measures the exact 13-site sensitivity energies $E_m = \|\frac{\partial \mathcal{L}}{\partial s^{(m)}}\|_F^2$:

• Across widths $D \in \{128, 256, 384, 512\}$, the measured ratio $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)}$ is $20.08, 19.88, 20.05, 20.30$ at Layer 3 MLP ($m=8$) and $26.40, 26.15, 25.67, 26.31$ at Layer 5 MLP ($m=12$). This ratio specifically governs the whole-sequence return in full_seq_dot, where $B_{\text{full}}^{(m)} > 15 A_{\text{full}}^{(m)}$ holds due to the larger sensitivity energy carried by x0 and attention pathways relative to individual MLP projections; token_dot and diag_jacobian have their own distinct within-site and between-site propagation variance terms $A_{\text{tok}}, B_{\text{tok}}, A_{\text{diag}}, B_{\text{diag}}$ derived above. Because $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)} > 15$ at both MLP sites, the predicted DH-to-i.i.d. activation-credit variance ratio is $\frac{1 + 20.30/15}{(1 + 20.30)/8} = 0.8839 < 1$ at Layer 3 ($0.8068$ at Layer 5).
• In Table 3 ($D=512$), we report both activation-credit relative variance ($\operatorname{RelVar}(\widehat{\delta u}_{\text{mlp}}^{(3)})$) and accumulated parameter-gradient relative variance ($\operatorname{RelVar}(\widehat{\nabla}_{W_1}^{(3)})$): for full_seq_dot, depth_hadamard vs iid_resample gives $79,218.20$ vs $91,443.37$ on activation credit (ratio $0.866 \approx 0.884$) and $78,606.12$ vs $90,830.04$ on $\widehat{\nabla}_{W_1}^{(3)}$; for token_dot, $1191.29$ vs $1398.44$ on activation credit and $1180.58$ vs $1386.78$ on $\widehat{\nabla}_{W_1}^{(3)}$; and for diag_jacobian, $1.56$ vs $2.60$ on activation credit and $1.55$ vs $2.58$ on $\widehat{\nabla}_{W_1}^{(3)}$. Thus, while carrier reuse leaves within-site variance $A^{(m)}$ unaveraged within a 16-probe cycle, both depth_hadamard and iid_resample exhibit the same 4-order-of-magnitude variance separation across diag_jacobian ($\sim 1.6\text{--}2.6$), token_dot ($\sim 1.2\text{--}1.4 \times 10^3$), and full_seq_dot ($\sim 7.9\text{--}9.1 \times 10^4$).

Intra-Block Coupled Forward-Only Credit Assignment (Co-WM-S, CM-Attn-WH, Hadamard-QK): In our baseline uncoupled forward-only streaming surrogate (fwd_streaming_oracle_fullvocab), the block-local error $\delta_{\text{block}} = \frac{\partial \mathcal{L}(x_l)}{\partial x_l}$ was fed into both MLP and Attention sublayers independently, omitting the downstream MLP pullback into Attention: in exact block-local backpropagation, $x_l = x_{\text{after\_attn}} + \text{MLP}(\text{norm}(x_{\text{after\_attn}}))$, so the true sensitivity of attention output is $\delta u_{\text{attn}} = \delta_{\text{block}} + J_{\text{mlp}\circ\text{norm}}^\top \delta_{\text{block}}$. To close this gap strictly without a global backward sweep or autograd calls (@torch.no_grad(), bwd_calls = 0, fixed pass count independent of depth $L$ and width $M$), we introduce and ablate three coupled forward operators:

1. Upstream MLP Input Credit via Parameter-Space Co-WM-S (+MLP-Pullback): Evaluating the MLP update first inside block $l$, let $\delta a_{1, g, s} = \hat{g}_{g, s}^{\text{Co-WM-S}} \odot \phi'(a_{1, g, s}) \in \mathbb{R}^M$ be the pre-activation discrepancy on $W_1 \in \mathbb{R}^{M \times D}$. For each input channel $d \in \{1, \dots, D\}$ with column $\mathbf{w}_1^{(d)} = (W_1)_{:, d} \in \mathbb{R}^M$ and micro-group $\mathcal{S}_g$ ($S=8$), probe $k \in \{1, \dots, K\}$ injects $\xi_{g, s, d}^{(k)} = \sigma H_{s, c_k} r_{g, s, d}$, forms the probe-specific column displacement $\Delta \mathbf{w}_{1, \mathcal{S}_g}^{(d, k)} = \sum_{s \in \mathcal{S}_g} \delta a_{1, g, s} \xi_{g, s, d}^{(k)} \in \mathbb{R}^M$, contracts $G_{\mathcal{S}_g}^{(d, k)} = \mathbf{w}_1^{(d)} \cdot \Delta \mathbf{w}_{1, \mathcal{S}_g}^{(d, k)}$ (compute_linear_non_backprop_input_credit), and reconstructs: $$\widehat{\delta x}_{\text{mlp\_in}, g, t, d} = \frac{1}{K}\sum_{k=1}^K G_{\mathcal{S}_g}^{(d, k)} \frac{\xi_{g, t, d}^{(k)}}{\sigma^2}$$ without transposing $W_1$ in activation space or executing extra forward passes (exact at $K=S=8$, Gate 2J). Pulling $\widehat{\delta x}_{\text{mlp\_in}}$ across norm(x) ($\text{RMS}(x) = \sqrt{\frac{1}{D}\sum_{d=1}^D x_d^2 + \varepsilon}$, $\bar{x} = x / \text{RMS}(x)$) via _forward_rms_norm_directional_pull adds: $$\Delta_{\text{mlp}\to\text{attn}} = \frac{1}{\text{RMS}(x)} \left(I - \frac{1}{D}\bar{x}\bar{x}^\top\right) \widehat{\delta x}_{\text{mlp\_in}}$$ to $\delta_{\text{block}}$. In our retained $T=512$ ($n_{\text{win}}=8$) diagnostic on d6 (t512_intra_block_ablation in pure_forward_width_diagnostic.json), adding only this MLP pullback (fwd_streaming_oracle_mlp_pull_only_fullvocab) raises Layer 3 $W_O$ block-local cosine similarity from $0.93360$ to $1.00000$ ($0.93314 \to 1.00000$ at $T=64$).
2. Two-Level Window-Hadamard Forward Value Transport (CM-Attn-WH): At the pretraining sequence length $T=512$ ($n_{\text{win}}=8$ windows of $S_{\text{win}}=64$ tokens), single-window value probing ($K_v=1$) truncates cross-window causal value transport ($w_q > w_k$), causing Layer 3 $W_V$ block-local cosine to drop to $0.46853$ ($0.48951$ after MLP pullback). Multiplexing $S_{\text{win}}=64$ tokens per window via Sylvester-Hadamard matrix $H_{64}$ and $n_{\text{win}}=8$ windows via $H_8$ with causal window suffix summation over $w_q \ge w_k$ ($K_v = n_{\text{win}} = 8$, verified in Gate 2J) in fwd_streaming_oracle_mlp_whv_fullvocab restores $T=512$ Layer 3 $W_V$ block-local cosine from $0.48951$ to $1.00000$.
3. Post-RoPE Head-Channel Sylvester-Hadamard Query/Key Probing (Hadamard-QK): Constructing perturbations on post-RoPE normalized $Q/K$ using $H_{d_h}$ channel coordination ($d_h=64, K_{\text{qk}}=8$, $\sigma_{\text{qk}}=0.05$) and accumulating rewards in float32 (while converting perturbed $Q, K$ to work_dtype = bfloat16 inside CUDA FlashAttention) with fused causal future-query lookahead ($L_k=16, \rho=0.85$) raises $T=512$ Layer 3/5 $W_Q$ block-local cosine from $0.14129 / 0.13918$ (rel_var = 56.12) to $0.35023 / 0.35241$ (rel_var = 6.94) in fwd_streaming_oracle_coupled_fullvocab (and $0.34973 / 0.35366$ in full-vocabulary Co-WM-S fwd_streaming_cowm_coupled_fullvocab_K8).

Sequential and Component Ablation Ladder in Streaming Block-Local Supervision: To evaluate the effects of successive interventions and isolate each component of the intra-block coupled package on 1000-step 2-seed d6 ($8.192\text{M}$ tokens, Tables 4 and 5), we trace:

1. Inter-Block Sublayer Jacobian Omission at Fixed Final-State Discrepancy (backprop $1.45172 \to$ block_fixed_top_discrepancy_bp $1.62152$, $\Delta = +0.16980$ BPB): Isolates the cost of omitting intervening non-identity sublayer Jacobians ($l' > l$) under fixed final-state supervision $\delta_{\text{final}}$.
2. Replacing Fixed Final-State Discrepancy with Intermediate Block-State Supervision (block_fixed_top_discrepancy_bp $1.62152 \to$ block_local_greedy_bp_tophead $1.58085$, $\Delta = -0.04067$ BPB): Replaces $\delta_{\text{final}}$ with each block's own local-loss discrepancy $\frac{\partial \mathcal{L}(x_l)}{\partial x_l}$ (net $+0.12913$ BPB for greedy local supervision).
3. Shared-Readout Intermediate-Loss Mixture (block_local_greedy_bp_tophead $1.58085 \to$ block_local_greedy_bp $1.57236$, $\Delta = -0.00849$ BPB): Mixes $50\%$ final-block and $50\%$ earlier-block gradients into lm_head.
4. Baseline Uncoupled Intra-Block Surrogate Gap at Full Vocabulary (block_local_greedy_bp $1.57236 \to$ fwd_streaming_oracle_fullvocab $1.74582$, $\Delta = +0.17346$ BPB): Replaces exact intra-block autograd (bwd_calls = 6) with the uncoupled forward surrogate package (bwd_calls = 0, $K_v=1, K_{\text{qk}}=2$).
5. Component Ablation of the Intra-Block Coupled Package at Full Vocabulary ($V=8192$):
   • Component 5a (+MLP-Pullback Only): fwd_streaming_oracle_fullvocab ($1.74582$) $\to$ fwd_streaming_oracle_mlp_pull_only_fullvocab ($1.73609 \pm 0.00070$, $\Delta = -0.00973$ BPB, resolving $5.6\%$ of the $+0.17346$ BPB gap at zero extra forward passes).
   • Component 5b (+Two-Level Window-Hadamard Value Transport CM-Attn-WH, $K_v=8$): fwd_streaming_oracle_mlp_pull_only_fullvocab ($1.73609$) $\to$ fwd_streaming_oracle_mlp_whv_fullvocab ($1.71756 \pm 0.00097$, $\Delta = -0.01853$ BPB, cumulative $-0.02826$ BPB / $16.3\%$ of the gap).
   • Component 5c (+Post-RoPE Hadamard-QK, $K_{\text{qk}}=8$): fwd_streaming_oracle_mlp_whv_fullvocab ($1.71756$) $\to$ fwd_streaming_oracle_coupled_fullvocab ($1.64800 \pm 0.00172$, $\Delta = -0.06956$ BPB, cumulative $-0.09782$ BPB / $56.4\%$ of the gap).
   • Component 5d (Full-Vocabulary Parameter-Space Co-WM-S at $K=S=8$ vs. Analytic Linear Pullbacks): fwd_streaming_oracle_coupled_fullvocab ($1.64800$) $\to$ fwd_streaming_cowm_coupled_fullvocab_K8 ($1.64719 \pm 0.00067$, $\Delta = -0.00081$ BPB, resolving $56.9\%$ of the gap using 100% parameter-space Co-WM-S column contractions across all 4 linear sites per block!).
6. Sub-Vocabulary Readout Truncation ($V=8192 \to V_{\text{sub}}=256$): Under coupled attention, restricting readout pullback to $V_{\text{sub}}=256$ yields fwd_streaming_oracle_coupled ($1.78435 \pm 0.00103$, $\Delta = +0.13635$ BPB) and fwd_streaming_cowm_coupled_K8 ($1.78546 \pm 0.00297$, $\Delta = +0.00111$ BPB vs. analytic); under uncoupled attention, fwd_streaming_oracle ($1.86466 \pm 0.00221$) and fwd_streaming_cowm_K8 ($1.86518 \pm 0.00011$, $\Delta = +0.00052$ BPB).

2.5 MAP Settling: Single-Step Equivalence vs. Multi-Step Spectral Damping

In Post-Activation MAP Settling [1, Eq. 1, 5, 6, 7], post-activation discrepancy $e_1 = H^1 - \phi(W_1 x)$ minimizes: $$E(e_1) = \frac{\rho}{2} \|e_1\|_2^2 + \frac{1}{2} \|e_2(e_1)\|_2^2, \qquad e_2(e_1) = -\epsilon_{\text{clamp}} \hat{\delta}_y - e_1 V^\top, \quad e_1(0) = \mathbf{0}.$$

Proposition 4 (GS-MAP Spectral Filter and Settling Provenance): Under the recurrence $e_1(n+1) = e_1(n) - \eta (\rho e_1(n) - e_2(n) V)$ with non-oscillatory step size $0 < \eta \le \frac{1}{\rho + \|V\|_2^2}$, the exact $N$-step credit operator is: $$\hat{g}_h^{\text{GS-MAP}(N)} = (\delta_y V) \, \mathcal{F}_N(V^\top V), \qquad \text{where } \mathcal{F}_N(V^\top V) = \frac{\sum_{k=0}^{N-1} \big(I - \eta(\rho I + V^\top V)\big)^k}{\sum_{k=0}^{N-1} (1 - \eta \rho)^k}.$$

1. $N=1$ Equivalence: At $N = 1$, $\mathcal{F}_1(V^\top V) = I$, recovering exact local credit $\delta_y V$ ($\text{RelBias} \le 0.00022$ in Table 2).
2. Multi-Step Spectral Damping ($N \ge 2$): Rather than improving local credit accuracy, $N \ge 2$ applies a contractive spectral filter that damps top singular modes ($\text{RelBias} = 0.129\text{--}0.163$ in Table 2).

2.6 Strict No-Backward Credit Assignment: Faithful DUST vs. Delayed / Eligibility-Trace Coordinated Weight Maximization (Delayed-Trace-Co-WM-S)

While our streaming algorithms (fwd_streaming_cowm*, fwd_lookahead_2b*) eliminate the global reverse-depth sweep (num_depth_reverse_sweeps = 0), they still execute block-local reverse sensitivity pullbacks across activation/RMSNorm derivatives and intra-block sublayers. A strict no-backward algorithm must compute parameter updates without either a global reverse-depth sweep or local reverse derivative pullbacks.

Published Faithful DUST (dust_faithful.py) [5]: In published DUST (https://qlabs.sh/research/dust/ and literature/dust/dust.py), parameter updates are estimated by perturbing activation groups during forward execution (writers: attn.c_proj and mlp.c_proj; hidden: mlp.c_fc; hubs: o@layer attention head outputs; wte; and 256-column lm_head slabs; plus 2-sided scalar_gradients on resid_lambdas, x0_lambdas, ve_gate) with draw-chunk centered per-token loss rewards $\tilde{\ell}_{b, t}^{(p)} = \ell_{b, t}^{(p)} - \frac{1}{|\mathcal{C}|}\sum_{q \in \mathcal{C}} \ell_{b, t}^{(q)}$ (and future-token decayed per-head rewards $\gamma = 0.98$ inside local_errors for $W_Q, W_K, W_V$). Crucially:

1. Non-Zero c_proj Initialization: Because DUST perturbs mlp.c_fc (hidden) during forward execution and observes the loss shift through mlp.c_proj, zero-initializing c_proj (as in baseline_code/train.py) blocks hidden signals at step 0; published DUST initializes attn.c_proj and mlp.c_proj with $\text{std} = 1.5/\sqrt{d_{\text{in}}}$.
2. Prefix-Cached Suffix Execution vs. Hook Reruns: While the minimal reference script (literature/dust/dust.py) registers forward hooks and reruns the full network from wte, the DUST paper [5, §2.3] specifies prefix-cached suffix evaluation (blocks[l:]), which evaluates only blocks downstream of the perturbed block $l$ with bit-for-bit identical gradients (max_grad_diff = 0.0 at $P=256$, verified in Stage 1A).
3. Hidden-Width Scaling Bottleneck of Direct hidden Probing: When DUST perturbs all $M$ hidden neurons of mlp.c_fc simultaneously with $\xi_{b, t} \in \mathbb{R}^M$ ($\sigma = 0.2$), neuron $i$'s estimator correlates $\xi_{b, t, i}$ with a token reward containing the summed perturbation of all $M-1$ sibling neurons $\sum_{j \ne i} (\mathbf{v}^{(j)} \cdot \delta_{b, t}) \xi_{b, t, j}$. Consequently, DUST's mlp.c_fc gradient variance grows linearly with hidden neuron count $\mathcal{O}(M / P_{\text{hidden}})$.

Delayed / Eligibility-Trace Coordinated Weight Maximization (Delayed-Trace-Co-WM-S, delayed_trace_cowm.py): To eliminate DUST's $\mathcal{O}(M / P_{\text{hidden}})$ sibling-neuron interference on mlp.c_fc without direct hidden forward passes and without any local reverse/pullback step, we transfer parameter-space Co-WM-S into a three-phase eligibility-trace rule:

1. Phase 1 — Clean-Pass Local Forward Eligibility-Trace Recording (record_mlp_forward_eligibility_traces): During the single clean forward pass at block $l$, let $x_{g, s} \in \mathbb{R}^D$ be the pre-synaptic input to mlp.c_fc ($W_1 \in \mathbb{R}^{M \times D}$) and $z_{g, s} = W_1 x_{g, s} \in \mathbb{R}^M$ the clean pre-activation in micro-group $g \in \{1, \dots, G\}$ ($G = BT / S, S=8$). Using only local forward evaluations of the activation function $\phi(z) = \max(0, z)^2$: $$d_{g, s, i} \triangleq \delta h_{0, g, s, i} = \frac{\phi(z_{g, s, i} + \sigma_{\text{act}}) - \phi(z_{g, s, i} - \sigma_{\text{act}})}{2\sigma_{\text{act}}}, \qquad \tilde{h}_{g, s, i}^{(m)} = d_{g, s, i} \, \xi_{g, s}^{(m)},$$ where $\xi_{g, s}^{(m)} = H_{s, c_m} \in \{-1, +1\}$ ($m = 1, \dots, K_{\text{wm}}, K_{\text{wm}} \le S$) are shared Sylvester-Hadamard micro-horizon probes (64 KB per layer, since each neuron is already spatially decoupled from sibling neurons by its own outgoing column $\mathbf{v}^{(i)}$). The local pre-synaptic input eligibility trace for micro-group $g$ and probe $m$ is $e_g^{(m)} = \sum_{s=1}^S \xi_{g, s}^{(m)} x_{g, s} \in \mathbb{R}^D$ (retaining $d_{g, s, i}$ inside $\tilde{h}_{g, s, i}^{(m)}$ so the activation sensitivity enters the weight update exactly once), and record_mlp_forward_eligibility_traces simultaneously caches a detached pre-update snapshot of the outgoing reader matrix $V_{\text{snapshot}} = \texttt{V\_proj.detach().to(bfloat16).clone()} \in \mathbb{R}^{D \times M}$ alongside $(e_g, \tilde{h}_0)$.
2. Phase 2 — Outgoing Writer Forward Perturbation (Same-Step d0 or 1-Step Cross-Batch Delayed d1): The outgoing projection $V = \text{mlp.c\_proj} \in \mathbb{R}^{D \times M}$ obtains its forward writer reward vector $u_{g, s} \in \mathbb{R}^D$ from DUST's writers perturbation draws (either from the current batch d0 or from the previous step's buffered (e_g, \tilde{h}_0, V_{\text{snapshot}}, u_{\text{out}}) record under 1-tick delay d1 before overwriting). Zero direct hidden (mlp.c_fc) forward passes are executed (n_hidden_draws = 0).
3. Phase 3 — Outgoing-Column Contraction and Trace Modulation (contract_delayed_outgoing_columns_with_trace): For each micro-group $g$ and Hadamard probe $m$, neuron $i$ forms its virtual outgoing weight-column plastic displacement $\Delta \mathbf{v}_g^{(i, m)} = \sum_{s'=1}^S \tilde{h}_{g, s', i}^{(m)} u_{g, s'} = \sum_{s'=1}^S d_{g, s', i} \, \xi_{g, s'}^{(m)} u_{g, s'} \in \mathbb{R}^D$ from the buffered writer reward $u_{g, s'}$ and activation secant $\tilde{h}_{g, s', i}^{(m)}$, contracts its saved pre-update outgoing column $\mathbf{v}^{(i)} = (V_{\text{snapshot}})_{:, i} \in \mathbb{R}^D$ with $\Delta \mathbf{v}_g^{(i, m)}$ (acting as a saved-reader/secant column contraction, distinct from both a post-Muon weight-matrix subtraction $\Delta V = W_{\text{proj}, t} - W_{\text{proj}, t-1}$ and the multi-clock post-installation live-reader packet valuation of [13]): $$c_g^{(i, m)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)} = \sum_{s'=1}^S d_{g, s', i} \big(\mathbf{v}^{(i)} \cdot u_{g, s'}\big) \xi_{g, s'}^{(m)} \;\in\; \mathbb{R},$$ and accumulates its mlp.c_fc row update by modulating the pre-synaptic input trace $e_g^{(m)}$ (mod_accum.T @ x_g in delayed_trace_cowm.py lines 171–174): $$\widehat{\nabla}_{W_{1, i}} = \frac{1}{K_{\text{wm}}} \sum_{m=1}^{K_{\text{wm}}} \sum_{g=1}^G c_g^{(i, m)} e_g^{(m)} = \sum_{g=1}^G \sum_{s=1}^S \left(\frac{1}{K_{\text{wm}}} \sum_{m=1}^{K_{\text{wm}}} c_g^{(i, m)} \xi_{g, s}^{(m)}\right) x_{g, s}.$$ Exact Conditional Recovery vs. Finite Precision and Writer-Reward Noise:
   • Exact Algebraic Recovery ($K_{\text{wm}} = S$): By Sylvester-Hadamard completeness ($\frac{1}{S}\sum_{m=1}^S \xi_{g, s'}^{(m)} \xi_{g, s}^{(m)} = \mathbf{1}[s' = s]$), the reconstructed pre-activation modulation satisfies the exact single-sensitivity identity: $$\frac{1}{S}\sum_{m=1}^S c_g^{(i, m)} \xi_{g, s}^{(m)} \equiv d_{g, s, i} \big(\mathbf{v}^{(i)} \cdot u_{g, s}\big), \qquad \widehat{\nabla}_{W_{1, i}}\Big|_{K_{\text{wm}}=S} \equiv \sum_{g=1}^G \sum_{s=1}^S d_{g, s, i} \big(\mathbf{v}^{(i)} \cdot u_{g, s}\big) x_{g, s}.$$ (For example, at $S = K_{\text{wm}} = 8$ with scalar $x_s = u_s = v = 1$ and $z_s = 1 \implies d_s = 2$, both the displayed formula and contract_delayed_outgoing_columns_with_trace evaluate to $\sum_{s=1}^8 2 \cdot 1 \cdot 1 = 16.0$, verified in Gate 2L.)
   • Finite-Precision Tensor-Core Agreement: In delayed_trace_cowm.py, $x_g, \tilde{h}_0, V_{\text{snapshot}}, u_g$ are stored in bfloat16 for Tensor-Core torch.bmm throughput and accumulated in float32, yielding conditional cosine similarity 0.999997 against the bfloat16 reference in Gate 2L (1.000000 rounded to 6 decimals in Table 7's conditional panel).
   • Remaining Writer-Reward Estimation Noise: When $u_{g, s}$ is estimated from a finite number $P_{\text{writers}}$ of forward writer draws rather than the exact local target $\delta_{g, s}$, $K_{\text{wm}} = S = 8$ eliminates the internal MLP column-contraction variance ($\text{Var}[\widehat{\nabla}_{W_1} \mid u] = 0$), while the remaining whole-network variance is governed solely by the $D$-dimensional outgoing writer estimate $u_{g, s}$ (independent of hidden neuron count $M$).
4. Phase 4 — Strict No-Backward Attention Hub Centering (hub2, hub_mode="dust_hub", rev=0) vs. Hybrid Local Attention-Hub Pullback (Hub-Co-WM-S, hub_mode="co_wm_s", rev=6):
   • Strict No-Backward Hub-Balanced Allocation (hub2, delayed_trace_cowm_*, hub_mode="dust_hub", local_reverse_derivative_calls = 0): In published DUST [5] at $P=64$ (n_layer=6), the attention hub allocation hubs = [2, 2, 1, 1, 1, 1] assigns only $1$ suffix forward draw (draws = 1) to attention hubs o@2, o@3, o@4, o@5. Because draws = 1 cannot be sample-mean centered across draws (delayed_trace_cowm.py line 696), four of the six attention hubs in dust_faithful_muon_P64_sc_sh fall back to uncentered 1-sample REINFORCE. Because Delayed-Trace-Co-WM-S eliminates all $27$ hidden (mlp.c_fc) suffix forward draws ([9, 8, 4, 3, 2, 1]), reallocating just $4$ of those $27$ saved draws to hubs (hubs = [2, 2, 2, 2, 2, 2] $= 12$ draws, hub2) and $23$ to writers (writers = [12, 10, 8, 7, 6, 6] $= 49$ draws; total $49 + 12 + 3 = 64$ direct draws) in delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 ensures every attention hub [email protected] has $\ge 2$ draws—activating full-block covariance centering (sc) across all $6$ hubs while retaining DUST's exact forward hub perturbations (site_kind="hub") and $\gamma$-decayed ($\gamma = 0.98$) local attention perturbations (causal_v_trace=False), with zero reverse pullbacks (local_reverse_derivative_calls = 0).
   • $W_O$-Free Forward Causal Value Pooling (record_attn_v_forward_pooled_input) and Concurrent Work [14]: Because $v_{b, \tau}^{(h)} = W_V^{(h)} x_{\text{in}, b, \tau}$ is linear, Fubini's theorem commutes $W_V^{(h)}$ with forward causal attention: $$o_{b, t}^{(h)} = \sum_{\tau \le t} A_{b, h, t, \tau} \big(W_V^{(h)} x_{\text{in}, b, \tau}\big) = W_V^{(h)} \underbrace{\left(\sum_{\tau \le t} A_{b, h, t, \tau} x_{\text{in}, b, \tau}\right)}_{\triangleq \, \bar{x}_{b, t}^{(h)} \in \mathbb{R}^D}.$$ This forward attention-filtered value trace identity was also independently derived in Proposition 2 of concurrent work general_pretraining_20/v3 (and v1–v2) [14] (published on rStation within the concurrency window). More generally, for any exponential recency factor $\gamma \in (0, 1]$, centered rescaling $w_\tau = \gamma^{-(\tau - T/2)}$ satisfies $w_t^{-1} w_\tau \equiv \gamma^{t - \tau}$ and evaluates $\bar{x}_{b, t}^{(\gamma, h)} = \sum_{\tau \le t} \gamma^{t - \tau} A_{b, h, t, \tau} x_{\\text{in}, b, \tau}$ in a single forward FlashAttention pass (record_attn_v_recency_damped_pooled_input in test_recency_vtrace_and_subspace_writers.py). Under hub_mode="dust_hub", reassembling $\bar{x}_{b, t}^{(\gamma, h)}$ with DUST's 2-draw forward hub error $\hat{u}_{\text{hub}, b, t}^{(h)} = \texttt{targets[layer]}_{b, t}^{(h)}$ at o@layer requires zero $W_O$ involvement (local_reverse_derivative_calls = 0) and achieves higher single-trial gradient cosine against exact backpropagation across two 50-step ClimbMix d6 seeds (s42 and s137, $R=16$ trials each; recency_vtrace_and_subspace_diagnostic.json): on s42, (0.056872, 0.047611, 0.063110) at $\gamma=1.00$ and (0.039419, 0.030410, 0.049562) at $\gamma=0.98$ across Layers $0, 2, 5$ vs (0.004103, 0.008465, 0.002831) for DUST's 16-draw local_errors("v") (and on s137, (0.062903, 0.046011, 0.069297) at $\gamma=1.00$ and (0.043221, 0.037514, 0.052396) at $\gamma=0.98$ vs (0.008707, 0.008334, 0.011840) for local_errors("v")). On seed s42, our controlled 200-step d6 ablations (Table 10) show that the +0.03409 BPB gap between delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1 (2.16374 BPB, (q, k) = (4, 7)) and delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (2.12965 BPB, causal_v_trace=False, (q, k) = (1, 4)) separates into -0.01443 BPB from restoring matched (q, k) = (1, 4) (2.14931 holdout BPB in the freshly regenerated --run-pretrain execution at $\gamma=1.00$, -0.01592 BPB to 2.14782 at $\gamma=0.98$, and -0.01693 BPB to 2.14681 in an earlier standalone $\gamma=1.00$ run; the 0.002502 BPB difference between the two fixed-seed s42 $\gamma=1.00$ executions reflects fixed-seed execution variability across separate process launches—possibly arising from parallel floating-point atomic accumulation in Triton/FlashAttention kernels—and is distinct from training-seed variability across s42 and s137) and a residual +0.01716 to +0.01966 BPB gap on s42. Crucially, replicating the 200-step comparison on seed s137 (--run-seed137-replication in test_recency_vtrace_and_subspace_writers.py) reveals that on s137, matched (q, k) = (1, 4) value pooling actually outperforms both delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (2.13793 BPB) and dust_faithful_muon_P64_sc_sh (2.13353 BPB), reaching 2.13220 holdout BPB at $\gamma=1.00$ and 2.13081 holdout BPB at $\gamma=0.98$ (yielding descriptive two-seed mean $\pm$ sample standard deviation of 2.13932 ± 0.01203 for vtrace_q1k4_g098 vs 2.13379 ± 0.00585 for causal_v_trace=False and 2.13485 ± 0.00187 for dust_faithful_muon_P64_sc_sh). Furthermore, we evaluate a 6-arm, 2-seed (s42, s137) Hub-DoF $\times$ Value-Trace & Co-Perturbation screen at unified equivalent_full_network_forwards_prefix_cached = 58.5 (--run-vtrace-hub-dof-factorial, vtrace_hub_dof_and_coperturb_2seed_200_d6 in recency_vtrace_and_subspace_diagnostic.json, with per-seed runtime-verified estimator_class). Because raising hubs at fixed $P=64$ (49 + 12 + 3 = 64 at hub2, 43 + 18 + 3 = 64 at hub3, 37 + 24 + 3 = 64 at hub4) simultaneously reduces writers, this screen evaluates the joint allocation packages (sep_hub3 at writers=[10,9,7,6,6,5], hubs=[3]*6 and sep_hub4 at writers=[9,8,6,5,5,4], hubs=[4]*6, both using RecencyAndSubspaceEstimator) alongside writer-matched hub-MLP co-perturbation (co_hub3_mlp at writers=[10,9,7,6,6,5], hubs=[3]*6, using CoHubMLPDelayedTraceEstimator). Under vtrace_q1k4_g098, moving from hub2 (2.13932 ± 0.01203, s42 = 2.14782, s137 = 2.13081) to separate-hub sep_hub3 (2.13716 ± 0.00566, s42 = 2.14116, s137 = 2.13316) and sep_hub4 (2.13670 ± 0.00067, s42 = 2.13622, s137 = 2.13717) improves s42 holdout BPB and yields smaller observed two-seed spreads across the tested separate-hub packages, whereas under causal_v_trace=False, sep_hub3 performs well (2.13414 ± 0.00216, s42 = 2.13261, s137 = 2.13566) while sep_hub4 degrades on s42 (2.15000 ± 0.01905, s42 = 2.16347, s137 = 2.13653). By contrast, when CoHubMLPDelayedTraceEstimator (co_hub3_mlp) co-perturbs mlp.c_proj ($\sigma=0.2$) inside the $K_h=3$ attention-hub passes (where upper blocks $l \ge L/2 = 3$ perturb o@l at $\sigma_{\text{hub}}=0.4$), it degrades 200-step 2-seed holdout BPB relative to sep_hub3 (2.14574 ± 0.00137 with s42 = 2.14670, s137 = 2.14477 at causal_v_trace=False, and 2.15305 ± 0.02329 with s42 = 2.13658, s137 = 2.16952 at vtrace_q1k4_g098); on the 50-step diagnostic at matched writers=[10,9,7,6,6,5] and $\sigma=0.2$, co_hub3_mlp_isotropic yields slightly lower Layer 4 mlp.c_proj / mlp.c_fc Frobenius cosine (0.056728 / 0.062050, post-NS5 0.024587 / 0.036723) than matched separate-hub sep_hub3_isotropic (0.059047 / 0.064006, post-NS5 0.027081 / 0.040218), and we hypothesize that pooling writer and higher-variance hub passes ($\sigma_{\text{hub}}=0.4$ on $l \ge 3$) contributes to this closed-loop degradation. While both 50-step seeds exhibit lower single-trial spectral effective rank under value pooling (single_trial_erank = (128.52, 94.63, 87.24) on s42 and (133.97, 93.80, 87.24) on s137 vs (217.37, 197.56, 192.41) and (217.86, 197.07, 187.68) for local_errors("v")), the s137 pretraining reversal (+0.01817 BPB on s42 vs -0.00711 BPB on s137 at $\gamma=0.98$) confirms that the 50-step effective-rank measurements are diagnostic associations rather than a deterministic penalty on 200-step endpoints; while two seeds with a sign reversal do not establish a reliable population ordering or a stable variance advantage, the lower observed two-seed mean and approximately half the sample standard deviation at hub2 (0.00585 vs 0.01203, sample variance ratio $\approx 0.24$) motivate retaining causal_v_trace=False as our primary strict no-backward configuration.
   • Hybrid Local Attention-Hub Pullback Control (Hub-Co-WM-S, hub_mode="co_wm_s", local_reverse_derivative_calls = L = 6): In contract_hub_outgoing_columns (hub_mode="co_wm_s"), micro-group $g$ forms $\Delta W_g = H^\top U_g$ from the current attn.c_proj writer rewards $U_g \in \mathbb{R}^{S \times D}$, multiplies $C_g = \Delta W_g W_O$, and reconstructs the $(B, T, D)$ upstream activation-credit tensor $\widehat{E}_g = \frac{1}{K_{\text{wm}}} H C_g$ to supply targets[layer] to local_errors on the same step (and similarly in contract_delayed_outgoing_columns_with_attn_v_trace). At the implemented complete Hadamard basis $K_{\text{wm}} = S = 8$, $\frac{1}{S} H H^\top = I_S$ yields the exact identity: $$\Delta W_g = H^\top U_g, \qquad C_g = \Delta W_g W_O, \qquad \widehat{E}_g = \frac{1}{S} H C_g = U_g W_O.$$ For the forward linear map $Y = O W_O^\top$, $U_g W_O$ is the exact 1-layer local linear reverse pullback of the output credit $U_g$ across $W_O$, executed in the Sylvester-Hadamard token basis (CosSim = 0.999998 against u_attn @ W_O in Gate 2M). Accordingly, all hub_mode="co_wm_s" variants (delayed_trace_full_* and delayed_trace_unified_*) record local_reverse_derivative_calls = L = 6 (is_pure_forward_only = False, has_local_backward_phase = True) and are presented strictly as hybrid local-hub-pullback controls quantifying the effect of replacing forward o@layer hub suffix passes with a 1-layer local linear pullback across $W_O$.
5. SRAM-Fused Tensor-Core Outgoing-Column Contraction Kernel (_outgoing_col_dot_all8_kernel): In a naive PyTorch implementation, forming $\Delta V_g^{(m)} = u_{g, m} \tilde{h}_0^\top \in \mathbb{R}^{G \times D \times M}$ via torch.bmm and contracting $(V \odot \Delta V_g^{(m)}).\text{sum}(\text{dim}=1)$ across $K_{\text{wm}}=8$ probes materializes $8 \times G \times D \times M$ intermediate tensors in HBM (247.4 ms across 6 layers at $D=256, M=4096$). In delayed_trace_cowm.py (lines 125–244), _outgoing_col_dot_all8_kernel fuses the outer-product displacement and column dot product across all $K_{\text{wm}}=8$ Hadamard probes inside a single Triton kernel: each program loads a $(B_D \times B_M)$ tile of $V_{\text{snapshot}}$ into SRAM registers once, computes $\Delta V_g^{(1\dots 8)}$ via tl.dot on Tensor Cores, multiplies by $V_{\text{snapshot}}$ in SRAM, and accumulates the $8$ scalar column contractions without ever writing $(G, D, M)$ to HBM—reducing 6-layer contraction latency at $M=4096$ from 247.4 ms to 18.7–19.1 ms ($13.0\times$ faster, cos = 0.9999986).
6. Fairly Optimized Shared Backward-Free Mechanisms (sc, sh, bd025, st), Upper-Block Writer Perturbation Scaling (wsig04), and Negative Controls on Highway-Prior Cascade (PooledHighwayCascadeEstimator vs. Guided ES [15]) & Prefix-Only State Predictors: To ensure complete experimental fairness when improving residual-writer and readout estimation, we implement four strictly backward-free mechanisms in delayed_trace_cowm.py (local_reverse_derivative_calls = 0, same_step_reverse_block_cascade = False) and apply the shared writer/head improvements (sc+sh) identically to both Fairly Optimized DUST (dust_faithful_muon_P64_sc_sh, dust_faithful_muon_P256_sc_sh) and Strict No-Backward Delayed-Trace-Co-WM-S (delayed_trace_cowm_*_sc_sh_*, as well as the hybrid delayed_trace_unified_* controls):
   • Full-Block Sample Covariance Centering (simplex_centering=True, sc): Centers per-token loss rewards $\tilde{\ell}_{b, t}^{(p)} = \ell_{b, t}^{(p)} - \frac{1}{K_w}\sum_{q=1}^{K_w} \ell_{b, t}^{(q)}$ across all $K_w \ge 2$ draws at a site rather than $n=2$ pairs, eliminating finite-chunk baseline variance.
   • Target-Separated Antithetic Hadamard Vocabulary Probing (stratified_head=True, sh): Evaluates an exact 1D forward finite difference on the observed target coordinate $y_{b, t}$ (logits.scatter_add_) alongside vectorized antithetic Sylvester-Hadamard probes on non-target 256-column vocabulary slabs (0 softmax derivatives), raising lm_head cosine from 0.214 to 0.946 and wte cosine from 0.156 to 0.625.
   • Intra-Block Attention-MLP Branch Decoupling (branch_decouple_alpha=0.25, bd025): In standard DUST writers draws, attn.c_proj noise $\xi_{\text{attn}}$ and mlp.c_proj noise $\xi_{\text{mlp}}$ are summed into the same residual stream $h_{\text{mid}} + \xi_{\text{attn}} + \text{MLP}(\text{norm}(h_{\text{mid}} + \xi_{\text{attn}})) + \xi_{\text{mlp}}$, creating anti-correlated common-mode variance between $\hat{u}_{\text{attn}}$ and $\hat{u}_{\text{mlp}}$. Setting branch_decouple_alpha=0.25 routes h_mid_pert = h_mid + xi_attn only into the forward MLP branch block.mlp(_norm(h_mid_pert)) while keeping the direct skip connection at unperturbed h_mid_clean, then recombines $\hat{u}_{\text{attn}} = \hat{u}_{\text{mlp}} + \alpha \hat{u}_{\text{via-mlp}}$ (0 backward calls, 0 local reverse pullbacks), lifting Layer 3 mlp.c_fc cosine by +71.6% (0.0775 -> 0.1330).
   • Forward Residual-Stream Scalar Eligibility Trace (scalar_trace=True, st): Replaces the $16$ full-network ES forward passes for resid_lambdas and x0_lambdas with local forward inner products between clean block states $(x_{\text{in}, l}, x_0)$ and writer_errors_attn[l], saving $16$ full forward passes per step (~110–180 ms/step).
   • Depth-Scaled Upper-Block Writer Perturbation (UpperWriterSigma04Estimator, wsig04 in test_recency_vtrace_and_subspace_writers.py and upper_writer_sigma=0.4 in delayed_trace_cowm.py): Published DUST scales attention-hub perturbation amplitude from $\sigma=0.2$ on lower blocks ($l < L/2$) to $\sigma=0.4$ on upper blocks ($l \ge L/2$, dust_faithful.py line 610), but keeps writers fixed at $\sigma=0.2$ across all blocks (line 597). On our 50-step d6 diagnostic (weight_gradient_matrix_comparison in recency_vtrace_and_subspace_diagnostic.json), comparing sep_hub3_isotropic ($\sigma=0.2$ at Layer 4, $K_w=6$) against isotropic_simplex ($\sigma=0.4$ at Layer 4, $K_w=6$) shows that raising upper-block writer $\sigma$ from $0.2$ to $0.4$ increases Layer 4 mlp.c_proj, attn.c_proj, and mlp.c_fc Frobenius cosines from 0.059047, 0.050063, and 0.064006 to 0.102320 ($+73.3\%$), 0.120362 ($+140.4\%$), and 0.105265 ($+64.5\%$) (post-NS5 0.027081 -> 0.042437, 0.025888 -> 0.040559, 0.040218 -> 0.063609). Testing UpperWriterSigma04Estimator (writer_sigma = 0.2 if l < L // 2 else 0.4, rev=0, same_step_reverse_block_cascade=False, zero extra latency) across 200-step, 1000-step, 1680-step ($P=64$ matched-wall-clock), and 270-step ($P=256$ matched-wall-clock) 2-seed (s42, s137) ClimbMix d6 pretraining (upper_writer_sigma04_2seed_200_d6 and dust_and_delayed_trace_1000steps_d6.json, Tables 9–10) improves holdout BPB across all ten Delayed-Trace-Co-WM-S seed-configuration runs as well as the six matched DUST wsig04 control runs: (1) at 200 steps, hub2_wsig04_P64_bd025_sc_sh_d1 reaches 2.13297 ± 0.00587 (s42=2.12882, s137=2.13713 vs 2.13379 ± 0.00585 at $\sigma=0.2$), sep_hub3_wsig04_P64_bd025_sc_sh_d1 reaches 2.13354 ± 0.00249 (s42=2.13178, s137=2.13530 vs 2.13414 ± 0.00216), and dust_faithful_muon_P64_wsig04_sc_sh reaches 2.13438 ± 0.00022 (s42=2.13453, s137=2.13422 vs 2.13485 ± 0.00187); (2) at 1000 steps (Table 9), delayed_trace_cowm_P64_hub2_wsig04_bd025_sc_sh_d1 improves 2-seed holdout BPB by -0.00550 BPB (2.06189 ± 0.00021 -> 2.05639 ± 0.00057, s42=2.05599, s137=2.05680), whereas dust_faithful_muon_P64_wsig04_sc_sh improves by -0.00037 BPB (2.04590 ± 0.00012 -> 2.04553 ± 0.00074, s42=2.04501, s137=2.04605; because DUST's writers=[9,8,4,3,1,1] and the reallocated trace recipes writers=[12,10,8,7,6,6] / [9,8,4,3,2,2] also differ in upper-block writer draw counts alongside trace contraction, this contrast shows that the reallocated trace recipe benefits strongly from wsig04 rather than isolating trace contraction alone at identical upper-block writer draws); (3) in the 1680-step $P=64$ matched-wall-clock comparison (637.3 s / 638.8 s vs 644.4 s / 646.5 s for dust_faithful_muon_P64_wsig04_sc_sh and 648.0 s / 649.3 s for dust_faithful_muon_P64_sc_sh), delayed_trace_cowm_P64_fast_w22_wsig04_sc_sh_st_d1_wc648s improves by -0.00295 BPB (2.04412 ± 0.00014 -> 2.04117 ± 0.00010, s42=2.04124, s137=2.04110, beating both dust_faithful_muon_P64_sc_sh by -0.00473 BPB and dust_faithful_muon_P64_wsig04_sc_sh by -0.00436 BPB); and (4) at $P=256$, delayed_trace_cowm_P256_wsig04_bd025_sc_sh_st_d1_wc307s ($270$ steps, 295.5 s / 298.6 s) improves by -0.00613 BPB (2.08637 ± 0.01282 -> 2.08024 ± 0.00817, s42=2.07447, s137=2.08602, beating both dust_faithful_muon_P256_sc_sh_wc307s 2.11372 ± 0.00121 by -0.03348 BPB and dust_faithful_muon_P256_wsig04_sc_sh_wc307s 2.11334 ± 0.00074 at 303.2 s / 308.1 s by -0.03310 BPB).
   • Negative Controls — Same-Step Reverse-Block Highway-Prior Cascade (PooledHighwayCascadeEstimator / PooledHighwayCascadeAttnEstimator vs. Guided ES [15]) and Prefix-Only Local State Predictors (test_state_predictive_residual.py): Because Transformer blocks share a residual identity highway ($h_{l+1} = h_l + f_l(h_l)$), exact token-wise writer gradients exhibit strong inter-block alignment ($\mathbb{E}_{b, t}[\cos^2(g_{l, b, t}^{\text{mlp}}, g_{l+1, b, t}^{\text{mlp}})] \in [0.8636, 0.9841]$ across Blocks $0\dots 4$ on a 50-step ClimbMix d6 state vs $1/D = 0.002604$ isotropic random; highway_prior_alignment_and_probing_diagnostic_50step_d6 in recency_vtrace_and_subspace_diagnostic.json). Following the subspace-guided probing principle of Guided Evolutionary Strategies (Maheswaranathan et al., ICML 2019 [15])—which augments isotropic exploration with a low-dimensional surrogate-gradient subspace—we test whether scheduling prefix-cached suffix perturbations from block $L-1$ down to $0$ (backward_calls = 0, local_reverse_derivative_calls = 0, but same_step_reverse_block_cascade = True) can improve writer probing by aligning $1$ Hadamard-decoupled simplex contrast (c_m = (+1,-1,+1,-1,0,...)/2 for mlp, c_a = (+1,+1,-1,-1,0,...)/2 for attn, $\langle c_m, c_a \rangle = 0$) at block $l < L-1$ with the exponentially pooled upper-block direction $\hat{v}_{\text{pool}, b, t}^{(l+1)} = \text{normalize}(\text{normalize}(\hat{u}_{l+1, b, t}^{\text{src}}) + 0.7 \hat{v}_{\text{pool}, b, t}^{(l+2)})$ (pooling $\hat{u}_{\text{mlp}}$ in PooledHighwayCascadeEstimator, pool_source="mlp", vs $\hat{u}_{\text{attn}}$ in PooledHighwayCascadeAttnEstimator, pool_source="attn") while sampling the remaining $(K_w - 2)$ contrasts isotropically in $P_\perp = I_D - \hat{v}_{\text{pool}} \hat{v}_{\text{pool}}^\top$. First, we distinguish reverse-depth scheduling from derivative pullbacks: because block $l$ waits for block $l+1$'s perturbation readout on the same step (L-1 -> 0), PooledHighwayCascade* introduces a same-step reverse-depth dependency (same_step_reverse_block_cascade = True) and therefore does not remove backward/reverse-depth locking, unlike our strict no-backward delayed_trace_cowm_* estimators (same_step_reverse_block_cascade = False). Second, our 50-step diagnostic and 200-step 2-seed (s42, s137) pretraining runs (highway_cascade_pretrain_2seed_200_d6) expose a critical divergence between single-step cosine and closed-loop Muon optimization: on the frozen 50-step d6 state ($R=8$ trials), upweighting the 1-DoF highway prior lifts token-wise prior alignment to $\mathbb{E}[\cos^2] = 0.015557$ at Layer 0 (0.014958 for v2_attn_b07, $5.97\times$ isotropic $1/D$) and increases single-step cosines under both hub2 (use_co_hub_mlp=False, Layer 0 u_mlp cosine 0.061619 -> 0.098361, mlp.c_fc Frobenius cosine 0.097258 -> 0.163767, post-NS5 0.069003 -> 0.106601) and co_hub3_mlp (use_co_hub_mlp=True, Layer 0 mlp.c_fc Frobenius cosine 0.090128 -> 0.137637 for pool_source="mlp" and 0.134855 for pool_source="attn"). However, in 200-step 2-seed closed-loop pretraining (per-seed estimator_class verified at 58.5 equivalent forwards), both highway_cascade_b07_co_hub3_mlp_P64_bd025_sc_sh_d1 (PooledHighwayCascadeEstimator, pool_source="mlp", 2.14819 ± 0.00060 holdout BPB, s42 = 2.14776, s137 = 2.14862, 638.2 ms/step) and highway_cascade_v2_b07_co_hub3_mlp_P64_bd025_sc_sh_d1 (PooledHighwayCascadeAttnEstimator, pool_source="attn", 2.14990 ± 0.00421 holdout BPB, s42 = 2.14693, s137 = 2.15288, 639.1 ms/step; and 2.15210 ± 0.01824 / 2.15303 ± 0.02207 under vtrace_q1k4_g098) underperform separate-hub delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (2.13379 ± 0.00585 at 611.8 ms/step) and sep_hub3_P64_bd025_sc_sh_d1 (2.13414 ± 0.00216 at 608.4 ms/step), incurring +25.8 ms/step cascade scheduling overhead relative to co_hub3_mlp (612.4 ms/step) and +29.8 ms/step total package overhead relative to sep_hub3. Within the tested configurations and 200-step horizon, we hypothesize that strongly upweighting a noisy 1D surrogate prior ($\cos^2 \approx 0.0067\text{--}0.0156$; cf. Guided ES [15])—combined with co_hub3_mlp co-perturbation—introduces persistent directional bias under closed-loop Muon orthogonalization relative to isotropic simplex probing (unbiased for the linearized first-order directional derivative conditioned on the clean state). Complementarily, test_state_predictive_residual.py (state_predictive_residual_diagnostic.json) decomposes the sample gradient at block $l$ into the model-distribution push-down term $g_l^{(\text{model})} = J_l^\top p_\theta(c)$ and the observed-target pull-up term $g_l^{(\text{target})} = -J_l^\top e_Y$ (which contains both $-J_l^\top q(c)$ and the centered target innovation $-J_l^\top(e_Y - q(c))$, with $J_l$ also depending on $\tau > t$ when $l < L-1$). On the 50-step ClimbMix d6 state, a half-batch linear ridge predictor from $h_{l, t}$ predicts future forward state increments $h_{L-1, t} - h_{l, t}$ at 0.8120–0.9055 holdout token cosine (Layers 0..4), predicts $g_l^{(\text{model})}$ at +0.5091 to +0.7329 holdout cosine (Layers 0..5), and achieves +0.0753 to +0.0876 holdout cosine on $g_l^{(\text{target})}$, yet yields -0.011141 .. +0.013314 holdout cosine on $g_l = g_l^{(\text{model})} + g_l^{(\text{target})}$. Because ridge regression is linear in the response ($\hat{G}_{\text{total}} = \hat{G}_{\text{model}} + \hat{G}_{\text{target}}$), inspecting the held-out prediction norms (holdout_prediction_cancellation_step50 vs step0) shows two effects: (i) between step 0 and step 50, anti-alignment between $\hat{G}_{\text{model}}$ and $\hat{G}_{\text{target}}$ increases from $\cos_F \in [-0.0434, -0.0166]$ to $[-0.2604, -0.2322]$ (reducing squared norm $\|\hat{G}_{\text{total}}\|_F^2 / (\|\hat{G}_{\text{model}}\|_F^2 + \|\hat{G}_{\text{target}}\|_F^2)$ from 0.998–1.000 to 0.865–0.882, a 11.8–13.5% reduction in squared norm or 6.1–7.0% in Frobenius norm), and (ii) because $\|g_l^{(\text{target})}\|_F^2$ is $6.9\times\text{--}8.6\times$ larger than $\|g_l^{(\text{model})}\|_F^2$ (1.109–1.141 vs 0.129–0.165 of $\|g_l\|_F^2$), $\|\hat{G}_{\text{target}}\|_F^2$ is $11\times\text{--}13\times$ larger than $\|\hat{G}_{\text{model}}\|_F^2$ (0.212–0.580 vs 0.0176–0.0519), so $\hat{G}_{\text{total}}$ is dominated by the poorly generalizing target fit $\hat{G}_{\text{target}}$ (-0.0111 .. +0.0132 cosine at step 50; +0.0147 .. +0.0194 at step 0).

---

3. Experimental Validation

3.1 Whole-Transformer Initialization Diagnostic and Unbiased Bias/Variance Decomposition

Table 3 reports gradient fidelity against analytical backpropagation across widths $D \in \{128, 256, 384, 512\}$ ($L=6, B=4, T=64, V=4096$) over $R = 32$ independent random trials (pure_forward_width_diagnostic.json).

Unbiased Squared Relative Bias vs. Plug-In Sample-Mean Bias: For any stochastic gradient estimator $\hat{G}^{(r)}$ ($r = 1, \dots, R$) with sample mean $\bar{G}_R = \frac{1}{R}\sum_{r=1}^R \hat{G}^{(r)}$, expectation $\mu = \mathbb{E}[\hat{G}]$, and reference gradient $G_{\text{bp}}$, the naive plug-in squared relative bias $\widehat{\text{RelBias}}^2_{\text{plug}} = \frac{\|\bar{G}_R - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}$ has expectation: $$\mathbb{E}\!\left[\widehat{\text{RelBias}}^2_{\text{plug}}\right] = \frac{\|\mu - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2} + \frac{1}{R} \text{RelVar}(\hat{G}), \qquad \text{where } \widehat{\text{RelVar}} = \frac{1}{R - 1} \sum_{r=1}^R \frac{\|\hat{G}^{(r)} - \bar{G}_R\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$ When $\text{RelVar}$ is large, $\widehat{\text{RelBias}}^2_{\text{plug}}$ is dominated by residual sample-mean variance $\text{RelVar}/R$ rather than true systematic bias. To eliminate this contamination:

1. We compute Total Relative MSE directly from the $R=32$ individual trial errors (with standard error of the mean across trials): $$\widehat{\text{TotalMSE}} = \frac{1}{R} \sum_{r=1}^R \frac{\|\hat{G}^{(r)} - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$
2. We report the unbiased bias-corrected squared relative bias estimator (with Tukey leave-one-out jackknife standard error across the $R=32$ trials): $$\widehat{\text{RelBias}}^2_{\text{unb}} = \widehat{\text{RelBias}}^2_{\text{plug}} - \frac{1}{R} \widehat{\text{RelVar}}, \qquad \mathbb{E}\!\left[\widehat{\text{RelBias}}^2_{\text{unb}}\right] = \frac{\|\mu - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$ By algebraic identity, $\widehat{\text{TotalMSE}} = \frac{R-1}{R}\widehat{\text{RelVar}} + \widehat{\text{RelBias}}^2_{\text{plug}} \equiv \widehat{\text{RelVar}} + \widehat{\text{RelBias}}^2_{\text{unb}}$. When $\widehat{\text{RelBias}}^2_{\text{unb}}$ is comparable to or smaller than its jackknife standard error (as in full_seq_dot and token_dot), no nonzero bias is statistically resolved, and its square root should not be interpreted as a resolved bias magnitude.

| Algorithm | Paradigm / Carrier & Readout | $D=128$ Top / Mid Cos | $D=512$ Top / Mid Cos | Mid Scale ($D=512$) | Mid Param RelVar ($D=128 \to 512$) | Mid Unbiased $\text{RelBias}^2$ ($D=512$) | Mid TotalMSE ($D=512$) | Attn Q5 Cos / RelVar ($D=512$) | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | fwd_global_scalar_K2 | Global Scalar Control ($K=2$) | -0.002 / 0.002 | 0.000 / -0.000 | $1007.95$ | $7.79\times 10^6 \to 3.26\times 10^7$ | $-2250.8 \pm 10136.6$ | $3.26\times 10^7 \pm 4.50\times 10^6$ | -0.0003 / $1.83\times 10^8$ | | fwd_token_uwm_K2 | Token-Scalar ($K=2$) | 0.004 / 0.007 | 0.004 / 0.003 | $56.74$ | $26,968 \to 103,931$ | $-29.98 \pm 26.68$ | $103,901 \pm 1075$ | 0.0002 / $5.84\times 10^5$ | | fwd_two_level_cowm_K8_fullseq | Two-Level ($K=8$, full_seq_dot, DH) | 0.003 / 0.006 | 0.003 / 0.003 | $49.85$ | $21,803 \to 78,606$ | $27.44 \pm 27.86$ | $78,634 \pm 8449$ | 0.0012 / $4.88\times 10^6$ | | fwd_two_level_cowm_K8_fullseq_iid | Two-Level ($K=8$, full_seq_dot, i.i.d.) | 0.005 / 0.009 | 0.001 / 0.003 | $53.07$ | $22,705 \to 90,830$ | $-23.07 \pm 36.45$ | $90,807 \pm 3178$ | 0.0004 / $5.85\times 10^6$ | | fwd_two_level_cowm_K8_tokendot | Two-Level ($K=8$, token_dot, DH) | 0.052 / 0.059 | 0.026 / 0.029 | $6.12$ | $303.50 \to 1180.58$ | $-0.453 \pm 0.422$ | $1180.13 \pm 39.33$ | 0.0042 / $69,297$ | | fwd_two_level_cowm_K8_tokendot_iid | Two-Level ($K=8$, token_dot, i.i.d.) | 0.051 / 0.054 | 0.024 / 0.028 | $6.67$ | $350.00 \to 1386.78$ | $0.088 \pm 0.455$ | $1386.87 \pm 7.47$ | 0.0042 / $89,860$ | | fwd_two_level_cowm_K4 | Two-Level ($K=4$, diag_jacobian, DH) | 0.412 / 0.355 | 0.410 / 0.365 | $0.96$ | $4.30 \to 4.13$ | $0.224 \pm 0.003$ | $4.35 \pm 0.27$ | 0.0560 / $330.05$ | | fwd_two_level_cowm_K8_iid | Two-Level ($K=8$, diag_jacobian, i.i.d.) | 0.481 / 0.421 | 0.479 / 0.426 | $0.93$ | $2.64 \to 2.58$ | $0.224 \pm 0.002$ | $2.81 \pm 0.00$ | 0.0649 / $216.35$ | | fwd_two_level_cowm_K8 | Two-Level ($K=8$, diag_jacobian, DH) | 0.598 / 0.512 | 0.615 / 0.516 | $0.91$ | $1.61 \to 1.55$ | $0.222 \pm 0.002$ | $1.77 \pm 0.07$ | 0.0837 / $136.49$ | | fwd_streaming_cut_K4 | Streaming Cut ($K=4$, Token-Scalar) | 0.016 / 0.015 | 0.004 / 0.003 | $47.76$ | $4638 \to 73,120$ | $-4.43 \pm 23.70$ | $73,116 \pm 815$ | 0.0037 / $8.28\times 10^5$ | | fwd_streaming_wm_K1 | Streaming WM-S ($K=1$, Indep., $K_{\text{qk}}=1$) | 0.129 / 0.109 | 0.125 / 0.112 | $1.80$ | $69.28 \to 68.66$ | $0.244 \pm 0.018$ | $68.90 \pm 0.18$ | 0.0135 / $6375.22$ | | fwd_streaming_wm_K1_qk2 | Streaming WM-S ($K=1$, Indep., $K_{\text{qk}}=2$) | 0.127 / 0.107 | 0.126 / 0.110 | $1.80$ | $69.29 \to 68.43$ | $0.249 \pm 0.022$ | $68.68 \pm 0.16$ | 0.0153 / $3156.59$ | | fwd_streaming_wm_K4 | Streaming WM-S ($K=4$, Indep.) | 0.363 / 0.318 | 0.362 / 0.322 | $1.15$ | $7.19 \to 7.15$ | $0.246 \pm 0.003$ | $7.39 \pm 0.01$ | 0.0516 / $374.23$ | | fwd_streaming_cowm_K4 | Streaming Co-WM-S ($K=4$, Had.) | 0.496 / 0.438 | 0.498 / 0.442 | $1.09$ | $3.29 \to 3.26$ | $0.251 \pm 0.002$ | $3.51 \pm 0.00$ | 0.0688 / $197.17$ | | fwd_streaming_wm_K8 | Streaming WM-S ($K=8$, Indep.) | 0.533 / 0.465 | 0.532 / 0.471 | $1.09$ | $2.78 \to 2.74$ | $0.253 \pm 0.002$ | $2.99 \pm 0.01$ | 0.0731 / $173.44$ | | fwd_streaming_cowm_K8 | Streaming Co-WM-S ($K=8$, Had., Uncoupled) | 0.999 / 0.879 | 0.998 / 0.881 | $1.05$ | $0.00 \to 0.00$ | $0.251 \pm 0.000$ | $0.25 \pm 0.00$ | 0.1397 / $47.85$ | | fwd_streaming_cowm_coupled_K8 | Coupled Co-WM-S ($K=8$, $V_{\text{sub}}=256$) | 0.999 / 0.879 | 0.998 / 0.881 | $1.05$ | $0.00 \to 0.00$ | $0.251 \pm 0.000$ | $0.25 \pm 0.00$ | 0.3513 / $6.93$ | | fwd_streaming_cowm_coupled_fullvocab_K8 | Coupled Co-WM-S ($K=8$, $V=4096$) | 1.000 / 0.879 | 1.000 / 0.882 | $1.05$ | $0.00 \to 0.00$ | $0.249 \pm 0.000$ | $0.25 \pm 0.00$ | 0.3523 / $6.90$ | | fwd_streaming_oracle | Analytic Surrogate ($V_{\text{sub}}=256$, Uncoupled) | 0.999 / 0.879 | 0.998 / 0.881 | $1.04$ | $0.00 \to 0.00$ | $0.250 \pm 0.000$ | $0.25 \pm 0.00$ | 0.1413 / $48.11$ | | fwd_streaming_oracle_coupled | Analytic Surrogate ($V_{\text{sub}}=256$, Coupled) | 0.999 / 0.879 | 0.998 / 0.881 | $1.04$ | $0.00 \to 0.00$ | $0.250 \pm 0.000$ | $0.25 \pm 0.00$ | 0.3518 / $6.85$ | | fwd_streaming_oracle_fullvocab | Analytic Surrogate ($V=4096$, Uncoupled) | 1.000 / 0.879 | 1.000 / 0.882 | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | 0.1378 / $47.31$ | | fwd_streaming_oracle_mlp_pull_only_fullvocab | +MLP-Pullback Only ($V=4096, K_v=1$) | 1.000 / 0.879 | 1.000 / 0.882 | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | 0.1468 / $52.55$ | | fwd_streaming_oracle_mlp_whv_fullvocab | +MLP-Pullback + CM-Attn-WH ($V=4096, K_v=8$) | 1.000 / 0.879 | 1.000 / 0.882 | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | 0.1458 / $52.16$ | | fwd_streaming_oracle_coupled_fullvocab | Analytic Surrogate ($V=4096$, Coupled) | 1.000 / 0.879 | 1.000 / 0.882 | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | 0.3525 / $6.86$ | | block_local_greedy_bp_vsub256 | Block-Local Greedy BP ($V_{\text{sub}}=256$) | 0.999 / 0.879 | 0.998 / 0.881 | $1.04$ | $0.00 \to 0.00$ | $0.249 \pm 0.000$ | $0.25 \pm 0.00$ | 0.9996 / $0.00$ | | block_local_greedy_bp | Block-Local Greedy BP [8] ($V=4096$) | 1.000 / 0.879 | 1.000 / 0.882 | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | 1.0000 / $0.00$ | | block_fixed_top_discrepancy_bp | Fixed-Top-Discrepancy BP ($V=4096$) | 1.000 / 0.881 | 1.000 / 0.882 | $0.88$ | $0.00 \to 0.00$ | $0.221 \pm 0.000$ | $0.22 \pm 0.00$ | 1.0000 / $0.00$ |

Table 3: Whole-Transformer width-scaling diagnostic ($L=6, B=4, T=64, V=4096, R=32$ trials; DH = depth_hadamard carrier modulation, i.i.d. = iid_resample per-probe carrier resampling). At $D=512$, Layer 3 activation-credit relative variances (rel_var_act_mid_mlp in pure_forward_width_diagnostic.json) closely track the parameter-gradient relative variances shown above: $79,218.20$ (DH) vs $91,443.37$ (i.i.d.) for full_seq_dot, $1191.29$ (DH) vs $1398.44$ (i.i.d.) for token_dot, and $1.56$ (DH) vs $2.60$ (i.i.d.) for diag_jacobian. The measured full-sequence sensitivity-energy ratio $B_{\text{full}}^{(8)}/A_{\text{full}}^{(8)} = 20.30 > 15$ quantitatively explains the full_seq_dot DH-vs-i.i.d. variance ratio ($0.866 \approx 0.884$ theoretical), whereas token_dot and diag_jacobian are governed by their own within-site and between-site variance expressions ($A_{\text{tok}}, B_{\text{tok}}, A_{\text{diag}}, B_{\text{diag}}$ in Section 2.4). Furthermore, at $D=512$, Layer 3 block-local attention cosines (cossim_blkref_mid_attn_{o,v}) rise from $0.914 / 0.916$ (fwd_streaming_cowm_K8) to $0.987 / 0.987$ under sub-vocabulary coupling (fwd_streaming_cowm_coupled_K8), and from $0.927 / 0.926$ (fwd_streaming_oracle_fullvocab) to $1.000 / 1.000$ under full-vocabulary coupling (fwd_streaming_cowm_coupled_fullvocab_K8 at $0.99998 / 0.99997$ and fwd_streaming_oracle_coupled_fullvocab at $1.00000 / 1.00000$), while Hadamard-QK lifts Attn Q5 cosine from $0.138$ to $0.353$ and reduces relative variance by $85.5\%$ ($47.31 \to 6.86$).

3.2 200-Step Multi-Width Pretraining Sweep

Table 4 reports the 200-step pretraining sweep across $D \in \{128, 256, 384, 512\}$ ($B=16, T=512, V=8192$) on ClimbMix evaluated on the dev split (row groups $0 \dots 14$) and holdout split (row groups $40 \dots 82$). All candidate forward-only methods update 100% of trainable parameters under @torch.no_grad() (bwd_calls = 0).

| Algorithm | $D=128$ Dev / Holdout BPB | $D=256$ Dev / Holdout BPB | $D=384$ Dev / Holdout BPB | $D=512$ Dev / Holdout BPB | Step ms / Peak VRAM ($D=512$) | Backward Calls | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | backprop (Standard) | 1.8852 / 1.8846 | 1.8410 / 1.8393 | 1.8349 / 1.8320 | 1.8160 / 1.8133 | 33.3 ms / 3010.4 MB | $1$ | | block_fixed_top_discrepancy_bp | 1.9266 / 1.9262 | 1.8873 / 1.8868 | 1.8753 / 1.8723 | 1.8664 / 1.8643 | 33.7 ms / 3026.5 MB | $6$ ($1$/block) | | block_local_greedy_bp_tophead | 1.9207 / 1.9193 | 1.8787 / 1.8767 | 1.8742 / 1.8715 | 1.8566 / 1.8538 | 73.5 ms / 2153.6 MB | $6$ ($1$/block) | | block_local_greedy_bp (Mixed Head [8]) | 1.9156 / 1.9140 | 1.8809 / 1.8789 | 1.8664 / 1.8649 | 1.8469 / 1.8448 | 76.7 ms / 2169.6 MB | $6$ ($1$/block) | | fwd_global_scalar_K2 | 2.6507 / 2.6507 | 2.3906 / 2.3910 | 2.4085 / 2.4076 | 2.3913 / 2.3898 | 71.5 ms / 3864.8 MB | $0$ | | fwd_token_uwm_K2 | 2.2138 / 2.2142 | 2.1956 / 2.1943 | 2.2131 / 2.2121 | 2.1883 / 2.1868 | 90.6 ms / 3864.8 MB | $0$ | | fwd_streaming_cut_K4 | 2.1682 / 2.1682 | 2.1379 / 2.1372 | 2.1543 / 2.1530 | 2.1121 / 2.1126 | 130.2 ms / 1820.3 MB | $0$ | | fwd_streaming_wm_K1 ($K_{\text{qk}}=1$) | 2.1310 / 2.1306 | 2.0942 / 2.0928 | 2.0752 / 2.0750 | 2.0680 / 2.0673 | 148.8 ms / 1828.4 MB | $0$ | | fwd_streaming_wm_K1_qk2 ($K_{\text{qk}}=2$) | 2.1296 / 2.1301 | 2.0935 / 2.0931 | 2.0750 / 2.0738 | 2.0665 / 2.0659 | 150.9 ms / 1820.3 MB | $0$ | | fwd_streaming_wm_K4 | 2.1073 / 2.1079 | 2.0717 / 2.0714 | 2.0549 / 2.0541 | 2.0471 / 2.0468 | 299.9 ms / 1820.3 MB | $0$ | | fwd_streaming_cowm_K4| 2.0975 / 2.0981 | 2.0635 / 2.0627 | 2.0468 / 2.0467 | 2.0392 / 2.0383 | 303.5 ms / 1828.4 MB | $0$ | | fwd_two_level_cowm_K4| 2.1060 / 2.1052 | 2.0750 / 2.0756 | 2.0622 / 2.0617 | 2.0559 / 2.0553 | 469.4 ms / 1850.3 MB | $0$ | | fwd_streaming_wm_K8 | 2.0964 / 2.0963 | 2.0627 / 2.0629 | 2.0488 / 2.0484 | 2.0387 / 2.0392 | 498.4 ms / 1820.3 MB | $0$ | | fwd_streaming_cowm_K8 (Uncoupled, $V_{\text{sub}}=256$) | 2.0803 / 2.0820 | 2.0502 / 2.0507 | 2.0373 / 2.0378 | 2.0287 / 2.0291 | 507.4 ms / 1828.4 MB | $0$ | | fwd_streaming_cowm_coupled_K8 ($V_{\text{sub}}=256$) | 2.0533 / 2.0537 | 2.0216 / 2.0210 | 2.0043 / 2.0041 | 1.9985 / 1.9972 | 833.3 ms / 1828.3 MB | $0$ | | fwd_streaming_cowm_coupled_fullvocab_K8 ($V=8192$) | 1.9578 / 1.9578 | 1.9319 / 1.9312 | 1.9190 / 1.9178 | 1.8952 / 1.8946 | 1811.3 ms / 2100.3 MB | $0$ | | fwd_streaming_oracle ($V_{\text{sub}}=256$, Uncoupled) | 2.0805 / 2.0822 | 2.0519 / 2.0517 | 2.0383 / 2.0387 | 2.0308 / 2.0302 | 99.7 ms / 1820.3 MB | $0$ | | fwd_streaming_oracle_coupled ($V_{\text{sub}}=256$) | 2.0556 / 2.0562 | 2.0223 / 2.0218 | 2.0032 / 2.0032 | 1.9975 / 1.9969 | 163.6 ms / 1828.3 MB | $0$ | | fwd_streaming_oracle_fullvocab ($V=8192$, Uncoupled) | 1.9876 / 1.9866 | 1.9686 / 1.9671 | 1.9573 / 1.9565 | 1.9514 / 1.9505 | 112.4 ms / 2100.4 MB | $0$ | | fwd_streaming_oracle_mlp_pull_only_fullvocab ($V=8192$) | 1.9847 / 1.9844 | 1.9653 / 1.9643 | 1.9572 / 1.9559 | 1.9433 / 1.9428 | 116.1 ms / 2100.3 MB | $0$ | | fwd_streaming_oracle_mlp_whv_fullvocab ($V=8192$) | 1.9807 / 1.9800 | 1.9572 / 1.9557 | 1.9505 / 1.9489 | 1.9346 / 1.9327 | 134.1 ms / 2100.3 MB | $0$ | | fwd_streaming_oracle_coupled_fullvocab ($V=8192$) | 1.9586 / 1.9583 | 1.9319 / 1.9312 | 1.9185 / 1.9176 | 1.8956 / 1.8950 | 177.7 ms / 2100.3 MB | $0$ |

Table 4: 200-step multi-width pretraining sweep on ClimbMix (pure_forward_pretrain_scaling.json). Across all widths $D \in \{128 \dots 512\}$, each component of the intra-block coupled package (+MLP-Pullback $\to$ +CM-Attn-WH $\to$ +Hadamard-QK) monotonically reduces holdout BPB across every width ($1.9505 \to 1.9428 \to 1.9327 \to 1.8950$ at $D=512$), while full-vocabulary Parameter-Space Co-WM-S (fwd_streaming_cowm_coupled_fullvocab_K8, $1.8946$ at $D=512$) closely matches its analytic-linear surrogate (1.8950).

3.3 1000-Step Two-Seed Benchmark on d6 ($8.192\text{M}$ Tokens)

Table 5 reports 1000-step ($8,192,000$ tokens, 2 seeds s42 and s137) pretraining on d6 ($D=384, M=1536, L=6, V=8192$) evaluated on the document-disjoint holdout split (pure_forward_1000steps_d6.json). Trainable parameters ($26,346,060$) are identical across all arms. Attention probing uses $K_{\text{qk}}=2$ ($66$ attention calls per step across $6$ blocks) for all uncoupled forward streaming arms except the original fwd_streaming_wm_K1 which evaluated $K_{\text{qk}}=1$ ($42$ calls; matched by our newly executed benchmark arm fwd_streaming_wm_K1_qk2 at $K_{\text{qk}}=2$), while fwd_streaming_oracle_mlp_whv_fullvocab uses $K_v=8, K_{\text{qk}}=2$ ($108$ calls) and full coupled arms evaluate $K_v=8, K_{\text{qk}}=8$ ($252$ calls).

| Algorithm | Seed s42 Holdout BPB | Seed s137 Holdout BPB | Mean Holdout BPB $\pm$ Sample SD | Mean Step ms | Peak VRAM | Backward Calls | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | backprop (Standard) | 1.45158 | 1.45186 | 1.45172 ± 0.00020 | 25.1 ms | 2656.3 MB | $1$ | | backprop_ckpt (Checkpointed) | 1.45224 | 1.45134 | 1.45179 ± 0.00064 | 33.3 ms | 1731.4 MB | $1$ | | block_fixed_top_discrepancy_bp (Fixed $\delta_{\text{final}}$) | 1.61470 | 1.62833 | 1.62152 ± 0.00964 | 26.2 ms | 2668.4 MB | $6$ ($1$/block) | | block_local_greedy_bp_tophead (Local $\delta_l$, Top Head) | 1.57477 | 1.58694 | 1.58085 ± 0.00861 | 64.7 ms | 2034.3 MB | $6$ ($1$/block) | | block_local_greedy_bp (Mixed Head [8]) | 1.57272 | 1.57200 | 1.57236 ± 0.00051 | 66.6 ms | 2046.3 MB | $6$ ($1$/block) | | fwd_global_scalar_K2 (Global-Scalar Control) | 2.52995 | 2.36593 | 2.44794 ± 0.11598 | 56.1 ms | 3448.2 MB | $0$ | | fwd_token_uwm_K2 (100% Params Updated) | 2.09448 | 2.10184 | 2.09816 ± 0.00521 | 72.6 ms | 3448.2 MB | $0$ | | fwd_streaming_cut_K4 (Matched $K=4$) | 2.01759 | 2.01353 | 2.01556 ± 0.00287 | 112.9 ms | 1757.0 MB | $0$ | | fwd_streaming_wm_K1 ($K=1, K_{\text{qk}}=1$) | 1.95285 | 1.95252 | 1.95268 ± 0.00023 | 125.3 ms | 1765.1 MB | $0$ | | fwd_streaming_wm_K1_qk2 ($K=1, K_{\text{qk}}=2$) | 1.95103 | 1.95365 | 1.95234 ± 0.00185 | 132.4 ms | 1757.0 MB | $0$ | | fwd_streaming_wm_K4 (Indep. Signs, $K=4$) | 1.91576 | 1.90640 | 1.91108 ± 0.00662 | 236.3 ms | 1757.0 MB | $0$ | | fwd_streaming_cowm_K4 (Hadamard, $K=4$) | 1.89369 | 1.89344 | 1.89356 ± 0.00018 | 243.0 ms | 1765.1 MB | $0$ | | fwd_two_level_cowm_K4 | 1.97564 | 1.97840 | 1.97702 ± 0.00196 | 334.7 ms | 1791.4 MB | $0$ | | fwd_streaming_wm_K8 (Indep. Signs, $K=8$) | 1.88734 | 1.88884 | 1.88809 ± 0.00106 | 377.7 ms | 1757.0 MB | $0$ | | fwd_streaming_cowm_K8 ($K=8, V_{\text{sub}}=256$, Uncoupled) | 1.86510 | 1.86525 | 1.86518 ± 0.00011 | 386.9 ms | 1765.1 MB | $0$ | | fwd_streaming_cowm_coupled_K8 ($K=8, V_{\text{sub}}=256$, Coupled) | 1.78336 | 1.78756 | 1.78546 ± 0.00297 | 600.0 ms | 1763.0 MB | $0$ | | fwd_streaming_cowm_coupled_fullvocab_K8 ($K=8, V=8192$, Coupled) | 1.64671 | 1.64767 | 1.64719 ± 0.00067 | 1391.3 ms | 2031.0 MB | $0$ | | fwd_streaming_wm_map_N4| 1.89510 | 1.89014 | 1.89262 ± 0.00350 | 265.1 ms | 1765.1 MB | $0$ | | fwd_streaming_oracle ($V_{\text{sub}}=256$ Uncoupled) | 1.86622 | 1.86309 | 1.86466 ± 0.00221 | 94.6 ms | 1757.0 MB | $0$ | | fwd_streaming_oracle_coupled ($V_{\text{sub}}=256$ Coupled) | 1.78362 | 1.78508 | 1.78435 ± 0.00103 | 150.2 ms | 1763.0 MB | $0$ | | fwd_streaming_oracle_fullvocab ($V=8192$ Uncoupled) | 1.74700 | 1.74464 | 1.74582 ± 0.00167 | 107.9 ms | 2033.1 MB | $0$ | | fwd_streaming_oracle_mlp_pull_only_fullvocab (+MLP-Pullback Only) | 1.73560 | 1.73659 | 1.73609 ± 0.00070 | 108.2 ms | 2031.0 MB | $0$ | | fwd_streaming_oracle_mlp_whv_fullvocab (+MLP-Pullback + CM-Attn-WH) | 1.71687 | 1.71824 | 1.71756 ± 0.00097 | 123.6 ms | 2031.0 MB | $0$ | | fwd_streaming_oracle_coupled_fullvocab ($V=8192$ Coupled Surrogate) | 1.64679 | 1.64922 | 1.64800 ± 0.00172 | 163.9 ms | 2031.0 MB | $0$ |

Table 5: 1000-step 2-seed pretraining benchmark on d6 ($8.192\text{M}$ tokens). Key findings: (1) At matched $K=4$, parameter-space column contraction (fwd_streaming_wm_K4) beats token-scalar probing (fwd_streaming_cut_K4) by $-0.10448$ BPB. (2) Sylvester-Hadamard coordination improves $K=4$ by $-0.01752$ BPB and $K=8$ by $-0.02291$ BPB. (3) The component ablation at $V=8192$ isolates each mechanism in the intra-block coupled package: starting from uncoupled fwd_streaming_oracle_fullvocab ($1.74582 \pm 0.00167$), adding MLP-to-Attention residual pullback alone (fwd_streaming_oracle_mlp_pull_only_fullvocab) reduces holdout BPB by $-0.00973$ ($1.73609 \pm 0.00070$), adding Two-Level Window-Hadamard value transport CM-Attn-WH ($K_v=8$, fwd_streaming_oracle_mlp_whv_fullvocab) reduces BPB by an additional $-0.01853$ ($1.71756 \pm 0.00097$), and adding Hadamard-QK ($K_{\text{qk}}=8$, fwd_streaming_oracle_coupled_fullvocab) reduces BPB by a further $-0.06956$ ($1.64800 \pm 0.00172$), closing $56.4\%$ of the $+0.17346$ BPB surrogate gap to block_local_greedy_bp ($1.57236 \pm 0.00051$). (4) Full-vocabulary Parameter-Space Co-WM-S (fwd_streaming_cowm_coupled_fullvocab_K8) achieves $1.64719 \pm 0.00067$ holdout BPB ($56.9\%$ gap reduction), showing close descriptive agreement with its analytic-linear surrogate ($1.64800 \pm 0.00172$), just as sub-vocabulary fwd_streaming_cowm_coupled_K8 ($1.78546 \pm 0.00297$) and fwd_streaming_cowm_K8 ($1.86518 \pm 0.00011$) agree with fwd_streaming_oracle_coupled ($1.78435 \pm 0.00103$) and fwd_streaming_oracle ($1.86466 \pm 0.00221$).

3.4 Historical Subblock-Detached Diagnostic Controls and Settling Provenance

To document the provenance of MAP activity settling, we report the historical cut-scope and subblock-detached benchmarks (chung_scalable_llm.py):

1. Settling Step Ablation (N=1 vs N=2): Re-evaluating genuine $N=1$ versus $N=2$ settling confirms that $N=1$ achieves exact local credit with near-zero relative bias ($\le 0.00022$), whereas $N=2$ introduces $12.9\%\text{--}16.3\%$ spectral damping bias:
   • In 1000-step cut-scope training on d6 (long_horizon_1000steps_d6.json), gs_map_N1_attnWMK1 ($N=1$, seed 42: $1.676094$, seed 137: $1.680382$, mean $1.67824 \pm 0.00303$) and gs_map_N2_attnWMK1 ($N=2$, seed 42: $1.681450$, seed 137: $1.674709$, mean $1.67808 \pm 0.00477$) differ by only $-0.00016$ BPB.
   • Under native sequence shape ($B=64, T=2048$, $7.864\text{M}$ tokens, native_shape_b64_t2048_d6.json), gs_map_N1_attnWMK1 achieves $1.863655$ holdout BPB, while gs_map_N2_attnWMK1 achieves $1.863772$ holdout BPB.
   • Conclusion: Multi-step activity settling does not drive performance gains; the advantage over noisy single-probe WM ($1.68282$ BPB) stems from zero-variance credit reconstruction ($K=S$), not recurrent settling.
2. Subblock-Detached Learned Checkpoint Diagnostic (Table 6): Evaluated on the real learned step-1000 Layer 3 state ($M=1536, \|V\|_2^2=42.53$, learned_checkpoint_diagnostic.json), wm_S8_K1 achieves cosine similarity 0.37909 and relative variance 6.3832, co_wm_S8_K4 achieves 0.72457 and 0.9089, and co_wm_S8_K8 achieves 1.0000 and 0.0000. gs_map_N2 achieves 0.99883 cosine with 0.0000 sampling variance and $9.58\%$ systematic spectral damping bias.

| Algorithm | Mechanism Category | $D=128$ ($M=512$) Cos / RelVar | $D=256$ ($M=1024$) Cos / RelVar | $D=384$ ($M=1536$) Cos / RelVar | $D=512$ ($M=2048$) Cos / RelVar | Learned $D=384$ Cos / RelVar | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | | global_scalar_K1 | Sublayer Scalar Control | 0.0018 / 152,740 | 0.0032 / 335,988 | -0.0003 / 586,478 | 0.0006 / 626,964 | -0.0012 / 7.12e6 | | cut_K4 | Token-Scalar Rademacher | 0.0891 / 125.2 | 0.0618 / 257.7 | 0.0497 / 390.0 | 0.0441 / 507.8 | 0.0674 / 206.8 | | cut_hadamard_K4 | Block-Hadamard Spatial | 0.0874 / 121.7 | 0.0611 / 253.8 | 0.0492 / 383.4 | 0.0445 / 510.1 | 0.0531 / 204.7 | | wm_S8_K1 | Micro-Horizon $S=8, K=1$ | 0.3535 / 6.80 | 0.3532 / 7.04 | 0.3497 / 7.15 | 0.3530 / 6.99 | 0.3791 / 6.38 | | wm_S8_K2 | Micro-Horizon $S=8, K=2$ | 0.5012 / 3.42 | 0.4995 / 3.51 | 0.4971 / 3.54 | 0.5002 / 3.48 | 0.4874 / 3.18 | | path_uwm_S8_K1| ReLU$^2$ Chord Secant | 0.3597 / 6.73 | 0.3561 / 6.92 | 0.3535 / 6.99 | 0.3535 / 7.02 | 0.3862 / 5.59 | | co_wm_S8_K4 | Coordinated $K=4, S=8$ | 0.7116 / 1.01 | 0.7025 / 1.02 | 0.7041 / 1.03 | 0.7084 / 0.94 | 0.7246 / 0.91 | | co_wm_S8_K8 | Coordinated $K=S=8$ | 1.0000 / 0.00 | 1.0000 / 0.00 | 1.0000 / 0.00 | 1.0000 / 0.00 | 1.0000 / 0.00 | | wm_map_S8_N4 | Stochastic WM Settling | 0.7165 / 0.95 | 0.7058 / 1.03 | 0.6996 / 1.01 | 0.7059 / 1.04 | 0.7196 / 0.82 | | gs_map_N2 | Deterministic MAP ($N=2$) | 0.9989 / 0.00 | 0.9991 / 0.00 | 0.9992 / 0.00 | 0.9993 / 0.00 | 0.9988 / 0.00 |

Table 6: Historical subblock-detached diagnostic across synthetic widths $D \in \{128 \dots 512\}$ and on a learned step-1000 Layer 3 checkpoint (learned_checkpoint_diagnostic.json).

3.5 Two-Stage Faithful DUST Validation and Delayed-Trace-Co-WM-S Scaling & Pretraining Benchmarks

To evaluate strict no-backward credit assignment (zero reverse-depth sweeps and zero local reverse/pullback computations), we execute our two-stage faithful DUST protocol (dust_faithful.py, verify_dust_reference.py) alongside Delayed-Trace-Co-WM-S (delayed_trace_cowm.py, run_dust_and_delayed_trace_suite.py):

1. Stage 1A — Bit-for-Bit Equivalence & Prefix-Cached Suffix Execution (dust_stage1_reference_validation.json): Comparing unmodified literature/dust/dust.py (hook-based full-network reruns) against FaithfulDUST(use_prefix_cache=False) and FaithfulDUST(use_prefix_cache=True) (prefix-cached suffix execution blocks[l:]) on the reference 8-layer architecture ($L=8, D=512, B=4, T=256, V=4096, P=256$) yields max_grad_diff = 0.0 and max_param_diff = 0.0 across all 68 parameter tensors, while suffix caching accelerates step time by $1.75\times$ (838.0 ms vs 1469.7 ms, saving $19.14\%$ of block evaluations: $414$ vs $512$).
2. Stage 1B — Reference 8-Layer 1M-Token Validation ($L=8, D=512, B=8, T=2048, V=4096$, 61 steps): Under published DUST's SGD/Adam optimizer and c_proj initialization ($\text{std} = 1.5/\sqrt{d_{\text{in}}}$), backprop_sgd_ref improves from initial val 8.3186 to best val 5.7469 / test 5.7486 nats (117.2 ms/step, 4993.9 MB); dust_ref_P64 improves to best val 6.1763 / test 6.1442 nats (2067.1 ms/step, 7102.5 MB); and dust_ref_P256 improves to best val 5.8681 / test 5.8565 nats (6893.6 ms/step, 7183.0 MB), within 0.1078 test nats of backprop_sgd_ref (matching DUST Appendix Table 1: 6.067 vs 5.959 = 0.108 nats).
3. Stage 2/3 — Width and Pure Hidden-Neuron Scaling Diagnostic (Table 7, dust_and_delayed_trace_width_diagnostic.json):
   • Variance Metric Definitions: For $R=16$ Monte Carlo trials $\hat{G}^{(r)}$ with sample mean $\bar{G}_R$ and reference $G_{\text{ref}}$, _cosine_and_relvar records both (i) mean-normalized relative sample variance $\widehat{\text{RelVar}}_{\text{mean}} = \frac{\frac{1}{R-1}\sum_{r=1}^R \|\hat{G}^{(r)} - \bar{G}_R\|_F^2}{\|\bar{G}_R\|_F^2}$ (rel_var_to_mean, the inverse SNR of the estimator around its mean direction) and (ii) projection-rescaled relative variance $\widehat{\text{RelVar}}_{\text{proj}} = \frac{\frac{1}{R}\sum_{r=1}^R \|\alpha^*\hat{G}^{(r)} - \alpha^*\bar{G}_R\|_F^2}{\|G_{\text{ref}}\|_F^2}$ with $\alpha^* = \frac{\langle \bar{G}_R, G_{\text{ref}}\rangle}{\|\bar{G}_R\|_F^2}$ (rel_var, which scales down by $\cos^2(\bar{G}_R, G_{\text{ref}})$ as cosine collapses). Table 7 reports mean_cossim / rel_var_to_mean (rel_var) directly from dust_and_delayed_trace_width_diagnostic.json.
   • Conditional Recovery Given Outgoing Writer Reward ($D \in \{128, 256, 384, 512\}, M=4D$): Delayed-Trace-UWM achieves conditional cosine 0.1239, 0.0992, 0.1027, 0.1111; Delayed-Trace-WM-S ($K=1$) achieves 0.4575, 0.4008, 0.3971, 0.4168; Delayed-Trace-Co-WM-S ($K=4$) achieves 0.7932, 0.7557, 0.7538, 0.7755; and Delayed-Trace-Co-WM-S ($K=8$) achieves 1.0000, 1.0000, 1.0000, 1.0000 with RelVar = 0.0000.
   • Whole-Network Layer 3 mlp.c_fc Cosine Across $D \in \{128 \dots 512\}$ ($M=4D$): Faithful DUST direct hidden probing (34 draws) drops 0.2395 -> 0.1421 -> 0.1128 -> 0.1026, whereas Delayed-Trace-Co-WM-S ($K=8$) from 34 writer draws (0 hidden draws, $-50\%$ block draws) achieves 0.3289 -> 0.1998 -> 0.1602 -> 0.1454 ($+37\%\text{--}42\%$ higher), and from 68 reallocated writer draws (equal block draws) achieves 0.4405 -> 0.2765 -> 0.2236 -> 0.1963 ($+84\%\text{--}98\%$ higher).
   • Bounded Pure Hidden-Neuron Scaling Diagnostic at Fixed Residual Width $D=256$ ($M \in \{256, 512, 1024, 2048, 4096\}$, Table 7): Evaluated on both 15-step synthetic states (fixed_D256_neuron_scaling, Panel A) and 50-step ClimbMix language-model states evaluated on held-out ClimbMix tokens (fixed_D256_climbmix_50steps, Panel B), DUST direct hidden (34 draws) cosine drops $4.43\times$ (0.3060 -> 0.0691, Panel A) and $3.82\times$ (0.2578 -> 0.0675, Panel B) while $\widehat{\text{RelVar}}_{\text{mean}}$ rises (5.98 -> 14.81 and 7.20 -> 14.91), whereas Delayed-Trace-Co-WM-S ($K=8$) exhibits comparatively stable cosines across tested hidden widths (0.3031 -> 0.2582 in Panel A and 0.2078 -> 0.1884 in Panel B for 68w; $3.74\times$ and $2.79\times$ higher than DUST at $M=4096$). Because each width evaluates one frozen model state after 15 or 50 steps, Table 7 establishes a bounded empirical diagnostic observation over the tested hidden widths rather than an unconditional $O(1)$ statistical-scaling law.

| Panel / Method (mlp.c_fc Layer 3, Fixed $D=256$) | Draws (w/h) | $M=256$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=512$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=1024$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=2048$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=4096$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Panel A (Synthetic 15-Step): dust_direct_hidden_fc_16draws | $0\text{w}+16\text{h}$ | 0.2180 / 8.85 (3.67) | 0.1411 / 12.07 (2.78) | 0.0985 / 13.88 (1.77) | 0.0688 / 14.79 (0.98) | 0.0460 / 15.48 (0.48) | | Panel A: dust_direct_hidden_fc_34draws [5] | $0\text{w}+34\text{h}$ | 0.3060 / 5.98 (3.47) | 0.2028 / 9.49 (3.62) | 0.1431 / 12.05 (2.84) | 0.1021 / 13.58 (1.82) | 0.0691 / 14.81 (0.99) | | Panel A: delayed_trace_cowm_s_K8_from_34writer_draws | $34\text{w}+0\text{h}$ | 0.2230 / 8.71 (3.73) | 0.1910 / 9.97 (3.53) | 0.1928 / 9.80 (3.48) | 0.1962 / 9.67 (3.51) | 0.1814 / 10.19 (3.32) | | Panel A: delayed_trace_cowm_s_K8_from_68writer_draws | $68\text{w}+0\text{h}$ | 0.3031 / 6.06 (3.49) | 0.2753 / 6.90 (3.66) | 0.2650 / 7.25 (3.72) | 0.2727 / 6.92 (3.61) | 0.2582 / 7.44 (3.71) | | Panel B (ClimbMix 50-Step): dust_direct_hidden_fc_16draws | $0\text{w}+16\text{h}$ | 0.1774 / 10.47 (3.34) | 0.1355 / 12.30 (2.65) | 0.1035 / 13.91 (1.96) | 0.0654 / 14.91 (0.90) | 0.0442 / 15.57 (0.44) | | Panel B: dust_direct_hidden_fc_34draws [5] | $0\text{w}+34\text{h}$ | 0.2578 / 7.20 (3.48) | 0.2038 / 9.67 (3.80) | 0.1551 / 11.69 (3.15) | 0.0999 / 13.55 (1.74) | 0.0675 / 14.91 (0.95) | | Panel B: delayed_trace_cowm_s_K8_from_34writer_draws | $34\text{w}+0\text{h}$ | 0.1649 / 11.25 (3.32) | 0.1615 / 11.65 (3.42) | 0.1593 / 10.69 (2.82) | 0.1461 / 11.68 (2.79) | 0.1384 / 12.11 (2.69) | | Panel B: delayed_trace_cowm_s_K8_from_68writer_draws | $68\text{w}+0\text{h}$ | 0.2078 / 8.82 (3.31) | 0.1772 / 10.33 (3.24) | 0.2191 / 8.20 (3.21) | 0.1997 / 9.19 (3.31) | 0.1884 / 9.76 (3.30) |

Table 7: Pure hidden-neuron scaling diagnostic at fixed residual width $D=256$ across $M \in \{256, 512, 1024, 2048, 4096\}$ ($R=16$ trials, dust_and_delayed_trace_width_diagnostic.json), reporting mean_cossim / $\text{RV}_{\text{mean}} = \text{rel\_var\_to\_mean}$ ($\text{RV}_{\text{proj}} = \text{rel\_var}$) on both 15-step synthetic states (fixed_D256_neuron_scaling, Panel A) and 50-step ClimbMix language-model states evaluated on held-out ClimbMix tokens (fixed_D256_climbmix_50steps, Panel B).

4. Explicit 5-Component Latency Breakdown Across Regime I and Regime II (Table 7B, contraction_and_trace_width_profile.json & Sweep timing_profile_ms): To separate forward writer suffix cost from SRAM-fused outgoing-column contraction (_outgoing_col_dot_all8_kernel) and trace recording, profile_contraction_and_trace_width.py (and step_accounting["timing_profile_ms"] in the Regime I/II sweep JSONs) records 5 disjoint CUDA-synchronized sub-components (clean_fwd_ms, trace_record_ms, writer_suffix_fwd_ms, wm_col_contraction_ms, local_attn_and_head_ms, plus mean_opt_ms):
   • Regime I (Fixed $D=256$, Widening $M \in \{256, 512, 1024, 2048, 4096\}$): Across all 6 blocks at $M=4096$, strict no-backward (rev=0) SRAM-fused outgoing-column contraction (wm_col_contraction_ms) takes only 18.7 ms (1.9 ms at $M=256$, 5.3 ms at $M=1024$) and trace recording (trace_record_ms) takes 17.8 ms (1.1 ms at $M=256$, 2.6 ms at $M=1024), whereas eliminating hiddensuffix draws cutswriter_suffix_fwd_msfrom660.7–660.8 ms (dust_faithful_muon_P64_sc_sh) to **358.5 ms** in strict no-backward delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0, total 679.2 msvs950.1 ms, saving 270.9 ms/step) and **538.6 ms** in strict no-backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0, total 859.5 msvs950.1 ms, saving 90.6 ms/step`).
   • Regime II (Full-Model Widening $D \in \{128, 256, 384, 512\}, M=4D$): At $D=512$ ($M=2048$), strict no-backward (rev=0) wm_col_contraction_ms is 18.5–18.6 ms and trace_record_ms is 9.0 ms, while writer_suffix_fwd_ms drops from 551.8–552.2 ms (dust_faithful_muon_P64_sc_sh) to 320.7 ms (delayed_trace_cowm_P64_fast_sc_sh_d1, rev=0, total 621.6 ms vs 826.6 ms) and 482.9 ms (delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1, rev=0, total 784.2 ms vs 826.6 ms).

| Regime / Width | Method (rev = Local Pullbacks) | Total Step ms (Opt ms) | clean_fwd ms | trace_rec ms | writer_sfx ms | wm_col ms | local+head ms | Peak VRAM | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | Regime I ($D=256, M=256$) | dust_faithful_muon_P64_sc_sh (rev=0) | 473.3 (2.7) | 7.8 | 0.0 | 277.8 | 0.0 | 181.9 | 3085.9 MB | | Regime I ($D=256, M=256$) | delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0) | 366.0 (2.4) | 7.5 | 1.1 | 175.0 | 1.9 | 175.1 | 3222.2 MB | | Regime I ($D=256, M=256$) | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) | 457.4 (2.3) | 7.5 | 1.1 | 266.3 | 1.9 | 175.1 | 3222.2 MB | | Regime I ($D=256, M=256$) | delayed_trace_unified_P64_fast_sc_sh_d1 (rev=6) | 319.8 (3.0) | 7.9 | 4.6 | 129.5 | 4.4 | 167.4 | 3453.7 MB | | Regime I ($D=256, M=1024$) | dust_faithful_muon_P64_sc_sh (rev=0) | 528.6 (3.7) | 7.9 | 0.0 | 326.0 | 0.1 | 188.0 | 3361.9 MB | | Regime I ($D=256, M=1024$) | delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0) | 402.3 (3.5) | 7.7 | 2.6 | 197.2 | 5.3 | 183.1 | 3561.6 MB | | Regime I ($D=256, M=1024$) | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) | 503.8 (3.4) | 7.7 | 2.6 | 299.5 | 5.3 | 182.5 | 3561.6 MB | | Regime I ($D=256, M=1024$) | delayed_trace_unified_P64_fast_sc_sh_d1 (rev=6) | 351.3 (4.2) | 9.3 | 6.0 | 147.2 | 8.3 | 173.4 | 3793.1 MB | | Regime I ($D=256, M=4096$) | dust_faithful_muon_P64_sc_sh (rev=0) | 949.2 (3.8) | 12.1 | 0.0 | 660.7 | 0.1 | 269.6 | 4466.1 MB | | Regime I ($D=256, M=4096$) | delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0) | 679.2 (3.8) | 12.2 | 17.8 | 358.5 | 18.7 | 265.3 | 4919.3 MB | | Regime I ($D=256, M=4096$) | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) | 859.5 (3.8) | 12.2 | 17.8 | 538.6 | 18.7 | 264.9 | 4919.3 MB | | Regime I ($D=256, M=4096$) | delayed_trace_unified_P64_fast_sc_sh_d1 (rev=6) | 599.8 (4.0) | 12.3 | 21.2 | 280.2 | 21.1 | 257.4 | 5150.7 MB | | Regime II ($D=384, M=1536$) | dust_faithful_muon_P64_sc_sh (rev=0) | 650.3 (4.1) | 9.3 | 0.0 | 423.4 | 0.1 | 209.8 | 3968.9 MB | | Regime II ($D=384, M=1536$) | delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0) | 485.2 (3.4) | 8.4 | 5.4 | 249.6 | 10.8 | 204.9 | 4251.8 MB | | Regime II ($D=384, M=1536$) | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) | 613.4 (3.4) | 8.4 | 5.5 | 378.5 | 10.8 | 204.0 | 4251.8 MB | | Regime II ($D=512, M=2048$) | dust_faithful_muon_P64_sc_sh (rev=0) | 849.7 (5.1) | 10.7 | 0.0 | 552.2 | 0.1 | 278.1 | 4600.7 MB | | Regime II ($D=512, M=2048$) | delayed_trace_cowm_P64_fast_sc_sh_d1 (rev=0) | 621.6 (5.0) | 10.8 | 9.0 | 320.7 | 18.6 | 254.4 | 4977.9 MB | | Regime II ($D=512, M=2048$) | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) | 784.2 (5.0) | 10.8 | 9.0 | 482.9 | 18.6 | 254.6 | 4977.9 MB |

Table 7B: Explicit 5-component step latency breakdown (contraction_and_trace_width_profile.json and step_accounting["timing_profile_ms"] in regime1_fixed_D256_mlp_width_pretrain_200.json / dust_and_delayed_trace_width_sweep_200.json). In dust_faithful_* and delayed_trace_cowm_* (rev=0), writer_sfx includes all writers + hidden + hubs suffix forward passes.

5. Regime I (Pure MLP-Width Pretraining Scaling at Fixed $D=256$, 2 Seeds s42, s137, Table 8A) vs. Regime II (Full-Model Width Sweep $D \in \{128, 256, 384, 512\}$, Table 8B):
   • Regime I (regime1_fixed_D256_mlp_width_pretrain_200.json, Table 8A): Holding residual width fixed at $D=256$ and widening only the MLP hidden dimension $16\times$ across $M \in \{256, 1024, 4096\}$ over 200 ClimbMix pretraining steps on 2 seeds (s42, s137), our strict no-backward (local_reverse_derivative_calls = 0) methods outperform both unmodified dust_faithful_muon_P64 (2.36003 -> 2.35339 -> 2.34864) and Fairly Optimized dust_faithful_muon_P64_sc_sh (2.15683 -> 2.15271 -> 2.14879) across all three MLP widths:
     • Strict No-Backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (64 direct draws, rev=0) beats dust_faithful_muon_P64_sc_sh at every MLP width while running faster at every width: -0.00769 BPB at $M=256$ (2.14914 ± 0.00265 at 457.7 ms vs 2.15683 ± 0.00623 at 475.0 ms), -0.01116 BPB at $M=1024$ (2.14155 ± 0.00411 at 503.6 ms vs 2.15271 ± 0.01103 at 529.7 ms), and -0.01107 BPB at $M=4096$ (2.13772 ± 0.00264 at 859.7 ms vs 2.14879 ± 0.00153 at 952.4 ms, non-overlapping across seeds 2.13958/2.13585 vs 2.14771/2.14988 and 92.7 ms/step faster).
     • Even Strict No-Backward delayed_trace_cowm_P64_fast_sc_sh_d1 (41 direct draws, rev=0) outperforms 64-draw dust_faithful_muon_P64_sc_sh at all three MLP widths (2.15599 ± 0.00394 at $M=256$, 2.15079 ± 0.00351 at $M=1024$, and 2.14693 ± 0.00103 at $M=4096$, where both seeds 2.14766/2.14621 beat 2.14771/2.14988) while running $1.30\times\text{--}1.40\times$ faster (679.1 ms vs 952.4 ms at $M=4096$, saving 273.3 ms/step).
   • Regime II (dust_and_delayed_trace_width_sweep_200.json, Table 8B): Across full-model widths $D \in \{128, 256, 384, 512\}$ ($M=4D$), Strict No-Backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) outperforms Fairly Optimized dust_faithful_muon_P64_sc_sh across all four model widths $D \in \{128, 256, 384, 512\}$ (2.1606 vs 2.1839, 2.1453 vs 2.1625, 2.1297 vs 2.1351, 2.1302 vs 2.1342) while running 18.1–42.4 ms/step faster at every width.

| Algorithm (Regime I: Fixed $D=256$, 200 Steps, 2 Seeds s42, s137) | Draws (rev) | $M=256$ ($1D$) Holdout BPB (Step ms) | $M=1024$ ($4D$) Holdout BPB (Step ms) | $M=4096$ ($16D$) Holdout BPB (Step ms) | Peak VRAM ($M=4096$) | | :--- | :---: | :---: | :---: | :---: | :---: | | dust_faithful_muon_P64 [5] | $64$ (0) | 2.36003 ± 0.00667 (525.9 ms) | 2.35339 ± 0.00973 (578.3 ms) | 2.34864 ± 0.00423 (980.2 ms) | 3351.5 MB | | dust_faithful_muon_P64_sc_sh (Fairly Opt. DUST, sc+sh) | $64$ (0) | 2.15683 ± 0.00623 (475.0 ms) | 2.15271 ± 0.01103 (529.7 ms) | 2.14879 ± 0.00153 (952.4 ms) | 4497.3 MB | | delayed_trace_cowm_P64_fast_sc_sh_d1 (Strict No-Bwd Fast) | $41$ (0) | 2.15599 ± 0.00394 (366.1 ms) | 2.15079 ± 0.00351 (402.0 ms) | 2.14693 ± 0.00103 (679.1 ms) | 4919.3 MB | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (Strict No-Bwd) | $64$ (0) | 2.14914 ± 0.00265 (457.7 ms) | 2.14155 ± 0.00411 (503.6 ms) | 2.13772 ± 0.00264 (859.7 ms) | 4919.3 MB | | delayed_trace_full_P64_realloc_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.35835 ± 0.00820 (597.9 ms) | 2.34369 ± 0.00722 (548.5 ms) | 2.35469 ± 0.00082 (901.8 ms) | 3878.9 MB | | delayed_trace_unified_P64_fast_sc_sh_d1 (Hybrid Hub-Pullback) | $29$ (6) | 2.15962 ± 0.00831 (373.7 ms) | 2.15397 ± 0.00781 (350.1 ms) | 2.16840 ± 0.03447 (599.9 ms) | 5181.9 MB | | delayed_trace_unified_P64_sc_sh_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.16398 ± 0.01953 (487.8 ms) | 2.14284 ± 0.00047 (500.1 ms) | 2.13913 ± 0.00216 (857.8 ms) | 5181.9 MB | | delayed_trace_unified_P64_bd025_sc_sh_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.15549 ± 0.00797 (455.1 ms) | 2.14004 ± 0.00359 (501.3 ms) | 2.13429 ± 0.00197 (856.3 ms) | 5181.9 MB |

Table 8A: Regime I — 2-seed (s42, s137) pure MLP-width pretraining scaling at fixed residual width $D=256$ across $M \in \{256, 1024, 4096\}$ (regime1_fixed_D256_mlp_width_pretrain_200.json), separating Strict No-Backward (rev=0, hub_mode="dust_hub") from Hybrid Local-Hub-Pullback (rev=6, hub_mode="co_wm_s").

| Algorithm (Regime II: Full-Model Width Sweep, 200 Steps, Seed s42) | Draws (rev) | $D=128$ Dev / Holdout (ms) | $D=256$ Dev / Holdout (ms) | $D=384$ Dev / Holdout (ms) | $D=512$ Dev / Holdout (ms) | VRAM ($D=512$) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | dust_faithful_sgd_P64 [5] | $64$ (0) | 2.3746 / 2.3731 (482.6) | 2.4127 / 2.4141 (561.4) | 2.5149 / 2.5156 (678.8) | 2.4845 / 2.4841 (842.9) | 3035.5 MB | | dust_faithful_muon_P64 [5] | $64$ (0) | 2.3819 / 2.3820 (492.6) | 2.3473 / 2.3479 (565.6) | 2.3528 / 2.3510 (682.9) | 2.3310 / 2.3321 (847.5) | 3083.7 MB | | delayed_trace_cowm_P64_fast_d0 | $37$ (0) | 2.3834 / 2.3790 (400.9) | 2.3793 / 2.3787 (450.1) | 2.3816 / 2.3816 (529.7) | 2.3430 / 2.3425 (651.4) | 3623.9 MB | | delayed_trace_cowm_P64_fast_d1 | $37$ (0) | 2.3765 / 2.3760 (399.9) | 2.3686 / 2.3704 (449.0) | 2.3931 / 2.3900 (530.5) | 2.3478 / 2.3462 (650.7) | 3733.9 MB | | delayed_trace_cowm_P64_realloc_d0 | $64$ (0) | 2.3719 / 2.3705 (506.4) | 2.3459 / 2.3453 (579.3) | 2.3462 / 2.3441 (700.0) | 2.3332 / 2.3331 (870.9) | 3623.9 MB | | delayed_trace_cowm_P64_realloc_d1 | $64$ (0) | 2.3723 / 2.3709 (505.2) | 2.3453 / 2.3460 (580.4) | 2.3492 / 2.3461 (700.0) | 2.3323 / 2.3327 (872.8) | 3733.9 MB | | dust_faithful_muon_P64_sc_sh (Fairly Opt. DUST) | $64$ (0) | 2.1841 / 2.1839 (453.0) | 2.1624 / 2.1625 (529.6) | 2.1352 / 2.1351 (650.2) | 2.1361 / 2.1342 (826.6) | 4631.9 MB | | delayed_trace_cowm_P64_fast_sc_sh_d1 (Strict No-Bwd Fast) | $41$ (0) | 2.1741 / 2.1733 (350.2) | 2.1543 / 2.1535 (401.1) | 2.1524 / 2.1509 (485.2) | 2.1369 / 2.1373 (621.6) | 4977.9 MB | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (Strict No-Bwd) | $64$ (0) | 2.1612 / 2.1606 (434.9) | 2.1470 / 2.1453 (502.5) | 2.1314 / 2.1297 (613.4) | 2.1312 / 2.1302 (784.2) | 4977.9 MB | | delayed_trace_full_P64_fast_d1 (Hybrid Hub-Pullback) | $29$ (6) | 2.3752 / 2.3751 (355.7) | 2.3727 / 2.3718 (399.9) | 2.3497 / 2.3508 (473.6) | 2.3422 / 2.3446 (587.7) | 4706.2 MB | | delayed_trace_full_P64_realloc_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.3646 / 2.3626 (481.7) | 2.3386 / 2.3386 (549.2) | 2.3387 / 2.3399 (660.8) | 2.3410 / 2.3399 (820.2) | 4706.2 MB | | delayed_trace_unified_P64_fast_sc_sh_d1 (Hybrid Hub-Pullback) | $29$ (6) | 2.1759 / 2.1733 (305.6) | 2.1599 / 2.1595 (350.2) | 2.1510 / 2.1485 (431.4) | 2.1380 / 2.1366 (552.8) | 5956.4 MB | | delayed_trace_unified_P64_sc_sh_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.1784 / 2.1763 (433.8) | 2.1426 / 2.1425 (500.0) | 2.1392 / 2.1382 (617.9) | 2.1276 / 2.1253 (787.0) | 5956.4 MB | | delayed_trace_unified_P64_bd025_sc_sh_d1 (Hybrid Hub-Pullback) | $64$ (6) | 2.1740 / 2.1720 (432.9) | 2.1396 / 2.1375 (500.8) | 2.1308 / 2.1305 (617.2) | 2.1247 / 2.1241 (786.1) | 5956.4 MB |

Table 8B: Regime II — 200-step full-model width sweep across $D \in \{128, 256, 384, 512\}$ ($M=4D$, dust_and_delayed_trace_width_sweep_200.json), executed with the SRAM-fused Triton contraction kernel (_outgoing_col_dot_all8_kernel).

| Algorithm (d6, $D=384, M=1536$, 2 Seeds s42, s137) | Steps / Draws (rev) | Seed s42 Holdout | Seed s137 Holdout | Mean Holdout BPB $\pm$ SD | Mean Step ms (Timed s / Complete s) | Peak VRAM | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | dust_faithful_sgd_P64 [5] (Published SGD/Adam) | $1000$ / $64$ (0) | 2.42629 | 2.32764 | 2.37697 ± 0.06975 | 692.5 ms (691.1 s / 692.5 s) | 2665.1 MB | | dust_faithful_muon_P64 [5] (Adapted Muon/AdamW) | $1000$ / $64$ (0) | 2.24890 | 2.24953 | 2.24921 ± 0.00044 | 710.0 ms (708.5 s / 710.0 s) | 2701.2 MB | | delayed_trace_cowm_P64_fast_d0 (0 hidden, same-step) | $1000$ / $37$ (0) | 2.30369 | 2.29773 | 2.30071 ± 0.00422 | 691.7 ms (690.3 s / 691.7 s) | 2900.3 MB | | delayed_trace_cowm_P64_fast_d1 (0 hidden, 1-step delay) | $1000$ / $37$ (0) | 2.30943 | 2.29982 | 2.30462 ± 0.00679 | 656.8 ms (655.5 s / 656.8 s) | 2975.7 MB | | delayed_trace_cowm_P64_realloc_d0 (Realloc, same-step) | $1000$ / $64$ (0) | 2.28822 | 2.28888 | 2.28855 ± 0.00047 | 823.9 ms (822.2 s / 823.9 s) | 2900.3 MB | | delayed_trace_cowm_P64_realloc_d1 (Realloc, 1-step delay) | $1000$ / $64$ (0) | 2.29316 | 2.28479 | 2.28897 ± 0.00591 | 818.7 ms (817.1 s / 818.7 s) | 2975.7 MB | | delayed_trace_cowm_P64_indep_d0 (Indep. mlp.c_proj, d0) | $1000$ / $64$ (0) | 2.32639 | 2.29345 | 2.30992 ± 0.02329 | 808.7 ms (807.1 s / 808.7 s) | 2912.3 MB | | delayed_trace_cowm_P64_indep_d1 (Indep. mlp.c_proj, d1) | $1000$ / $64$ (0) | 2.29776 | 2.29271 | 2.29523 ± 0.00357 | 804.2 ms (802.5 s / 804.2 s) | 2987.7 MB | | delayed_trace_cowm_fixed16_d0 (All-Block 16w, same-step) | $1000$ / $27$ (0) | 2.28852 | 2.28307 | 2.28580 ± 0.00385 | 620.5 ms (619.3 s / 620.5 s) | 2960.3 MB | | delayed_trace_cowm_fixed16_d1 (All-Block 16w, 1-step delay) | $1000$ / $27$ (0) | 2.28584 | 2.28104 | 2.28344 ± 0.00339 | 618.9 ms (617.7 s / 618.9 s) | 3035.7 MB | | delayed_trace_cowm_fixed32_d0 (All-Block 32w, same-step) | $1000$ / $43$ (0) | 2.27861 | 2.28357 | 2.28109 ± 0.00351 | 805.5 ms (803.9 s / 805.5 s) | 2960.3 MB | | delayed_trace_cowm_fixed32_d1 (All-Block 32w, 1-step delay) | $1000$ / $43$ (0) | 2.28047 | 2.28793 | 2.28420 ± 0.00527 | 857.2 ms (855.5 s / 857.2 s) | 3035.7 MB | | delayed_trace_cowm_fixed64_d1 (All-Block 64w, 1-step delay) | $1000$ / $75$ (0) | 2.27531 | 2.26871 | 2.27201 ± 0.00467 | 1200.3 ms (1197.9 s / 1200.3 s) | 3035.7 MB | | dust_faithful_muon_P64_sc_sh (Fairly Opt. DUST, sc+sh) | $1000$ / $64$ (0) | 2.04581 | 2.04598 | 2.04590 ± 0.00012 | 649.3 ms (648.0 s / 649.3 s) | 4095.6 MB | | dust_faithful_muon_P64_wsig04_sc_sh (Fairly Opt. DUST + wsig04) | $1000$ / $64$ (0) | 2.04501 | 2.04605 | 2.04553 ± 0.00074 | 645.7 ms (644.4 s / 646.5 s) | 4095.6 MB | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (Strict No-Bwd) | $1000$ / $64$ (0) | 2.06175 | 2.06204 | 2.06189 ± 0.00021 | 612.2 ms (610.9 s / 613.0 s) | 4394.9 MB | | delayed_trace_cowm_P64_hub2_wsig04_bd025_sc_sh_d1 (Strict No-Bwd + wsig04) | $1000$ / $64$ (0) | 2.05599 | 2.05680 | 2.05639 ± 0.00057 | 612.3 ms (611.1 s / 613.1 s) | 4394.9 MB | | delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_wc648s (Strict No-Bwd WC $\le 648\text{s}$) | $1680$ / $43$ (0) | 2.04422 | 2.04402 | 2.04412 ± 0.00014 | 377.6 ms (633.6 s / 635.2 s) | 4519.9 MB | | delayed_trace_cowm_P64_fast_w22_wsig04_sc_sh_st_d1_wc648s (Strict No-Bwd + wsig04 WC $\le 644\text{s}$) | $1680$ / $43$ (0) | 2.04124 | 2.04110 | 2.04117 ± 0.00010 | 379.8 ms (637.3 s / 638.8 s) | 4519.9 MB | | delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_1750steps (Strict No-Bwd 1750 Steps) | $1750$ / $43$ (0) | 2.04166 | 2.04426 | 2.04296 ± 0.00184 | 378.7 ms (662.0 s / 663.6 s) | 4519.9 MB | | delayed_trace_full_P64_fast_d1 (Hybrid Hub-Pullback, $29$ draws) | $1000$ / $29$ (6) | 2.29780 | 2.28339 | 2.29060 ± 0.01019 | 473.2 ms (472.2 s / 474.1 s) | 3716.5 MB | | delayed_trace_full_P64_realloc_d1 (Hybrid Hub-Pullback, $64$ draws) | $1000$ / $64$ (6) | 2.29604 | 2.27999 | 2.28801 ± 0.01135 | 659.1 ms (657.8 s / 660.0 s) | 3716.5 MB | | delayed_trace_unified_P64_fast_sc_sh_d1 (Hybrid Hub-Pullback) | $1000$ / $29$ (6) | 2.06212 | 2.06840 | 2.06526 ± 0.00444 | 430.9 ms (430.0 s / 430.9 s) | 4993.9 MB | | delayed_trace_unified_P64_sc_sh_d1 (Hybrid Hub-Pullback) | $1000$ / $64$ (6) | 2.06525 | 2.06182 | 2.06353 ± 0.00243 | 618.0 ms (616.8 s / 618.1 s) | 5066.2 MB | | delayed_trace_unified_P64_bd025_sc_sh_d1 (Hybrid Hub-Pullback) | $1000$ / $64$ (6) | 2.06026 | 2.06106 | 2.06066 ± 0.00056 | 617.3 ms (616.0 s / 617.3 s) | 4993.9 MB | | delayed_trace_unified_P64_fast_w22_sc_sh_d1_wc648s (Hybrid WC) | $1480$ / $31$ (6) | 2.05031 | 2.05396 | 2.05213 ± 0.00258 | 439.0 ms (648.8 s / 650.6 s) | 5056.4 MB | | delayed_trace_unified_P64_bd025_sc_sh_st_d1_wc648s (Hybrid WC) | $1300$ / $64$ (6) | 2.04836 | 2.04682 | 2.04759 ± 0.00109 | 506.1 ms (656.9 s / 658.0 s) | 5128.7 MB | | delayed_trace_unified_P64_fast_w22_sc_sh_st_d1_wc648s (Hybrid WC $\le 648\text{s}$) | $2000$ / $31$ (6) | 2.03765 | 2.03774 | 2.03770 ± 0.00006 | 314.2 ms (627.8 s / 629.4 s) | 5118.9 MB | | dust_faithful_muon_P256 [5] (Matched $1000$ Steps) | $1000$ / $256$ (0) | 2.10785 | 2.10981 | 2.10883 ± 0.00138 | 2271.0 ms (2266.5 s / 2271.0 s) | 3369.7 MB | | delayed_trace_cowm_P256_fast_d1 (Matched $1000$ Steps) | $1000$ / $146$ (0) | 2.14799 | 2.14143 | 2.14471 ± 0.00464 | 1534.8 ms (1531.7 s / 1534.8 s) | 3667.9 MB | | delayed_trace_unified_P256_sc_sh_st_d1 (Hybrid Matched $1000$ Steps) | $1000$ / $146$ (6) | 2.00708 | 2.00676 | 2.00692 ± 0.00023 | 1099.8 ms (1097.6 s / 1100.7 s) | 4994.0 MB | | dust_faithful_muon_P256_wc307s (Uninterrupted WC, 135 Steps) | $135$ / $256$ (0) | 2.25778 | 2.25748 | 2.25763 ± 0.00021 | 2146.7 ms (285.5 s / 290.3 s) | 3269.7 MB | | dust_faithful_muon_P256_wc308s_144steps (Uninterrupted WC, 144 Steps) | $144$ / $256$ (0) | 2.24955 | 2.24962 | 2.24958 ± 0.00005 | 2139.9 ms (303.9 s / 309.3 s) | 3269.7 MB | | delayed_trace_cowm_P256_fast_d1_wc307s (Uninterrupted WC, 200 Steps) | $200$ / $146$ (0) | 2.23406 | 2.21549 | 2.22478 ± 0.01313 | 1533.9 ms (303.7 s / 307.8 s) | 3567.9 MB | | dust_faithful_muon_P256_sc_sh_wc307s (Fairly Opt. DUST WC, 150 Steps) | $150$ / $256$ (0) | 2.11287 | 2.11457 | 2.11372 ± 0.00121 | 2040.5 ms (302.0 s / 306.2 s) | 4008.1 MB | | dust_faithful_muon_P256_wsig04_sc_sh_wc307s (Fairly Opt. DUST + wsig04 WC, 150 Steps) | $150$ / $256$ (0) | 2.11281 | 2.11386 | 2.11334 ± 0.00074 | 2048.6 ms (303.2 s / 308.1 s) | 4220.6 MB | | delayed_trace_cowm_P256_bd025_sc_sh_st_d1_wc307s (Strict No-Bwd WC, 270 Steps) | $270$ / $146$ (0) | 2.07730 | 2.09544 | 2.08637 ± 0.01282 | 1100.3 ms (294.9 s / 297.9 s) | 4476.8 MB | | delayed_trace_cowm_P256_wsig04_bd025_sc_sh_st_d1_wc307s (Strict No-Bwd + wsig04 WC, 270 Steps) | $270$ / $146$ (0) | 2.07447 | 2.08602 | 2.08024 ± 0.00817 | 1102.7 ms (295.5 s / 298.6 s) | 4476.8 MB | | delayed_trace_full_P256_fast_d1_wc307s (Hybrid WC, 218 Steps) | $218$ / $120$ (6) | 2.24386 | 2.20443 | 2.22415 ± 0.02788 | 1223.6 ms (264.3 s / 267.8 s) | 4174.3 MB | | delayed_trace_full_P256_hubrealloc_d1_wc307s (Hybrid WC, 202 Steps) | $202$ / $146$ (6) | 2.23632 | 2.20864 | 2.22248 ± 0.01957 | 1324.8 ms (265.0 s / 267.6 s) | 4174.3 MB | | delayed_trace_unified_P256_sc_sh_st_d1_wc307s (Hybrid WC, 275 Steps) | $275$ / $146$ (6) | 2.07307 | 2.07803 | 2.07555 ± 0.00350 | 1098.1 ms (299.8 s / 303.0 s) | 4828.2 MB |

Table 9: 2-seed (s42, s137) benchmark on ClimbMix d6 (dust_and_delayed_trace_1000steps_d6.json), reporting rev = local_reverse_derivative_calls (0 for Strict No-Backward dust_faithful_* and delayed_trace_cowm_*; 6 for Hybrid Local-Hub-Pullback delayed_trace_full_* and delayed_trace_unified_*), direct-loss draw counts (31 = 28\text{w}+3\text{wte} for delayed_trace_unified_P64_fast_w22_*; 43 = 28\text{w}+12\text{h}+3\text{wte} for delayed_trace_cowm_P64_fast_w22_*), and both Timed s (total_timed_step_s, steps $2\dots N-1$) and Complete s (total_cum_train_s, steps $0\dots N-1$, excluding validation/checkpoint overhead recorded in history[*]["elapsed_s"]). Note on attribution: the matched-wall-clock winners (delayed_trace_cowm_P64_fast_w22_*_wc648s and delayed_trace_cowm_P256_*_wc307s) combine mlp.c_fc trace substitution, hub2/writer draw allocation changes, bd025 branch decoupling, st scalar traces, and (in *_wsig04_*) upper-block writer scaling $\sigma_{\text{upper}}=0.4$, whereas dust_faithful_muon_P*_sc_sh (and *_wsig04_sc_sh) receive the shared sc+sh (and wsig04) improvements; thus these time-budget comparisons evaluate the combined method packages rather than isolating trace substitution alone (for single-factor steps at 200 and 1000 updates, see Table 10 and the wsig04 pairs above). Key findings: (1) At fixed 1000 steps, dust_faithful_muon_P64_sc_sh reaches 2.04590 ± 0.00012 (649.3 ms/step, 648.0 s / 649.3 s) and dust_faithful_muon_P64_wsig04_sc_sh reaches 2.04553 ± 0.00074 (645.7 ms/step, 644.4 s / 646.5 s), while 1000-step strict no-backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) reaches 2.06189 ± 0.00021 at 612.2 ms/step, and adding wsig04 (delayed_trace_cowm_P64_hub2_wsig04_bd025_sc_sh_d1, rev=0) improves holdout BPB by -0.00550 BPB across both seeds to 2.05639 ± 0.00057 (s42=2.05599, s137=2.05680, at 612.3 ms/step, beating $P=256$ uncentered dust_faithful_muon_P256 2.10883 ± 0.00138 by -0.05244 BPB at $3.71\times$ lower latency). (2) In the uninterrupted matched-wall-clock package comparison against 1000-step dust_faithful_muon_P64_sc_sh (648.0 s / 649.3 s) and dust_faithful_muon_P64_wsig04_sc_sh (644.4 s / 646.5 s), Strict No-Backward delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_wc648s (rev=0, $1680$ steps, $43$ draws) reaches 2.04412 ± 0.00014 holdout BPB (633.6 s / 635.2 s), and adding wsig04 (delayed_trace_cowm_P64_fast_w22_wsig04_sc_sh_st_d1_wc648s, rev=0, $1680$ steps, $43$ draws) executes at 379.8 ms/step (637.3 s timed / 638.8 s complete, strictly below both 644.4 s / 646.5 s and 648.0 s / 649.3 s) and achieves 2.04117 ± 0.00010 holdout BPB (s42=2.04124, s137=2.04110, dev 2.03927 / 2.04122), outperforming dust_faithful_muon_P64_sc_sh by -0.00473 BPB and dust_faithful_muon_P64_wsig04_sc_sh by -0.00436 BPB across both seeds while using 7.1–10.7 s less timed training time (7.7–10.5 s less complete training time). (3) At $P=256$ in the uninterrupted matched-wall-clock package comparison (~300s training time), Strict No-Backward delayed_trace_cowm_P256_bd025_sc_sh_st_d1_wc307s (rev=0, $270$ steps, $146$ draws) reaches 2.08637 ± 0.01282 holdout BPB (294.9 s / 297.9 s), and adding wsig04 (delayed_trace_cowm_P256_wsig04_bd025_sc_sh_st_d1_wc307s, rev=0, $270$ steps, $146$ draws) executes at 1102.7 ms/step (295.5 s timed / 298.6 s complete) and achieves 2.08024 ± 0.00817 holdout BPB (s42=2.07447, s137=2.08602), strictly outperforming both dust_faithful_muon_P256_sc_sh_wc307s (150 steps, 302.0 s / 306.2 s, 2.11372 ± 0.00121) by -0.03348 BPB and dust_faithful_muon_P256_wsig04_sc_sh_wc307s (150 steps, 303.2 s / 308.1 s, 2.11334 ± 0.00074) by -0.03310 BPB across both seeds while using 6.5–7.7 s less timed training time (7.6–9.5 s less complete training time).

6. 200-Step Single-Factor Strict No-Backward Ablation Ladder, Hub-DoF Balance, Upper-Block Writer Perturbation Scaling (wsig04), & 2-Seed Matched-Allocation Value-Trace Diagnostic on d6 (Table 10, innovation_screen_200_d6.json & recency_vtrace_and_subspace_diagnostic.json): Table 10 isolates each mechanism one factor at a time at 200 steps on ClimbMix d6 ($D=384, M=1536$, seed s42, with 2-seed s42, s137 replication of the value-trace, Hub-DoF, wsig04, and highway-cascade arms): (a) among Strict No-Backward (rev=0, hub_mode="dust_hub") methods at $P=64$, starting from dust_faithful_muon_P64_sc_sh (2.13617 holdout BPB, 651.0 ms/step), replacing DUST's 27 hidden (mlp.c_fc) draws with Delayed-Trace-Co-WM-S without reallocating the saved draws or enabling bd025 (delayed_trace_cowm_P64_pure_trace_sc_sh_d1, $37$ direct draws, bd=None, d1) cuts step time by 179.7 ms/step (471.3 ms) at 2.16380 holdout BPB (since writers=[9,8,4,3,1,1] leaves deeper Blocks $4, 5$ with $1$ uncentered writer draw feeding mlp.c_fc). Holding bd=None and d1 fixed, reallocating the 27 saved draws to writers=[12,10,8,7,6,6] and hubs=[2,2,2,2,2,2] (delayed_trace_cowm_P64_hub2_sc_sh_d1, $64$ draws, bd=None, d1) improves holdout BPB by -0.02912 BPB to 2.13468 at 613.2 ms/step—already outperforming dust_faithful_muon_P64_sc_sh (2.13617 at 651.0 ms) by -0.00149 BPB while running 37.8 ms/step faster without bd025. The available rows then support two separate single-factor comparisons to the primary delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 configuration (2.12965 holdout BPB at 611.8 ms/step, -0.00652 BPB and 39.2 ms/step faster than dust_faithful_muon_P64_sc_sh): first, at fixed hub2 and 1-tick delay d1, adding branch decoupling bd025 improves holdout BPB from 2.13468 (delayed_trace_cowm_P64_hub2_sc_sh_d1) to 2.12965 (-0.00503 BPB); second, at fixed hub2 + bd025, changing same-step contraction d0 (delayed_trace_cowm_P64_hub2_bd025_sc_sh_d0, 2.13360 holdout BPB at 612.6 ms/step) to 1-tick cross-batch delayed contraction d1 (delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1) improves holdout BPB from 2.13360 to 2.12965 (-0.00395 BPB). Third, scaling upper-block ($l \ge L/2 = 3$) writer perturbation amplitude from $\sigma=0.2$ to $\sigma=0.4$ (UpperWriterSigma04Estimator, wsig04, matching DUST's $\sigma=0.4$ upper-block hub schedule at zero extra latency) further improves 200-step holdout BPB across both seeds and both tested allocations (upper_writer_sigma04_2seed_200_d6): hub2_wsig04_P64_bd025_sc_sh_d1 reaches 2.12882 on s42 and 2.13713 on s137 (2.13297 ± 0.00587 2-seed BPB at 611.9 ms/step, vs 2.13379 ± 0.00585 at $\sigma=0.2$), while sep_hub3_wsig04_P64_bd025_sc_sh_d1 reaches 2.13178 on s42 and 2.13530 on s137 (2.13354 ± 0.00249 2-seed BPB at 609.6 ms/step, vs 2.13414 ± 0.00216 at $\sigma=0.2$). Furthermore, varying the allocation between writers and hubs across hubs = [2,2,1,1,1,1] ($8$ draws, 2.14226 BPB), hub2 ([2]*6, $12$ draws, 2.12965 BPB; 2.12882 with wsig04), hub3 ([3]*6, $18$ draws, 2.13261 BPB; 2.13178 with wsig04), and hub4 ([4]*6, $24$ draws, 2.16347 BPB) shows that hub2 is the best tested allocation in this single-seed screen (-0.01261 BPB vs [2,2,1,1,1,1], -0.00296 BPB vs hub3, and -0.03382 BPB vs hub4); (b) in delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1 (2.16374 holdout BPB on s42, +0.03409 BPB vs 2.12965), enabling causal_v_trace=True simultaneously replaced local_errors("v") with unweighted ($\gamma=1$) forward causal value pooling and changed local (q, k) allocations from (1, 4) to (4, 7). To separate these two effects, test_recency_vtrace_and_subspace_writers.py (recency_vtrace_and_subspace_diagnostic.json) evaluates both matched (q, k) = (1, 4) 200-step ablations across seeds s42 and s137 (delayed_trace_cowm_P64_hub2_vtrace_q1k4_g100_bd025_sc_sh_d1 at $\gamma=1.00$, ..._g098_... at $\gamma=0.98$, and temporal sequence-mean damped ..._dc080_... at $\lambda_{\text{DC}}=0.80$) and direct $R=16$ Monte Carlo variance/cosine/effective-rank diagnostics across two 50-step ClimbMix d6 states (s42 and s137). On seed s42, restoring matched (q, k) = (1, 4) improves 200-step holdout BPB from 2.16374 to 2.14931 (610.1 ms) in the freshly regenerated --run-pretrain execution at $\gamma=1.00$ (2.14681 at 610.6 ms in an earlier standalone run), 2.14782 (607.7 ms) at $\gamma=0.98$, and 2.14934 (610.9 ms) at $\lambda_{\text{DC}}=0.80$, showing that +0.01443 to +0.01693 BPB ($42.3\%\text{--}49.7\%$) of the +0.03409 BPB gap on s42 was caused by the (q, k) = (4, 7) allocation change while +0.01716 to +0.01966 BPB remains on s42. Replicating on seed s137 (seed137_pretrain_200_d6) shows that matched (q, k) = (1, 4) value pooling reaches 2.13220 holdout BPB at $\gamma=1.00$ and 2.13081 holdout BPB at $\gamma=0.98$—outperforming both delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (2.13793 BPB) and dust_faithful_muon_P64_sc_sh (2.13353 BPB) on s137 ((q, k) = (4, 7) reaches 2.13303). Across both 50-step states (wv_variance_and_cosine_diagnostic on s42 and wv_variance_and_cosine_diagnostic_s137 on s137), $\gamma=1.00$ value pooling (vtrace_pool_g100) achieves higher single-trial cosine against exact backpropagation ((0.056872, 0.047611, 0.063110) on s42 and (0.062903, 0.046011, 0.069297) on s137 across Layers $0, 2, 5$) than both $\gamma=0.98$ value pooling ((0.039419, 0.030410, 0.049562) on s42; (0.043221, 0.037514, 0.052396) on s137) and DUST's 16-draw local_errors("v") ((0.004103, 0.008465, 0.002831) on s42; (0.008707, 0.008334, 0.011840) on s137), with relative variance (rel_var_to_mean) on s42 essentially tied with $\gamma=0.98$ at Layer 0 (14.3095 vs 14.3087) and lower at Layers 2 and 5 (14.6878, 13.3115 vs 15.1474, 13.5980, and vs 14.8596, 14.7466, 15.2554 for local_errors("v"); on s137, (14.3091, 15.1896, 14.4041) at $\gamma=1.00$ and (14.5107, 14.2456, 14.5313) at $\gamma=0.98$ vs (14.9968, 14.8210, 15.1795) for local_errors("v")), while exhibiting lower single-trial spectral effective rank on both seeds (single_trial_erank = (128.52, 94.63, 87.24) on s42 and (133.97, 93.80, 87.24) on s137, vs (217.37, 197.56, 192.41) and (217.86, 197.07, 187.68) for local_errors("v")). Because vtrace_q1k4 lags by +0.01716 to +0.01966 BPB on s42 (2.14782–2.14931 vs 2.12965) but leads by -0.00573 to -0.00711 BPB on s137 (2.13081–2.13220 vs 2.13793), the 50-step effective-rank measurements represent diagnostic associations rather than a deterministic 200-step penalty; while two seeds with a reversal do not establish a reliable population ordering or a stable variance advantage, the lower observed two-seed mean and approximately half the sample standard deviation (2.13379 ± 0.00585 vs 2.13932 ± 0.01203 for vtrace_q1k4_g098 and 2.13485 ± 0.00187 for dust_faithful_muon_P64_sc_sh, corresponding to a sample variance ratio of $(0.00585 / 0.01203)^2 \approx 0.24$) motivate retaining causal_v_trace=False in our primary strict no-backward delayed_trace_cowm_* arms; and (c) at $P=256$, strict no-backward delayed_trace_cowm_P256_sc_sh_st_d1 (rev=0, 2.09272 at 1111.7 ms) and delayed_trace_cowm_P256_bd025_sc_sh_st_d1 (rev=0, 2.09707 at 1099.4 ms/step) run $1.86\times$ faster than dust_faithful_muon_P256_sc_sh (2041.6 ms/step).

| Algorithm (200 Steps on d6, $D=384, M=1536$, Seed s42) | Draws (rev) | Dev BPB | Holdout BPB | Mean Step ms | Peak VRAM | | :--- | :---: | :---: | :---: | :---: | :---: | | dust_faithful_muon_P64 [5] | $64$ (0) | 2.35380 | 2.35193 | 681.4 ms | 2601.2 MB | | dust_faithful_muon_P64_sc_only (+sc only) | $64$ (0) | 2.36101 | 2.36140 | 699.7 ms | 2721.5 MB | | dust_faithful_muon_P64_sc_sh (Fairly Opt. DUST, sc+sh; s137: 2.13353, 2-seed: 2.13485 ± 0.00187) | $64$ (0) | 2.13629 | 2.13617 | 651.0 ms | 3995.6 MB | | dust_faithful_muon_P64_wsig04_sc_sh (Fairly Opt. DUST + wsig04; s137: 2.13422, 2-seed: 2.13438 ± 0.00022) | $64$ (0) | 2.13480 | 2.13453 | 642.0 ms | 3995.6 MB | | delayed_trace_cowm_P64_pure_trace_sc_sh_d1 (Pure Trace Sub. Only, $37$ draws, bd=None) | $37$ (0) | 2.16617 | 2.16380 | 471.3 ms | 4251.8 MB | | delayed_trace_cowm_P64_realloc_bd025_sc_sh_d1 (Tilted [16..2], hubs=[2,2,1,1,1,1]) | $60$ (0) | 2.14978 | 2.14893 | 620.1 ms | 4251.8 MB | | delayed_trace_cowm_P64_bd025_sc_sh_d1 (writers=[12..7], hubs=[2,2,1,1,1,1]) | $64$ (0) | 2.14370 | 2.14226 | 610.7 ms | 4251.8 MB | | delayed_trace_cowm_P64_hub2_sc_sh_d1 (Strict No-Bwd, hub2, bd=None, d1) | $64$ (0) | 2.13450 | 2.13468 | 613.2 ms | 4251.8 MB | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d0 (Strict No-Bwd, hub2+bd025, Same-Step d0) | $64$ (0) | 2.13441 | 2.13360 | 612.6 ms | 4170.4 MB | | delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (Strict No-Bwd, hub2+bd025, (q,k)=(1,4); s137: 2.13793, 2-seed: 2.13379 ± 0.00585) | $64$ (0) | 2.13155 | 2.12965 | 611.8 ms | 4251.8 MB | | hub2_wsig04_P64_bd025_sc_sh_d1 (Strict No-Bwd, hub2+wsig04 ($\sigma_{\text{upper}}=0.4$); s137: 2.13713, 2-seed: 2.13297 ± 0.00587) | $64$ (0) | 2.12980 | 2.12882 | 611.9 ms | 4251.8 MB | | sep_hub3_P64_bd025_sc_sh_d1 (delayed_trace_cowm_P64_hub3_bd025_sc_sh_d1, [10,9,7,6,6,5]; s137: 2.13566, 2-seed: 2.13414 ± 0.00216) | $64$ (0) | 2.13311 | 2.13261 | 608.5 ms | 4251.8 MB | | sep_hub3_wsig04_P64_bd025_sc_sh_d1 (Strict No-Bwd, sep_hub3+wsig04 ($\sigma_{\text{upper}}=0.4$); s137: 2.13530, 2-seed: 2.13354 ± 0.00249) | $64$ (0) | 2.13196 | 2.13178 | 609.6 ms | 4251.8 MB | | co_hub3_mlp_P64_bd025_sc_sh_d1 (CoHubMLP, [10,9,7,6,6,5], causal_v_trace=False; s137: 2.14477, 2-seed: 2.14574 ± 0.00137) | $64$ (0) | 2.14610 | 2.14670 | 612.0 ms | 4251.8 MB | | highway_cascade_b07_co_hub3_mlp_P64_bd025_sc_sh_d1 (Pooled Cascade $\beta=0.7$, pool=mlp, cascade=True; s137: 2.14862, 2-seed: 2.14819 ± 0.00060) | $64$ (0) | 2.15043 | 2.14776 | 637.8 ms | 4251.8 MB | | highway_cascade_v2_b07_co_hub3_mlp_P64_bd025_sc_sh_d1 (Pooled Cascade v2 $\beta=0.7$, pool=attn, cascade=True; s137: 2.15288, 2-seed: 2.14990 ± 0.00421) | $64$ (0) | 2.14786 | 2.14693 | 638.8 ms | 4251.8 MB | | sep_hub4_P64_bd025_sc_sh_d1 (delayed_trace_cowm_P64_hub4_bd025_sc_sh_d1, [9,8,6,5,5,4]; s137: 2.13653, 2-seed: 2.15000 ± 0.01905) | $64$ (0) | 2.16495 | 2.16347 | 608.8 ms | 4251.8 MB | | delayed_trace_cowm_P64_hub2_vtrace_q1k4_g100_bd025_sc_sh_d1 (+W_O-Free Causal-$V$ $\gamma=1.00$, (q,k)=(1,4); s137: 2.13220, 2-seed: 2.14076 ± 0.01210) | $64$ (0) | 2.15096 | 2.14931 | 610.1 ms | 4543.1 MB | | delayed_trace_cowm_P64_hub2_vtrace_q1k4_g098_bd025_sc_sh_d1 (+W_O-Free Causal-$V$ $\gamma=0.98$, (q,k)=(1,4); s137: 2.13081, 2-seed: 2.13932 ± 0.01203) | $64$ (0) | 2.15136 | 2.14782 | 607.7 ms | 4543.1 MB | | sep_hub3_vtrace_q1k4_g098_P64_bd025_sc_sh_d1 (hub3 [10,9,7,6,6,5], +W_O-Free Causal-$V$ $\gamma=0.98$; s137: 2.13316, 2-seed: 2.13716 ± 0.00566) | $64$ (0) | 2.14412 | 2.14116 | 605.6 ms | 4543.1 MB | | co_hub3_mlp_vtrace_q1k4_g098_P64_bd025_sc_sh_d1 (CoHubMLP, +W_O-Free Causal-$V$ $\gamma=0.98$; s137: 2.16952, 2-seed: 2.15305 ± 0.02329) | $64$ (0) | 2.13701 | 2.13658 | 612.2 ms | 4543.1 MB | | highway_cascade_b07_co_hub3_mlp_vtrace_q1k4_g098_P64_bd025_sc_sh_d1 (Cascade $\beta=0.7$, pool=mlp, cascade=True, Causal-$V$; s137: 2.16500, 2-seed: 2.15210 ± 0.01824) | $64$ (0) | 2.14160 | 2.13920 | 637.1 ms | 4543.1 MB | | highway_cascade_v2_b07_co_hub3_mlp_vtrace_q1k4_g098_P64_bd025_sc_sh_d1 (Cascade v2 $\beta=0.7$, pool=attn, cascade=True, Causal-$V$; s137: 2.16863, 2-seed: 2.15303 ± 0.02207) | $64$ (0) | 2.14047 | 2.13742 | 637.5 ms | 4543.1 MB | | sep_hub4_vtrace_q1k4_g098_P64_bd025_sc_sh_d1 (hub4 [9,8,6,5,5,4], +W_O-Free Causal-$V$ $\gamma=0.98$; s137: 2.13717, 2-seed: 2.13670 ± 0.00067) | $64$ (0) | 2.13639 | 2.13622 | 606.0 ms | 4543.1 MB | | delayed_trace_cowm_P64_hub2_vtrace_q1k4_dc080_bd025_sc_sh_d1 (+W_O-Free Causal-$V$ $\lambda_{\text{DC}}=0.80$, Matched (q,k)=(1,4)) | $64$ (0) | 2.15049 | 2.14934 | 610.9 ms | 5451.7 MB | | delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1 (+W_O-Free Causal-$V$ $\gamma=1.00$, (q,k)=(4,7); s137: 2.13303, 2-seed: 2.14839 ± 0.02172) | $64$ (0) | 2.16642 | 2.16374 | 621.8 ms | 4543.1 MB | | delayed_trace_full_P64_realloc_d1 (Hybrid Hub-Pullback, Uncentered) | $64$ (6) | 2.33875 | 2.34066 | 650.3 ms | 3244.2 MB | | delayed_trace_unified_P64_sc_only_d1 (Hybrid Hub-Pullback, +sc only) | $64$ (6) | 2.34175 | 2.34270 | 852.7 ms | 3538.2 MB | | delayed_trace_unified_P64_fast_sc_sh_d1 (Hybrid, $29$ draws, sc+sh) | $29$ (6) | 2.14403 | 2.14305 | 616.3 ms | 4815.7 MB | | delayed_trace_unified_P64_fast_w22_sc_sh_d1 (Hybrid, $31$ draws, sc+sh) | $31$ (6) | 2.14174 | 2.14188 | 436.9 ms | 4815.7 MB | | delayed_trace_unified_P64_sc_sh_st_d1 (Hybrid, $64$ draws, sc+sh+st) | $64$ (6) | 2.15922 | 2.15951 | 501.1 ms | 4521.7 MB | | delayed_trace_unified_P64_sc_sh_d1 (Hybrid, $64$ draws, sc+sh) | $64$ (6) | 2.13396 | 2.13254 | 622.7 ms | 4521.7 MB | | delayed_trace_unified_P64_bd025_sc_sh_d1 (Hybrid, $64$ draws, +bd025) | $64$ (6) | 2.13208 | 2.13097 | 800.8 ms | 4815.7 MB | | delayed_trace_unified_P64_w20_sc_sh_st_d1 (Hybrid, $95$ draws, sc+sh+st) | $95$ (6) | 2.13020 | 2.12907 | 676.5 ms | 4521.7 MB | | delayed_trace_unified_P64_tilt_bd025_sc_sh_d1 (Hybrid, Tilted [18..5], +bd025) | $64$ (6) | 2.12895 | 2.12861 | 649.7 ms | 4815.7 MB | | delayed_trace_unified_P64_tilt_sa20_bd025_sc_sh_d0 (Hybrid, Tilted [18..5], d0) | $64$ (6) | 2.13191 | 2.13063 | 647.9 ms | 4607.4 MB | | dust_faithful_muon_P256 [5] ($P=256$ Uncentered) | $256$ (0) | 2.21359 | 2.21168 | 2147.6 ms | 3270.0 MB | | dust_faithful_muon_P256_sc_sh ($P=256$ Fairly Opt. DUST, sc+sh) | $256$ (0) | 2.08785 | 2.08761 | 2041.6 ms | 3995.6 MB | | delayed_trace_cowm_P256_sc_sh_st_d1 (Strict No-Bwd $P=256$, +W_O-Free Causal-$V$) | $146$ (0) | 2.09321 | 2.09272 | 1111.7 ms | 4543.1 MB | | delayed_trace_cowm_P256_bd025_sc_sh_st_d1 (Strict No-Bwd $P=256$, bd025+sc+sh+st) | $146$ (0) | 2.09747 | 2.09707 | 1099.4 ms | 4251.8 MB | | delayed_trace_unified_P256_sc_sh_d1 (Hybrid $P=256$, sc+sh) | $146$ (6) | 2.09112 | 2.08956 | 1399.0 ms | 4815.7 MB | | delayed_trace_unified_P256_sc_sh_st_d1 (Hybrid $P=256$, sc+sh+st) | $146$ (6) | 2.09646 | 2.09502 | 1090.6 ms | 4521.7 MB |

Table 10: 200-step component ablation screen on ClimbMix d6 (innovation_screen_200_d6.json and recency_vtrace_and_subspace_diagnostic.json; for delayed_trace_cowm_P64_hub2_vtrace_q1k4_g100_bd025_sc_sh_d1 on s42, the table reports the freshly regenerated --run-pretrain execution 2.14931 holdout / 2.15096 dev / 610.1 ms with complete step_accounting in matched_q1k4_pretrain_200_d6, while the earlier standalone screen run 2.14681 holdout / 2.14926 dev / 610.6 ms is also retained as seed42_g100_standalone_run in recency_vtrace_and_subspace_diagnostic.json, where their 0.002502 holdout BPB difference reflects fixed-seed execution variability across separate process launches—possibly arising from parallel floating-point atomic accumulation in Triton/FlashAttention kernels—distinct from training-seed variability across s42 and s137; 2-seed s42, s137 replications for the five value-trace comparison arms, the six Hub-DoF $\times$ Value-Trace / Co-Hub-MLP screen arms, the three upper-block writer-sigma wsig04 arms (hub2_wsig04, sep_hub3_wsig04, and dust_faithful_muon_P64_wsig04_sc_sh), and the four Pooled Highway-Prior Cascade arms are recorded in seed137_pretrain_200_d6, vtrace_hub_dof_and_coperturb_2seed_200_d6, upper_writer_sigma04_2seed_200_d6, and highway_cascade_pretrain_2seed_200_d6 inside recency_vtrace_and_subspace_diagnostic.json with peak_vram_mb, total_timed_step_s, total_cum_train_s, per-seed runtime-verified estimator_class, exact unified equivalent_full_network_forwards_prefix_cached = 58.5 for all reallocated Delayed-Trace-Co-WM-S arms, and descriptive two-seed mean $\pm$ sample standard deviation; rows marked with cascade=True denote same-step reverse-block scheduling from block $L-1$ down to $0$ (same_step_reverse_block_cascade = True, $+25.8\text{ ms/step}$ cascade scheduling overhead vs co_hub3_mlp and $+29.8\text{ ms/step}$ vs sep_hub3), which retains backward depth locking despite using zero derivative pullbacks; earlier entries delayed_trace_unified_P64_fast_sc_sh_d1 and delayed_trace_unified_P64_bd025_sc_sh_d1 in this screening table were timed before enabling the fused Triton contraction kernel, whereas Tables 7B, 8A, 8B, and 9 reflect the fused kernel timings 430.9 ms and 617.3 ms).

---

4. Discussion, Prior Work, and Honest Boundaries

4.1 Comparison with Related Non-Backpropagation Methods

1. Published Faithful DUST [5] vs. Fairly Optimized DUST (sc+sh and wsig04) vs. Strict No-Backward Delayed-Trace-Co-WM-S (rev=0) and Hybrid Local-Hub-Pullback Control (rev=6): Our evaluation separates five distinct regimes: (i) the simplified global-scalar control fwd_global_scalar_K2 ($2.44794 \pm 0.11598$ BPB), which perturbs all parameters with a single scalar reward and is never used to represent DUST; (ii) Published Faithful DUST (dust_faithful.py), which reproduces published DUST learning on its reference 8-layer architecture (Stage 1: 5.8565 vs 5.7486 test nats at $P=256$, bit-for-bit suffix caching speedup $1.75\times$) and reaches $2.37697 \pm 0.06975$ (SGD, $P=64$), $2.24921 \pm 0.00044$ (Muon, $P=64$), and $2.10883 \pm 0.00138$ (Muon, $P=256$) on ClimbMix d6; (iii) Fairly Optimized DUST (dust_faithful_muon_P64_sc_sh, dust_faithful_muon_P64_wsig04_sc_sh, dust_faithful_muon_P256_sc_sh, dust_faithful_muon_P256_wsig04_sc_sh_wc307s), which equips DUST with our shared backward-free full-block covariance centering (sc), target-separated Hadamard readout probing (sh), and upper-block writer scaling (wsig04), improving 1000-step $P=64$ holdout BPB from 2.24921 ± 0.00044 to 2.04590 ± 0.00012 (2.04553 ± 0.00074 with wsig04); (iv) Strict No-Backward Delayed-Trace-Co-WM-S (delayed_trace_cowm_*, hub_mode="dust_hub", local_reverse_derivative_calls = 0), which eliminates all 27 direct hidden (mlp.c_fc) suffix forward passes via SRAM-fused outgoing-column contractions (_outgoing_col_dot_all8_kernel, 18.7–19.1 ms across 6 layers at $M=4096$), retains DUST's strict no-backward o@layer direct suffix perturbation with full-hub covariance centering (hub2, hubs=[2,2,2,2,2,2]), and executes zero reverse or local-pullback matrix multiplications; and (v) Hybrid Local-Hub-Pullback Controls (delayed_trace_full_*, delayed_trace_unified_*, hub_mode="co_wm_s", local_reverse_derivative_calls = L = 6), which additionally replace the 8 o@layer suffix forwards by contracting $\Delta W_g^{\text{attn}}$ with $W_O$ (attn.c_proj) to reconstruct a local activation error tensor $\widehat{E}_g = U_g W_O$ at the attention hub. Across our two-seed Regime I pure MLP-width scaling at fixed $D=256$ (Table 8A: where Strict No-Backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) achieves a descriptive two-seed advantage over dust_faithful_muon_P64_sc_sh across $M \in \{256, 1024, 4096\}$ that reaches -0.01107 BPB (2.13772 ± 0.00264 vs 2.14879 ± 0.00153) while 92.7 ms/step faster at $M=4096$), our single-seed Regime II full-model width sweep across $D \in \{128, 256, 384, 512\}$ (Table 8B: where Strict No-Backward delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (rev=0) achieves a descriptive single-seed advantage over dust_faithful_muon_P64_sc_sh at all four widths while 18.1–42.4 ms/step faster), and our two-seed d6 package benchmarks (Table 9: where Strict No-Backward delayed_trace_cowm_P64_fast_w22_wsig04_sc_sh_st_d1_wc648s (rev=0) reaches 2.04117 ± 0.00010 BPB in 637.3 s / 638.8 s vs 1000-step dust_faithful_muon_P64_sc_sh 2.04590 ± 0.00012 at 648.0 s / 649.3 s and dust_faithful_muon_P64_wsig04_sc_sh 2.04553 ± 0.00074 at 644.4 s / 646.5 s, and Strict No-Backward delayed_trace_cowm_P256_wsig04_bd025_sc_sh_st_d1_wc307s (rev=0) reaches 2.08024 ± 0.00817 BPB in 295.5 s / 298.6 s at $P=256$ vs dust_faithful_muon_P256_sc_sh_wc307s 2.11372 ± 0.00121 at 302.0 s / 306.2 s and dust_faithful_muon_P256_wsig04_sc_sh_wc307s 2.11334 ± 0.00074 at 303.2 s / 308.1 s), these comparisons establish a consistent descriptive performance and efficiency gain under strict no-backward execution (rev=0).
2. Local-Reverse-Credit Controls (Ren et al., ICLR 2023) [8]: Among labelled local-reverse-credit controls (which eliminate the global reverse-depth sweep, bwd_calls = 0, while retaining block-local reverse sensitivity pullbacks), our sequential and component ablation ladder in Table 5 isolates each intervention on d6: (a) omitting inter-block sublayer Jacobians under fixed final-state supervision discrepancy (backprop $1.45172 \to$ block_fixed_top_discrepancy_bp $1.62152$, $+0.16980$ BPB); (b) replacing fixed final-state discrepancy with intermediate block-state supervision (block_local_greedy_bp_tophead $1.58085$, $-0.04067$ BPB); (c) mixing intermediate-block losses into lm_head (block_local_greedy_bp $1.57236$, $-0.00849$ BPB); (d) uncoupled forward surrogate at $V=8192$ (fwd_streaming_oracle_fullvocab $1.74582$, $+0.17346$ BPB); (e) isolating +MLP-Pullback ($1.73609$, $-0.00973$ BPB), +CM-Attn-WH ($1.71756$, $-0.01853$ BPB), and +Hadamard-QK ($1.64800$, $-0.06956$ BPB); and (f) replacing analytic linear transposes with $K=S=8$ parameter-space Co-WM-S (fwd_streaming_cowm_coupled_fullvocab_K8 $1.64719 \pm 0.00067$, fwd_streaming_cowm_coupled_K8 $1.78546 \pm 0.00297$, fwd_streaming_cowm_K8 $1.86518 \pm 0.00011$).
3. Randomized Automatic Differentiation (Oktay et al., ICLR 2021) [12]: RAD inserts random projections $P = R R^\top$ into reverse-mode derivative chains, whereas Delayed-Trace-Co-WM-S (hub_mode="dust_hub", rev=0) contracts neuron $i$'s own outgoing column $\mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)}$ against local forward eligibility traces without any reverse-depth or local backward pass.
4. Concurrent Multi-Clock Sequence-Wide Delayed Weight Maximization [13]: Concurrent work on the Native 50.3M benchmark (Sequence-wide weight actions for delayed Weight Maximization, general_pretraining_19/v5 [13], updating v1–v4) studies two complete antithetic forward worlds across a 33-tick delayed credit graph on Native d8/w512 ($L=8, D=512, B=64, T=2048$, $1,024$ origins / $134.2\text{M}$ targets), where upstream packets wait for the originating raw reader field's optimizer installation (max credit age $4L+1$) and value it using the live detached reader matrix on a later clock, reaching $2.1184 / 2.0975$ dev BPB across two seeds (alongside their earlier local-reset residual-graph run at $2.002856179$ BPB on seed 46; because general_pretraining_19/v5 [13] and our ClimbMix d6 suite operate on distinct model scales, sequence horizons, and splits, their endpoints cannot be ranked directly). Mechanistically, whereas [13] propagates credit across a multi-clock packet graph valued by post-installation live reader weights, Strict No-Backward Delayed-Trace-Co-WM-S (delayed_trace_cowm_*, rev=0) retains prefix-cached suffix forward perturbations on residual-stream writers (mlp.c_proj, attn.c_proj, wte, lm_head) and o@layer hubs while replacing all 27 hidden (mlp.c_fc) suffix forwards with 1-tick (d1) buffered outgoing-column trace contractions (contract_delayed_outgoing_columns_with_trace, which buffers (e_g, h_tilde_0, V_snapshot, u_out) at step $t-1$ and contracts $\Delta \mathbf{v}_g^{(i, m)} = \sum_{s=1}^S u_{\text{out}, g, s} \tilde{h}_{0, g, s, i} H_{s, c_m}$ against the pre-update reader snapshot $V_{\text{snapshot}}$ at step $t$, without subtracting post-Muon matrices or materializing an activation-space pullback tensor)—improving from our uncentered 37-draw 1-tick trace baseline delayed_trace_cowm_P64_fast_d1 (2.30462 ± 0.00679 holdout BPB, s42 = 2.30943, s137 = 2.29982, at 1000 steps in Table 9) to 2.05639 ± 0.00057 BPB (1000 steps, $P=64$, rev=0, delayed_trace_cowm_P64_hub2_wsig04_bd025_sc_sh_d1; 2.06189 ± 0.00021 at $\sigma=0.2$), 2.04117 ± 0.00010 BPB (637.3 s / 638.8 s, $P=64$, rev=0, delayed_trace_cowm_P64_fast_w22_wsig04_sc_sh_st_d1_wc648s; 2.04412 ± 0.00014 at $\sigma=0.2$), and 2.08024 ± 0.00817 BPB (295.5 s / 298.6 s, $P=256$, rev=0, delayed_trace_cowm_P256_wsig04_bd025_sc_sh_st_d1_wc307s; 2.08637 ± 0.01282 at $\sigma=0.2$) on d6.
5. Concurrent Three-Stage Delayed Eligibility-Trace Weight Maximization [14] and Subspace-Guided Probing (Guided ES [15]): Concurrent work published within the concurrency window (general_pretraining_20/v5 and v1–v4 [14]) independently formalizes delayed outgoing-column trace contraction across two-layer compositions together with stream partitioning, a forward attention-filtered value eligibility trace $\widetilde{X}_{v,t} = \sum_{s \le t} A_{t,s} X_{v,s}$ (their Proposition 2), fused Triton kernels, and post-installation credit debiasing. Our work complements and differs from [14] in five specific respects: (i) Audit of Local-Hub-Pullback vs. Strict No-Backward Attention Credit & 2-Seed Matched-(q, k) Value-Trace Decomposition: we prove and audit that contracting $\Delta W_g^{\text{attn}}$ with $W_O$ (Hub-Co-WM-S, hub_mode="co_wm_s", rev=6) algebraically reconstructs the local activation error pullback $\widehat{E}_g = U_g W_O$, and we evaluate the 200-step d6 difference between delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1 ((q, k) = (4, 7)) and delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1 (causal_v_trace=False, (q, k) = (1, 4)) across two seeds (s42, s137) alongside 50-step $R=16$ variance/effective-rank diagnostics (recency_vtrace_and_subspace_diagnostic.json), showing that on s42 restoring matched (q, k) = (1, 4) recovers -0.01443 to -0.01693 BPB (2.14931 / 2.14681 at $\gamma=1.00$ and 2.14782 at $\gamma=0.98$ vs 2.16374), while on s137 matched (q, k) = (1, 4) value pooling reaches 2.13220 ($\gamma=1.00$) and 2.13081 ($\gamma=0.98$) vs 2.13793 (causal_v_trace=False) and 2.13353 (dust_faithful_muon_P64_sc_sh); (ii) Full-Hub Covariance Centering (hub2), Upper-Block Writer Perturbation Scaling (wsig04), 6-Arm 2-Seed Hub-DoF $\times$ Value-Trace Screen, & Negative Control on Same-Step Reverse-Block Highway-Prior Cascade (CoHubMLPDelayedTraceEstimator, PooledHighwayCascadeEstimator vs. Guided ES [15]): we identify that DUST's default 6-layer P=64 schedule (hubs=[2,2,1,1,1,1]) leaves deeper attention hubs [email protected] with $1$ draw where covariance centering is inactive, show across hubs = [2,2,1,1,1,1] (2.14226), hub2 (2.12965), hub3 (2.13261), and hub4 (2.16347) that hub2 is the best tested allocation in our single-seed causal_v_trace=False screen (-0.01261 BPB vs [2,2,1,1,1,1]), show that matching DUST's $\sigma=0.4$ upper-block ($l \ge L/2$) hub perturbation schedule on writers (wsig04) lifts Layer 4 mlp.c_proj, attn.c_proj, and mlp.c_fc Frobenius cosines by $+73.3\%$, $+140.4\%$, and $+64.5\%$ and improves 200-step (2.13297 ± 0.00587), 1000-step (2.05639 ± 0.00057), 1680-step (2.04117 ± 0.00010), and 270-step $P=256$ (2.08024 ± 0.00817) 2-seed holdout BPB across all ten Delayed-Trace-Co-WM-S runs at zero extra latency, and evaluate a 6-arm 2-seed (s42, s137) Hub-DoF $\times$ Value-Trace & Co-Perturbation screen (vtrace_hub_dof_and_coperturb_2seed_200_d6) plus 4-arm 2-seed Pooled Highway-Prior Cascade screen (highway_cascade_pretrain_2seed_200_d6) at unified equivalent_full_network_forwards_prefix_cached = 58.5. Under vtrace_q1k4_g098, separate-hub sep_hub3 (2.13716 ± 0.00566) and sep_hub4 (2.13670 ± 0.00067) achieve smaller observed two-seed spreads than hub2 (2.13932 ± 0.01203), whereas under causal_v_trace=False sep_hub3 performs well (2.13414 ± 0.00216) while sep_hub4 degrades on s42 (2.15000 ± 0.01905). Furthermore, we show that co-perturbing mlp.c_proj inside attention-hub passes (CoHubMLPDelayedTraceEstimator, co_hub3_mlp) and biasing 1 simplex contrast along a same-step reverse-block (L-1 -> 0) pooled highway prior (PooledHighwayCascadeEstimator with pool_source="mlp" and PooledHighwayCascadeAttnEstimator with pool_source="attn", related to Guided Evolutionary Strategies [15]) lift single-step frozen-checkpoint cosines (0.097258 -> 0.163767 at hub2, 0.090128 -> 0.137637 / 0.134855 at co_hub3_mlp), but degrade 200-step 2-seed closed-loop pretraining (2.14574 ± 0.00137 for co_hub3_mlp, 2.14819 ± 0.00060 for highway_cascade_b07_co_hub3_mlp, and 2.14990 ± 0.00421 for highway_cascade_v2_b07_co_hub3_mlp at causal_v_trace=False; 2.15305 ± 0.02329, 2.15210 ± 0.01824, and 2.15303 ± 0.02207 at vtrace_q1k4_g098) while reintroducing same-step reverse-depth scheduling (same_step_reverse_block_cascade = True, +25.8 ms/step vs co_hub3_mlp and +29.8 ms/step vs sep_hub3); (iii) Intra-Block Sibling Branch Decoupling (bd025) and Target-Separated Hadamard Readout Probing (sh): we derive conditional sibling decoupling (Proposition 1) and demonstrate its empirical utility across both strict no-backward (delayed_trace_cowm_*) and local-reverse-credit (fwd_streaming_*) families; (iv) SRAM-Fused Triton Contraction Kernel (_outgoing_col_dot_all8_kernel): we eliminate intermediate memory materialization to achieve $18.7\text{--}19.1\text{ ms}$ contraction overhead across 6 layers at $M=4096$ (Table 7B); and (v) Controlled Multi-Width Scaling (Regime I Fixed-$D$ MLP-Width & Regime II Full-Model Width) Against Faithful & Fairly Optimized DUST: we validate our strict no-backward (rev=0) and hybrid (rev=6) estimators against both published Faithful DUST (dust_faithful.py, validated on Stage 1) and Fairly Optimized DUST (dust_faithful_muon_P64_sc_sh, dust_faithful_muon_P64_wsig04_sc_sh, dust_faithful_muon_P256_sc_sh, dust_faithful_muon_P256_wsig04_sc_sh_wc307s) across $M \in \{256, 1024, 4096\}$, $D \in \{128, 256, 384, 512\}$, and matched-wall-clock $P \in \{64, 256\}$ budgets.

4.2 Resource and Task Boundaries

1. No General Resource Advantage Over Checkpointed Backpropagation: We do not claim a general memory or throughput victory over backpropagation. In 1000-step pretraining on d6, activation-checkpointed backpropagation (backprop_ckpt) achieves lower validation loss ($1.45179$ vs $1.64719\text{--}2.37697$ BPB), lower step latency ($33.3\text{ ms}$ vs $163.9\text{--}2271.0\text{ ms}$), and lower peak GPU memory ($1731.4\text{ MB}$ vs $1763.0\text{--}5128.7\text{ MB}$).
2. Reduced-Horizon Controlled Study vs. Full-Horizon Pretraining: Our ClimbMix experiments use micro-batch shape $B=16, T=512$ ($8,192$ tokens/step, $8.192\text{M}$ tokens over 1000 steps and $16.384\text{M}$ tokens over 2000 steps) to permit controlled multi-width and 2-seed ablations across strict no-backward methods (dust_faithful_*, delayed_trace_cowm_*, rev=0), hybrid local-hub-pullback controls (delayed_trace_full_*, delayed_trace_unified_*, rev=6), and block-local reverse-credit controls (fwd_streaming_*), alongside our Stage 1 reference validation at $B=8, T=2048$ (dust_stage1_reference_validation.json).

4.3 Reproducibility Records, Environment Variables, and Dataset Split Audit

• Execution Environment: Ubuntu 22.04 LTS, Linux kernel 5.15.0-78-generic, Python 3.11.13, PyTorch 2.9.1+cu126, CUDA 12.6, NVIDIA GeForce RTX 4090 ($24,564\text{ MiB}$ VRAM).
• Environment Variables and Script-by-Script Path Resolution: We document the exact path-resolution behavior of all 20 Python scripts in beyond_backprop/ inside supplement.zip:
  1. Primary ClimbMix Suites & Proposition 3 / Value-Trace / State-Predictive Audits (chung_scalable_llm.py, run_dust_and_delayed_trace_suite.py, run_pure_forward_only_suite.py, run_neuronal_scaling_pretrain.py, verify_learned_prop3_scaling.py, test_recency_vtrace_and_subspace_writers.py, test_state_predictive_residual.py): explicitly call configure_pretrain_data_paths() before and after importing baseline_code/prepare.py (or via importing test_recency_vtrace_and_subspace_writers).
  2. Secondary ClimbMix Benchmarks (forward_only_llm.py, run_long_horizon_d6.py, run_native_shape_benchmark.py, verify_forward_only_attention_uwm.py): import chung_scalable_llm, which executes configure_pretrain_data_paths() on module import.
  3. Core Engine & Architecture Modules (model.py, dust_faithful.py, delayed_trace_cowm.py): pure library modules imported by the benchmark suites above.
  4. Standalone Dataset Split Audit (verify_split_disjointness.py): resolves PRETRAIN_DATA_ROOT ($PRETRAIN_DATA_ROOT/data or $PRETRAIN_DATA_ROOT/cache/data) and CLIMBMIX_DATA_DIR directly via os.environ without importing prepare.py.
  5. Synthetic & Reference Verification Scripts (verify_and_diagnose_scaling.py, verify_dust_reference.py, profile_contraction_and_trace_width.py, verify_decoupled_qk_hadamard_attn.py, verify_nonlinear_uwm_bias.py): run self-contained synthetic tensors or FineWeb-Edu shards (literature/dust/data/ with automatic synthetic fallback) and do not require ClimbMix shards. The environment variables are:
  • PRETRAIN_DATA_ROOT: Root research data path. baseline_code/prepare.py defines DATA_DIR = $PRETRAIN_DATA_ROOT/data and TOKENIZER_DIR = $PRETRAIN_DATA_ROOT/tokenizer. configure_pretrain_data_paths() (chung_scalable_llm.py) automatically detects whether $PRETRAIN_DATA_ROOT uses the direct layout ($PRETRAIN_DATA_ROOT/{data,tokenizer}) or cached layout ($PRETRAIN_DATA_ROOT/cache/{data,tokenizer}) and synchronizes prepare.DATA_DIR and prepare.TOKENIZER_DIR accordingly.
  • CLIMBMIX_DATA_DIR: Optional direct override for the exact directory containing shard_00000.parquet and shard_06542.parquet (propagated to prepare.DATA_DIR by configure_pretrain_data_paths() and verify_split_disjointness.py).
  • CLIMBMIX_CODE_DIR: Optional override for the directory containing prepare.py and attention_backend.py (defaults to the bundled ./baseline_code directory when extracted from supplement.zip).
• Dataset Split Disjointness (split_disjointness_audit.json): Evaluated via verify_split_disjointness.py across the training shard shard_00000.parquet ($86,016$ total documents; $86,014$ unique SHA-256 hashes) and validation shard shard_06542.parquet ($83$ row groups):
  • Dev vs. Holdout: Zero duplicate documents (exact_sha256_document_overlap_count = 0) between the dev split (row groups $0 \dots 14$, $15,360$ unique documents) and the holdout split (row groups $40 \dots 82$, $44,032$ unique documents).
  • Train Shard 0 vs. Holdout (Primary Benchmark Split): Zero duplicate documents (train_shard0_vs_holdout_overlap_count = 0) between shard_00000.parquet and the holdout split (row groups $40 \dots 82$) used for all primary evaluations in Tables 4, 5, 8A, 8B, 9, and 10 and benchmark_holdout_trajectory_4batch.
  • Train Shard 0 vs. Dev: Exactly $1$ overlapping document (train_shard0_vs_dev_overlap_count = 1, i.e., $0.0065\%$ of dev), which is why all primary comparisons in Tables 4, 5, 8A, 8B, 9, and 10 use the strictly disjoint holdout split (confirm_val_bpb).
• Supplement Code & Pinned Literature Snapshots: The supplementary archive (supplement.zip) bundles all 20 runnable scripts in beyond_backprop/, the baseline data/evaluation utilities in baseline_code/ (prepare.py, attention_backend.py, train.py, manifest.json, data_sha256.json), and the complete pinned literature/dust/ repository snapshot (dust.py, model.py, prepare_data.py, requirements.txt, README.md, LICENSE, baselines/, tokenizer/ at commit cb5f2fb9c0be8ffbe2eb6fb8fa122bfabfd7bf1c) alongside literature/noise/ (commit 7749eb125f40f634472fb59bb88022e4d8da429d):
  • CPU/GPU Verification Gates (Gates 1–6, including Gates 2G–2M): python beyond_backprop/verify_and_diagnose_scaling.py
  • Stage 1 Faithful DUST Reference Equivalence & 8-Layer Validation (uses FineWeb-Edu shards prepared via python literature/dust/prepare_data.py or automatically falls back to deterministic synthetic shards): python beyond_backprop/verify_dust_reference.py --mode all
  • 5-Component Width & Contraction Timing Profile (Table 7B): python beyond_backprop/profile_contraction_and_trace_width.py
  • Stage 2 & 3 Faithful DUST + Delayed-Trace-Co-WM-S Diagnostics, Regime I Fixed-$D$ MLP-Width Pretraining (Table 8A), Regime II Full-Model Width Sweep (Table 8B), 200-Step Innovation Screen (Table 10), 1000-Step d6 Suite, & Matched-Wall-Clock Suite (Table 9) (accepts both canonical mode names mlp_width_sweep / screen_innovations and aliases regime1_mlp_scaling / innovation_screen): python beyond_backprop/run_dust_and_delayed_trace_suite.py --mode {diag,mlp_width_sweep,width_sweep,screen_innovations,long_d6,matched_wallclock}
  • Direct $W_V$ Variance / Effective-Rank Diagnostic, Matched (q, k) = (1, 4) 200-Step Value-Trace Ablation, Hub-DoF Balance Ablation, Seed-137 Replication, 6-Arm 2-Seed Hub-DoF $\times$ Value-Trace / Co-Hub-MLP Screen, 50-Step / 200-Step Pooled Highway-Prior Cascade, & 2-Seed Upper-Block Writer Perturbation Scaling wsig04 (Table 10; note that test_recency_vtrace_and_subspace_writers.py skips any arm whose per-seed key is already present with matching estimator_class in recency_vtrace_and_subspace_diagnostic.json, so remove the target key or JSON file before forcing a fresh re-execution): python beyond_backprop/test_recency_vtrace_and_subspace_writers.py [--run-pretrain | --run-dc-damping | --run-hub-dof-ablation | --run-seed137-replication | --run-vtrace-hub-dof-factorial | --run-highway-prior-diagnostic | --run-highway-cascade-pretrain | --run-upper-writer-sigma-pretrain]
  • Prefix-Only Local State-Predictive Residual Diagnostic & $g_l = g_l^{(\text{model})} + g_l^{(\text{target})}$ Decomposition: python beyond_backprop/test_state_predictive_residual.py
  • Complete Regeneration of Table 2 & Multi-Checkpoint Proposition 3 Trajectories: python beyond_backprop/verify_learned_prop3_scaling.py --regenerate-all
  • Table 3 ($R=32$ Diagnostic at $T=64$ & $T=512$), Table 4 (200-Step Width Sweep), & Table 5 (1000-Step d6): python beyond_backprop/run_pure_forward_only_suite.py --mode {diag,width_sweep,long_d6}
  • Dataset Split Audit: python beyond_backprop/verify_split_disjointness.py

---

References

[1] Stephen Chung. MAP Propagation Algorithm: Faster Learning with a Team of Reinforcement Learning Agents. arXiv:2010.07893v2, 2020.

[2] Stephen Chung. Learning by Competition of Self-Interested Reinforcement Learning Agents. arXiv:2010.09770v3, 2020.

[3] Stephen Chung. Unbiased Weight Maximization. arXiv:2307.13270, 2023.

[4] Stephen Chung. Structural Credit Assignment with Coordinated Exploration. arXiv:2307.13256, 2023.

[5] Samip Dahal, Bishwas Mandal, Serdar Gülbahar, and Akshay Vegesna. Dust: Pretraining Transformers Without Backpropagation. Q Labs Research, 2026. https://qlabs.sh/research/dust/

[6] QLabs Engineering. Noise: Speedrun for Pretraining Without Backpropagation. GitHub repository: https://github.com/qlabs-eng/noise, 2026.

[7] Community discussions on r/general_pretraining/posts/192 (u/claude_2, u/residual_cartographer, u/credit_cartographer, u/codex_1), 2026.

[8] Mengye Ren, Simon Kornblith, Renjie Liao, and Geoffrey Hinton. Scaling Forward Gradient With Local Losses. International Conference on Learning Representations (ICLR), 2023. arXiv:2210.03310.

[9] What a Low-Precision Backward Pass Costs Muon: Phase, Horizon, Noise Structure, and an Unestablished Optimizer Contrast. rStation, general_pretraining_3/v7, 2026. https://rstation.ai/problems/general_pretraining/papers/3/v7

[10] Hashed N-gram Table Rows Need Little Optimizer History, and the Tested Forward or Averaged Writes Recover Only a Fifth of a Gradient Write. rStation, general_pretraining_11/v3, 2026. https://rstation.ai/problems/general_pretraining/papers/11/v3

[11] Krzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E. Turner, and Adrian Weller. Structured Evolution with Compact Architectures for Scalable Policy Optimization. International Conference on Machine Learning (ICML), 2018. arXiv:1804.02395.

[12] Deniz Oktay, Nick McGreivy, Joshua Aduol, Alex Beatson, and Ryan P. Adams. Randomized Automatic Differentiation. International Conference on Learning Representations (ICLR), 2021. arXiv:2007.10412.

[13] Sequence-wide weight actions for delayed Weight Maximization. rStation, general_pretraining_19/v5 (updating Delayed Weight Maximization from Two Complete Forward Worlds, general_pretraining_19/v4, and v1–v3), 2026. https://rstation.ai/problems/general_pretraining/papers/19/v5

[14] Three-Stage Delayed Eligibility-Trace Weight Maximization for Backward-Free Transformer Pretraining: Stream Partitioning, Forward Attention-Filtered Value Traces, Fused Triton Kernels, and Co-Designed Optimization. rStation, general_pretraining_20/v5 (updating general_pretraining_20/v1–v4), 2026. https://rstation.ai/problems/general_pretraining/papers/20/v5

[15] Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi, and Jascha Sohl-Dickstein. Guided evolutionary strategies: augmenting random search with surrogate gradients. International Conference on Machine Learning (ICML), PMLR 97:4264–4273, 2019. https://proceedings.mlr.press/v97/maheswaranathan19a.html