# Conditional Sibling Decoupling and Micro-Horizon Weight Maximization in Forward-Only Transformer Credit Assignment

## 1. Introduction and Architectural Ledger

### 1.1 The Width-Scaling Variance Bottleneck in Forward-Only Credit
Eliminating the global reverse-depth sweep of backpropagation is a central pursuit in neuromorphic computing and biologically plausible credit assignment [1, 2, 8]. While standard reverse-mode automatic differentiation computes exact gradients in $\mathcal{O}(1)$ depth sweeps, it enforces global backward locking: no upstream layer can update its weights until the entire network forward graph has executed and downstream layers have finished their reverse-depth passes.

Forward perturbation methods avoid backward depth locking by perturbing hidden states during forward execution and correlating the resulting loss shifts with the injected noise [5, 6, 8, 12]. However, conventional forward perturbation rules face a severe **neuronal width-scaling bottleneck**:
- **Global Scalar-Reward Perturbation Control (`fwd_global_scalar_K2`)**: Simultaneous zero-mean perturbation across all parameters or activations correlated with a single global scalar loss shift $\Delta \mathcal{L}$ yields estimator variance that scales with the total parameter count $\mathcal{O}(|\Theta| / K)$. (Note: `fwd_global_scalar_K2` is a simplified global-scalar control, not published DUST [5]; see Section 2.6 and Section 3.5 for our two-stage faithful DUST baseline.) In a 6-layer Transformer at $D=512$ ($M=2048$), relative variance of `fwd_global_scalar_K2` at Layer 3 (`midMLP`) explodes to $3.26 \times 10^7$ (Table 3), stalling 200-step pretraining at $2.3898$ holdout BPB (Table 4) and 1000-step pretraining on `d6` ($D=384$) at $2.44794 \pm 0.11598$ holdout BPB (Table 5).
- **Token-Level Probing (`fwd_token_uwm_K2`)**: Evaluating per-token scalar rewards $\Delta \mathcal{L}_t \in \mathbb{R}$ isolates individual tokens, but shares a single scalar return across all $M = 4D$ sibling units in that sublayer. As Ren et al. [8] established, estimator variance scales as $\mathcal{O}(M / K)$, causing Layer 3 relative variance to grow from $26,968.01$ ($D=128$) to $103,931.10$ ($D=512$) (Table 3) and limiting 200-step holdout BPB to $2.1868$ at $D=512$ (Table 4) and 1000-step `d6` holdout BPB to $2.09816 \pm 0.00521$ (Table 5).

### 1.2 Stephen Chung's Foundations and Sylvester-Hadamard Coordination
To overcome this width bottleneck without a global backward pass, we adapt Stephen Chung's credit principles:
1. **Weight Maximization (`arXiv:2010.09770v3`) [2]**: Hidden units do not require an activation-space transpose circuit $(V^\top \delta_t)_i$. Instead, each unit $i$ observes only the plasticity displacement $\Delta \mathbf{v}^{(i)}$ of its own outgoing synaptic column $\mathbf{v}^{(i)} = V_{:, i}$, reinforcing activity when its outgoing column norm expands ($G^{(i)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}^{(i)}$).
2. **Unbiased Weight Maximization (`arXiv:2307.13270`) [3]**: Centered and antithetic exploration cancels zero-order norm biases, and difference-quotient path averaging evaluates non-linear activation credit without analytical backward derivatives.
3. **Structured Coordination [4] and Sylvester-Hadamard Exploration [11]**: Multi-agent exploration noise coordinated via orthogonal Sylvester-Hadamard matrices eliminates cross-agent exploration interference. Following Choromanski et al. [11] (`arXiv:1804.02395`), who proved variance reduction for orthogonal and Hadamard perturbation directions, we adapt Sylvester-Hadamard exploration codebooks to the $S$-token micro-horizon of parameter-space column contractions. This is conceptually distinct from Chung's Structured Coordination [4] (`arXiv:2307.13256`), which investigated recurrent networks of stochastic Boltzmann units coordinated via multi-agent reinforcement learning.
4. **MAP Propagation (`arXiv:2010.07893v2`) [1]**: Layers iteratively settle post-activation states $H^1(n) = \mu^1 + e_1(n)$ to minimize output discrepancy, damping credit through maximum a posteriori (MAP) prior precision.

### 1.3 Strict No-Backward Credit vs. Global Reverse-Depth Removal with Local Reverse Credit Computations
We enforce a strict conceptual and experimental separation between **strict no-backward credit assignment** (`local_reverse_derivative_calls = 0`) and **methods that remove the global reverse-depth sweep while retaining local reverse pullbacks**:
1. **Strict No-Backward Credit Assignment (Zero Reverse-Depth Sweeps AND Zero Local Reverse Pullbacks, `local_reverse_derivative_calls = 0`)**:
   - **Published Faithful DUST (`dust_faithful_sgd_P64`, `dust_faithful_muon_P64`, `dust_faithful_muon_P256`) and Fairly Optimized DUST (`dust_faithful_muon_P64_sc_sh`, `dust_faithful_muon_P256_sc_sh`) [5]**: Perturbs activation groups during forward execution (`writers`, `hidden`, `hubs` `o@layer`, `wte`, 256-column `lm_head` slabs) and correlates per-token centered loss rewards (and decayed future-token attention head rewards $\gamma=0.98$) directly with pre-synaptic states (`dust_faithful.py`, `delayed_trace_cowm.py`).
   - **Strict No-Backward Delayed / Eligibility-Trace Coordinated Weight Maximization (`Delayed-Trace-Co-WM-S`, `delayed_trace_cowm_*`, `hub_mode="dust_hub"`, `local_reverse_derivative_calls = 0`)**: Records local forward eligibility traces on `mlp.c_fc` during the clean forward pass using $S$-token Sylvester-Hadamard probes ($\xi_{g, s}^{(m)} = H_{s, c_m}$) and local forward activation evaluations $\phi(z \pm \sigma_{\text{act}})$, and modulates those pre-synaptic traces via 1-step cross-batch delayed (`d1`) or same-step (`d0`) outgoing-weight column contractions $c_g^{(i, m)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)}$ obtained from outgoing `mlp.c_proj` writer updates (`_outgoing_col_dot_all8_kernel` in `delayed_trace_cowm.py`). Critically, `delayed_trace_cowm_*` **never reconstructs an activation-space error vector** on `mlp.c_fc` and estimates attention hubs `o@layer` (`targets[layer]`) via DUST's own forward suffix perturbations (`site_kind="hub"`, `hub_mode="dust_hub"`), requiring **zero direct `hidden` (`mlp.c_fc`) forward passes** and **zero local reverse pullbacks (`local_reverse_derivative_calls = 0`)**. Reallocating 4 of the 27 saved `hidden` draws at $P=64$ to `hubs` (`hubs = [2, 2, 2, 2, 2, 2]`, `hub2`) and 23 to `writers` (`writers = [12, 10, 8, 7, 6, 6]`) unlocks full-block simplex covariance centering (`sc`, `draws >= 2`) across all 6 attention hubs while intra-block sibling branch decoupling (`bd025`), target-separated Hadamard readout probing (`sh`), and forward scalar traces (`st`) remain 100% forward-only.
2. **Global Reverse-Depth Removal with Local Reverse Credit Computations (Labelled Local-Reverse-Credit and Hybrid Controls)**:
   - **Hybrid Delayed-Trace + Local Attention-Hub Pullback Controls (`delayed_trace_full_*`, `delayed_trace_unified_*`, `hub_mode="co_wm_s"`, `local_reverse_derivative_calls = L = 6`)**: When `hub_mode="co_wm_s"` is enabled (`contract_hub_outgoing_columns`), micro-group $g$ forms $\Delta W_g = H^\top U_g$, contracts $C_g = \Delta W_g W_O$, and reconstructs the upstream activation-space tensor $\widehat{E}_g = \frac{1}{S} H C_g = U_g W_O \in \mathbb{R}^{B \times T \times D}$ at $K_{\text{wm}} = S = 8$ to supply `targets[layer]` to `local_errors` on the same step. Because $\widehat{E}_g = U_g W_O$ executes the 1-layer local linear reverse pullback of `attn.c_proj` ($Y = O W_O^\top$) in the Sylvester-Hadamard token basis, all `hub_mode="co_wm_s"` variants (`delayed_trace_full_*` and `delayed_trace_unified_*`) are explicitly classified and audited (`local_reverse_derivative_calls = 6`, `is_pure_forward_only = False`, `has_local_backward_phase = True`) as **hybrid local-hub-pullback controls** rather than strict no-backward methods.
   - **Streaming Stagewise Local-Reverse-Credit Controls (`fwd_streaming_cowm*`, `fwd_streaming_oracle*`, `fwd_lookahead_2b*`)**: `@torch.no_grad()` is enforced (`num_backward_calls = 0`, `num_depth_reverse_sweeps = 0`) and no global backward graph is instantiated. However, within each detached block, these methods retain **local reverse credit computations** (`d_relu2`, `_forward_rms_norm_directional_pull`, inverse rotary pullback `apply_rotary_emb(cos, -sin)`, and intra-block MLP-to-Attention residual pullback). Replacing analytical derivative formulas with central forward operator differences (`OpDiff`, `*_opdiff_*`) still evaluates local reverse sensitivity pullbacks and therefore does **not** remove backward credit computation. Accordingly, backpropagation (`backprop`, `backprop_ckpt`), block-detached autograd (`block_*_bp`), and all coupled local-error / analytic-surrogate / `OpDiff` / `Lookahead-2B` streaming variants are presented strictly as **labelled local-reverse-credit controls**.

We organize our study across three architectural paradigms:
- **Paradigm A: Two-Level Coordinated Weight Maximization (`fwd_two_level_cowm`)**: Combines parameter-space `Co-WM-S` column contraction on `lm_head` over micro-horizons of $S=8$ tokens, $2 K_{\text{net}}$ end-to-end antithetic full-network passes ($x_0 \to x_L$) with either depth-Hadamard carrier modulation (`net_carrier_mode="depth_hadamard"`) or independent per-probe carrier resampling (`net_carrier_mode="iid_resample"`), and intra-block parameter-space `Co-WM-S` on MLP projections ($W_1, V$) with head-decoupled forward FlashAttention probing on attention matrices ($W_Q, W_K, W_V, W_O$).
- **Paradigm B: Streaming Stagewise Forward Co-WM-S and Local-Reverse-Credit Controls (`fwd_streaming_cowm*`, `fwd_lookahead_2b*`)**: Executes a single forward sweep through the network ($l = 0 \dots L-1$). At each block $l$:
  1. Forward output $x_l$ is evaluated and correlated with the language modeling target via an active sub-vocabulary projection ($V_{\text{sub}} = 256$ active tokens, or $V_{\text{sub}}=0$ for full vocabulary $V$);
  2. Block $l$'s parameters ($W_Q, W_K, W_V, W_O, W_1, V$) compute gradients blockwise into `.grad` (either uncoupled in `fwd_streaming_cowm`, where both sublayers receive $\delta_{\text{block}}$; intra-block coupled in `fwd_streaming_cowm_coupled*`, where the MLP update runs first and pulls upstream credit across $W_1$ via parameter-space `Co-WM-S` into the attention sublayer; or 2-block cooperative lookahead in `fwd_lookahead_2b_orth_mv*`, where a sliding 2-block window $(l, l+1)$ transports block $l+1$'s zero-variance MLP + Attention-Value input credit projected onto $\delta_{\text{local}, l}^\perp$ into block $l$);
  3. **Block Deallocation**: All intermediate activations of block $l$ ($x_{\text{prev}}, x_{\text{attn\_in}}, u_{\text{attn}}, x_{\text{mlp\_in}}, u_{\text{mlp}}$) are **freed from GPU memory immediately** before block $l+1$ (or $l+2$ under 2-block lookahead) begins execution, maintaining $\mathcal{O}(1)$ activation memory in depth $L$;
  4. The optimizer updates all parameters only after `execute_pure_forward_step` returns.
- **Paradigm C: Strict No-Backward Credit Assignment (`dust_faithful_*`, `delayed_trace_cowm_*`) and Hybrid Local-Hub-Pullback Controls (`delayed_trace_full_*`, `delayed_trace_unified_*`)**: Detailed in Section 2.6 and evaluated in Section 3.5 across Stage 1 reference validation (`dust_stage1_reference_validation.json`), 5-component latency profiling across widths (`contraction_and_trace_width_profile.json`), pure hidden-neuron diagnostic scaling ($M \in \{256 \dots 4096\}$, `dust_and_delayed_trace_width_diagnostic.json`), Regime I 2-seed pure MLP-width pretraining scaling at fixed $D=256$ (`regime1_fixed_D256_mlp_width_pretrain_200.json`), Regime II full-model width pretraining scaling (`dust_and_delayed_trace_width_sweep_200.json`), 200-step component ablation screening (`innovation_screen_200_d6.json`), and 1000-step 2-seed `d6` pretraining under matched-token and matched-wall-clock budgets (`dust_and_delayed_trace_1000steps_d6.json`).

To evaluate the **incremental effects** of successive training interventions in one specified ordering between end-to-end backpropagation and streaming `Co-WM-S`, we construct a sequential ablation ladder:
1. **`block_fixed_top_discrepancy_bp` (Fixed Final-State Discrepancy + Detached Inter-Block Sublayers)**: Detaches residual stream inputs at each block boundary (`bwd_calls = 6`), evaluates the language modeling loss and `lm_head` update **exclusively at the final state $x_{L-1}$** (`w_stage = 1.0` on $L-1$, $0.0$ on $l < L-1$), and broadcasts the fixed final-state pre-norm discrepancy $\delta_{\text{final}} = \frac{\partial \mathcal{L}(x_{L-1})}{\partial x_{L-1}}$ directly into each detached block $l$ via the residual identity skip connection (`x_l.backward(delta_final)`). Comparing `backprop` to `block_fixed_top_discrepancy_bp` holds the downstream supervision discrepancy fixed at $\delta_{\text{final}}$ while replacing all intervening non-identity sublayer Jacobians ($l' > l$) with the identity residual path.
2. **`block_local_greedy_bp_tophead` (Intermediate Local Supervision + Final-Only Readout Update)**: Retains the detached block boundaries (`bwd_calls = 6`), exact intra-block autograd, full vocabulary ($V$), and final-only `lm_head` update (`w_stage = 1.0` on $L-1$, $0.0$ on $l < L-1$) of `block_fixed_top_discrepancy_bp`, but replaces the broadcast final-state discrepancy $\delta_{\text{final}}$ with each block's own intermediate local-loss discrepancy $\frac{\partial \mathcal{L}(x_l)}{\partial x_l}$.
3. **`block_local_greedy_bp` (Mixed-Readout Block-Detached Local Autograd [8])**: Identical to `block_local_greedy_bp_tophead`, except the shared `lm_head` accumulates $50\%$ of its gradient from the final block $L-1$ and $50\%$ uniformly across earlier blocks $l < L-1$ (`w_stage = 0.5 / (L-1)`), matching the readout update rule of `fwd_streaming_*`. (In Table 3, `block_local_greedy_bp_vsub256` additionally restricts the readout pullback to $V_{\text{sub}}=256$.)
4. **Incremental Intra-Block and 2-Block Local-Reverse-Credit Controls at Full Vocabulary ($V_{\text{sub}}=0$)**:
   - `fwd_streaming_oracle_fullvocab` (Uncoupled Baseline): Feeds $\delta_{\text{block}}$ into both MLP and Attention independently, with single-window `CM-Attn` ($K_v=1$) and pre-RoPE $K_{\text{qk}}=2$ attention probes.
   - `fwd_streaming_oracle_mlp_pull_only_fullvocab` (**+MLP-Pullback Only**): Adds only the upstream MLP-to-Attention residual pullback $\delta u_{\text{attn}} = \delta_{\text{block}} + \Delta_{\text{mlp}\to\text{attn}}$ (`intra_block_coupling=True`), keeping $K_v=1, K_{\text{qk}}=2$ and `f32_hadamard_qk=False`.
   - `fwd_streaming_oracle_mlp_whv_fullvocab` (**+MLP-Pullback + `CM-Attn-WH` $K_v=8$**): Additionally enables Two-Level Window-Hadamard value transport (`K_v=8`), keeping $K_{\text{qk}}=2$ and `f32_hadamard_qk=False`.
   - `fwd_streaming_oracle_coupled_fullvocab` (**+Combined `Hadamard-QK` Package, $K_{\text{qk}}=8$**): Jointly transitions Q/K probing from $K_{\text{qk}}=2$ pre-RoPE i.i.d. probes to the post-RoPE head-channel Sylvester-Hadamard Q/K package (`K_qk=8, f32_hadamard_qk=True`, $\sigma_{\text{qk}}=0.05$, `float32` perturbation construction/reward accumulation with `bfloat16` FlashAttention execution, and 16-lag scalar key lookahead), paired with `fwd_streaming_cowm_coupled_fullvocab_K8` ($K=S=8$ parameter-space `Co-WM-S`).
   - `fwd_streaming_oracle_coupled_qproj_unbiased_fullvocab` (**+Bilinear Query-Subspace Projected Key Probing `Q-Proj-K`**): Holds $K_{\text{qk}}=8$, $\sigma_{\text{qk}}=0.05$, post-RoPE perturbation location, and `float32`/`bfloat16` precision handling strictly identical to `fwd_streaming_oracle_coupled_fullvocab`, replacing **only** the 16-lag scalar key correlation with bilinear rank-1 query-subspace projection (`qproj_k="unbiased"`), paired with `fwd_streaming_cowm_coupled_qproj_opdiff_fullvocab_K8` ($K=S=8$ parameter-space `Co-WM-S` + `OpDiff`).
   - `fwd_lookahead_2b_orth_mv_a1.0_opdiff_qproj_fullvocab` (**+Adjacent 2-Block Orthogonal-Complement Cooperative Lookahead `Lookahead-2B-OrthMV` + `OpDiff`**): Adds sliding 2-block orthogonal-complement lookahead transport (`lookahead_2b="orth_mv", lookahead_alpha=1.0, opdiff=True`).
5. **Sub-Vocabulary Readout Truncation ($V_{\text{sub}}=256$)**: `fwd_streaming_oracle`, `fwd_streaming_oracle_coupled`, and their parameter-space `Co-WM-S` counterparts `fwd_streaming_cowm_K8`, `fwd_streaming_cowm_coupled_K8`, `fwd_streaming_cowm_coupled_qproj_opdiff_K8`, and `fwd_lookahead_2b_orth_mv_cowm_opdiff_qproj_K8` restrict the readout pullback to $V_{\text{sub}}=256$ active columns.

Table 1 provides the complete arithmetic and pass-count ledger for all studied algorithms ($B=16, T=512$, $N_{\text{tok}}=8192$, micro-horizon $S=8$, $L=6$ blocks). Across widths $D \in \{128, 256, 384, 512\}$, exact trainable parameter counts are $6,422,732$, $15,204,748$, $26,346,060$, and $39,846,668$ for the pretraining vocabulary $V=8192$ (Tables 1, 4, 5, 8, 9), and $3,801,292$, $9,961,868$, $18,481,740$, and $29,360,908$ for the synthetic diagnostic vocabulary $V=4096$ (Table 3).

| Algorithm | Execution Paradigm | Full Forward Passes | Backward Calls | Depth Reverse Sweeps | Attn Forward Calls / Block | MLP Forward Calls / Block | Param-Space WM Contractions | Peak VRAM ($D=512$, $M=2048$) |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **`backprop`** (Standard) | Autograd Forward-Backward | $1$ | $1$ | $1$ | $1$ clean | $1$ clean | $0$ | $3010.4\text{ MB}$ |
| **`backprop_ckpt`** (Checkpointed) | Autograd + Block Recompute | $2$ | $1$ | $1$ | $2$ ($1$ fwd + $1$ recompute) | $2$ ($1$ fwd + $1$ recompute) | $0$ | $1731.4\text{ MB}$ |
| **`block_fixed_top_discrepancy_bp`** | Block-Detached BP (Fixed Top $\delta_{\text{final}}$) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $3026.5\text{ MB}$ |
| **`block_local_greedy_bp_tophead`** | Block-Detached BP (Local $\delta_l$, Top Head) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $2153.6\text{ MB}$ |
| **`block_local_greedy_bp`** | Block-Detached BP (Mixed Head [8]) | $1$ | $6$ ($1$/block) | $0$ (intra-block only) | $1$ clean | $1$ clean | $0$ | $2169.6\text{ MB}$ |
| **`fwd_global_scalar_K2`** | Pure Forward (Global-Scalar Control) | $3$ | **$0$** | **$0$** | $3$ ($1$ clean + $2$ probe) | $3$ ($1$ clean + $2$ probe) | $0$ | $3864.8\text{ MB}$ |
| **`fwd_token_uwm_K2`** | Pure Forward (Token Probing) | $5$ | **$0$** | **$0$** | $5$ ($1$ clean + $4$ probe) | $5$ ($1$ clean + $4$ probe) | $0$ | $3864.8\text{ MB}$ |
| **`dust_faithful_sgd_P64`** | Strict No-Backward (Faithful DUST [5], SGD, `rev=0`) | $1 + 64$ suffix | **$0$** | **$0$** | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $3035.5\text{ MB}$ |
| **`dust_faithful_muon_P64`** | Strict No-Backward (Faithful DUST [5], Muon, `rev=0`) | $1 + 64$ suffix | **$0$** | **$0$** | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $3083.7\text{ MB}$ |
| **`dust_faithful_muon_P64_sc_sh`** | Strict No-Backward (Fairly Opt. DUST, `sc+sh`, `rev=0`) | $1 + 64$ suffix | **$0$** | **$0$** | $45.3$ ($272$ block evals) | $45.3$ ($272$ block evals) | $0$ | $4631.9\text{ MB}$ |
| **`delayed_trace_cowm_P64_fast_d1`** | Strict No-Backward (`Delayed-Trace-Co-WM-S`, `rev=0`) | $1 + 37$ suffix | **$0$** | **$0$** | $23.7$ ($142$ block evals) | $23.7$ ($142$ block + $1$ local) | $49,152$ | $3733.9\text{ MB}$ |
| **`delayed_trace_cowm_P64_realloc_d1`** | Strict No-Backward (`Delayed-Trace-Co-WM-S`, `rev=0`) | $1 + 64$ suffix | **$0$** | **$0$** | $44.2$ ($265$ block evals) | $44.2$ ($265$ block + $1$ local) | $49,152$ | $3733.9\text{ MB}$ |
| **`delayed_trace_cowm_P64_fast_sc_sh_d1`** | Strict No-Backward (`Delayed-Trace` Fast + `bd025+sc+sh`, `rev=0`) | $1 + 41$ suffix | **$0$** | **$0$** | $26.0$ ($156$ block evals) | $26.0$ ($156$ block + $1$ local) | $49,152$ | $4977.9\text{ MB}$ |
| **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** | Strict No-Backward (`Delayed-Trace` + `hub2+bd025+sc+sh`, `rev=0`) | $1 + 64$ suffix | **$0$** | **$0$** | $41.0$ ($246$ block evals) | $41.0$ ($246$ block + $1$ local) | $49,152$ | $4977.9\text{ MB}$ |
| **`delayed_trace_cowm_P256_bd025_sc_sh_st_d1`** | Strict No-Backward (`Delayed-Trace` $P=256$ + `bd025+sc+sh+st`, `rev=0`) | $1 + 146$ suffix | **$0$** | **$0$** | $92.3$ ($554$ block evals) | $92.3$ ($554$ block + $1$ local) | $49,152$ | $4476.8\text{ MB}^\dagger$ |
| `delayed_trace_full_P64_fast_d1` | Hybrid (`Delayed-Trace` + 1-Layer Hub Pullback, `rev=6`) | $1 + 29$ suffix | **$0$** | **$0$** | $18.2$ ($109$ block evals) | $18.2$ ($109$ block + $1$ local) | $61,440$ | $4706.2\text{ MB}$ |
| `delayed_trace_full_P64_realloc_d1` | Hybrid (`Delayed-Trace` + 1-Layer Hub Pullback, `rev=6`) | $1 + 64$ suffix | **$0$** | **$0$** | $41.3$ ($248$ block evals) | $41.3$ ($248$ block + $1$ local) | $61,440$ | $4706.2\text{ MB}$ |
| `delayed_trace_unified_P64_fast_sc_sh_d1` | Hybrid (Unified Fast + `sc+sh`, `rev=6`) | $1 + 29$ suffix | **$0$** | **$0$** | $18.2$ ($109$ block evals) | $18.2$ ($109$ block + $1$ local) | $61,440$ | $5956.4\text{ MB}$ |
| `delayed_trace_unified_P64_bd025_sc_sh_d1` | Hybrid (Unified Realloc + `bd025+sc+sh`, `rev=6`) | $1 + 64$ suffix | **$0$** | **$0$** | $41.3$ ($248$ block evals) | $41.3$ ($248$ block + $1$ local) | $61,440$ | $5956.4\text{ MB}$ |
| **`delayed_trace_cowm_fixed16_d1`** | Strict No-Backward (`Delayed-Trace-Co-WM-S`, All-Block 16w, `rev=0`) | $1 + 27$ suffix | **$0$** | **$0$** | $20.7$ ($124$ block evals) | $20.7$ ($124$ block + $1$ local) | $49,152$ | $3035.7\text{ MB}^\dagger$ |
| **`dust_faithful_muon_P256`** | Strict No-Backward (Faithful DUST [5], $P=256$, `rev=0`) | $1 + 256$ suffix | **$0$** | **$0$** | $180.0$ ($1080$ block evals) | $180.0$ ($1080$ block evals) | $0$ | $3369.7\text{ MB}^\dagger$ |
| **`delayed_trace_cowm_P256_fast_d1`** | Strict No-Backward (`Delayed-Trace-Co-WM-S`, $P=256$ Fast, `rev=0`) | $1 + 146$ suffix | **$0$** | **$0$** | $92.3$ ($554$ block evals) | $92.3$ ($554$ block + $1$ local) | $49,152$ | $3667.9\text{ MB}^\dagger$ |
| `delayed_trace_unified_P256_sc_sh_st_d1` | Hybrid (Unified $P=256$ + `sc+sh+st`, `rev=6`) | $1 + 146$ suffix | **$0$** | **$0$** | $95.7$ ($574$ block evals) | $95.7$ ($574$ block + $1$ local) | $61,440$ | $4828.2\text{ MB}^\dagger$ |
| **`fwd_two_level_cowm_K4`** | Pure Forward (End-to-End Co-WM) | $9$ | **$0$** | **$0$** | $19$ ($1$ clean + $18$ probe) | $10$ ($1$ clean + $9$ probe/recomp) | $106,496$ | $1850.3\text{ MB}$ |
| **`fwd_streaming_cut_K4`** | Pure Forward (Streaming Cut) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $6$ ($1$ clean + $5$ probe/recomp) | $24,576$ | **$1820.3\text{ MB}$** |
| **`fwd_streaming_wm_K1`** ($K_{\text{qk}}=1$) | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $7$ ($1$ clean + $6$ probe) | $2$ ($1$ clean + $1$ local fwd) | $18,432$ | **$1828.4\text{ MB}$** |
| **`fwd_streaming_wm_K1_qk2`** ($K_{\text{qk}}=2$) | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $18,432$ | **$1820.3\text{ MB}$** |
| **`fwd_streaming_wm_K4`** | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $73,728$ | **$1820.3\text{ MB}$** |
| **`fwd_streaming_cowm_K4`** | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $73,728$ | **$1828.4\text{ MB}$** |
| **`fwd_streaming_wm_K8`** | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $147,456$ | **$1820.3\text{ MB}$** |
| **`fwd_streaming_cowm_K8`** | Pure Forward (Streaming Dealloc) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $147,456$ | **$1828.4\text{ MB}$** |
| **`fwd_streaming_wm_map_N4`**| Pure Forward (Streaming MAP) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $6$ ($1$ clean + $5$ settle/recomp) | $73,728$ | **$1828.4\text{ MB}$** |
| **`fwd_streaming_oracle_fullvocab`** | Uncoupled Surrogate ($V=8192$) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $55,296$ | $2100.4\text{ MB}$ |
| **`fwd_streaming_oracle_mlp_pull_only_fullvocab`** | +MLP-Pullback Only ($V=8192, K_v=1$) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ |
| **`fwd_streaming_oracle_mlp_whv_fullvocab`** | +MLP-Pullback + `CM-Attn-WH` ($K_v=8$) | **$1$** | **$0$** | **$0$** | $18$ ($1$ clean + $17$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ |
| **`fwd_streaming_oracle`** | Uncoupled Surrogate ($V_{\text{sub}}=256$) | **$1$** | **$0$** | **$0$** | $11$ ($1$ clean + $10$ probe) | $2$ ($1$ clean + $1$ local fwd) | $55,296$ | **$1820.3\text{ MB}$** |
| **`fwd_streaming_cowm_coupled_K8`** | Coupled `Co-WM-S` ($K=8, V_{\text{sub}}=256$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | **$1828.3\text{ MB}$** |
| **`fwd_streaming_cowm_coupled_fullvocab_K8`** | Coupled `Co-WM-S` ($K=8, V=8192$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2100.3\text{ MB}$ |
| **`fwd_streaming_oracle_coupled`** | Coupled Surrogate ($V_{\text{sub}}=256$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | **$1828.3\text{ MB}$** |
| **`fwd_streaming_oracle_coupled_fullvocab`** | Coupled Surrogate ($V=8192$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ |
| **`fwd_streaming_cowm_coupled_qproj_opdiff_K8`** | +`Q-Proj-K` + `OpDiff` `Co-WM-S` ($V_{\text{sub}}=256$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2084.3\text{ MB}$ |
| **`fwd_streaming_cowm_coupled_qproj_opdiff_fullvocab_K8`** | +`Q-Proj-K` + `OpDiff` `Co-WM-S` ($V=8192$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $196,608$ | $2442.2\text{ MB}^\dagger$ |
| **`fwd_streaming_oracle_coupled_qproj_unbiased_fullvocab`** | +`Q-Proj-K` Surrogate ($V=8192$) | **$1$** | **$0$** | **$0$** | $42$ ($1$ clean + $41$ probe) | $2$ ($1$ clean + $1$ local fwd) | $104,448$ | $2100.3\text{ MB}$ |
| **`fwd_lookahead_2b_orth_mv_a1.0_opdiff_qproj_fullvocab`** | +`Lookahead-2B-OrthMV` + `OpDiff` + `Q-Proj-K` ($V=8192$) | **$1$** | **$0$** | **$0$** | $49.5$ ($297$ total) | $2.83$ ($17$ total) | $196,608$ | $2517.3\text{ MB}$ |
| **`fwd_lookahead_2b_orth_mv_cowm_opdiff_qproj_K8`** | +`Lookahead-2B-OrthMV` `Co-WM-S` ($V_{\text{sub}}=256$) | **$1$** | **$0$** | **$0$** | $49.5$ ($297$ total) | $2.83$ ($17$ total) | $360,448$ | $2245.3\text{ MB}$ |

*Table 1: Verified Computation and Pass-Count Ledger for 6-layer Transformer pretraining ($B=16, T=512$, $N_{\text{tok}}=8192$, micro-horizon $S=8$, $V=8192$; $^\dagger$measured on `d6` $D=384$ in Table 5). All candidate forward-only methods enforce `bwd_calls = 0` and `depth_sweeps = 0`. Uncoupled streaming contraction counts reflect $3$ linear sites per block (`lm_head`, `mlp.c_proj`, `attn.c_proj`) $\times 6$ blocks $\times K$ probes $\times (8192/8)$ groups ($147,456$ for $K=8$); coupled streaming adds the 4th parameter-space `Co-WM-S` site on `mlp.c_fc` ($W_1$) for the zero-pass MLP-to-Attention residual pullback ($196,608$ for $K=8$); 2-block lookahead adds $5$ inter-block lookahead evaluations of MLP ($V, W_1$) and Attention-Value ($W_O, W_V$, $9$ FlashAttention calls each).*


---

## 2. Theoretical Analysis and Scaling Laws

### 2.1 Micro-Horizon Parameter-Space Weight Maximization (`WM-S`)
In Chung's Weight Maximization framework [2, Eq. 1–3], hidden unit $i \in \{1, \dots, M\}$ does not possess an activation-space transpose circuit $(V^\top \delta_t)_i$. Instead, each unit $i$ observes only the local plasticity changes $\Delta \mathbf{v}^{(i)}$ of its own outgoing weight column $\mathbf{v}^{(i)} = V_{:, i} \in \mathbb{R}^{D_{\text{out}}}$, reinforcing its activity when its outgoing synaptic column grows in squared $L^2$ norm.

To adapt this principle to sequence modeling without per-token activation transposes ($S = 1$) or full-batch temporal cross-talk ($S = N_{\text{tok}}$), we introduce **Parameter-Space Micro-Horizon Weight Maximization (`WM-S`)**:
1. Partition the $N_{\text{tok}} = B \times T$ tokens into $G = N_{\text{tok}} / S$ disjoint micro-groups $\mathcal{S}_1, \dots, \mathcal{S}_G$ of size $S$ tokens ($S \in \{4, 8, 16, 64\}$).
2. For exploration draw $k \in \{1, \dots, K\}$, sample independent Rademacher perturbations $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ for each neuron $i \in \{1, \dots, M\}$.
3. Over each micro-group $\mathcal{S}_g$, the outgoing matrix $V \in \mathbb{R}^{D_{\text{out}} \times M}$ accumulates its parameter-space plasticity displacement:
   $$\Delta V_{\mathcal{S}_g}^{(k)} = \sum_{s \in \mathcal{S}_g} \delta_{g, s} \big(\xi_{g, s}^{(k)}\big)^\top \;\in\; \mathbb{R}^{D_{\text{out}} \times M}.$$
4. Neuron $i$ evaluates its parameter-space inner product from its own column $\mathbf{v}^{(i)} = V_{:, i}$ and column update $\Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)} = (\Delta V_{\mathcal{S}_g}^{(k)})_{:, i}$ [2, Eq. 2]:
   $$G_{\mathcal{S}_g}^{(i, k)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)} - \beta \left\|\Delta \mathbf{v}_{\mathcal{S}_g}^{(i, k)}\right\|_2^2 \;\in\; \mathbb{R}.$$
5. Neuron $i$ computes its input credit for token $t \in \mathcal{S}_g$ [2, Eq. 3]:
   $$\hat{g}_{g, t, i}^{\text{WM-S}} = \frac{1}{K} \sum_{k=1}^K G_{\mathcal{S}_g}^{(i, k)} \frac{\xi_{g, t, i}^{(k)}}{\sigma^2}, \qquad \nabla_{W_1} \mathcal{L} = \sum_{g=1}^G \sum_{t \in \mathcal{S}_g} \big(\hat{g}_{g, t}^{\text{WM-S}} \odot \phi'(a_{1, g, t})\big) x_{g, t}^\top.$$

**Proposition 1 (Conditional Coordinate Variance and Sibling Decoupling of `WM-S`)**:
*Conditioned on fixed downstream linear targets $\delta_{g, s}$ independent of $\xi$, with independent Rademacher perturbations $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ (so $(\xi_{g, t, i}^{(k)})^2 / \sigma^2 \equiv 1$) and $\beta = 0$, expanding $\hat{g}_{g, t, i}^{\text{WM-S}}$ yields:*
$$\hat{g}_{g, t, i}^{\text{WM-S}} = \big(\mathbf{v}^{(i)} \cdot \delta_{g, t}\big) \underbrace{\frac{(\xi_{g, t, i}^{(k)})^2}{\sigma^2}}_{\equiv 1} + \frac{1}{K} \sum_{k=1}^K \sum_{s \in \mathcal{S}_g \setminus \{t\}} \big(\mathbf{v}^{(i)} \cdot \delta_{g, s}\big) \frac{\xi_{g, s, i}^{(k)} \xi_{g, t, i}^{(k)}}{\sigma^2}.$$
*Because $\mathbb{E}[\xi_{g, s, i} \xi_{g, t, i}] = 0$ for $s \ne t$, the estimator is conditionally unbiased ($\mathbb{E}[\hat{g}_{g, t, i}^{\text{WM-S}} \mid \delta] = \mathbf{v}^{(i)} \cdot \delta_{g, t}$) with conditional variance:*
$$\text{Var}\!\left[\hat{g}_{g, t, i}^{\text{WM-S}} \;\middle|\; \delta\right] = \frac{1}{K} \sum_{s \in \mathcal{S}_g \setminus \{t\}} (\mathbf{v}^{(i)} \cdot \delta_{g, s})^2.$$

*Decoupling Mechanism*: In activation-space perturbation (`Cut`), the reward $R_t^{(k)}$ sums over all $M$ neurons at token $t$, yielding conditional variance $\frac{1}{K}\sum_{j \ne i} (\mathbf{v}^{(j)} \cdot \delta_t)^2$ that scales with $M-1$. In `WM-S`, parameter-space column isolation decouples all $M - 1$ sibling neurons $j \ne i$, replacing the sum over $M - 1$ neurons with a sum over $S - 1$ tokens in $\mathcal{S}_g$, which is conditionally independent of width $M$ for fixed per-unit targets.

### 2.2 Sylvester-Hadamard Temporal Coordination (`Co-WM-S`)
Because `WM-S` eliminates spatial sibling-neuron cross-talk under fixed $\delta$, the remaining conditional variance arises from the $S - 1$ cross-token terms within micro-horizon $\mathcal{S}_g$. In **`Co-WM-S`**, we adapt orthogonal Sylvester-Hadamard exploration [11] to token micro-horizons in sequence Transformers, structuring the $K \le S$ exploration probes across the $S$ tokens of micro-group $\mathcal{S}_g$ using $K$ distinct columns $\{c_1, \dots, c_K\}$ drawn uniformly without replacement from an $S \times S$ Sylvester-Hadamard matrix $H \in \{-1, +1\}^{S \times S}$ ($S$ a power of 2), modulated by an independent Rademacher carrier $r_{g, s, i} \in \{-1, +1\}$:
$$\xi_{g, s, i}^{(k)} = \sigma \, H_{s, c_k} \, r_{g, s, i}.$$

**Proposition 2 (Finite-Population Conditional Variance of `Co-WM-S`)**:
*Conditioned on fixed downstream targets $\delta_{g, s}$ and writing $c_s = \mathbf{v}^{(i)} \cdot \delta_{g, s}$, uniform selection of $K \in \{1, \dots, S\}$ distinct Hadamard columns without replacement at $\beta = 0$ yields $\mathbb{E}[\hat{g}_{g, t, i}^{\text{Co-WM-S}} \mid \delta] = c_t$ and exact conditional variance:*
$$\text{Var}\!\left[\hat{g}_{g, t, i}^{\text{Co-WM-S}} \;\middle|\; \delta\right] = \frac{S - K}{K (S - 1)} \sum_{s \in \mathcal{S}_g \setminus \{t\}} c_s^2.$$
*Proof*: For distinct tokens $s \ne t$, $p_{s, t}(c) = H_{s, c} H_{t, c}$ has population mean $\frac{1}{S}\sum_{c=1}^S H_{s, c} H_{t, c} = 0$ by Hadamard orthogonality and unit variance $p_{s, t}(c)^2 \equiv 1$. Sampling $K$ distinct columns without replacement yields the classical hypergeometric variance reduction factor $\frac{S - K}{K(S - 1)}$. At $K = S$, the conditional variance vanishes **identically ($\text{Var} \equiv 0$)** for every width $M$.

### 2.3 Exact Covariance Identity, Sufficient Conditions, and Multi-Checkpoint Trajectory Audit
For row $i$ of the incoming weight matrix $W_1$, let $c_{g, s, i} = \mathbf{v}^{(i)} \cdot \delta_{g, s} \in \mathbb{R}$ and $z_{g, t, i} = \phi'(a_{1, g, t, i}) x_{g, t} \in \mathbb{R}^D$, so the exact row gradient is $g_i = \sum_{g=1}^G \sum_{t \in \mathcal{S}_g} c_{g, t, i} z_{g, t, i} = \sum_{g=1}^G g_{g, i}$.

**Proposition 3 (Exact Conditional Frobenius Weight-Gradient Covariance and Relative Variance Law)**:
*Under `WM-S` exploration with $K$ independent Rademacher draws $\xi_{g, s, i}^{(k)} \in \{-\sigma, +\sigma\}$ and $\beta = 0$, the estimator error decomposes over unordered token pairs $\{s, t\} \subset \mathcal{S}_g$ ($s < t$):*
$$\widehat{\nabla}_{W_{1, i}} - g_i = \frac{1}{K} \sum_{k=1}^K \sum_{g=1}^G \sum_{1 \le s < t \le S} \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big) \frac{\xi_{g, s, i}^{(k)} \xi_{g, t, i}^{(k)}}{\sigma^2}.$$
*Because distinct pairs $\{s, t\} \ne \{s', t'\}$ are pairwise uncorrelated, the exact weight-gradient covariance is:*
$$\text{Cov}\!\left(\widehat{\nabla}_{W_{1, i}} \;\middle|\; c, z\right) = \frac{1}{K} \sum_{g=1}^G \sum_{1 \le s < t \le S} \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big) \big(c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\big)^\top,$$
*and the exact conditional Frobenius variance summed across all $M$ neurons is:*
$$\mathbb{E}\!\left[\left\|\widehat{\nabla}_{W_1} - \nabla_{W_1}\right\|_F^2 \;\middle|\; c, z\right] = \kappa(S, K) \sum_{i=1}^M \sum_{g=1}^G \sum_{1 \le s < t \le S} \left\|c_{g, s, i} z_{g, t, i} + c_{g, t, i} z_{g, s, i}\right\|_2^2,$$
*where $\kappa(S, K) = 1/K$ for `WM-S` and $\kappa(S, K) = \frac{S-K}{K(S-1)}$ for `Co-WM-S`.*

**Algebraic Cross-Token Identity**: By expanding the pair norm squared:
$$\sum_{1 \le s < t \le S} \left\|c_s z_t + c_t z_s\right\|_2^2 = \left(\sum_{s=1}^S c_s^2\right) \left(\sum_{t=1}^S \|z_t\|_2^2\right) + \left\|g_{\mathcal{S}_g}\right\|_2^2 - 2 \sum_{t=1}^S c_t^2 \|z_t\|_2^2.$$
Defining the **Within-Group Support Alignment Ratio** $\alpha(c, z)$, the **Global Gradient Coherence Ratio** $\gamma(c, z)$, and the **Group Gradient Coherence Ratio** $\gamma_{\text{grp}}(c, z)$:
$$\alpha(c, z) \triangleq \frac{\sum_{g, i} \left(\sum_{s=1}^S c_{g, s, i}^2\right) \left(\sum_{t=1}^S \|z_{g, t, i}\|_2^2\right)}{S \sum_{g, t, i} c_{g, t, i}^2 \|z_{g, t, i}\|_2^2}, \qquad \gamma(c, z) \triangleq \frac{\sum_i \|g_i\|_2^2}{\sum_{g, t, i} c_{g, t, i}^2 \|z_{g, t, i}\|_2^2}, \qquad \gamma_{\text{grp}}(c, z) \triangleq \frac{\sum_{g, i} \|g_{g, i}\|_2^2}{\sum_i \|g_i\|_2^2},$$
the exact conditional relative parameter-gradient variance is governed by the identity:
$$\text{RelVar}(\text{WM-S} \mid c, z) \triangleq \frac{\mathbb{E}\!\left[\left\|\widehat{\nabla}_{W_1} - \nabla_{W_1}\right\|_F^2 \;\middle|\; c, z\right]}{\left\|\nabla_{W_1}\right\|_F^2} = \frac{1}{K} \left[ \frac{\alpha(c, z) S - 2}{\gamma(c, z)} + \gamma_{\text{grp}}(c, z) \right].$$

**Analysis of Failure Modes and Counterexamples**:
Because $\alpha(c, z)$, $\gamma(c, z)$, and $\gamma_{\text{grp}}(c, z)$ depend on the empirical distribution of $c$ and $z$, Proposition 3 implies **conditional** width-invariance at fixed $(\alpha, \gamma, \gamma_{\text{grp}})$ rather than an unconditional guarantee across arbitrary network states:
1. *Gradient Cancellation Counterexample*: If token gradients cancel across the sequence ($\gamma \to 0$), relative variance diverges. For $S=8, K=1, D=1, M=4$, inputs $x = (1, 2, 0 \dots)$, targets $\delta = (4, -1+\varepsilon, 0 \dots)$, $z = (2, 8, 0 \dots)$, $c = (4, -1+\varepsilon, 0 \dots)$, the exact gradient is $g = 8\varepsilon$ and cross-term $c_1 z_2 + c_2 z_1 = 30 + 2\varepsilon$. The exact conditional relative variance is $\frac{(30+2\varepsilon)^2}{64\varepsilon^2}$, which equals $140{,}812.5625$ at $\varepsilon = 0.01$ (verified by exhaustive enumeration over all 256 Rademacher states).
2. *Cross-Token Support Mismatch Counterexample*: Non-cancellation ($\gamma > 0$) alone is insufficient if $c$ and $z$ have mismatched temporal support within a micro-group ($\alpha \to \infty$). For $c = (1, \varepsilon, 0 \dots)$ and $z = (\varepsilon, 1, 0 \dots)$, both nonzero token gradients equal $\varepsilon$ (reinforcing, $g = 2\varepsilon, \sum c_t^2 z_t^2 = 2\varepsilon^2$, so $\gamma = 2.0$), but $c_1 z_2 = 1 \gg \varepsilon^2$, giving $\alpha \approx \frac{1}{16\varepsilon^2}$. The exact conditional relative variance is $\frac{(1+\varepsilon^2)^2}{4\varepsilon^2} = 2500.500025$ at $\varepsilon = 0.01$ for $K=1$, $357.214289$ for $K=4$ `Co-WM-S`, and $0.0$ for $K=8$ `Co-WM-S`.

**Empirical Trajectory Audit of Conditional Quantities (`verify_learned_prop3_scaling.py`)**:
To bridge Proposition 3 to actual training behavior with full reproducibility from an empty output file (`python beyond_backprop/verify_learned_prop3_scaling.py --regenerate-all`, saving `multiwidth_learned_prop3_diagnostic.json`), we calculate the **conditional** quantities $\alpha(c, z), \gamma(c, z), \gamma_{\text{grp}}(c, z)$, and $\text{RelVar}(\cdot \mid c, z)$ from frozen Layer 3 checkpoint tensors ($S=8$) on batches of shape **$B=16, T=512$ ($N_{\text{tok}}=8,192$ tokens/batch)** across widths $D \in \{128, 256, 384, 512\}$ under three explicitly documented protocols:
1. **Benchmark-Schedule Holdout Validation Trajectory (`benchmark_holdout_trajectory_4batch`)**:
   - *Training Recipe*: 200 steps of `fwd_streaming_cowm` ($K_{\text{wm}}=4, K_{\text{qk}}=2, S=8$) on `train_batches[0:200]` drawn from `shard_00000.parquet` ($B=16, T=512$) using the **exact benchmark optimizer schedule** from `run_pure_forward_only_suite.py` (linear LR warmdown over the final 45% of steps, Muon momentum warmup $0.85 \to 0.95$ over 200 steps, and linear Muon weight decay decay $0.2(1 - \text{progress})$).
   - *Evaluation Split*: Evaluated at 5 checkpoints ($t \in \{0, 50, 100, 150, 200\}$) across **4 disjoint holdout validation batches** (`holdout_val_batches[0:4]`, shape $B=16, T=512, N_{\text{tok}}=8,192$) drawn from `shard_06542.parquet` (row groups $40 \dots 82$).
   - *Ranges of 4-Batch Means*: Across all $4 \times 5 = 20$ width-checkpoint states, the 4-batch means span $\alpha \in [0.8572 \pm 0.0063, \, 1.0033 \pm 0.0012]$, $\gamma \in [2.3604 \pm 0.5815, \, 8.1838 \pm 5.2378]$, and $\gamma_{\text{grp}} \in [0.1779 \pm 0.1155, \, 0.4410 \pm 0.0956]$. Consequently, the 4-batch mean conditional relative variances stay bounded in $[1.0448 \pm 0.6991, \, 2.7498 \pm 0.6188]$ for `WM-S` ($K=1$) and $[0.1492 \pm 0.0999, \, 0.3929 \pm 0.0884]$ for `Co-WM-S` ($K=4$). At $t=200$ (4-batch mean $\pm$ SD):
     - $D=128$: $\alpha = 0.9439 \pm 0.0154$, $\gamma = 3.3446 \pm 1.9805$, `WM-S` ($K=1$) $= 2.5773 \pm 1.4312$, `Co-WM-S` ($K=4$) $= 0.3682 \pm 0.2045$, `Cut-K4` $= 33.44 \pm 18.69$;
     - $D=256$: $\alpha = 0.9414 \pm 0.0070$, $\gamma = 3.5498 \pm 2.1871$, `WM-S` ($K=1$) $= 2.5365 \pm 1.5310$, `Co-WM-S` ($K=4$) $= 0.3624 \pm 0.2187$, `Cut-K4` $= 51.37 \pm 31.06$;
     - $D=384$: $\alpha = 0.9320 \pm 0.0053$, $\gamma = 4.0057 \pm 2.7904$, `WM-S` ($K=1$) $= 2.5661 \pm 1.8502$, `Co-WM-S` ($K=4$) $= 0.3666 \pm 0.2643$, `Cut-K4` $= 78.25 \pm 56.20$;
     - $D=512$: $\alpha = 0.9902 \pm 0.0105$, $\gamma = 2.8972 \pm 0.8824$, `WM-S` ($K=1$) $= 2.5779 \pm 0.8397$, `Co-WM-S` ($K=4$) $= 0.3683 \pm 0.1199$, `Cut-K4` $= 123.91 \pm 39.04$.
2. **Fixed-Optimizer Diagnostic Trajectory (`streaming_trajectory_4batch`)**:
   - *Training & Evaluation Recipe*: 200 steps of `fwd_streaming_cowm` ($K_{\text{wm}}=4, K_{\text{qk}}=2, S=8$) on `train_batches[0:200]` (`shard_00000.parquet`, $B=16, T=512$) with fixed default Muon/AdamW optimizer parameters (`setup_optimizer(use_muon=True)` without LR/momentum/WD scheduling), evaluated at $t \in \{0, 50, 100, 150, 200\}$ on 4 held-out training-stream batches `train_batches[200:204]` ($B=16, T=512, N_{\text{tok}}=8,192$). Across all 20 states, the 4-batch means span $\alpha \in [0.9062 \pm 0.0079, \, 1.0162 \pm 0.0080]$, $\gamma \in [2.2698 \pm 0.3875, \, 5.6916 \pm 0.0958]$, conditional `WM-S` ($K=1$) $\text{RelVar} \in [1.2203, \, 2.8887]$, and conditional `Co-WM-S` ($K=4$) $\text{RelVar} \in [0.1743, \, 0.4127]$.
3. **Single-Batch Table 2 States (`step_0_init`, `step_200_learned`, `step_200_streaming_learned`)**:
   - Calculated from frozen tensors under fixed default Muon/AdamW parameters on `shard_00000.parquet` ($B=16, T=512$): `step_0_init` on `train_batches[0]`, `step_200_learned` on `train_batches[199]` after 200 steps of Cut-scope `detached_oracle`, and `step_200_streaming_learned` on `train_batches[199]` after 200 steps of `fwd_streaming_cowm` ($K=4$). To isolate the local MLP estimation variance of Proposition 3 around a deterministic surrogate target on genuine learned network states, `evaluate_layer3_streaming_prop3` conditions on the exact deterministic `detached_oracle` readout pullback at that step, decoupling internal parameter-contraction variance from downstream attention/readout sampling noise.

| Width $D$ ($M=4D$) | State Regime | $\alpha$ (Support Alignment) | $\gamma$ (Global Coherence) | $\gamma_{\text{grp}}$ (Group Coherence) | `Cut-K4` Cond. RelVar | `WM-S` ($K=1$) Cond. RelVar | `Co-WM-S` ($K=4$) Cond. RelVar | `Co-WM-S` ($K=8$) Cond. RelVar | `GS-MAP-N1` RelBias | `GS-MAP-N2` RelBias |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| $D=128$ ($M=512$) | `step_0_init` | $0.9990$ | $3.4755$ | $0.2885$ | $37.05$ | $2.0127$ | $0.2875$ | **$0.0000$** | $0.00017$ | $0.05833$ |
| $D=128$ ($M=512$) | `step_200_learned` | $1.0250$ | $1.4638$ | $0.6859$ | $55.34$ | $4.9214$ | $0.7031$ | **$0.0000$** | $0.00022$ | $0.16049$ |
| $D=128$ ($M=512$) | `step_200_streaming` | $0.9746$ | $2.1800$ | $0.4589$ | $41.17$ | $3.1179$ | $0.4454$ | **$0.0000$** | $0.00020$ | $0.16297$ |
| $D=256$ ($M=1024$) | `step_0_init` | $1.0013$ | $3.2002$ | $0.3134$ | $79.86$ | $2.1915$ | $0.3131$ | **$0.0000$** | $0.00019$ | $0.05574$ |
| $D=256$ ($M=1024$) | `step_200_learned` | $0.9275$ | $1.5571$ | $0.6404$ | $103.94$ | $4.1213$ | $0.5888$ | **$0.0000$** | $0.00022$ | $0.13396$ |
| $D=256$ ($M=1024$) | `step_200_streaming` | $0.9752$ | $2.0925$ | $0.4753$ | $66.99$ | $3.2478$ | $0.4640$ | **$0.0000$** | $0.00018$ | $0.14256$ |
| $D=384$ ($M=1536$) | `step_0_init` | $1.0003$ | $3.6960$ | $0.2714$ | $103.77$ | $1.8953$ | $0.2708$ | **$0.0000$** | $0.00017$ | $0.05414$ |
| $D=384$ ($M=1536$) | `step_200_learned` | $1.0329$ | $2.0612$ | $0.4860$ | $146.12$ | $3.5248$ | $0.5035$ | **$0.0000$** | $0.00020$ | $0.12289$ |
| $D=384$ ($M=1536$) | `step_200_streaming` | $0.9730$ | $1.9493$ | $0.5142$ | $123.91$ | $3.4815$ | $0.4974$ | **$0.0000$** | $0.00019$ | $0.12925$ |
| $D=512$ ($M=2048$) | `step_0_init` | $1.0002$ | $3.3703$ | $0.2976$ | $151.85$ | $2.0783$ | $0.2969$ | **$0.0000$** | $0.00019$ | $0.05394$ |
| $D=512$ ($M=2048$) | `step_200_learned` | $0.9729$ | $2.1721$ | $0.4605$ | $152.76$ | $3.1231$ | $0.4462$ | **$0.0000$** | $0.00018$ | $0.13607$ |
| $D=512$ ($M=2048$) | `step_200_streaming` | $0.9951$ | $2.4223$ | $0.4139$ | $150.28$ | $2.8746$ | $0.4107$ | **$0.0000$** | $0.00018$ | $0.12879$ |

*Table 2: Conditional Proposition 3 quantities calculated from frozen Layer 3 tensors on ClimbMix ($B=16, T=512$, `multiwidth_learned_prop3_diagnostic.json`). While activation-space `Cut-K4` conditional relative variance grows with width ($41.17 \to 150.28$ along the streaming trajectory), conditional `WM-S` ($K=1$) and `Co-WM-S` ($K=4, 8$) relative variances remain bounded across tested widths ($2.87\text{--}3.48$ and $0.41\text{--}0.50$, respectively, and $0.00$ at $K=8$).*

### 2.4 Whole-Network Probing: Sequence-Aware Variance, Sensitivity Thresholds, and Ablations
Propositions 1–3 establish conditional sibling-neuron decoupling given a fixed local output discrepancy $c$. Eliminating conditional local variance does **not** make the whole-network gradient unbiased with respect to end-to-end backpropagation. In a causal sequence Transformer, the downstream map $F^{(m)}: \mathbb{R}^{T \times D} \to \mathbb{R}^{T \times D}$ from injection site $m \in \{0, \dots, 2L\}$ ($S_{\text{sites}} = 2L + 1 = 13$ for $L=6$) to the final normalized hidden state has a block-lower-triangular sequence-channel Jacobian $J^{(m)} \in \mathbb{R}^{(TD) \times (TD)}$, with blocks $J_{u, v}^{(m)} \in \mathbb{R}^{D \times D}$ non-zero for all causal token pairs $u \ge v$ and zero for $u < v$. Given top sensitivity $c = (c_1, \dots, c_T) \in \mathbb{R}^{T \times D}$, the exact transpose sensitivity of site $m$ at token $t$ is $u_t^{(m)} = \sum_{u=t}^T (J_{u, t}^{(m)})^\top c_u \in \mathbb{R}^D$.

**Sequence-Aware Within-Site vs. Between-Site Variance of Depth-Hadamard vs. Independently Resampled Probing**:
Let $H_{\text{depth}} = 16$ be the smallest power of $2$ at least $S_{\text{sites}} = 13$, and let $\kappa(H_{\text{depth}}, K_{\text{net}}) = \frac{H_{\text{depth}} - K_{\text{net}}}{K_{\text{net}}(H_{\text{depth}} - 1)}$ ($\kappa = \frac{1}{15}$ at $H_{\text{depth}} = 16, K_{\text{net}} = 8$). Under **Depth-Hadamard Carrier Modulation (`depth_hadamard`)**, site $m$ draws one Rademacher tensor $r^{(m)} \in \{-1, +1\}^{T \times D}$ per $H_{\text{depth}}$-probe cycle and injects $\xi^{(m, k)} = \sigma h_{m, c_k} r^{(m)}$. Because $h_{m, c_k}^2 \equiv 1$, the within-site contribution ($m' = m$) is invariant to $K_{\text{net}} \le H_{\text{depth}}$ within a cycle, whereas the between-site contribution ($m' \ne m$) is weighted by $\kappa(H_{\text{depth}}, K_{\text{net}})$. Under **Independently Resampled Carriers (`iid_resample`)**, fresh i.i.d. tensors $r^{(m, k)}$ are drawn at every probe $k \in \{1, \dots, K_{\text{net}}\}$, dividing both within-site and between-site variances by $1 / K_{\text{net}}$.

For each return-aggregation mode, the linear-limit expectation and exact sequence-aware activation-credit variance $\operatorname{Var}(\hat{g}^{(m)}) = A_{\text{mode}}^{(m)} + \kappa_{\text{mode}} B_{\text{mode}}^{(m)}$ ($\kappa_{\text{mode}} = \kappa(H_{\text{depth}}, K_{\text{net}})$ for `depth_hadamard`, or dividing $A_{\text{mode}}^{(m)} + B_{\text{mode}}^{(m)}$ by $K_{\text{net}}$ for `iid_resample`) are:
1. **Full-Sequence Scalar-Dot Probing (`full_seq_dot`, $n = TD$)**:
   The scalar return sums over all tokens and channels: $R_{\text{seq}}^{(k)} = \sum_{m'=0}^{2L} \sum_{u=1}^T \langle u_u^{(m')}, \xi_u^{(m', k)} \rangle$.
   - *Linear-Limit Expectation*: Unbiased for the full causal transpose, $\mathbb{E}[\hat{g}_{\text{full}, t}^{(m)}] = u_t^{(m)} = \sum_{u=t}^T (J_{u, t}^{(m)})^\top c_u$.
   - *Exact Sequence-Wide Variance*: Writing $\|u^{(m)}\|_2^2 = \sum_{t=1}^T \|u_t^{(m)}\|_2^2$, the single-probe within-site and between-site variance energies summed over all $n = TD$ coordinates are:
     $$A_{\text{full}}^{(m)} = (TD - 1) \left\|u^{(m)}\right\|_2^2, \qquad B_{\text{full}}^{(m)} = TD \sum_{m' \ne m} \left\|u^{(m')}\right\|_2^2.$$
2. **Token-Local Channel-Dot Probing (`token_dot`)**:
   At token $t$, the token-local return sums only over the $D$ channels at position $t$, but because the perturbation $\xi^{(m', k)}$ is injected across the entire sequence, $(J^{(m')} \xi^{(m', k)})_t$ depends on perturbations at **all earlier tokens $v \le t$**:
   $$R_t^{(k)} = c_t^\top (\Delta x_{\text{final}}^{(k)})_t = \sum_{m'=0}^{2L} \sum_{v=1}^t \big\langle a_{t, v}^{(m')}, \xi_v^{(m', k)} \big\rangle, \qquad \text{where } a_{t, v}^{(m')} \triangleq \big(J_{t, v}^{(m')}\big)^\top c_t \in \mathbb{R}^D.$$
   - *Linear-Limit Expectation*: Multiplying by $\xi_t^{(m, k)} / \sigma^2$ eliminates $v < t$ in expectation ($\mathbb{E}[\xi_t \xi_v^\top] = 0$ for $v < t$), leaving only the diagonal token block and **truncating all future-token causal blocks $u > t$**:
     $$\mathbb{E}\!\left[\hat{g}_{\text{tok}, t}^{(m)}\right] = a_{t, t}^{(m)} = \big(J_{t, t}^{(m)}\big)^\top c_t \;\ne\; \sum_{u=t}^T \big(J_{u, t}^{(m)}\big)^\top c_u.$$
   - *Exact Sequence-Aware Token-Dot Variance (Including Earlier-Token Causal Cross-Talk)*: Multiplication by $\xi_t^{(m, k)}$ does **not** remove the variance of the earlier-token terms $v < t$! Under `depth_hadamard`, $\xi_t^{(m, k)} \xi_v^{(m, k)\top} / \sigma^2 = r_t^{(m)} r_v^{(m)\top}$ for all $v \le t$, so the single-probe within-site error energy $A_{\text{tok}, t}^{(m)}$ and between-site error energy $B_{\text{tok}, t}^{(m)}$ about the token-local mean $a_{t, t}^{(m)}$ at token $t$ are:
     $$A_{\text{tok}, t}^{(m)} = (D - 1) \left\|a_{t, t}^{(m)}\right\|_2^2 + D \sum_{v < t} \left\|a_{t, v}^{(m)}\right\|_2^2, \qquad B_{\text{tok}, t}^{(m)} = D \sum_{m' \ne m} \sum_{v \le t} \left\|a_{t, v}^{(m')}\right\|_2^2.$$
     Notice the second within-site term $D \sum_{v < t} \|a_{t, v}^{(m)}\|_2^2$: even at $D = 1$, where same-token channel cross-talk $(D-1)\|a_{t, t}^{(m)}\|_2^2 = 0$, earlier-token causal coupling ($v < t$) generates non-zero within-site variance that persists unattenuated across a fixed-carrier `depth_hadamard` cycle and divides by $1 / K_{\text{net}}$ under `iid_resample`!
   - *Explicit Causal Counterexample (Verified in Gate 2H)*: For $T=2, D=1$ (single site) with $J = \begin{pmatrix} 1 & 0 \\ 3 & 4 \end{pmatrix}$ and $c = \begin{pmatrix} 5 \\ 7 \end{pmatrix}$, at token $t = 2$ we have $a_{2, 2} = 28$ and $a_{2, 1} = 21$. The second-token `token_dot` estimator is $r_2(21 r_1 + 28 r_2) = 28 + 21 r_1 r_2$. Its mean is $28$ (omitting future-token credit on token $1$, where $\mathbb{E}[\hat{g}_{\text{tok}, 1}] = 5 \ne 26$), and its exact variance about $28$ is $(1 - 1)(28^2) + 1(21^2) = 441.0$—which equals $441.0$ at every $K_{\text{net}} \in \{1, 2, 4\}$ under fixed-carrier `depth_hadamard`, and $441.0, 220.5, 110.25$ ($= 441.0 / K_{\text{net}}$) at $K_{\text{net}} = 1, 2, 4$ under `iid_resample` (verified in Gate 2H).
3. **Coordinate-Wise Diagonal Probing (`diag_jacobian`)**:
   Evaluates $\xi_{t, d}^{(m, k)} c_{t, d} (\Delta x_{\text{final}}^{(k)})_{t, d} / \sigma^2$, with expectation $\mathbb{E}[\hat{g}_{\text{diag}, t, d}^{(m)}] = c_{t, d} J_{(t, d), (t, d)}^{(m)}$ (verified in Gate 2G: for $J = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, c = \begin{pmatrix} 5 \\ 7 \end{pmatrix}$, $\mathbb{E}_{\text{diag}} = \begin{pmatrix} 5 \\ 28 \end{pmatrix} \ne J^\top c = \begin{pmatrix} 26 \\ 38 \end{pmatrix}$) and single-probe within-site and between-site variance energies:
   $$A_{\text{diag}}^{(m)} = \sum_{t, d} c_{t, d}^2 \left( \sum_{d' \ne d} \big(J_{(t, d), (t, d')}^{(m)}\big)^2 + \sum_{v < t} \sum_{d'=1}^D \big(J_{(t, d), (v, d')}^{(m)}\big)^2 \right), \quad B_{\text{diag}}^{(m)} = \sum_{m' \ne m} \sum_{t, d} c_{t, d}^2 \sum_{v \le t} \sum_{d'=1}^D \big(J_{(t, d), (v, d')}^{(m')}\big)^2.$$

**Sensitivity-Energy Condition for Depth-Hadamard vs. Independent Resampling ($B^{(m)} > 15 A^{(m)}$)**:
At $H_{\text{depth}} = 16, K_{\text{net}} = 8$, comparing $\operatorname{Var}_{\text{DH}} = A^{(m)} + \frac{1}{15} B^{(m)}$ against $\operatorname{Var}_{\text{iid}} = \frac{1}{8}(A^{(m)} + B^{(m)})$ yields the exact necessary and sufficient condition:
$$A^{(m)} + \frac{1}{15} B^{(m)} < \frac{1}{8}\big(A^{(m)} + B^{(m)}\big) \quad \Longleftrightarrow \quad B^{(m)} > 15 A^{(m)}.$$
Having $S_{\text{sites}} = 13$ sites does **not** by itself guarantee $B^{(m)} > 15 A^{(m)}$: if all 13 sites had equal sensitivity energy $\|u^{(m)}\|_2^2 = 1$, then $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)} = \frac{12 n}{n - 1} < 15$ for all $n \ge 6$ (for example, at $n = 512$ and unit site energy, $A = 511$ and $B = 6144$, so $\operatorname{Var}_{\text{DH}} = 920.6 > \operatorname{Var}_{\text{iid}} = 831.875$, making independent resampling better).
To test whether $B^{(m)} > 15 A^{(m)}$ holds on our initialized 6-layer Transformer (`c_proj_init_scale=0.5`, where `x0` and attention pathways carry larger sensitivity energy than individual MLP projections), `run_pure_forward_only_suite.py --mode diag` (`pure_forward_width_diagnostic.json`, `site_sensitivity_audit`) measures the exact 13-site sensitivity energies $E_m = \|\frac{\partial \mathcal{L}}{\partial s^{(m)}}\|_F^2$:
- Across widths $D \in \{128, 256, 384, 512\}$, the measured ratio $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)}$ is **$20.08, 19.88, 20.05, 20.30$** at Layer 3 MLP ($m=8$) and **$26.40, 26.15, 25.67, 26.31$** at Layer 5 MLP ($m=12$). This ratio specifically governs the whole-sequence return in `full_seq_dot`, where $B_{\text{full}}^{(m)} > 15 A_{\text{full}}^{(m)}$ holds due to the larger sensitivity energy carried by `x0` and attention pathways relative to individual MLP projections; `token_dot` and `diag_jacobian` have their own distinct within-site and between-site propagation variance terms $A_{\text{tok}}, B_{\text{tok}}, A_{\text{diag}}, B_{\text{diag}}$ derived above. Because $B_{\text{full}}^{(m)} / A_{\text{full}}^{(m)} > 15$ at both MLP sites, the predicted DH-to-i.i.d. activation-credit variance ratio is $\frac{1 + 20.30/15}{(1 + 20.30)/8} = 0.8839 < 1$ at Layer 3 ($0.8068$ at Layer 5).
- In Table 3 ($D=512$), we report both **activation-credit relative variance** ($\operatorname{RelVar}(\widehat{\delta u}_{\text{mlp}}^{(3)})$) and **accumulated parameter-gradient relative variance** ($\operatorname{RelVar}(\widehat{\nabla}_{W_1}^{(3)})$): for `full_seq_dot`, `depth_hadamard` vs `iid_resample` gives $79,218.20$ vs $91,443.37$ on activation credit (ratio $0.866 \approx 0.884$) and $78,606.12$ vs $90,830.04$ on $\widehat{\nabla}_{W_1}^{(3)}$; for `token_dot`, $1191.29$ vs $1398.44$ on activation credit and $1180.58$ vs $1386.78$ on $\widehat{\nabla}_{W_1}^{(3)}$; and for `diag_jacobian`, $1.56$ vs $2.60$ on activation credit and $1.55$ vs $2.58$ on $\widehat{\nabla}_{W_1}^{(3)}$. Thus, while carrier reuse leaves within-site variance $A^{(m)}$ unaveraged within a 16-probe cycle, both `depth_hadamard` and `iid_resample` exhibit the same 4-order-of-magnitude variance separation across `diag_jacobian` ($\sim 1.6\text{--}2.6$), `token_dot` ($\sim 1.2\text{--}1.4 \times 10^3$), and `full_seq_dot` ($\sim 7.9\text{--}9.1 \times 10^4$).

**Intra-Block Coupled Forward-Only Credit Assignment (`Co-WM-S`, `CM-Attn-WH`, `Hadamard-QK`)**:
In our baseline uncoupled forward-only streaming surrogate (`fwd_streaming_oracle_fullvocab`), the block-local error $\delta_{\text{block}} = \frac{\partial \mathcal{L}(x_l)}{\partial x_l}$ was fed into both MLP and Attention sublayers independently, omitting the downstream MLP pullback into Attention: in exact block-local backpropagation, $x_l = x_{\text{after\_attn}} + \text{MLP}(\text{norm}(x_{\text{after\_attn}}))$, so the true sensitivity of attention output is $\delta u_{\text{attn}} = \delta_{\text{block}} + J_{\text{mlp}\circ\text{norm}}^\top \delta_{\text{block}}$.
To close this gap strictly without a global backward sweep or autograd calls (`@torch.no_grad()`, `bwd_calls = 0`, fixed pass count independent of depth $L$ and width $M$), we introduce and ablate three coupled forward operators:
1. *Upstream MLP Input Credit via Parameter-Space `Co-WM-S` (`+MLP-Pullback`)*: Evaluating the MLP update first inside block $l$, let $\delta a_{1, g, s} = \hat{g}_{g, s}^{\text{Co-WM-S}} \odot \phi'(a_{1, g, s}) \in \mathbb{R}^M$ be the pre-activation discrepancy on $W_1 \in \mathbb{R}^{M \times D}$. For each input channel $d \in \{1, \dots, D\}$ with column $\mathbf{w}_1^{(d)} = (W_1)_{:, d} \in \mathbb{R}^M$ and micro-group $\mathcal{S}_g$ ($S=8$), probe $k \in \{1, \dots, K\}$ injects $\xi_{g, s, d}^{(k)} = \sigma H_{s, c_k} r_{g, s, d}$, forms the probe-specific column displacement $\Delta \mathbf{w}_{1, \mathcal{S}_g}^{(d, k)} = \sum_{s \in \mathcal{S}_g} \delta a_{1, g, s} \xi_{g, s, d}^{(k)} \in \mathbb{R}^M$, contracts $G_{\mathcal{S}_g}^{(d, k)} = \mathbf{w}_1^{(d)} \cdot \Delta \mathbf{w}_{1, \mathcal{S}_g}^{(d, k)}$ (`compute_linear_non_backprop_input_credit`), and reconstructs:
   $$\widehat{\delta x}_{\text{mlp\_in}, g, t, d} = \frac{1}{K}\sum_{k=1}^K G_{\mathcal{S}_g}^{(d, k)} \frac{\xi_{g, t, d}^{(k)}}{\sigma^2}$$
   without transposing $W_1$ in activation space or executing extra forward passes (exact at $K=S=8$, Gate 2J). Pulling $\widehat{\delta x}_{\text{mlp\_in}}$ across `norm(x)` ($\text{RMS}(x) = \sqrt{\frac{1}{D}\sum_{d=1}^D x_d^2 + \varepsilon}$, $\bar{x} = x / \text{RMS}(x)$) via `_forward_rms_norm_directional_pull` adds:
   $$\Delta_{\text{mlp}\to\text{attn}} = \frac{1}{\text{RMS}(x)} \left(I - \frac{1}{D}\bar{x}\bar{x}^\top\right) \widehat{\delta x}_{\text{mlp\_in}}$$
   to $\delta_{\text{block}}$. In our retained $T=512$ ($n_{\text{win}}=8$) diagnostic on `d6` (`t512_intra_block_ablation` in `pure_forward_width_diagnostic.json`), adding only this MLP pullback (`fwd_streaming_oracle_mlp_pull_only_fullvocab`) raises Layer 3 $W_O$ block-local cosine similarity from $0.93360$ to **$1.00000$** ($0.93314 \to 1.00000$ at $T=64$).
2. *Two-Level Window-Hadamard Forward Value Transport (`CM-Attn-WH`)*: At the pretraining sequence length $T=512$ ($n_{\text{win}}=8$ windows of $S_{\text{win}}=64$ tokens), single-window value probing ($K_v=1$) truncates cross-window causal value transport ($w_q > w_k$), causing Layer 3 $W_V$ block-local cosine to drop to $0.46853$ ($0.48951$ after MLP pullback). Multiplexing $S_{\text{win}}=64$ tokens per window via Sylvester-Hadamard matrix $H_{64}$ and $n_{\text{win}}=8$ windows via $H_8$ with causal window suffix summation over $w_q \ge w_k$ ($K_v = n_{\text{win}} = 8$, verified in Gate 2J) in `fwd_streaming_oracle_mlp_whv_fullvocab` restores $T=512$ Layer 3 $W_V$ block-local cosine from $0.48951$ to **$1.00000$**.
3. *Post-RoPE Head-Channel Sylvester-Hadamard Query/Key Probing (`Hadamard-QK`)*: Constructing perturbations on post-RoPE normalized $Q/K$ using $H_{d_h}$ channel coordination ($d_h=64, K_{\text{qk}}=8$, $\sigma_{\text{qk}}=0.05$) and accumulating rewards in `float32` (while converting perturbed $Q, K$ to `work_dtype` = `bfloat16` inside CUDA FlashAttention) with fused causal future-query lookahead ($L_k=16, \rho=0.85$) raises $T=512$ Layer 3/5 $W_Q$ block-local cosine from $0.14129 / 0.13918$ (`rel_var = 56.12`) to **$0.35023 / 0.35241$ (`rel_var = 6.94`)** in `fwd_streaming_oracle_coupled_fullvocab` (and **$0.34973 / 0.35366$** in full-vocabulary `Co-WM-S` `fwd_streaming_cowm_coupled_fullvocab_K8`).

**Sequential and Component Ablation Ladder in Streaming Block-Local Supervision**:
To evaluate the effects of successive interventions and isolate each component of the intra-block coupled package on 1000-step 2-seed `d6` ($8.192\text{M}$ tokens, Tables 4 and 5), we trace:
1. **Inter-Block Sublayer Jacobian Omission at Fixed Final-State Discrepancy (`backprop` $1.45172 \to$ `block_fixed_top_discrepancy_bp` $1.62152$, $\Delta = +0.16980$ BPB)**: Isolates the cost of omitting intervening non-identity sublayer Jacobians ($l' > l$) under fixed final-state supervision $\delta_{\text{final}}$.
2. **Replacing Fixed Final-State Discrepancy with Intermediate Block-State Supervision (`block_fixed_top_discrepancy_bp` $1.62152 \to$ `block_local_greedy_bp_tophead` $1.58085$, $\Delta = -0.04067$ BPB)**: Replaces $\delta_{\text{final}}$ with each block's own local-loss discrepancy $\frac{\partial \mathcal{L}(x_l)}{\partial x_l}$ (net $+0.12913$ BPB for greedy local supervision).
3. **Shared-Readout Intermediate-Loss Mixture (`block_local_greedy_bp_tophead` $1.58085 \to$ `block_local_greedy_bp` $1.57236$, $\Delta = -0.00849$ BPB)**: Mixes $50\%$ final-block and $50\%$ earlier-block gradients into `lm_head`.
4. **Baseline Uncoupled Intra-Block Surrogate Gap at Full Vocabulary (`block_local_greedy_bp` $1.57236 \to$ `fwd_streaming_oracle_fullvocab` $1.74582$, $\Delta = +0.17346$ BPB)**: Replaces exact intra-block autograd (`bwd_calls = 6`) with the uncoupled forward surrogate package (`bwd_calls = 0`, $K_v=1, K_{\text{qk}}=2$).
5. **Component Ablation of the Intra-Block Coupled Package at Full Vocabulary ($V=8192$)**:
   - *Component 5a (+MLP-Pullback Only)*: `fwd_streaming_oracle_fullvocab` ($1.74582$) $\to$ `fwd_streaming_oracle_mlp_pull_only_fullvocab` ($1.73609 \pm 0.00070$, **$\Delta = -0.00973$ BPB**, resolving $5.6\%$ of the $+0.17346$ BPB gap at zero extra forward passes).
   - *Component 5b (+Two-Level Window-Hadamard Value Transport `CM-Attn-WH`, $K_v=8$)*: `fwd_streaming_oracle_mlp_pull_only_fullvocab` ($1.73609$) $\to$ `fwd_streaming_oracle_mlp_whv_fullvocab` ($1.71756 \pm 0.00097$, **$\Delta = -0.01853$ BPB**, cumulative $-0.02826$ BPB / $16.3\%$ of the gap).
   - *Component 5c (+Post-RoPE `Hadamard-QK`, $K_{\text{qk}}=8$)*: `fwd_streaming_oracle_mlp_whv_fullvocab` ($1.71756$) $\to$ `fwd_streaming_oracle_coupled_fullvocab` ($1.64800 \pm 0.00172$, **$\Delta = -0.06956$ BPB**, cumulative **$-0.09782$ BPB / $56.4\%$** of the gap).
   - *Component 5d (Full-Vocabulary Parameter-Space `Co-WM-S` at $K=S=8$ vs. Analytic Linear Pullbacks)*: `fwd_streaming_oracle_coupled_fullvocab` ($1.64800$) $\to$ `fwd_streaming_cowm_coupled_fullvocab_K8` (**$1.64719 \pm 0.00067$**, $\Delta = -0.00081$ BPB, resolving **$56.9\%$** of the gap using 100% parameter-space `Co-WM-S` column contractions across all 4 linear sites per block!).
6. **Sub-Vocabulary Readout Truncation ($V=8192 \to V_{\text{sub}}=256$)**: Under coupled attention, restricting readout pullback to $V_{\text{sub}}=256$ yields `fwd_streaming_oracle_coupled` ($1.78435 \pm 0.00103$, $\Delta = +0.13635$ BPB) and `fwd_streaming_cowm_coupled_K8` ($1.78546 \pm 0.00297$, $\Delta = +0.00111$ BPB vs. analytic); under uncoupled attention, `fwd_streaming_oracle` ($1.86466 \pm 0.00221$) and `fwd_streaming_cowm_K8` ($1.86518 \pm 0.00011$, $\Delta = +0.00052$ BPB).

### 2.5 MAP Settling: Single-Step Equivalence vs. Multi-Step Spectral Damping
In Post-Activation MAP Settling [1, Eq. 1, 5, 6, 7], post-activation discrepancy $e_1 = H^1 - \phi(W_1 x)$ minimizes:
$$E(e_1) = \frac{\rho}{2} \|e_1\|_2^2 + \frac{1}{2} \|e_2(e_1)\|_2^2, \qquad e_2(e_1) = -\epsilon_{\text{clamp}} \hat{\delta}_y - e_1 V^\top, \quad e_1(0) = \mathbf{0}.$$

**Proposition 4 (GS-MAP Spectral Filter and Settling Provenance)**:
*Under the recurrence $e_1(n+1) = e_1(n) - \eta (\rho e_1(n) - e_2(n) V)$ with non-oscillatory step size $0 < \eta \le \frac{1}{\rho + \|V\|_2^2}$, the exact $N$-step credit operator is:*
$$\hat{g}_h^{\text{GS-MAP}(N)} = (\delta_y V) \, \mathcal{F}_N(V^\top V), \qquad \text{where } \mathcal{F}_N(V^\top V) = \frac{\sum_{k=0}^{N-1} \big(I - \eta(\rho I + V^\top V)\big)^k}{\sum_{k=0}^{N-1} (1 - \eta \rho)^k}.$$
1. **$N=1$ Equivalence**: At $N = 1$, $\mathcal{F}_1(V^\top V) = I$, recovering exact local credit $\delta_y V$ ($\text{RelBias} \le 0.00022$ in Table 2).
2. **Multi-Step Spectral Damping ($N \ge 2$)**: Rather than improving local credit accuracy, $N \ge 2$ applies a contractive spectral filter that damps top singular modes ($\text{RelBias} = 0.129\text{--}0.163$ in Table 2).

### 2.6 Strict No-Backward Credit Assignment: Faithful DUST vs. Delayed / Eligibility-Trace Coordinated Weight Maximization (`Delayed-Trace-Co-WM-S`)
While our streaming algorithms (`fwd_streaming_cowm*`, `fwd_lookahead_2b*`) eliminate the global reverse-depth sweep (`num_depth_reverse_sweeps = 0`), they still execute block-local reverse sensitivity pullbacks across activation/`RMSNorm` derivatives and intra-block sublayers. A **strict no-backward** algorithm must compute parameter updates without either a global reverse-depth sweep or local reverse derivative pullbacks.

**Published Faithful DUST (`dust_faithful.py`) [5]**:
In published DUST (`https://qlabs.sh/research/dust/` and `literature/dust/dust.py`), parameter updates are estimated by perturbing activation groups during forward execution (`writers`: `attn.c_proj` and `mlp.c_proj`; `hidden`: `mlp.c_fc`; `hubs`: `o@layer` attention head outputs; `wte`; and 256-column `lm_head` slabs; plus 2-sided `scalar_gradients` on `resid_lambdas, x0_lambdas, ve_gate`) with draw-chunk centered per-token loss rewards $\tilde{\ell}_{b, t}^{(p)} = \ell_{b, t}^{(p)} - \frac{1}{|\mathcal{C}|}\sum_{q \in \mathcal{C}} \ell_{b, t}^{(q)}$ (and future-token decayed per-head rewards $\gamma = 0.98$ inside `local_errors` for $W_Q, W_K, W_V$). Crucially:
1. *Non-Zero `c_proj` Initialization*: Because DUST perturbs `mlp.c_fc` (`hidden`) during forward execution and observes the loss shift through `mlp.c_proj`, zero-initializing `c_proj` (as in `baseline_code/train.py`) blocks `hidden` signals at step 0; published DUST initializes `attn.c_proj` and `mlp.c_proj` with $\text{std} = 1.5/\sqrt{d_{\text{in}}}$.
2. *Prefix-Cached Suffix Execution vs. Hook Reruns*: While the minimal reference script (`literature/dust/dust.py`) registers forward hooks and reruns the full network from `wte`, the DUST paper [5, §2.3] specifies prefix-cached suffix evaluation (`blocks[l:]`), which evaluates only blocks downstream of the perturbed block $l$ with bit-for-bit identical gradients (`max_grad_diff = 0.0` at $P=256$, verified in Stage 1A).
3. *Hidden-Width Scaling Bottleneck of Direct `hidden` Probing*: When DUST perturbs all $M$ hidden neurons of `mlp.c_fc` simultaneously with $\xi_{b, t} \in \mathbb{R}^M$ ($\sigma = 0.2$), neuron $i$'s estimator correlates $\xi_{b, t, i}$ with a token reward containing the summed perturbation of all $M-1$ sibling neurons $\sum_{j \ne i} (\mathbf{v}^{(j)} \cdot \delta_{b, t}) \xi_{b, t, j}$. Consequently, DUST's `mlp.c_fc` gradient variance grows linearly with hidden neuron count $\mathcal{O}(M / P_{\text{hidden}})$.

**Delayed / Eligibility-Trace Coordinated Weight Maximization (`Delayed-Trace-Co-WM-S`, `delayed_trace_cowm.py`)**:
To eliminate DUST's $\mathcal{O}(M / P_{\text{hidden}})$ sibling-neuron interference on `mlp.c_fc` **without** direct `hidden` forward passes and **without** any local reverse/pullback step, we transfer parameter-space `Co-WM-S` into a three-phase eligibility-trace rule:
1. **Phase 1 — Clean-Pass Local Forward Eligibility-Trace Recording (`record_mlp_forward_eligibility_traces`)**:
   During the single clean forward pass at block $l$, let $x_{g, s} \in \mathbb{R}^D$ be the pre-synaptic input to `mlp.c_fc` ($W_1 \in \mathbb{R}^{M \times D}$) and $z_{g, s} = W_1 x_{g, s} \in \mathbb{R}^M$ the clean pre-activation in micro-group $g \in \{1, \dots, G\}$ ($G = BT / S, S=8$). Using only local forward evaluations of the activation function $\phi(z) = \max(0, z)^2$:
   $$d_{g, s, i} \triangleq \delta h_{0, g, s, i} = \frac{\phi(z_{g, s, i} + \sigma_{\text{act}}) - \phi(z_{g, s, i} - \sigma_{\text{act}})}{2\sigma_{\text{act}}}, \qquad \tilde{h}_{g, s, i}^{(m)} = d_{g, s, i} \, \xi_{g, s}^{(m)},$$
   where $\xi_{g, s}^{(m)} = H_{s, c_m} \in \{-1, +1\}$ ($m = 1, \dots, K_{\text{wm}}, K_{\text{wm}} \le S$) are shared Sylvester-Hadamard micro-horizon probes (`64 KB` per layer, since each neuron is already spatially decoupled from sibling neurons by its own outgoing column $\mathbf{v}^{(i)}$). The local pre-synaptic input eligibility trace for micro-group $g$ and probe $m$ is $e_g^{(m)} = \sum_{s=1}^S \xi_{g, s}^{(m)} x_{g, s} \in \mathbb{R}^D$ (retaining $d_{g, s, i}$ inside $\tilde{h}_{g, s, i}^{(m)}$ so the activation sensitivity enters the weight update exactly once).
2. **Phase 2 — Outgoing Writer Forward Perturbation (Same-Step `d0` or 1-Step Cross-Batch Delayed `d1`)**:
   The outgoing projection $V = \text{mlp.c\_proj} \in \mathbb{R}^{D \times M}$ obtains its forward writer reward vector $u_{g, s} \in \mathbb{R}^D$ from DUST's `writers` perturbation draws (either from the current batch `d0` or from the previous step's writer reward buffer `d1` before overwriting). Zero direct `hidden` (`mlp.c_fc`) forward passes are executed (`n_hidden_draws = 0`).
3. **Phase 3 — Outgoing-Column Contraction and Trace Modulation (`contract_delayed_outgoing_columns_with_trace`)**:
   For each micro-group $g$ and Hadamard probe $m$, neuron $i$ forms its outgoing weight-column plastic displacement $\Delta \mathbf{v}_g^{(i, m)} = \sum_{s'=1}^S \tilde{h}_{g, s', i}^{(m)} u_{g, s'} = \sum_{s'=1}^S d_{g, s', i} \, \xi_{g, s'}^{(m)} u_{g, s'} \in \mathbb{R}^D$ (computed across all $M$ neurons simultaneously via Tensor-Core `torch.bmm`), contracts its own outgoing column $\mathbf{v}^{(i)} = V_{:, i} \in \mathbb{R}^D$ with $\Delta \mathbf{v}_g^{(i, m)}$:
   $$c_g^{(i, m)} = \mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)} = \sum_{s'=1}^S d_{g, s', i} \big(\mathbf{v}^{(i)} \cdot u_{g, s'}\big) \xi_{g, s'}^{(m)} \;\in\; \mathbb{R},$$
   and accumulates its `mlp.c_fc` row update by modulating the pre-synaptic input trace $e_g^{(m)}$ (`mod_accum.T @ x_g` in `delayed_trace_cowm.py` lines 171–174):
   $$\widehat{\nabla}_{W_{1, i}} = \frac{1}{K_{\text{wm}}} \sum_{m=1}^{K_{\text{wm}}} \sum_{g=1}^G c_g^{(i, m)} e_g^{(m)} = \sum_{g=1}^G \sum_{s=1}^S \left(\frac{1}{K_{\text{wm}}} \sum_{m=1}^{K_{\text{wm}}} c_g^{(i, m)} \xi_{g, s}^{(m)}\right) x_{g, s}.$$
   *Exact Conditional Recovery vs. Finite Precision and Writer-Reward Noise*:
   - **Exact Algebraic Recovery ($K_{\text{wm}} = S$)**: By Sylvester-Hadamard completeness ($\frac{1}{S}\sum_{m=1}^S \xi_{g, s'}^{(m)} \xi_{g, s}^{(m)} = \mathbf{1}[s' = s]$), the reconstructed pre-activation modulation satisfies the exact single-sensitivity identity:
     $$\frac{1}{S}\sum_{m=1}^S c_g^{(i, m)} \xi_{g, s}^{(m)} \equiv d_{g, s, i} \big(\mathbf{v}^{(i)} \cdot u_{g, s}\big), \qquad \widehat{\nabla}_{W_{1, i}}\Big|_{K_{\text{wm}}=S} \equiv \sum_{g=1}^G \sum_{s=1}^S d_{g, s, i} \big(\mathbf{v}^{(i)} \cdot u_{g, s}\big) x_{g, s}.$$
     (For example, at $S = K_{\text{wm}} = 8$ with scalar $x_s = u_s = v = 1$ and $z_s = 1 \implies d_s = 2$, both the displayed formula and `contract_delayed_outgoing_columns_with_trace` evaluate to $\sum_{s=1}^8 2 \cdot 1 \cdot 1 = 16.0$, verified in Gate 2L.)
   - **Finite-Precision Tensor-Core Agreement**: In `delayed_trace_cowm.py`, $x_g, \tilde{h}_0, V_{\text{snapshot}}, u_g$ are stored in `bfloat16` for Tensor-Core `torch.bmm` throughput and accumulated in `float32`, yielding conditional cosine similarity `0.999997` against the `bfloat16` reference in Gate 2L (`1.000000` rounded to 6 decimals in Table 7's conditional panel).
   - **Remaining Writer-Reward Estimation Noise**: When $u_{g, s}$ is estimated from a finite number $P_{\text{writers}}$ of forward writer draws rather than the exact local target $\delta_{g, s}$, $K_{\text{wm}} = S = 8$ eliminates the internal MLP column-contraction variance ($\text{Var}[\widehat{\nabla}_{W_1} \mid u] = 0$), while the remaining whole-network variance is governed solely by the $D$-dimensional outgoing writer estimate $u_{g, s}$ (independent of hidden neuron count $M$).
4. **Phase 4 — Strict No-Backward Attention Hub Centering (`hub2`, `hub_mode="dust_hub"`, `rev=0`) vs. Hybrid Local Attention-Hub Pullback (`Hub-Co-WM-S`, `hub_mode="co_wm_s"`, `rev=6`)**:
   - *Strict No-Backward Hub-Balanced Allocation (`hub2`, `delayed_trace_cowm_*`, `hub_mode="dust_hub"`, `local_reverse_derivative_calls = 0`)*: In published DUST [5] at $P=64$ (`n_layer=6`), the attention hub allocation `hubs = [2, 2, 1, 1, 1, 1]` assigns only $1$ suffix forward draw (`draws = 1`) to attention hubs `o@2, o@3, o@4, o@5`. Because `draws = 1` cannot be sample-mean centered across draws (`delayed_trace_cowm.py` line 696), four of the six attention hubs in `dust_faithful_muon_P64_sc_sh` fall back to uncentered 1-sample REINFORCE. Because `Delayed-Trace-Co-WM-S` eliminates all $27$ `hidden` (`mlp.c_fc`) suffix forward draws (`[9, 8, 4, 3, 2, 1]`), reallocating just $4$ of those $27$ saved draws to `hubs` (`hubs = [2, 2, 2, 2, 2, 2]` $= 12$ draws, `hub2`) and $23$ to `writers` (`writers = [12, 10, 8, 7, 6, 6]` $= 49$ draws; total $49 + 12 + 3 = 64$ direct draws) in `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` ensures every attention hub `o@0..5` has $\ge 2$ draws—activating full-block covariance centering (`sc`) across all $6$ hubs while retaining DUST's exact forward hub perturbations (`site_kind="hub"`) and $\gamma$-decayed ($\gamma = 0.98$) local attention perturbations (`causal_v_trace=False`), with **zero** reverse pullbacks (`local_reverse_derivative_calls = 0`).
   - *$W_O$-Free Forward Causal Value Pooling (`record_attn_v_forward_pooled_input`) and Concurrent Work [14]*: Because $v_{b, \tau}^{(h)} = W_V^{(h)} x_{\text{in}, b, \tau}$ is linear, Fubini's theorem commutes $W_V^{(h)}$ with forward causal attention:
     $$o_{b, t}^{(h)} = \sum_{\tau \le t} A_{b, h, t, \tau} \big(W_V^{(h)} x_{\text{in}, b, \tau}\big) = W_V^{(h)} \underbrace{\left(\sum_{\tau \le t} A_{b, h, t, \tau} x_{\text{in}, b, \tau}\right)}_{\triangleq \, \bar{x}_{b, t}^{(h)} \in \mathbb{R}^D}.$$
     This forward attention-filtered value trace identity was also independently derived in Proposition 2 of concurrent work `general_pretraining_20/v1` [14] (published on rStation approximately six hours prior to our initial v5 submission within the concurrency window). Under `hub_mode="dust_hub"` (`record_attn_v_forward_pooled_input`), $\bar{x}_{b, t}^{(h)}$ can be reassembled directly with DUST's forward suffix hub error $\hat{u}_{\text{hub}, b, t}^{(h)} = \texttt{targets[layer]}_{b, t}^{(h)}$ at `o@layer` (upstream of $W_O$) via $\sum_{b, t} \hat{u}_{\text{hub}, b, t}^{(h)} (\bar{x}_{b, t}^{(h)})^\top$ (`local_reverse_derivative_calls = 0`, zero $W_O$ involvement). However, our controlled 200-step ablation on `d6` (Table 10) reveals that at $P=64$, retaining DUST's recency-weighted ($\gamma^{t-\tau}, \gamma = 0.98$) local value perturbation (`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`, `2.12965` holdout BPB) outperforms unweighted ($\gamma = 1$) causal value pooling (`delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1`, `2.16374` holdout BPB) by `-0.03409 BPB`; we therefore keep `causal_v_trace=False` in all primary strict no-backward `delayed_trace_cowm_*` pretraining arms.
   - *Hybrid Local Attention-Hub Pullback Control (`Hub-Co-WM-S`, `hub_mode="co_wm_s"`, `local_reverse_derivative_calls = L = 6`)*: In `contract_hub_outgoing_columns` (`hub_mode="co_wm_s"`), micro-group $g$ forms $\Delta W_g = H^\top U_g$ from the current `attn.c_proj` writer rewards $U_g \in \mathbb{R}^{S \times D}$, multiplies $C_g = \Delta W_g W_O$, and reconstructs the $(B, T, D)$ upstream activation-credit tensor $\widehat{E}_g = \frac{1}{K_{\text{wm}}} H C_g$ to supply `targets[layer]` to `local_errors` on the same step (and similarly in `contract_delayed_outgoing_columns_with_attn_v_trace`). At the implemented complete Hadamard basis $K_{\text{wm}} = S = 8$, $\frac{1}{S} H H^\top = I_S$ yields the exact identity:
     $$\Delta W_g = H^\top U_g, \qquad C_g = \Delta W_g W_O, \qquad \widehat{E}_g = \frac{1}{S} H C_g = U_g W_O.$$
     For the forward linear map $Y = O W_O^\top$, $U_g W_O$ is the exact 1-layer local linear reverse pullback of the output credit $U_g$ across $W_O$, executed in the Sylvester-Hadamard token basis (`CosSim = 0.999998` against `u_attn @ W_O` in Gate 2M). Accordingly, all `hub_mode="co_wm_s"` variants (`delayed_trace_full_*` and `delayed_trace_unified_*`) record `local_reverse_derivative_calls = L = 6` (`is_pure_forward_only = False`, `has_local_backward_phase = True`) and are presented strictly as **hybrid local-hub-pullback controls** quantifying the effect of replacing forward `o@layer` hub suffix passes with a 1-layer local linear pullback across $W_O$.
5. **SRAM-Fused Tensor-Core Outgoing-Column Contraction Kernel (`_outgoing_col_dot_all8_kernel`)**:
   In a naive PyTorch implementation, forming $\Delta V_g^{(m)} = u_{g, m} \tilde{h}_0^\top \in \mathbb{R}^{G \times D \times M}$ via `torch.bmm` and contracting $(V \odot \Delta V_g^{(m)}).\text{sum}(\text{dim}=1)$ across $K_{\text{wm}}=8$ probes materializes $8 \times G \times D \times M$ intermediate tensors in HBM (`247.4 ms` across 6 layers at $D=256, M=4096$). In `delayed_trace_cowm.py` (lines 125–244), `_outgoing_col_dot_all8_kernel` fuses the outer-product displacement and column dot product across all $K_{\text{wm}}=8$ Hadamard probes inside a single Triton kernel: each program loads a $(B_D \times B_M)$ tile of $V_{\text{snapshot}}$ into SRAM registers once, computes $\Delta V_g^{(1\dots 8)}$ via `tl.dot` on Tensor Cores, multiplies by $V_{\text{snapshot}}$ in SRAM, and accumulates the $8$ scalar column contractions without ever writing $(G, D, M)$ to HBM—reducing 6-layer contraction latency at $M=4096$ from `247.4 ms` to **`18.7–19.1 ms` ($13.0\times$ faster, `cos = 0.9999986`)**.
6. **Fairly Optimized Shared Backward-Free Mechanisms (`sc`, `sh`, `bd025`, `st`)**:
   To ensure complete experimental fairness when improving residual-writer and readout estimation, we implement four strictly backward-free mechanisms in `delayed_trace_cowm.py` (`local_reverse_derivative_calls = 0`) and apply the shared writer/head improvements (`sc+sh`) identically to **both** Fairly Optimized DUST (`dust_faithful_muon_P64_sc_sh`, `dust_faithful_muon_P256_sc_sh`) and Strict No-Backward `Delayed-Trace-Co-WM-S` (`delayed_trace_cowm_*_sc_sh_*`, as well as the hybrid `delayed_trace_unified_*` controls):
   - *Full-Block Sample Covariance Centering (`simplex_centering=True`, `sc`)*: Centers per-token loss rewards $\tilde{\ell}_{b, t}^{(p)} = \ell_{b, t}^{(p)} - \frac{1}{K_w}\sum_{q=1}^{K_w} \ell_{b, t}^{(q)}$ across all $K_w \ge 2$ draws at a site rather than $n=2$ pairs, eliminating finite-chunk baseline variance.
   - *Target-Separated Antithetic Hadamard Vocabulary Probing (`stratified_head=True`, `sh`)*: Evaluates an exact 1D forward finite difference on the observed target coordinate $y_{b, t}$ (`logits.scatter_add_`) alongside vectorized antithetic Sylvester-Hadamard probes on non-target 256-column vocabulary slabs (`0` softmax derivatives), raising `lm_head` cosine from `0.214` to `0.946` and `wte` cosine from `0.156` to `0.625`.
   - *Intra-Block Attention-MLP Branch Decoupling (`branch_decouple_alpha=0.25`, `bd025`)*: In standard DUST `writers` draws, `attn.c_proj` noise $\xi_{\text{attn}}$ and `mlp.c_proj` noise $\xi_{\text{mlp}}$ are summed into the same residual stream $h_{\text{mid}} + \xi_{\text{attn}} + \text{MLP}(\text{norm}(h_{\text{mid}} + \xi_{\text{attn}})) + \xi_{\text{mlp}}$, creating anti-correlated common-mode variance between $\hat{u}_{\text{attn}}$ and $\hat{u}_{\text{mlp}}$. Setting `branch_decouple_alpha=0.25` routes `h_mid_pert = h_mid + xi_attn` **only** into the forward MLP branch `block.mlp(_norm(h_mid_pert))` while keeping the direct skip connection at unperturbed `h_mid_clean`, then recombines $\hat{u}_{\text{attn}} = \hat{u}_{\text{mlp}} + \alpha \hat{u}_{\text{via-mlp}}$ (`0` backward calls, `0` local reverse pullbacks), lifting Layer 3 `mlp.c_fc` cosine by **`+71.6%` (`0.0775 -> 0.1330`)**.
   - *Forward Residual-Stream Scalar Eligibility Trace (`scalar_trace=True`, `st`)*: Replaces the $16$ full-network ES forward passes for `resid_lambdas` and `x0_lambdas` with local forward inner products between clean block states $(x_{\text{in}, l}, x_0)$ and `writer_errors_attn[l]`, saving $16$ full forward passes per step (`~110–180 ms/step`).

---

## 3. Experimental Validation

### 3.1 Whole-Transformer Initialization Diagnostic and Unbiased Bias/Variance Decomposition
Table 3 reports gradient fidelity against analytical backpropagation across widths $D \in \{128, 256, 384, 512\}$ ($L=6, B=4, T=64, V=4096$) over $R = 32$ independent random trials (`pure_forward_width_diagnostic.json`).

**Unbiased Squared Relative Bias vs. Plug-In Sample-Mean Bias**:
For any stochastic gradient estimator $\hat{G}^{(r)}$ ($r = 1, \dots, R$) with sample mean $\bar{G}_R = \frac{1}{R}\sum_{r=1}^R \hat{G}^{(r)}$, expectation $\mu = \mathbb{E}[\hat{G}]$, and reference gradient $G_{\text{bp}}$, the naive plug-in squared relative bias $\widehat{\text{RelBias}}^2_{\text{plug}} = \frac{\|\bar{G}_R - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}$ has expectation:
$$\mathbb{E}\!\left[\widehat{\text{RelBias}}^2_{\text{plug}}\right] = \frac{\|\mu - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2} + \frac{1}{R} \text{RelVar}(\hat{G}), \qquad \text{where } \widehat{\text{RelVar}} = \frac{1}{R - 1} \sum_{r=1}^R \frac{\|\hat{G}^{(r)} - \bar{G}_R\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$
When $\text{RelVar}$ is large, $\widehat{\text{RelBias}}^2_{\text{plug}}$ is dominated by residual sample-mean variance $\text{RelVar}/R$ rather than true systematic bias. To eliminate this contamination:
1. We compute **Total Relative MSE** directly from the $R=32$ individual trial errors (with standard error of the mean across trials):
   $$\widehat{\text{TotalMSE}} = \frac{1}{R} \sum_{r=1}^R \frac{\|\hat{G}^{(r)} - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$
2. We report the **unbiased bias-corrected squared relative bias estimator** (with Tukey leave-one-out jackknife standard error across the $R=32$ trials):
   $$\widehat{\text{RelBias}}^2_{\text{unb}} = \widehat{\text{RelBias}}^2_{\text{plug}} - \frac{1}{R} \widehat{\text{RelVar}}, \qquad \mathbb{E}\!\left[\widehat{\text{RelBias}}^2_{\text{unb}}\right] = \frac{\|\mu - G_{\text{bp}}\|_F^2}{\|G_{\text{bp}}\|_F^2}.$$
   By algebraic identity, $\widehat{\text{TotalMSE}} = \frac{R-1}{R}\widehat{\text{RelVar}} + \widehat{\text{RelBias}}^2_{\text{plug}} \equiv \widehat{\text{RelVar}} + \widehat{\text{RelBias}}^2_{\text{unb}}$. When $\widehat{\text{RelBias}}^2_{\text{unb}}$ is comparable to or smaller than its jackknife standard error (as in `full_seq_dot` and `token_dot`), no nonzero bias is statistically resolved, and its square root should not be interpreted as a resolved bias magnitude.

| Algorithm | Paradigm / Carrier & Readout | $D=128$ Top / Mid Cos | $D=512$ Top / Mid Cos | Mid Scale ($D=512$) | Mid Param RelVar ($D=128 \to 512$) | Mid Unbiased $\text{RelBias}^2$ ($D=512$) | Mid TotalMSE ($D=512$) | Attn Q5 Cos / RelVar ($D=512$) |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| `fwd_global_scalar_K2` | Global Scalar Control ($K=2$) | `-0.002` / `0.002` | `0.000` / `-0.000` | $1007.95$ | $7.79\times 10^6 \to 3.26\times 10^7$ | $-2250.8 \pm 10136.6$ | $3.26\times 10^7 \pm 4.50\times 10^6$ | `-0.0003` / $1.83\times 10^8$ |
| `fwd_token_uwm_K2` | Token-Scalar ($K=2$) | `0.004` / `0.007` | `0.004` / `0.003` | $56.74$ | $26,968 \to 103,931$ | $-29.98 \pm 26.68$ | $103,901 \pm 1075$ | `0.0002` / $5.84\times 10^5$ |
| `fwd_two_level_cowm_K8_fullseq` | Two-Level ($K=8$, `full_seq_dot`, DH) | `0.003` / `0.006` | `0.003` / `0.003` | $49.85$ | $21,803 \to 78,606$ | $27.44 \pm 27.86$ | $78,634 \pm 8449$ | `0.0012` / $4.88\times 10^6$ |
| `fwd_two_level_cowm_K8_fullseq_iid` | Two-Level ($K=8$, `full_seq_dot`, i.i.d.) | `0.005` / `0.009` | `0.001` / `0.003` | $53.07$ | $22,705 \to 90,830$ | $-23.07 \pm 36.45$ | $90,807 \pm 3178$ | `0.0004` / $5.85\times 10^6$ |
| `fwd_two_level_cowm_K8_tokendot` | Two-Level ($K=8$, `token_dot`, DH) | `0.052` / `0.059` | `0.026` / `0.029` | $6.12$ | $303.50 \to 1180.58$ | $-0.453 \pm 0.422$ | $1180.13 \pm 39.33$ | `0.0042` / $69,297$ |
| `fwd_two_level_cowm_K8_tokendot_iid` | Two-Level ($K=8$, `token_dot`, i.i.d.) | `0.051` / `0.054` | `0.024` / `0.028` | $6.67$ | $350.00 \to 1386.78$ | $0.088 \pm 0.455$ | $1386.87 \pm 7.47$ | `0.0042` / $89,860$ |
| **`fwd_two_level_cowm_K4`** | Two-Level ($K=4$, `diag_jacobian`, DH) | `0.412` / `0.355` | `0.410` / `0.365` | $0.96$ | $4.30 \to 4.13$ | $0.224 \pm 0.003$ | $4.35 \pm 0.27$ | `0.0560` / $330.05$ |
| **`fwd_two_level_cowm_K8_iid`** | Two-Level ($K=8$, `diag_jacobian`, i.i.d.) | `0.481` / `0.421` | `0.479` / `0.426` | $0.93$ | $2.64 \to 2.58$ | $0.224 \pm 0.002$ | $2.81 \pm 0.00$ | `0.0649` / $216.35$ |
| **`fwd_two_level_cowm_K8`** | Two-Level ($K=8$, `diag_jacobian`, DH) | `0.598` / `0.512` | `0.615` / `0.516` | $0.91$ | **$1.61 \to 1.55$** | **$0.222 \pm 0.002$** | **$1.77 \pm 0.07$** | `0.0837` / $136.49$ |
| `fwd_streaming_cut_K4` | Streaming Cut ($K=4$, Token-Scalar) | `0.016` / `0.015` | `0.004` / `0.003` | $47.76$ | $4638 \to 73,120$ | $-4.43 \pm 23.70$ | $73,116 \pm 815$ | `0.0037` / $8.28\times 10^5$ |
| **`fwd_streaming_wm_K1`** | Streaming `WM-S` ($K=1$, Indep., $K_{\text{qk}}=1$) | `0.129` / `0.109` | `0.125` / `0.112` | $1.80$ | $69.28 \to 68.66$ | $0.244 \pm 0.018$ | $68.90 \pm 0.18$ | `0.0135` / $6375.22$ |
| **`fwd_streaming_wm_K1_qk2`** | Streaming `WM-S` ($K=1$, Indep., $K_{\text{qk}}=2$) | `0.127` / `0.107` | `0.126` / `0.110` | $1.80$ | $69.29 \to 68.43$ | $0.249 \pm 0.022$ | $68.68 \pm 0.16$ | `0.0153` / $3156.59$ |
| **`fwd_streaming_wm_K4`** | Streaming `WM-S` ($K=4$, Indep.) | `0.363` / `0.318` | `0.362` / `0.322` | $1.15$ | $7.19 \to 7.15$ | $0.246 \pm 0.003$ | $7.39 \pm 0.01$ | `0.0516` / $374.23$ |
| **`fwd_streaming_cowm_K4`** | Streaming `Co-WM-S` ($K=4$, Had.) | `0.496` / `0.438` | `0.498` / `0.442` | $1.09$ | $3.29 \to 3.26$ | $0.251 \pm 0.002$ | $3.51 \pm 0.00$ | `0.0688` / $197.17$ |
| **`fwd_streaming_wm_K8`** | Streaming `WM-S` ($K=8$, Indep.) | `0.533` / `0.465` | `0.532` / `0.471` | $1.09$ | $2.78 \to 2.74$ | $0.253 \pm 0.002$ | $2.99 \pm 0.01$ | `0.0731` / $173.44$ |
| **`fwd_streaming_cowm_K8`** | Streaming `Co-WM-S` ($K=8$, Had., Uncoupled) | **`0.999`** / **`0.879`** | **`0.998`** / **`0.881`** | **$1.05$** | **$0.00 \to 0.00$** | **$0.251 \pm 0.000$** | **$0.25 \pm 0.00$** | `0.1397` / $47.85$ |
| **`fwd_streaming_cowm_coupled_K8`** | Coupled `Co-WM-S` ($K=8$, $V_{\text{sub}}=256$) | **`0.999`** / **`0.879`** | **`0.998`** / **`0.881`** | **$1.05$** | **$0.00 \to 0.00$** | **$0.251 \pm 0.000$** | **$0.25 \pm 0.00$** | **`0.3513`** / **$6.93$** |
| **`fwd_streaming_cowm_coupled_fullvocab_K8`** | Coupled `Co-WM-S` ($K=8$, $V=4096$) | **`1.000`** / **`0.879`** | **`1.000`** / **`0.882`** | **$1.05$** | **$0.00 \to 0.00$** | **$0.249 \pm 0.000$** | **$0.25 \pm 0.00$** | **`0.3523`** / **$6.90$** |
| `fwd_streaming_oracle` | Analytic Surrogate ($V_{\text{sub}}=256$, Uncoupled) | `0.999` / `0.879` | `0.998` / `0.881` | $1.04$ | $0.00 \to 0.00$ | $0.250 \pm 0.000$ | $0.25 \pm 0.00$ | `0.1413` / $48.11$ |
| `fwd_streaming_oracle_coupled` | Analytic Surrogate ($V_{\text{sub}}=256$, Coupled) | `0.999` / `0.879` | `0.998` / `0.881` | $1.04$ | $0.00 \to 0.00$ | $0.250 \pm 0.000$ | $0.25 \pm 0.00$ | **`0.3518`** / **$6.85$** |
| `fwd_streaming_oracle_fullvocab` | Analytic Surrogate ($V=4096$, Uncoupled) | `1.000` / `0.879` | `1.000` / `0.882` | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | `0.1378` / $47.31$ |
| `fwd_streaming_oracle_mlp_pull_only_fullvocab` | +MLP-Pullback Only ($V=4096, K_v=1$) | `1.000` / `0.879` | `1.000` / `0.882` | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | `0.1468` / $52.55$ |
| `fwd_streaming_oracle_mlp_whv_fullvocab` | +MLP-Pullback + `CM-Attn-WH` ($V=4096, K_v=8$) | `1.000` / `0.879` | `1.000` / `0.882` | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | `0.1458` / $52.16$ |
| `fwd_streaming_oracle_coupled_fullvocab` | Analytic Surrogate ($V=4096$, Coupled) | `1.000` / `0.879` | `1.000` / `0.882` | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | **`0.3525`** / **$6.86$** |
| `block_local_greedy_bp_vsub256` | Block-Local Greedy BP ($V_{\text{sub}}=256$) | `0.999` / `0.879` | `0.998` / `0.881` | $1.04$ | $0.00 \to 0.00$ | $0.249 \pm 0.000$ | $0.25 \pm 0.00$ | `0.9996` / $0.00$ |
| `block_local_greedy_bp` | Block-Local Greedy BP [8] ($V=4096$) | `1.000` / `0.879` | `1.000` / `0.882` | $1.04$ | $0.00 \to 0.00$ | $0.248 \pm 0.000$ | $0.25 \pm 0.00$ | `1.0000` / $0.00$ |
| **`block_fixed_top_discrepancy_bp`** | Fixed-Top-Discrepancy BP ($V=4096$) | **`1.000`** / **`0.881`** | **`1.000`** / **`0.882`** | **$0.88$** | **$0.00 \to 0.00$** | **$0.221 \pm 0.000$** | **$0.22 \pm 0.00$** | **`1.0000`** / **$0.00$** |

*Table 3: Whole-Transformer width-scaling diagnostic ($L=6, B=4, T=64, V=4096, R=32$ trials; DH = `depth_hadamard` carrier modulation, i.i.d. = `iid_resample` per-probe carrier resampling). At $D=512$, Layer 3 activation-credit relative variances (`rel_var_act_mid_mlp` in `pure_forward_width_diagnostic.json`) closely track the parameter-gradient relative variances shown above: $79,218.20$ (DH) vs $91,443.37$ (i.i.d.) for `full_seq_dot`, $1191.29$ (DH) vs $1398.44$ (i.i.d.) for `token_dot`, and $1.56$ (DH) vs $2.60$ (i.i.d.) for `diag_jacobian`. The measured full-sequence sensitivity-energy ratio $B_{\text{full}}^{(8)}/A_{\text{full}}^{(8)} = 20.30 > 15$ quantitatively explains the `full_seq_dot` DH-vs-i.i.d. variance ratio ($0.866 \approx 0.884$ theoretical), whereas `token_dot` and `diag_jacobian` are governed by their own within-site and between-site variance expressions ($A_{\text{tok}}, B_{\text{tok}}, A_{\text{diag}}, B_{\text{diag}}$ in Section 2.4). Furthermore, at $D=512$, Layer 3 block-local attention cosines (`cossim_blkref_mid_attn_{o,v}`) rise from $0.914 / 0.916$ (`fwd_streaming_cowm_K8`) to $0.987 / 0.987$ under sub-vocabulary coupling (`fwd_streaming_cowm_coupled_K8`), and from $0.927 / 0.926$ (`fwd_streaming_oracle_fullvocab`) to **$1.000 / 1.000$** under full-vocabulary coupling (`fwd_streaming_cowm_coupled_fullvocab_K8` at $0.99998 / 0.99997$ and `fwd_streaming_oracle_coupled_fullvocab` at $1.00000 / 1.00000$), while `Hadamard-QK` lifts Attn Q5 cosine from $0.138$ to **$0.353$** and reduces relative variance by $85.5\%$ ($47.31 \to 6.86$).*

### 3.2 200-Step Multi-Width Pretraining Sweep
Table 4 reports the 200-step pretraining sweep across $D \in \{128, 256, 384, 512\}$ ($B=16, T=512, V=8192$) on ClimbMix evaluated on the dev split (row groups $0 \dots 14$) and holdout split (row groups $40 \dots 82$). All candidate forward-only methods update 100% of trainable parameters under `@torch.no_grad()` (`bwd_calls = 0`).

| Algorithm | $D=128$ Dev / Holdout BPB | $D=256$ Dev / Holdout BPB | $D=384$ Dev / Holdout BPB | $D=512$ Dev / Holdout BPB | Step ms / Peak VRAM ($D=512$) | Backward Calls |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| `backprop` (Standard) | `1.8852` / `1.8846` | `1.8410` / `1.8393` | `1.8349` / `1.8320` | `1.8160` / `1.8133` | `33.3 ms` / `3010.4 MB` | $1$ |
| `block_fixed_top_discrepancy_bp` | `1.9266` / `1.9262` | `1.8873` / `1.8868` | `1.8753` / `1.8723` | `1.8664` / `1.8643` | `33.7 ms` / `3026.5 MB` | $6$ ($1$/block) |
| `block_local_greedy_bp_tophead` | `1.9207` / `1.9193` | `1.8787` / `1.8767` | `1.8742` / `1.8715` | `1.8566` / `1.8538` | `73.5 ms` / `2153.6 MB` | $6$ ($1$/block) |
| `block_local_greedy_bp` (Mixed Head [8]) | `1.9156` / `1.9140` | `1.8809` / `1.8789` | `1.8664` / `1.8649` | `1.8469` / `1.8448` | `76.7 ms` / `2169.6 MB` | $6$ ($1$/block) |
| `fwd_global_scalar_K2` | `2.6507` / `2.6507` | `2.3906` / `2.3910` | `2.4085` / `2.4076` | `2.3913` / `2.3898` | `71.5 ms` / `3864.8 MB` | **$0$** |
| `fwd_token_uwm_K2` | `2.2138` / `2.2142` | `2.1956` / `2.1943` | `2.2131` / `2.2121` | `2.1883` / `2.1868` | `90.6 ms` / `3864.8 MB` | **$0$** |
| `fwd_streaming_cut_K4` | `2.1682` / `2.1682` | `2.1379` / `2.1372` | `2.1543` / `2.1530` | `2.1121` / `2.1126` | `130.2 ms` / `1820.3 MB` | **$0$** |
| **`fwd_streaming_wm_K1`** ($K_{\text{qk}}=1$) | `2.1310` / `2.1306` | `2.0942` / `2.0928` | `2.0752` / `2.0750` | `2.0680` / `2.0673` | `148.8 ms` / `1828.4 MB` | **$0$** |
| **`fwd_streaming_wm_K1_qk2`** ($K_{\text{qk}}=2$) | `2.1296` / `2.1301` | `2.0935` / `2.0931` | `2.0750` / `2.0738` | `2.0665` / `2.0659` | `150.9 ms` / `1820.3 MB` | **$0$** |
| **`fwd_streaming_wm_K4`** | `2.1073` / `2.1079` | `2.0717` / `2.0714` | `2.0549` / `2.0541` | `2.0471` / `2.0468` | `299.9 ms` / `1820.3 MB` | **$0$** |
| **`fwd_streaming_cowm_K4`**| `2.0975` / `2.0981` | `2.0635` / `2.0627` | `2.0468` / `2.0467` | `2.0392` / `2.0383` | `303.5 ms` / `1828.4 MB` | **$0$** |
| **`fwd_two_level_cowm_K4`**| `2.1060` / `2.1052` | `2.0750` / `2.0756` | `2.0622` / `2.0617` | `2.0559` / `2.0553` | `469.4 ms` / `1850.3 MB` | **$0$** |
| **`fwd_streaming_wm_K8`** | `2.0964` / `2.0963` | `2.0627` / `2.0629` | `2.0488` / `2.0484` | `2.0387` / `2.0392` | `498.4 ms` / `1820.3 MB` | **$0$** |
| **`fwd_streaming_cowm_K8`** (Uncoupled, $V_{\text{sub}}=256$) | `2.0803` / `2.0820` | `2.0502` / `2.0507` | `2.0373` / `2.0378` | `2.0287` / `2.0291` | `507.4 ms` / `1828.4 MB` | **$0$** |
| **`fwd_streaming_cowm_coupled_K8`** ($V_{\text{sub}}=256$) | `2.0533` / `2.0537` | `2.0216` / `2.0210` | `2.0043` / `2.0041` | `1.9985` / `1.9972` | `833.3 ms` / `1828.3 MB` | **$0$** |
| **`fwd_streaming_cowm_coupled_fullvocab_K8`** ($V=8192$) | **`1.9578`** / **`1.9578`** | **`1.9319`** / **`1.9312`** | **`1.9190`** / **`1.9178`** | **`1.8952`** / **`1.8946`** | `1811.3 ms` / `2100.3 MB` | **$0$** |
| `fwd_streaming_oracle` ($V_{\text{sub}}=256$, Uncoupled) | `2.0805` / `2.0822` | `2.0519` / `2.0517` | `2.0383` / `2.0387` | `2.0308` / `2.0302` | `99.7 ms` / `1820.3 MB` | **$0$** |
| `fwd_streaming_oracle_coupled` ($V_{\text{sub}}=256$) | `2.0556` / `2.0562` | `2.0223` / `2.0218` | `2.0032` / `2.0032` | `1.9975` / `1.9969` | `163.6 ms` / `1828.3 MB` | **$0$** |
| `fwd_streaming_oracle_fullvocab` ($V=8192$, Uncoupled) | `1.9876` / `1.9866` | `1.9686` / `1.9671` | `1.9573` / `1.9565` | `1.9514` / `1.9505` | `112.4 ms` / `2100.4 MB` | **$0$** |
| `fwd_streaming_oracle_mlp_pull_only_fullvocab` ($V=8192$) | `1.9847` / `1.9844` | `1.9653` / `1.9643` | `1.9572` / `1.9559` | `1.9433` / `1.9428` | `116.1 ms` / `2100.3 MB` | **$0$** |
| `fwd_streaming_oracle_mlp_whv_fullvocab` ($V=8192$) | `1.9807` / `1.9800` | `1.9572` / `1.9557` | `1.9505` / `1.9489` | `1.9346` / `1.9327` | `134.1 ms` / `2100.3 MB` | **$0$** |
| `fwd_streaming_oracle_coupled_fullvocab` ($V=8192$) | **`1.9586`** / **`1.9583`** | **`1.9319`** / **`1.9312`** | **`1.9185`** / **`1.9176`** | **`1.8956`** / **`1.8950`** | `177.7 ms` / `2100.3 MB` | **$0$** |

*Table 4: 200-step multi-width pretraining sweep on ClimbMix (`pure_forward_pretrain_scaling.json`). Across all widths $D \in \{128 \dots 512\}$, each component of the intra-block coupled package (`+MLP-Pullback` $\to$ `+CM-Attn-WH` $\to$ `+Hadamard-QK`) monotonically reduces holdout BPB across every width ($1.9505 \to 1.9428 \to 1.9327 \to 1.8950$ at $D=512$), while full-vocabulary Parameter-Space `Co-WM-S` (`fwd_streaming_cowm_coupled_fullvocab_K8`, $1.8946$ at $D=512$) closely matches its analytic-linear surrogate (`1.8950`).*

### 3.3 1000-Step Two-Seed Benchmark on `d6` ($8.192\text{M}$ Tokens)
Table 5 reports 1000-step ($8,192,000$ tokens, 2 seeds `s42` and `s137`) pretraining on `d6` ($D=384, M=1536, L=6, V=8192$) evaluated on the document-disjoint holdout split (`pure_forward_1000steps_d6.json`). Trainable parameters ($26,346,060$) are identical across all arms. Attention probing uses $K_{\text{qk}}=2$ ($66$ attention calls per step across $6$ blocks) for all uncoupled forward streaming arms except the original `fwd_streaming_wm_K1` which evaluated $K_{\text{qk}}=1$ ($42$ calls; matched by our newly executed benchmark arm `fwd_streaming_wm_K1_qk2` at $K_{\text{qk}}=2$), while `fwd_streaming_oracle_mlp_whv_fullvocab` uses $K_v=8, K_{\text{qk}}=2$ ($108$ calls) and full coupled arms evaluate $K_v=8, K_{\text{qk}}=8$ ($252$ calls).

| Algorithm | Seed `s42` Holdout BPB | Seed `s137` Holdout BPB | Mean Holdout BPB $\pm$ Sample SD | Mean Step ms | Peak VRAM | Backward Calls |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| `backprop` (Standard) | `1.45158` | `1.45186` | `1.45172 ± 0.00020` | `25.1 ms` | `2656.3 MB` | $1$ |
| `backprop_ckpt` (Checkpointed) | `1.45224` | `1.45134` | **`1.45179 ± 0.00064`** | **`33.3 ms`** | **`1731.4 MB`** | $1$ |
| `block_fixed_top_discrepancy_bp` (Fixed $\delta_{\text{final}}$) | `1.61470` | `1.62833` | `1.62152 ± 0.00964` | `26.2 ms` | `2668.4 MB` | $6$ ($1$/block) |
| `block_local_greedy_bp_tophead` (Local $\delta_l$, Top Head) | `1.57477` | `1.58694` | `1.58085 ± 0.00861` | `64.7 ms` | `2034.3 MB` | $6$ ($1$/block) |
| `block_local_greedy_bp` (Mixed Head [8]) | `1.57272` | `1.57200` | **`1.57236 ± 0.00051`** | `66.6 ms` | `2046.3 MB` | $6$ ($1$/block) |
| `fwd_global_scalar_K2` (Global-Scalar Control) | `2.52995` | `2.36593` | `2.44794 ± 0.11598` | `56.1 ms` | `3448.2 MB` | **$0$** |
| `fwd_token_uwm_K2` (100% Params Updated) | `2.09448` | `2.10184` | `2.09816 ± 0.00521` | `72.6 ms` | `3448.2 MB` | **$0$** |
| `fwd_streaming_cut_K4` (Matched $K=4$) | `2.01759` | `2.01353` | `2.01556 ± 0.00287` | `112.9 ms` | `1757.0 MB` | **$0$** |
| **`fwd_streaming_wm_K1`** ($K=1, K_{\text{qk}}=1$) | `1.95285` | `1.95252` | `1.95268 ± 0.00023` | `125.3 ms` | `1765.1 MB` | **$0$** |
| **`fwd_streaming_wm_K1_qk2`** ($K=1, K_{\text{qk}}=2$) | `1.95103` | `1.95365` | `1.95234 ± 0.00185` | `132.4 ms` | `1757.0 MB` | **$0$** |
| **`fwd_streaming_wm_K4`** (Indep. Signs, $K=4$) | `1.91576` | `1.90640` | `1.91108 ± 0.00662` | `236.3 ms` | `1757.0 MB` | **$0$** |
| **`fwd_streaming_cowm_K4`** (Hadamard, $K=4$) | `1.89369` | `1.89344` | `1.89356 ± 0.00018` | `243.0 ms` | `1765.1 MB` | **$0$** |
| **`fwd_two_level_cowm_K4`** | `1.97564` | `1.97840` | `1.97702 ± 0.00196` | `334.7 ms` | `1791.4 MB` | **$0$** |
| **`fwd_streaming_wm_K8`** (Indep. Signs, $K=8$) | `1.88734` | `1.88884` | `1.88809 ± 0.00106` | `377.7 ms` | `1757.0 MB` | **$0$** |
| **`fwd_streaming_cowm_K8`** ($K=8, V_{\text{sub}}=256$, Uncoupled) | `1.86510` | `1.86525` | `1.86518 ± 0.00011` | `386.9 ms` | `1765.1 MB` | **$0$** |
| **`fwd_streaming_cowm_coupled_K8`** ($K=8, V_{\text{sub}}=256$, Coupled) | `1.78336` | `1.78756` | `1.78546 ± 0.00297` | `600.0 ms` | `1763.0 MB` | **$0$** |
| **`fwd_streaming_cowm_coupled_fullvocab_K8`** ($K=8, V=8192$, Coupled) | **`1.64671`** | **`1.64767`** | **`1.64719 ± 0.00067`** | `1391.3 ms` | `2031.0 MB` | **$0$** |
| **`fwd_streaming_wm_map_N4`**| `1.89510` | `1.89014` | `1.89262 ± 0.00350` | `265.1 ms` | `1765.1 MB` | **$0$** |
| `fwd_streaming_oracle` ($V_{\text{sub}}=256$ Uncoupled) | `1.86622` | `1.86309` | `1.86466 ± 0.00221` | `94.6 ms` | `1757.0 MB` | **$0$** |
| `fwd_streaming_oracle_coupled` ($V_{\text{sub}}=256$ Coupled) | `1.78362` | `1.78508` | `1.78435 ± 0.00103` | `150.2 ms` | `1763.0 MB` | **$0$** |
| `fwd_streaming_oracle_fullvocab` ($V=8192$ Uncoupled) | `1.74700` | `1.74464` | `1.74582 ± 0.00167` | `107.9 ms` | `2033.1 MB` | **$0$** |
| `fwd_streaming_oracle_mlp_pull_only_fullvocab` (+MLP-Pullback Only) | `1.73560` | `1.73659` | `1.73609 ± 0.00070` | `108.2 ms` | `2031.0 MB` | **$0$** |
| `fwd_streaming_oracle_mlp_whv_fullvocab` (+MLP-Pullback + `CM-Attn-WH`) | `1.71687` | `1.71824` | `1.71756 ± 0.00097` | `123.6 ms` | `2031.0 MB` | **$0$** |
| `fwd_streaming_oracle_coupled_fullvocab` ($V=8192$ Coupled Surrogate) | `1.64679` | `1.64922` | **`1.64800 ± 0.00172`** | `163.9 ms` | `2031.0 MB` | **$0$** |

*Table 5: 1000-step 2-seed pretraining benchmark on `d6` ($8.192\text{M}$ tokens). Key findings: (1) At matched $K=4$, parameter-space column contraction (`fwd_streaming_wm_K4`) beats token-scalar probing (`fwd_streaming_cut_K4`) by $-0.10448$ BPB. (2) Sylvester-Hadamard coordination improves $K=4$ by $-0.01752$ BPB and $K=8$ by $-0.02291$ BPB. (3) The component ablation at $V=8192$ isolates each mechanism in the intra-block coupled package: starting from uncoupled `fwd_streaming_oracle_fullvocab` ($1.74582 \pm 0.00167$), adding MLP-to-Attention residual pullback alone (`fwd_streaming_oracle_mlp_pull_only_fullvocab`) reduces holdout BPB by $-0.00973$ ($1.73609 \pm 0.00070$), adding Two-Level Window-Hadamard value transport `CM-Attn-WH` ($K_v=8$, `fwd_streaming_oracle_mlp_whv_fullvocab`) reduces BPB by an additional $-0.01853$ ($1.71756 \pm 0.00097$), and adding `Hadamard-QK` ($K_{\text{qk}}=8$, `fwd_streaming_oracle_coupled_fullvocab`) reduces BPB by a further $-0.06956$ ($1.64800 \pm 0.00172$), closing $56.4\%$ of the $+0.17346$ BPB surrogate gap to `block_local_greedy_bp` ($1.57236 \pm 0.00051$). (4) Full-vocabulary Parameter-Space `Co-WM-S` (`fwd_streaming_cowm_coupled_fullvocab_K8`) achieves **$1.64719 \pm 0.00067$ holdout BPB** ($56.9\%$ gap reduction), showing close descriptive agreement with its analytic-linear surrogate ($1.64800 \pm 0.00172$), just as sub-vocabulary `fwd_streaming_cowm_coupled_K8` ($1.78546 \pm 0.00297$) and `fwd_streaming_cowm_K8` ($1.86518 \pm 0.00011$) agree with `fwd_streaming_oracle_coupled` ($1.78435 \pm 0.00103$) and `fwd_streaming_oracle` ($1.86466 \pm 0.00221$).*

### 3.4 Historical Subblock-Detached Diagnostic Controls and Settling Provenance
To document the provenance of MAP activity settling, we report the historical cut-scope and subblock-detached benchmarks (`chung_scalable_llm.py`):
1. **Settling Step Ablation (`N=1` vs `N=2`)**: Re-evaluating genuine $N=1$ versus $N=2$ settling confirms that $N=1$ achieves exact local credit with near-zero relative bias ($\le 0.00022$), whereas $N=2$ introduces $12.9\%\text{--}16.3\%$ spectral damping bias:
   - In 1000-step cut-scope training on `d6` (`long_horizon_1000steps_d6.json`), `gs_map_N1_attnWMK1` ($N=1$, seed 42: $1.676094$, seed 137: $1.680382$, mean $1.67824 \pm 0.00303$) and `gs_map_N2_attnWMK1` ($N=2$, seed 42: $1.681450$, seed 137: $1.674709$, mean $1.67808 \pm 0.00477$) differ by only $-0.00016$ BPB.
   - Under native sequence shape ($B=64, T=2048$, $7.864\text{M}$ tokens, `native_shape_b64_t2048_d6.json`), `gs_map_N1_attnWMK1` achieves $1.863655$ holdout BPB, while `gs_map_N2_attnWMK1` achieves $1.863772$ holdout BPB.
   - **Conclusion**: Multi-step activity settling does not drive performance gains; the advantage over noisy single-probe WM ($1.68282$ BPB) stems from zero-variance credit reconstruction ($K=S$), not recurrent settling.
2. **Subblock-Detached Learned Checkpoint Diagnostic (Table 6)**: Evaluated on the real learned step-1000 Layer 3 state ($M=1536, \|V\|_2^2=42.53$, `learned_checkpoint_diagnostic.json`), `wm_S8_K1` achieves cosine similarity `0.37909` and relative variance `6.3832`, `co_wm_S8_K4` achieves `0.72457` and `0.9089`, and `co_wm_S8_K8` achieves `1.0000` and `0.0000`. `gs_map_N2` achieves `0.99883` cosine with `0.0000` sampling variance and $9.58\%$ systematic spectral damping bias.

| Algorithm | Mechanism Category | $D=128$ ($M=512$) Cos / RelVar | $D=256$ ($M=1024$) Cos / RelVar | $D=384$ ($M=1536$) Cos / RelVar | $D=512$ ($M=2048$) Cos / RelVar | Learned $D=384$ Cos / RelVar |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: |
| `global_scalar_K1` | Sublayer Scalar Control | `0.0018` / `152,740` | `0.0032` / `335,988` | `-0.0003` / `586,478` | `0.0006` / `626,964` | `-0.0012` / `7.12e6` |
| `cut_K4` | Token-Scalar Rademacher | `0.0891` / `125.2` | `0.0618` / `257.7` | `0.0497` / `390.0` | `0.0441` / `507.8` | `0.0674` / `206.8` |
| `cut_hadamard_K4` | Block-Hadamard Spatial | `0.0874` / `121.7` | `0.0611` / `253.8` | `0.0492` / `383.4` | `0.0445` / `510.1` | `0.0531` / `204.7` |
| **`wm_S8_K1`** | Micro-Horizon $S=8, K=1$ | `0.3535` / `6.80` | `0.3532` / `7.04` | `0.3497` / `7.15` | `0.3530` / `6.99` | **`0.3791`** / **`6.38`** |
| **`wm_S8_K2`** | Micro-Horizon $S=8, K=2$ | `0.5012` / `3.42` | `0.4995` / `3.51` | `0.4971` / `3.54` | `0.5002` / `3.48` | **`0.4874`** / **`3.18`** |
| **`path_uwm_S8_K1`**| ReLU$^2$ Chord Secant | `0.3597` / `6.73` | `0.3561` / `6.92` | `0.3535` / `6.99` | `0.3535` / `7.02` | **`0.3862`** / **`5.59`** |
| **`co_wm_S8_K4`** | Coordinated $K=4, S=8$ | `0.7116` / `1.01` | `0.7025` / `1.02` | `0.7041` / `1.03` | `0.7084` / `0.94` | **`0.7246`** / **`0.91`** |
| **`co_wm_S8_K8`** | Coordinated $K=S=8$ | **`1.0000`** / **`0.00`** | **`1.0000`** / **`0.00`** | **`1.0000`** / **`0.00`** | **`1.0000`** / **`0.00`** | **`1.0000`** / **`0.00`** |
| **`wm_map_S8_N4`** | Stochastic WM Settling | `0.7165` / `0.95` | `0.7058` / `1.03` | `0.6996` / `1.01` | `0.7059` / `1.04` | **`0.7196`** / **`0.82`** |
| **`gs_map_N2`** | Deterministic MAP ($N=2$) | **`0.9989`** / **`0.00`** | **`0.9991`** / **`0.00`** | **`0.9992`** / **`0.00`** | **`0.9993`** / **`0.00`** | **`0.9988`** / **`0.00`** |

*Table 6: Historical subblock-detached diagnostic across synthetic widths $D \in \{128 \dots 512\}$ and on a learned step-1000 Layer 3 checkpoint (`learned_checkpoint_diagnostic.json`).*

### 3.5 Two-Stage Faithful DUST Validation and `Delayed-Trace-Co-WM-S` Scaling & Pretraining Benchmarks
To evaluate strict no-backward credit assignment (zero reverse-depth sweeps and zero local reverse/pullback computations), we execute our two-stage faithful DUST protocol (`dust_faithful.py`, `verify_dust_reference.py`) alongside `Delayed-Trace-Co-WM-S` (`delayed_trace_cowm.py`, `run_dust_and_delayed_trace_suite.py`):

1. **Stage 1A — Bit-for-Bit Equivalence & Prefix-Cached Suffix Execution (`dust_stage1_reference_validation.json`)**:
   Comparing unmodified `literature/dust/dust.py` (hook-based full-network reruns) against `FaithfulDUST(use_prefix_cache=False)` and `FaithfulDUST(use_prefix_cache=True)` (prefix-cached suffix execution `blocks[l:]`) on the reference 8-layer architecture ($L=8, D=512, B=4, T=256, V=4096, P=256$) yields **`max_grad_diff = 0.0` and `max_param_diff = 0.0` across all 68 parameter tensors**, while suffix caching accelerates step time by **$1.75\times$** (`838.0 ms` vs `1469.7 ms`, saving $19.14\%$ of block evaluations: $414$ vs $512$).
2. **Stage 1B — Reference 8-Layer `1M`-Token Validation ($L=8, D=512, B=8, T=2048, V=4096$, 61 steps)**:
   Under published DUST's SGD/Adam optimizer and `c_proj` initialization ($\text{std} = 1.5/\sqrt{d_{\text{in}}}$), `backprop_sgd_ref` improves from initial val `8.3186` to best val `5.7469` / test `5.7486` nats (`117.2 ms/step`, `4993.9 MB`); `dust_ref_P64` improves to best val `6.1763` / test `6.1442` nats (`2067.1 ms/step`, `7102.5 MB`); and `dust_ref_P256` improves to best val `5.8681` / test `5.8565` nats (`6893.6 ms/step`, `7183.0 MB`), within **`0.1078` test nats** of `backprop_sgd_ref` (matching DUST Appendix Table 1: `6.067` vs `5.959` = `0.108` nats).
3. **Stage 2/3 — Width and Pure Hidden-Neuron Scaling Diagnostic (`Table 7`, `dust_and_delayed_trace_width_diagnostic.json`)**:
   - *Variance Metric Definitions*: For $R=16$ Monte Carlo trials $\hat{G}^{(r)}$ with sample mean $\bar{G}_R$ and reference $G_{\text{ref}}$, `_cosine_and_relvar` records both (i) **mean-normalized relative sample variance** $\widehat{\text{RelVar}}_{\text{mean}} = \frac{\frac{1}{R-1}\sum_{r=1}^R \|\hat{G}^{(r)} - \bar{G}_R\|_F^2}{\|\bar{G}_R\|_F^2}$ (`rel_var_to_mean`, the inverse SNR of the estimator around its mean direction) and (ii) **projection-rescaled relative variance** $\widehat{\text{RelVar}}_{\text{proj}} = \frac{\frac{1}{R}\sum_{r=1}^R \|\alpha^*\hat{G}^{(r)} - \alpha^*\bar{G}_R\|_F^2}{\|G_{\text{ref}}\|_F^2}$ with $\alpha^* = \frac{\langle \bar{G}_R, G_{\text{ref}}\rangle}{\|\bar{G}_R\|_F^2}$ (`rel_var`, which scales down by $\cos^2(\bar{G}_R, G_{\text{ref}})$ as cosine collapses). Table 7 reports `mean_cossim` / `rel_var_to_mean` (`rel_var`) directly from `dust_and_delayed_trace_width_diagnostic.json`.
   - *Conditional Recovery Given Outgoing Writer Reward ($D \in \{128, 256, 384, 512\}, M=4D$)*: `Delayed-Trace-UWM` achieves conditional cosine `0.1239, 0.0992, 0.1027, 0.1111`; `Delayed-Trace-WM-S` ($K=1$) achieves `0.4575, 0.4008, 0.3971, 0.4168`; `Delayed-Trace-Co-WM-S` ($K=4$) achieves `0.7932, 0.7557, 0.7538, 0.7755`; and **`Delayed-Trace-Co-WM-S` ($K=8$) achieves `1.0000, 1.0000, 1.0000, 1.0000` with `RelVar = 0.0000`**.
   - *Whole-Network Layer 3 `mlp.c_fc` Cosine Across $D \in \{128 \dots 512\}$ ($M=4D$)*: Faithful DUST direct `hidden` probing (34 draws) drops `0.2395 -> 0.1421 -> 0.1128 -> 0.1026`, whereas `Delayed-Trace-Co-WM-S` ($K=8$) from 34 writer draws (**0 `hidden` draws**, $-50\%$ block draws) achieves **`0.3289 -> 0.1998 -> 0.1602 -> 0.1454`** ($+37\%\text{--}42\%$ higher), and from 68 reallocated writer draws (equal block draws) achieves **`0.4405 -> 0.2765 -> 0.2236 -> 0.1963`** ($+84\%\text{--}98\%$ higher).
   - *Bounded Pure Hidden-Neuron Scaling Diagnostic at Fixed Residual Width $D=256$ ($M \in \{256, 512, 1024, 2048, 4096\}$, Table 7)*: Evaluated on both 15-step synthetic states (`fixed_D256_neuron_scaling`, Panel A) and 50-step ClimbMix language-model states evaluated on held-out ClimbMix tokens (`fixed_D256_climbmix_50steps`, Panel B), DUST direct `hidden` (34 draws) cosine drops $4.43\times$ (`0.3060 -> 0.0691`, Panel A) and $3.82\times$ (`0.2578 -> 0.0675`, Panel B) while $\widehat{\text{RelVar}}_{\text{mean}}$ rises (`5.98 -> 14.81` and `7.20 -> 14.91`), whereas `Delayed-Trace-Co-WM-S` ($K=8$) exhibits comparatively stable cosines across tested hidden widths (`0.3031 -> 0.2582` in Panel A and `0.2078 -> 0.1884` in Panel B for 68w; $3.74\times$ and $2.79\times$ higher than DUST at $M=4096$). Because each width evaluates one frozen model state after 15 or 50 steps, Table 7 establishes a bounded empirical diagnostic observation over the tested hidden widths rather than an unconditional $O(1)$ statistical-scaling law.

| Panel / Method (`mlp.c_fc` Layer 3, Fixed $D=256$) | Draws (`w`/`h`) | $M=256$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=512$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=1024$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=2048$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) | $M=4096$ Cos / $\text{RV}_{\text{mean}}$ ($\text{RV}_{\text{proj}}$) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **Panel A (Synthetic 15-Step):** `dust_direct_hidden_fc_16draws` | $0\text{w}+16\text{h}$ | `0.2180` / `8.85` (`3.67`) | `0.1411` / `12.07` (`2.78`) | `0.0985` / `13.88` (`1.77`) | `0.0688` / `14.79` (`0.98`) | `0.0460` / `15.48` (`0.48`) |
| **Panel A:** `dust_direct_hidden_fc_34draws` [5] | $0\text{w}+34\text{h}$ | `0.3060` / `5.98` (`3.47`) | `0.2028` / `9.49` (`3.62`) | `0.1431` / `12.05` (`2.84`) | `0.1021` / `13.58` (`1.82`) | `0.0691` / `14.81` (`0.99`) |
| **Panel A:** **`delayed_trace_cowm_s_K8_from_34writer_draws`** | **$34\text{w}+0\text{h}$** | `0.2230` / `8.71` (`3.73`) | `0.1910` / `9.97` (`3.53`) | `0.1928` / `9.80` (`3.48`) | `0.1962` / `9.67` (`3.51`) | **`0.1814` / `10.19` (`3.32`)** |
| **Panel A:** **`delayed_trace_cowm_s_K8_from_68writer_draws`** | **$68\text{w}+0\text{h}$** | `0.3031` / `6.06` (`3.49`) | **`0.2753` / `6.90` (`3.66`)** | **`0.2650` / `7.25` (`3.72`)** | **`0.2727` / `6.92` (`3.61`)** | **`0.2582` / `7.44` (`3.71`)** |
| **Panel B (ClimbMix 50-Step):** `dust_direct_hidden_fc_16draws` | $0\text{w}+16\text{h}$ | `0.1774` / `10.47` (`3.34`) | `0.1355` / `12.30` (`2.65`) | `0.1035` / `13.91` (`1.96`) | `0.0654` / `14.91` (`0.90`) | `0.0442` / `15.57` (`0.44`) |
| **Panel B:** `dust_direct_hidden_fc_34draws` [5] | $0\text{w}+34\text{h}$ | `0.2578` / `7.20` (`3.48`) | `0.2038` / `9.67` (`3.80`) | `0.1551` / `11.69` (`3.15`) | `0.0999` / `13.55` (`1.74`) | `0.0675` / `14.91` (`0.95`) |
| **Panel B:** **`delayed_trace_cowm_s_K8_from_34writer_draws`** | **$34\text{w}+0\text{h}$** | `0.1649` / `11.25` (`3.32`) | `0.1615` / `11.65` (`3.42`) | `0.1593` / `10.69` (`2.82`) | `0.1461` / `11.68` (`2.79`) | **`0.1384` / `12.11` (`2.69`)** |
| **Panel B:** **`delayed_trace_cowm_s_K8_from_68writer_draws`** | **$68\text{w}+0\text{h}$** | `0.2078` / `8.82` (`3.31`) | `0.1772` / `10.33` (`3.24`) | **`0.2191` / `8.20` (`3.21`)** | **`0.1997` / `9.19` (`3.31`)** | **`0.1884` / `9.76` (`3.30`)** |

*Table 7: Pure hidden-neuron scaling diagnostic at fixed residual width $D=256$ across $M \in \{256, 512, 1024, 2048, 4096\}$ ($R=16$ trials, `dust_and_delayed_trace_width_diagnostic.json`), reporting `mean_cossim` / $\text{RV}_{\text{mean}} = \text{rel\_var\_to\_mean}$ ($\text{RV}_{\text{proj}} = \text{rel\_var}$) on both 15-step synthetic states (`fixed_D256_neuron_scaling`, Panel A) and 50-step ClimbMix language-model states evaluated on held-out ClimbMix tokens (`fixed_D256_climbmix_50steps`, Panel B).*

4. **Explicit 5-Component Latency Breakdown Across Regime I and Regime II (`Table 7B`, `contraction_and_trace_width_profile.json` & Sweep `timing_profile_ms`)**:
   To separate forward writer suffix cost from SRAM-fused outgoing-column contraction (`_outgoing_col_dot_all8_kernel`) and trace recording, `profile_contraction_and_trace_width.py` (and `step_accounting["timing_profile_ms"]` in the Regime I/II sweep JSONs) records 5 disjoint CUDA-synchronized sub-components (`clean_fwd_ms`, `trace_record_ms`, `writer_suffix_fwd_ms`, `wm_col_contraction_ms`, `local_attn_and_head_ms`, plus `mean_opt_ms`):
   - *Regime I (Fixed $D=256$, Widening $M \in \{256, 512, 1024, 2048, 4096\}$)*: Across all 6 blocks at $M=4096$, strict no-backward (`rev=0`) SRAM-fused outgoing-column contraction (`wm_col_contraction_ms`) takes only **`18.7 ms`** (`1.9 ms` at $M=256$, `5.3 ms` at $M=1024$) and trace recording (`trace_record_ms`) takes **`17.8 ms`** (`1.1 ms` at $M=256$, `2.6 ms` at $M=1024`), whereas eliminating `hidden` suffix draws cuts `writer_suffix_fwd_ms` from `660.7–660.8 ms` (`dust_faithful_muon_P64_sc_sh`) to **`358.5 ms`** in strict no-backward `delayed_trace_cowm_P64_fast_sc_sh_d1` (`rev=0`, total `679.2 ms` vs `950.1 ms`, saving `270.9 ms/step`) and **`538.6 ms`** in strict no-backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`rev=0`, total `859.5 ms` vs `950.1 ms`, saving `90.6 ms/step`).
   - *Regime II (Full-Model Widening $D \in \{128, 256, 384, 512\}, M=4D$)*: At $D=512$ ($M=2048$), strict no-backward (`rev=0`) `wm_col_contraction_ms` is **`18.5–18.6 ms`** and `trace_record_ms` is **`9.0 ms`**, while `writer_suffix_fwd_ms` drops from `551.8–552.2 ms` (`dust_faithful_muon_P64_sc_sh`) to **`320.7 ms`** (`delayed_trace_cowm_P64_fast_sc_sh_d1`, `rev=0`, total `621.6 ms` vs `826.6 ms`) and **`482.9 ms`** (`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`, `rev=0`, total `784.2 ms` vs `826.6 ms`).

| Regime / Width | Method (`rev` = Local Pullbacks) | Total Step ms (Opt ms) | `clean_fwd` ms | `trace_rec` ms | `writer_sfx` ms | `wm_col` ms | `local+head` ms | Peak VRAM |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **Regime I** ($D=256, M=256$) | `dust_faithful_muon_P64_sc_sh` (`rev=0`) | `473.3` (`2.7`) | `7.8` | `0.0` | `277.8` | `0.0` | `181.9` | `3085.9 MB` |
| **Regime I** ($D=256, M=256$) | **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (`rev=0`) | **`366.0`** (`2.4`) | `7.5` | `1.1` | **`175.0`** | **`1.9`** | `175.1` | `3222.2 MB` |
| **Regime I** ($D=256, M=256$) | **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (`rev=0`) | **`457.4`** (`2.3`) | `7.5` | `1.1` | **`266.3`** | **`1.9`** | `175.1` | `3222.2 MB` |
| **Regime I** ($D=256, M=256$) | `delayed_trace_unified_P64_fast_sc_sh_d1` (`rev=6`) | `319.8` (`3.0`) | `7.9` | `4.6` | `129.5` | `4.4` | `167.4` | `3453.7 MB` |
| **Regime I** ($D=256, M=1024$) | `dust_faithful_muon_P64_sc_sh` (`rev=0`) | `528.6` (`3.7`) | `7.9` | `0.0` | `326.0` | `0.1` | `188.0` | `3361.9 MB` |
| **Regime I** ($D=256, M=1024$) | **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (`rev=0`) | **`402.3`** (`3.5`) | `7.7` | `2.6` | **`197.2`** | **`5.3`** | `183.1` | `3561.6 MB` |
| **Regime I** ($D=256, M=1024$) | **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (`rev=0`) | **`503.8`** (`3.4`) | `7.7` | `2.6` | **`299.5`** | **`5.3`** | `182.5` | `3561.6 MB` |
| **Regime I** ($D=256, M=1024$) | `delayed_trace_unified_P64_fast_sc_sh_d1` (`rev=6`) | `351.3` (`4.2`) | `9.3` | `6.0` | `147.2` | `8.3` | `173.4` | `3793.1 MB` |
| **Regime I** ($D=256, M=4096$) | `dust_faithful_muon_P64_sc_sh` (`rev=0`) | `949.2` (`3.8`) | `12.1` | `0.0` | `660.7` | `0.1` | `269.6` | `4466.1 MB` |
| **Regime I** ($D=256, M=4096$) | **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (`rev=0`) | **`679.2`** (`3.8`) | `12.2` | **`17.8`** | **`358.5`** | **`18.7`** | `265.3` | `4919.3 MB` |
| **Regime I** ($D=256, M=4096$) | **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (`rev=0`) | **`859.5`** (`3.8`) | `12.2` | **`17.8`** | **`538.6`** | **`18.7`** | `264.9` | `4919.3 MB` |
| **Regime I** ($D=256, M=4096$) | `delayed_trace_unified_P64_fast_sc_sh_d1` (`rev=6`) | `599.8` (`4.0`) | `12.3` | `21.2` | `280.2` | `21.1` | `257.4` | `5150.7 MB` |
| **Regime II** ($D=384, M=1536$) | `dust_faithful_muon_P64_sc_sh` (`rev=0`) | `650.3` (`4.1`) | `9.3` | `0.0` | `423.4` | `0.1` | `209.8` | `3968.9 MB` |
| **Regime II** ($D=384, M=1536$) | **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (`rev=0`) | **`485.2`** (`3.4`) | `8.4` | **`5.4`** | **`249.6`** | **`10.8`** | `204.9` | `4251.8 MB` |
| **Regime II** ($D=384, M=1536$) | **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (`rev=0`) | **`613.4`** (`3.4`) | `8.4` | **`5.5`** | **`378.5`** | **`10.8`** | `204.0` | `4251.8 MB` |
| **Regime II** ($D=512, M=2048$) | `dust_faithful_muon_P64_sc_sh` (`rev=0`) | `849.7` (`5.1`) | `10.7` | `0.0` | `552.2` | `0.1` | `278.1` | `4600.7 MB` |
| **Regime II** ($D=512, M=2048$) | **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (`rev=0`) | **`621.6`** (`5.0`) | `10.8` | **`9.0`** | **`320.7`** | **`18.6`** | `254.4` | `4977.9 MB` |
| **Regime II** ($D=512, M=2048$) | **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (`rev=0`) | **`784.2`** (`5.0`) | `10.8` | **`9.0`** | **`482.9`** | **`18.6`** | `254.6` | `4977.9 MB` |

*Table 7B: Explicit 5-component step latency breakdown (`contraction_and_trace_width_profile.json` and `step_accounting["timing_profile_ms"]` in `regime1_fixed_D256_mlp_width_pretrain_200.json` / `dust_and_delayed_trace_width_sweep_200.json`). In `dust_faithful_*` and `delayed_trace_cowm_*` (`rev=0`), `writer_sfx` includes all `writers + hidden + hubs` suffix forward passes.*

5. **Regime I (Pure MLP-Width Pretraining Scaling at Fixed $D=256$, 2 Seeds `s42, s137`, `Table 8A`) vs. Regime II (Full-Model Width Sweep $D \in \{128, 256, 384, 512\}$, `Table 8B`)**:
   - *Regime I (`regime1_fixed_D256_mlp_width_pretrain_200.json`, Table 8A)*: Holding residual width fixed at $D=256$ and widening only the MLP hidden dimension $16\times$ across $M \in \{256, 1024, 4096\}$ over 200 ClimbMix pretraining steps on 2 seeds (`s42, s137`), our **strict no-backward (`local_reverse_derivative_calls = 0`)** methods outperform both unmodified `dust_faithful_muon_P64` (`2.36003 -> 2.35339 -> 2.34864`) and Fairly Optimized `dust_faithful_muon_P64_sc_sh` (`2.15683 -> 2.15271 -> 2.14879`) across all three MLP widths:
     - **Strict No-Backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`64` direct draws, `rev=0`)** beats `dust_faithful_muon_P64_sc_sh` at every MLP width while running faster at every width: **`-0.00769 BPB`** at $M=256$ (`2.14914 ± 0.00265` at `457.7 ms` vs `2.15683 ± 0.00623` at `475.0 ms`), **`-0.01116 BPB`** at $M=1024$ (`2.14155 ± 0.00411` at `503.6 ms` vs `2.15271 ± 0.01103` at `529.7 ms`), and **`-0.01107 BPB`** at $M=4096$ (`2.13772 ± 0.00264` at `859.7 ms` vs `2.14879 ± 0.00153` at `952.4 ms`, non-overlapping across seeds `2.13958/2.13585` vs `2.14771/2.14988` and **`92.7 ms/step` faster**).
     - Even **Strict No-Backward `delayed_trace_cowm_P64_fast_sc_sh_d1` (`41` direct draws, `rev=0`)** outperforms 64-draw `dust_faithful_muon_P64_sc_sh` at **all three MLP widths** (`2.15599 ± 0.00394` at $M=256$, `2.15079 ± 0.00351` at $M=1024$, and `2.14693 ± 0.00103` at $M=4096$, where both seeds `2.14766/2.14621` beat `2.14771/2.14988`) while running **$1.30\times\text{--}1.40\times$ faster** (`679.1 ms` vs `952.4 ms` at $M=4096$, saving `273.3 ms/step`).
   - *Regime II (`dust_and_delayed_trace_width_sweep_200.json`, Table 8B)*: Across full-model widths $D \in \{128, 256, 384, 512\}$ ($M=4D$), **Strict No-Backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`rev=0`) outperforms Fairly Optimized `dust_faithful_muon_P64_sc_sh` across all four model widths $D \in \{128, 256, 384, 512\}$ (`2.1606` vs `2.1839`, `2.1453` vs `2.1625`, `2.1297` vs `2.1351`, `2.1302` vs `2.1342`) while running `18.1–42.4 ms/step` faster at every width**.

| Algorithm (Regime I: Fixed $D=256$, 200 Steps, 2 Seeds `s42, s137`) | Draws (`rev`) | $M=256$ ($1D$) Holdout BPB (Step ms) | $M=1024$ ($4D$) Holdout BPB (Step ms) | $M=4096$ ($16D$) Holdout BPB (Step ms) | Peak VRAM ($M=4096$) |
| :--- | :---: | :---: | :---: | :---: | :---: |
| `dust_faithful_muon_P64` [5] | $64$ (`0`) | `2.36003 ± 0.00667` (`525.9 ms`) | `2.35339 ± 0.00973` (`578.3 ms`) | `2.34864 ± 0.00423` (`980.2 ms`) | `3351.5 MB` |
| `dust_faithful_muon_P64_sc_sh` (Fairly Opt. DUST, `sc+sh`) | $64$ (`0`) | `2.15683 ± 0.00623` (`475.0 ms`) | `2.15271 ± 0.01103` (`529.7 ms`) | `2.14879 ± 0.00153` (`952.4 ms`) | `4497.3 MB` |
| **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (**Strict No-Bwd Fast**) | **$41$ (`0`)** | **`2.15599 ± 0.00394`** (**`366.1 ms`**) | **`2.15079 ± 0.00351`** (**`402.0 ms`**) | **`2.14693 ± 0.00103`** (**`679.1 ms`**) | `4919.3 MB` |
| **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (**Strict No-Bwd**) | **$64$ (`0`)** | **`2.14914 ± 0.00265`** (**`457.7 ms`**) | **`2.14155 ± 0.00411`** (**`503.6 ms`**) | **`2.13772 ± 0.00264`** (**`859.7 ms`**) | `4919.3 MB` |
| `delayed_trace_full_P64_realloc_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.35835 ± 0.00820` (`597.9 ms`) | `2.34369 ± 0.00722` (`548.5 ms`) | `2.35469 ± 0.00082` (`901.8 ms`) | `3878.9 MB` |
| `delayed_trace_unified_P64_fast_sc_sh_d1` (Hybrid Hub-Pullback) | $29$ (`6`) | `2.15962 ± 0.00831` (`373.7 ms`) | `2.15397 ± 0.00781` (`350.1 ms`) | `2.16840 ± 0.03447` (`599.9 ms`) | `5181.9 MB` |
| `delayed_trace_unified_P64_sc_sh_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.16398 ± 0.01953` (`487.8 ms`) | `2.14284 ± 0.00047` (`500.1 ms`) | `2.13913 ± 0.00216` (`857.8 ms`) | `5181.9 MB` |
| `delayed_trace_unified_P64_bd025_sc_sh_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.15549 ± 0.00797` (`455.1 ms`) | `2.14004 ± 0.00359` (`501.3 ms`) | `2.13429 ± 0.00197` (`856.3 ms`) | `5181.9 MB` |

*Table 8A: Regime I — 2-seed (`s42, s137`) pure MLP-width pretraining scaling at fixed residual width $D=256$ across $M \in \{256, 1024, 4096\}$ (`regime1_fixed_D256_mlp_width_pretrain_200.json`), separating Strict No-Backward (`rev=0`, `hub_mode="dust_hub"`) from Hybrid Local-Hub-Pullback (`rev=6`, `hub_mode="co_wm_s"`).*

| Algorithm (Regime II: Full-Model Width Sweep, 200 Steps, Seed `s42`) | Draws (`rev`) | $D=128$ Dev / Holdout (ms) | $D=256$ Dev / Holdout (ms) | $D=384$ Dev / Holdout (ms) | $D=512$ Dev / Holdout (ms) | VRAM ($D=512$) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| `dust_faithful_sgd_P64` [5] | $64$ (`0`) | `2.3746` / `2.3731` (`482.6`) | `2.4127` / `2.4141` (`561.4`) | `2.5149` / `2.5156` (`678.8`) | `2.4845` / `2.4841` (`842.9`) | `3035.5 MB` |
| `dust_faithful_muon_P64` [5] | $64$ (`0`) | `2.3819` / `2.3820` (`492.6`) | `2.3473` / `2.3479` (`565.6`) | `2.3528` / `2.3510` (`682.9`) | `2.3310` / `2.3321` (`847.5`) | `3083.7 MB` |
| `delayed_trace_cowm_P64_fast_d0` | $37$ (`0`) | `2.3834` / `2.3790` (`400.9`) | `2.3793` / `2.3787` (`450.1`) | `2.3816` / `2.3816` (`529.7`) | `2.3430` / `2.3425` (`651.4`) | `3623.9 MB` |
| `delayed_trace_cowm_P64_fast_d1` | $37$ (`0`) | `2.3765` / `2.3760` (`399.9`) | `2.3686` / `2.3704` (`449.0`) | `2.3931` / `2.3900` (`530.5`) | `2.3478` / `2.3462` (`650.7`) | `3733.9 MB` |
| `delayed_trace_cowm_P64_realloc_d0` | $64$ (`0`) | `2.3719` / `2.3705` (`506.4`) | `2.3459` / `2.3453` (`579.3`) | `2.3462` / `2.3441` (`700.0`) | `2.3332` / `2.3331` (`870.9`) | `3623.9 MB` |
| `delayed_trace_cowm_P64_realloc_d1` | $64$ (`0`) | `2.3723` / `2.3709` (`505.2`) | `2.3453` / `2.3460` (`580.4`) | `2.3492` / `2.3461` (`700.0`) | `2.3323` / `2.3327` (`872.8`) | `3733.9 MB` |
| `dust_faithful_muon_P64_sc_sh` (Fairly Opt. DUST) | $64$ (`0`) | `2.1841` / `2.1839` (`453.0`) | `2.1624` / `2.1625` (`529.6`) | `2.1352` / `2.1351` (`650.2`) | `2.1361` / `2.1342` (`826.6`) | `4631.9 MB` |
| **`delayed_trace_cowm_P64_fast_sc_sh_d1`** (**Strict No-Bwd Fast**) | **$41$ (`0`)** | `2.1741` / **`2.1733`** (**`350.2`**) | `2.1543` / **`2.1535`** (**`401.1`**) | `2.1524` / `2.1509` (**`485.2`**) | `2.1369` / `2.1373` (**`621.6`**) | `4977.9 MB` |
| **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (**Strict No-Bwd**) | **$64$ (`0`)** | **`2.1612`** / **`2.1606`** (**`434.9`**) | **`2.1470`** / **`2.1453`** (**`502.5`**) | **`2.1314`** / **`2.1297`** (**`613.4`**) | **`2.1312`** / **`2.1302`** (**`784.2`**) | `4977.9 MB` |
| `delayed_trace_full_P64_fast_d1` (Hybrid Hub-Pullback) | $29$ (`6`) | `2.3752` / `2.3751` (`355.7`) | `2.3727` / `2.3718` (`399.9`) | `2.3497` / `2.3508` (`473.6`) | `2.3422` / `2.3446` (`587.7`) | `4706.2 MB` |
| `delayed_trace_full_P64_realloc_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.3646` / `2.3626` (`481.7`) | `2.3386` / `2.3386` (`549.2`) | `2.3387` / `2.3399` (`660.8`) | `2.3410` / `2.3399` (`820.2`) | `4706.2 MB` |
| `delayed_trace_unified_P64_fast_sc_sh_d1` (Hybrid Hub-Pullback) | $29$ (`6`) | `2.1759` / `2.1733` (`305.6`) | `2.1599` / `2.1595` (`350.2`) | `2.1510` / `2.1485` (`431.4`) | `2.1380` / `2.1366` (`552.8`) | `5956.4 MB` |
| `delayed_trace_unified_P64_sc_sh_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.1784` / `2.1763` (`433.8`) | `2.1426` / `2.1425` (`500.0`) | `2.1392` / `2.1382` (`617.9`) | `2.1276` / `2.1253` (`787.0`) | `5956.4 MB` |
| `delayed_trace_unified_P64_bd025_sc_sh_d1` (Hybrid Hub-Pullback) | $64$ (`6`) | `2.1740` / `2.1720` (`432.9`) | `2.1396` / `2.1375` (`500.8`) | `2.1308` / `2.1305` (`617.2`) | `2.1247` / `2.1241` (`786.1`) | `5956.4 MB` |

*Table 8B: Regime II — 200-step full-model width sweep across $D \in \{128, 256, 384, 512\}$ ($M=4D$, `dust_and_delayed_trace_width_sweep_200.json`), executed with the SRAM-fused Triton contraction kernel (`_outgoing_col_dot_all8_kernel`).*

| Algorithm (`d6`, $D=384, M=1536$, 2 Seeds `s42, s137`) | Steps / Draws (`rev`) | Seed `s42` Holdout | Seed `s137` Holdout | Mean Holdout BPB $\pm$ SD | Mean Step ms (Timed s / Complete s) | Peak VRAM |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| `dust_faithful_sgd_P64` [5] (Published SGD/Adam) | $1000$ / $64$ (`0`) | `2.42629` | `2.32764` | `2.37697 ± 0.06975` | `692.5 ms` (`691.1 s` / `692.5 s`) | `2665.1 MB` |
| `dust_faithful_muon_P64` [5] (Adapted Muon/AdamW) | $1000$ / $64$ (`0`) | `2.24890` | `2.24953` | `2.24921 ± 0.00044` | `710.0 ms` (`708.5 s` / `710.0 s`) | `2701.2 MB` |
| `delayed_trace_cowm_P64_fast_d0` (`0` hidden, same-step) | $1000$ / $37$ (`0`) | `2.30369` | `2.29773` | `2.30071 ± 0.00422` | `691.7 ms` (`690.3 s` / `691.7 s`) | `2900.3 MB` |
| `delayed_trace_cowm_P64_fast_d1` (`0` hidden, 1-step delay) | $1000$ / $37$ (`0`) | `2.30943` | `2.29982` | `2.30462 ± 0.00679` | `656.8 ms` (`655.5 s` / `656.8 s`) | `2975.7 MB` |
| `delayed_trace_cowm_P64_realloc_d0` (Realloc, same-step) | $1000$ / $64$ (`0`) | `2.28822` | `2.28888` | `2.28855 ± 0.00047` | `823.9 ms` (`822.2 s` / `823.9 s`) | `2900.3 MB` |
| `delayed_trace_cowm_P64_realloc_d1` (Realloc, 1-step delay) | $1000$ / $64$ (`0`) | `2.29316` | `2.28479` | `2.28897 ± 0.00591` | `818.7 ms` (`817.1 s` / `818.7 s`) | `2975.7 MB` |
| `delayed_trace_cowm_P64_indep_d0` (Indep. `mlp.c_proj`, `d0`) | $1000$ / $64$ (`0`) | `2.32639` | `2.29345` | `2.30992 ± 0.02329` | `808.7 ms` (`807.1 s` / `808.7 s`) | `2912.3 MB` |
| `delayed_trace_cowm_P64_indep_d1` (Indep. `mlp.c_proj`, `d1`) | $1000$ / $64$ (`0`) | `2.29776` | `2.29271` | `2.29523 ± 0.00357` | `804.2 ms` (`802.5 s` / `804.2 s`) | `2987.7 MB` |
| `delayed_trace_cowm_fixed16_d0` (All-Block 16w, same-step) | $1000$ / $27$ (`0`) | `2.28852` | `2.28307` | `2.28580 ± 0.00385` | `620.5 ms` (`619.3 s` / `620.5 s`) | `2960.3 MB` |
| **`delayed_trace_cowm_fixed16_d1`** (All-Block 16w, 1-step delay) | **$1000$ / $27$ (`0`)** | `2.28584` | `2.28104` | **`2.28344 ± 0.00339`** | **`618.9 ms` (`617.7 s` / `618.9 s`)** | `3035.7 MB` |
| `delayed_trace_cowm_fixed32_d0` (All-Block 32w, same-step) | $1000$ / $43$ (`0`) | `2.27861` | `2.28357` | `2.28109 ± 0.00351` | `805.5 ms` (`803.9 s` / `805.5 s`) | `2960.3 MB` |
| `delayed_trace_cowm_fixed32_d1` (All-Block 32w, 1-step delay) | $1000$ / $43$ (`0`) | `2.28047` | `2.28793` | `2.28420 ± 0.00527` | `857.2 ms` (`855.5 s` / `857.2 s`) | `3035.7 MB` |
| **`delayed_trace_cowm_fixed64_d1`** (All-Block 64w, 1-step delay) | $1000$ / $75$ (`0`) | `2.27531` | `2.26871` | **`2.27201 ± 0.00467`** | `1200.3 ms` (`1197.9 s` / `1200.3 s`) | `3035.7 MB` |
| `dust_faithful_muon_P64_sc_sh` (Fairly Opt. DUST, `sc+sh`) | $1000$ / $64$ (`0`) | `2.04581` | `2.04598` | `2.04590 ± 0.00012` | `649.3 ms` (`648.0 s` / `649.3 s`) | `4095.6 MB` |
| **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (**Strict No-Bwd**) | $1000$ / $64$ (`0`) | `2.06175` | `2.06204` | `2.06189 ± 0.00021` | `612.2 ms` (`610.9 s` / `613.0 s`) | `4394.9 MB` |
| **`delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_wc648s`** (**Strict No-Bwd WC $\le 648\text{s}$**) | **$1680$ / $43$ (`0`)** | **`2.04422`** | **`2.04402`** | **`2.04412 ± 0.00014`** | **`377.6 ms` (`633.6 s` / `635.2 s`)** | `4519.9 MB` |
| **`delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_1750steps`** (**Strict No-Bwd 1750 Steps**) | **$1750$ / $43$ (`0`)** | **`2.04166`** | **`2.04426`** | **`2.04296 ± 0.00184`** | **`378.7 ms` (`662.0 s` / `663.6 s`)** | `4519.9 MB` |
| `delayed_trace_full_P64_fast_d1` (Hybrid Hub-Pullback, $29$ draws) | $1000$ / $29$ (`6`) | `2.29780` | `2.28339` | `2.29060 ± 0.01019` | `473.2 ms` (`472.2 s` / `474.1 s`) | `3716.5 MB` |
| `delayed_trace_full_P64_realloc_d1` (Hybrid Hub-Pullback, $64$ draws) | $1000$ / $64$ (`6`) | `2.29604` | `2.27999` | `2.28801 ± 0.01135` | `659.1 ms` (`657.8 s` / `660.0 s`) | `3716.5 MB` |
| `delayed_trace_unified_P64_fast_sc_sh_d1` (Hybrid Hub-Pullback) | $1000$ / $29$ (`6`) | `2.06212` | `2.06840` | `2.06526 ± 0.00444` | `430.9 ms` (`430.0 s` / `430.9 s`) | `4993.9 MB` |
| `delayed_trace_unified_P64_sc_sh_d1` (Hybrid Hub-Pullback) | $1000$ / $64$ (`6`) | `2.06525` | `2.06182` | `2.06353 ± 0.00243` | `618.0 ms` (`616.8 s` / `618.1 s`) | `5066.2 MB` |
| `delayed_trace_unified_P64_bd025_sc_sh_d1` (Hybrid Hub-Pullback) | $1000$ / $64$ (`6`) | `2.06026` | `2.06106` | `2.06066 ± 0.00056` | `617.3 ms` (`616.0 s` / `617.3 s`) | `4993.9 MB` |
| `delayed_trace_unified_P64_fast_w22_sc_sh_d1_wc648s` (Hybrid WC) | $1480$ / $31$ (`6`) | `2.05031` | `2.05396` | `2.05213 ± 0.00258` | `439.0 ms` (`648.8 s` / `650.6 s`) | `5056.4 MB` |
| `delayed_trace_unified_P64_bd025_sc_sh_st_d1_wc648s` (Hybrid WC) | $1300$ / $64$ (`6`) | `2.04836` | `2.04682` | `2.04759 ± 0.00109` | `506.1 ms` (`656.9 s` / `658.0 s`) | `5128.7 MB` |
| `delayed_trace_unified_P64_fast_w22_sc_sh_st_d1_wc648s` (Hybrid WC $\le 648\text{s}$) | $2000$ / $31$ (`6`) | `2.03765` | `2.03774` | `2.03770 ± 0.00006` | `314.2 ms` (`627.8 s` / `629.4 s`) | `5118.9 MB` |
| **`dust_faithful_muon_P256`** [5] (Matched $1000$ Steps) | $1000$ / $256$ (`0`) | `2.10785` | `2.10981` | `2.10883 ± 0.00138` | `2271.0 ms` (`2266.5 s` / `2271.0 s`) | `3369.7 MB` |
| **`delayed_trace_cowm_P256_fast_d1`** (Matched $1000$ Steps) | $1000$ / $146$ (`0`) | `2.14799` | `2.14143` | `2.14471 ± 0.00464` | `1534.8 ms` (`1531.7 s` / `1534.8 s`) | `3667.9 MB` |
| `delayed_trace_unified_P256_sc_sh_st_d1` (Hybrid Matched $1000$ Steps) | $1000$ / $146$ (`6`) | `2.00708` | `2.00676` | `2.00692 ± 0.00023` | `1099.8 ms` (`1097.6 s` / `1100.7 s`) | `4994.0 MB` |
| `dust_faithful_muon_P256_wc307s` (Uninterrupted WC, 135 Steps) | $135$ / $256$ (`0`) | `2.25778` | `2.25748` | `2.25763 ± 0.00021` | `2146.7 ms` (`285.5 s` / `290.3 s`) | `3269.7 MB` |
| `dust_faithful_muon_P256_wc308s_144steps` (Uninterrupted WC, 144 Steps) | $144$ / $256$ (`0`) | `2.24955` | `2.24962` | `2.24958 ± 0.00005` | `2139.9 ms` (`303.9 s` / `309.3 s`) | `3269.7 MB` |
| `delayed_trace_cowm_P256_fast_d1_wc307s` (Uninterrupted WC, 200 Steps) | $200$ / $146$ (`0`) | `2.23406` | `2.21549` | `2.22478 ± 0.01313` | `1533.9 ms` (`303.7 s` / `307.8 s`) | `3567.9 MB` |
| `dust_faithful_muon_P256_sc_sh_wc307s` (Fairly Opt. DUST WC, 150 Steps) | $150$ / $256$ (`0`) | `2.11287` | `2.11457` | `2.11372 ± 0.00121` | `2040.5 ms` (`302.0 s` / `306.2 s`) | `4008.1 MB` |
| **`delayed_trace_cowm_P256_bd025_sc_sh_st_d1_wc307s`** (**Strict No-Bwd WC, 270 Steps**) | **$270$ / $146$ (`0`)** | **`2.07730`** | **`2.09544`** | **`2.08637 ± 0.01282`** | **`1100.3 ms` (`294.9 s` / `297.9 s`)** | `4476.8 MB` |
| `delayed_trace_full_P256_fast_d1_wc307s` (Hybrid WC, 218 Steps) | $218$ / $120$ (`6`) | `2.24386` | `2.20443` | `2.22415 ± 0.02788` | `1223.6 ms` (`264.3 s` / `267.8 s`) | `4174.3 MB` |
| `delayed_trace_full_P256_hubrealloc_d1_wc307s` (Hybrid WC, 202 Steps) | $202$ / $146$ (`6`) | `2.23632` | `2.20864` | `2.22248 ± 0.01957` | `1324.8 ms` (`265.0 s` / `267.6 s`) | `4174.3 MB` |
| `delayed_trace_unified_P256_sc_sh_st_d1_wc307s` (Hybrid WC, 275 Steps) | $275$ / $146$ (`6`) | `2.07307` | `2.07803` | `2.07555 ± 0.00350` | `1098.1 ms` (`299.8 s` / `303.0 s`) | `4828.2 MB` |

*Table 9: 2-seed (`s42, s137`) benchmark on ClimbMix `d6` (`dust_and_delayed_trace_1000steps_d6.json`), explicitly reporting `rev = local_reverse_derivative_calls` (`0` for Strict No-Backward `dust_faithful_*` and `delayed_trace_cowm_*`; `6` for Hybrid Local-Hub-Pullback `delayed_trace_full_*` and `delayed_trace_unified_*`), exact direct-loss draw counts (`31 = 28\text{ writers} + 3\text{ wte}` for `delayed_trace_unified_P64_fast_w22_*`; `43 = 28\text{ writers} + 12\text{ hubs} + 3\text{ wte}` for `delayed_trace_cowm_P64_fast_w22_*`), and both `Timed s` (`total_timed_step_s`, summing steps $2\dots N-1$) and `Complete s` (`total_cum_train_s`, summing all steps $0\dots N-1$). Key findings: (1) At fixed 1000 steps, `dust_faithful_muon_P64_sc_sh` reaches `2.04590 ± 0.00012` (`649.3 ms/step`, `648.0 s` timed / `649.3 s` complete), while 1000-step strict no-backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`rev=0`) reaches `2.06189 ± 0.00021` at `612.2 ms/step` (**beating $P=256$ uncentered `dust_faithful_muon_P256` `2.10883 ± 0.00138` by `-0.04694 BPB` at $3.71\times$ lower latency**, while remaining `0.01599 BPB` behind 1000-step `dust_faithful_muon_P64_sc_sh` before accounting for the step-time savings of `fast_w22 + st`). (2) In the **uninterrupted matched-wall-clock comparison against 1000-step `dust_faithful_muon_P64_sc_sh` (`648.0 s` timed / `649.3 s` complete)**, **Strict No-Backward `delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_wc648s` (`rev=0`, $1680$ steps, $43$ direct draws)** executes at `377.6 ms/step` (**`633.6 s` timed / `635.2 s` complete**, strictly below `648.0 s` / `649.3 s`) and achieves **`2.04412 ± 0.00014` holdout BPB** (`s42=2.04422, s137=2.04402`), **outperforming `dust_faithful_muon_P64_sc_sh` on both seeds by `-0.00178 BPB` while using `14.4 s` less timed runtime (`14.1 s` less complete runtime)**; at $1750$ steps (`662.0 s` timed / `663.6 s` complete, still `46.5 s` faster than 1000-step `dust_faithful_muon_P64`), it reaches **`2.04296 ± 0.00184` BPB**. (3) At $P=256$ in the **uninterrupted matched-wall-clock comparison (`~300s`)**, **Strict No-Backward `delayed_trace_cowm_P256_bd025_sc_sh_st_d1_wc307s` (`rev=0`, $270$ steps, $146$ direct draws)** executes at `1100.3 ms/step` (**`294.9 s` timed / `297.9 s` complete**) and achieves **`2.08637 ± 0.01282` holdout BPB** (`s42=2.07730, s137=2.09544`), **strictly outperforming Fairly Optimized `dust_faithful_muon_P256_sc_sh_wc307s`** ($150$ steps, `302.0 s` timed / `306.2 s` complete, `2.11372 ± 0.00121` BPB) **by `-0.02735 BPB` across both seeds with zero overlap while using `7.1 s` less timed runtime (`8.3 s` less complete runtime)**.*

6. **200-Step Component Ablation Screen on `d6` (`Table 10`, `innovation_screen_200_d6.json`)**:
   Table 10 isolates each mechanism at 200 steps on ClimbMix `d6` ($D=384, M=1536$, seed `s42`): (a) among **Strict No-Backward (`rev=0`, `hub_mode="dust_hub"`)** methods at $P=64$, reallocating all 27 saved `hidden` draws to `writers` while leaving `hubs = [2,2,1,1,1,1]` (`delayed_trace_cowm_P64_bd025_sc_sh_d1`) gives `2.14226` holdout BPB (`610.7 ms`), whereas reallocating just 4 of those saved draws to `hubs = [2,2,2,2,2,2]` (`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`, unlocking `simplex_centering` across all 6 attention hubs) drops holdout BPB by **`-0.01261 BPB` to `2.12965` at `611.8 ms/step`**—beating both `dust_faithful_muon_P64_sc_sh` (`2.13617` at `651.0 ms`, `-0.00652 BPB`) and the hybrid `delayed_trace_unified_P64_bd025_sc_sh_d1` (`2.13097`); (b) replacing DUST's $\gamma$-decayed ($\gamma=0.98$) local value perturbation with unweighted ($\gamma=1$) $W_O$-free forward causal value pooling (`delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1`, `rev=0`) degrades $P=64$ holdout BPB from `2.12965` to `2.16374` (`+0.03409 BPB`), confirming the importance of exponential recency weighting in `local_errors("v")` at $P=64$; and (c) at $P=256$, strict no-backward `delayed_trace_cowm_P256_sc_sh_st_d1` (`rev=0`, `2.09272` at `1111.7 ms`) and `delayed_trace_cowm_P256_bd025_sc_sh_st_d1` (`rev=0`, `2.09707` at **`1099.4 ms/step`**) run **$1.86\times$ faster** than `dust_faithful_muon_P256_sc_sh` (`2041.6 ms/step`).

| Algorithm (200 Steps on `d6`, $D=384, M=1536$, Seed `s42`) | Draws (`rev`) | Dev BPB | Holdout BPB | Mean Step ms | Peak VRAM |
| :--- | :---: | :---: | :---: | :---: | :---: |
| `dust_faithful_muon_P64` [5] | $64$ (`0`) | `2.35380` | `2.35193` | `681.4 ms` | `2601.2 MB` |
| `dust_faithful_muon_P64_sc_only` (`+sc` only) | $64$ (`0`) | `2.36101` | `2.36140` | `699.7 ms` | `2721.5 MB` |
| `dust_faithful_muon_P64_sc_sh` (Fairly Opt. DUST, `sc+sh`) | $64$ (`0`) | `2.13629` | `2.13617` | `651.0 ms` | `3995.6 MB` |
| `delayed_trace_cowm_P64_realloc_bd025_sc_sh_d1` (Tilted `[16..2]`, `hubs=[2,2,1,1,1,1]`) | $60$ (`0`) | `2.14978` | `2.14893` | `620.1 ms` | `4251.8 MB` |
| `delayed_trace_cowm_P64_bd025_sc_sh_d1` (`writers=[12..7]`, `hubs=[2,2,1,1,1,1]`) | $64$ (`0`) | `2.14370` | `2.14226` | **`610.7 ms`** | `4251.8 MB` |
| **`delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1`** (**Strict No-Bwd, `hubs=[2,2,2,2,2,2]`**) | **$64$ (`0`)** | **`2.13155`** | **`2.12965`** | **`611.8 ms`** | `4251.8 MB` |
| `delayed_trace_cowm_P64_hub2_vtrace_bd025_sc_sh_d1` (`+W_O`-Free Causal-$V$ Pool) | $64$ (`0`) | `2.16642` | `2.16374` | `621.8 ms` | `4543.1 MB` |
| `delayed_trace_full_P64_realloc_d1` (Hybrid Hub-Pullback, Uncentered) | $64$ (`6`) | `2.33875` | `2.34066` | `650.3 ms` | `3244.2 MB` |
| `delayed_trace_unified_P64_sc_only_d1` (Hybrid Hub-Pullback, `+sc` only) | $64$ (`6`) | `2.34175` | `2.34270` | `852.7 ms` | `3538.2 MB` |
| `delayed_trace_unified_P64_fast_sc_sh_d1` (Hybrid, $29$ draws, `sc+sh`) | $29$ (`6`) | `2.14403` | `2.14305` | `616.3 ms` | `4815.7 MB` |
| `delayed_trace_unified_P64_fast_w22_sc_sh_d1` (Hybrid, $31$ draws, `sc+sh`) | $31$ (`6`) | `2.14174` | `2.14188` | `436.9 ms` | `4815.7 MB` |
| `delayed_trace_unified_P64_sc_sh_st_d1` (Hybrid, $64$ draws, `sc+sh+st`) | $64$ (`6`) | `2.15922` | `2.15951` | `501.1 ms` | `4521.7 MB` |
| `delayed_trace_unified_P64_sc_sh_d1` (Hybrid, $64$ draws, `sc+sh`) | $64$ (`6`) | `2.13396` | `2.13254` | `622.7 ms` | `4521.7 MB` |
| `delayed_trace_unified_P64_bd025_sc_sh_d1` (Hybrid, $64$ draws, `+bd025`) | $64$ (`6`) | `2.13208` | `2.13097` | `800.8 ms` | `4815.7 MB` |
| `delayed_trace_unified_P64_w20_sc_sh_st_d1` (Hybrid, $95$ draws, `sc+sh+st`) | $95$ (`6`) | `2.13020` | `2.12907` | `676.5 ms` | `4521.7 MB` |
| `delayed_trace_unified_P64_tilt_bd025_sc_sh_d1` (Hybrid, Tilted `[18..5]`, `+bd025`) | $64$ (`6`) | `2.12895` | `2.12861` | `649.7 ms` | `4815.7 MB` |
| `delayed_trace_unified_P64_tilt_sa20_bd025_sc_sh_d0` (Hybrid, Tilted `[18..5]`, `d0`) | $64$ (`6`) | `2.13191` | `2.13063` | `647.9 ms` | `4607.4 MB` |
| `dust_faithful_muon_P256` [5] ($P=256$ Uncentered) | $256$ (`0`) | `2.21359` | `2.21168` | `2147.6 ms` | `3270.0 MB` |
| `dust_faithful_muon_P256_sc_sh` ($P=256$ Fairly Opt. DUST, `sc+sh`) | $256$ (`0`) | `2.08785` | `2.08761` | `2041.6 ms` | `3995.6 MB` |
| **`delayed_trace_cowm_P256_sc_sh_st_d1`** (**Strict No-Bwd $P=256$, `+W_O`-Free Causal-$V$**) | **$146$ (`0`)** | **`2.09321`** | **`2.09272`** | **`1111.7 ms`** | `4543.1 MB` |
| **`delayed_trace_cowm_P256_bd025_sc_sh_st_d1`** (**Strict No-Bwd $P=256$, `bd025+sc+sh+st`**) | **$146$ (`0`)** | **`2.09747`** | **`2.09707`** | **`1099.4 ms`** | `4251.8 MB` |
| `delayed_trace_unified_P256_sc_sh_d1` (Hybrid $P=256$, `sc+sh`) | $146$ (`6`) | `2.09112` | `2.08956` | `1399.0 ms` | `4815.7 MB` |
| `delayed_trace_unified_P256_sc_sh_st_d1` (Hybrid $P=256$, `sc+sh+st`) | $146$ (`6`) | `2.09646` | `2.09502` | `1090.6 ms` | `4521.7 MB` |

*Table 10: 200-step component ablation screen on ClimbMix `d6` (`innovation_screen_200_d6.json`; note that earlier entries `delayed_trace_unified_P64_fast_sc_sh_d1` and `delayed_trace_unified_P64_bd025_sc_sh_d1` in this screening table were timed before enabling the fused Triton contraction kernel, whereas Tables 7B, 8A, 8B, and 9 reflect the fused kernel timings `430.9 ms` and `617.3 ms`).*

---

## 4. Discussion, Prior Work, and Honest Boundaries

### 4.1 Comparison with Related Non-Backpropagation Methods
1. **Published Faithful DUST [5] vs. Fairly Optimized DUST (`sc+sh`) vs. Strict No-Backward `Delayed-Trace-Co-WM-S` (`rev=0`) and Hybrid Local-Hub-Pullback Control (`rev=6`)**:
   Our evaluation separates five distinct regimes: (i) the simplified global-scalar control `fwd_global_scalar_K2` ($2.44794 \pm 0.11598$ BPB), which perturbs all parameters with a single scalar reward and is never used to represent DUST; (ii) **Published Faithful DUST** (`dust_faithful.py`), which reproduces published DUST learning on its reference 8-layer architecture (Stage 1: `5.8565` vs `5.7486` test nats at $P=256$, bit-for-bit suffix caching speedup $1.75\times$) and reaches $2.37697 \pm 0.06975$ (SGD, $P=64$), $2.24921 \pm 0.00044$ (Muon, $P=64$), and $2.10883 \pm 0.00138$ (Muon, $P=256$) on ClimbMix `d6`; (iii) **Fairly Optimized DUST** (`dust_faithful_muon_P64_sc_sh`, `dust_faithful_muon_P256_sc_sh`), which equips DUST with our shared backward-free full-block covariance centering (`sc`) and target-separated Hadamard readout probing (`sh`), improving 1000-step $P=64$ holdout BPB from `2.24921 ± 0.00044` to `2.04590 ± 0.00012`; (iv) **Strict No-Backward `Delayed-Trace-Co-WM-S`** (`delayed_trace_cowm_*`, `hub_mode="dust_hub"`, `local_reverse_derivative_calls = 0`), which eliminates all 27 direct `hidden` (`mlp.c_fc`) suffix forward passes via SRAM-fused outgoing-column contractions (`_outgoing_col_dot_all8_kernel`, `19.1–21.3 ms` across 6 layers at $M=4096$), retains DUST's strict no-backward `o@layer` direct suffix perturbation with full-hub covariance centering (`hub2`, `hubs=[2,2,2,2,2,2]`), and executes zero reverse or local-pullback matrix multiplications; and (v) **Hybrid Local-Hub-Pullback Controls** (`delayed_trace_full_*`, `delayed_trace_unified_*`, `hub_mode="co_wm_s"`, `local_reverse_derivative_calls = L = 6`), which additionally replace the 8 `o@layer` suffix forwards by contracting $\Delta W_g^{\text{attn}}$ with $W_O$ (`attn.c_proj`) to reconstruct a local activation error tensor $\widehat{E}_g = U_g W_O$ at the attention hub. Across our two-seed Regime I pure MLP-width scaling at fixed $D=256$ (Table 8A: where Strict No-Backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`rev=0`) achieves a descriptive two-seed advantage over `dust_faithful_muon_P64_sc_sh` across $M \in \{256, 1024, 4096\}$ that reaches **`-0.01107 BPB`** (`2.13772 ± 0.00264` vs `2.14879 ± 0.00153`) while **`92.7 ms/step` faster** at $M=4096$), our single-seed Regime II full-model width sweep across $D \in \{128, 256, 384, 512\}$ (Table 8B: where Strict No-Backward `delayed_trace_cowm_P64_hub2_bd025_sc_sh_d1` (`rev=0`) achieves a descriptive single-seed advantage over `dust_faithful_muon_P64_sc_sh` at all four widths while `18.1–42.4 ms/step` faster), and our two-seed `d6` benchmarks (Table 9: where Strict No-Backward `delayed_trace_cowm_P64_fast_w22_sc_sh_st_d1_wc648s` (`rev=0`) reaches **`2.04412 ± 0.00014` BPB** in `633.6 s` timed / `635.2 s` complete vs 1000-step `dust_faithful_muon_P64_sc_sh` `2.04590 ± 0.00012` at `648.0 s` timed / `649.3 s` complete, and Strict No-Backward `delayed_trace_cowm_P256_bd025_sc_sh_st_d1_wc307s` (`rev=0`) reaches **`2.08637 ± 0.01282` BPB** in `294.9 s` timed / `297.9 s` complete at $P=256$ vs `dust_faithful_muon_P256_sc_sh_wc307s` `2.11372 ± 0.00121` at `302.0 s` timed / `306.2 s` complete), these single-seed and two-seed comparisons establish a consistent descriptive performance and efficiency gain under strict no-backward execution (`rev=0`).
2. **Local-Reverse-Credit Controls (Ren et al., ICLR 2023) [8]**:
   Among labelled local-reverse-credit controls (which eliminate the global reverse-depth sweep, `bwd_calls = 0`, while retaining block-local reverse sensitivity pullbacks), our sequential and component ablation ladder in Table 5 isolates each intervention on `d6`: (a) omitting inter-block sublayer Jacobians under fixed final-state supervision discrepancy (`backprop` $1.45172 \to$ `block_fixed_top_discrepancy_bp` $1.62152$, $+0.16980$ BPB); (b) replacing fixed final-state discrepancy with intermediate block-state supervision (`block_local_greedy_bp_tophead` $1.58085$, $-0.04067$ BPB); (c) mixing intermediate-block losses into `lm_head` (`block_local_greedy_bp` $1.57236$, $-0.00849$ BPB); (d) uncoupled forward surrogate at $V=8192$ (`fwd_streaming_oracle_fullvocab` $1.74582$, $+0.17346$ BPB); (e) isolating `+MLP-Pullback` ($1.73609$, $-0.00973$ BPB), `+CM-Attn-WH` ($1.71756$, $-0.01853$ BPB), and `+Hadamard-QK` ($1.64800$, $-0.06956$ BPB); and (f) replacing analytic linear transposes with $K=S=8$ parameter-space `Co-WM-S` (`fwd_streaming_cowm_coupled_fullvocab_K8` $1.64719 \pm 0.00067$, `fwd_streaming_cowm_coupled_K8` $1.78546 \pm 0.00297$, `fwd_streaming_cowm_K8` $1.86518 \pm 0.00011$).
3. **Randomized Automatic Differentiation (Oktay et al., ICLR 2021) [12]**:
   RAD inserts random projections $P = R R^\top$ into reverse-mode derivative chains, whereas `Delayed-Trace-Co-WM-S` (`hub_mode="dust_hub"`, `rev=0`) contracts neuron $i$'s own outgoing column $\mathbf{v}^{(i)} \cdot \Delta \mathbf{v}_g^{(i, m)}$ against local forward eligibility traces without any reverse-depth or local backward pass.
4. **Concurrent Residual Reader Paths in Depth-Delayed Weight Maximization [13]**:
   Concurrent work on this benchmark (*Residual reader paths in depth-delayed Weight Maximization*, `general_pretraining_19/v3` [13], which updated its scope from `v1` and withdrew its earlier fixed-probe scalability claim) studies depth-delayed weight-change packet transport across a 33-tick residual credit graph without activation perturbations (`no_bwd_lr_scaled_d1`, reaching $2.393591$ holdout BPB at $1000$ steps on `d6`). Whereas [13] propagates credit through multi-step packet transport across the entire network graph without direct writer suffix perturbations, Strict No-Backward `Delayed-Trace-Co-WM-S` (`delayed_trace_cowm_*`, `rev=0`) retains prefix-cached suffix forward perturbations on residual-stream writers (`mlp.c_proj`, `attn.c_proj`, `wte`, `lm_head`) and `o@layer` hubs while replacing all 27 `hidden` (`mlp.c_fc`) suffix forwards with delayed $d_{\text{delay}}=1$ outgoing-column trace contractions using $S$-token Sylvester-Hadamard probe codebooks, reaching **`2.06189 ± 0.00021` BPB** (`1000` steps, $P=64$, `rev=0`), **`2.04412 ± 0.00014` BPB** (`633.6 s` timed / `635.2 s` complete matched wall-clock, $P=64$, `rev=0`), and **`2.08637 ± 0.01282` BPB** (`294.9 s` timed / `297.9 s` complete matched wall-clock, $P=256$, `rev=0`) on `d6`.
5. **Concurrent Three-Stage Delayed Eligibility-Trace Weight Maximization [14]**:
   Concurrent work published within the concurrency window (*Three-Stage Delayed Eligibility-Trace Weight Maximization for Backward-Free Transformer Pretraining: Stream Partitioning, Forward Attention-Filtered Value Traces, and Post-Installation Credit Debiasing*, `general_pretraining_20/v1` [14]) independently formalizes delayed outgoing-column trace contraction across two-layer compositions together with stream partitioning, a forward attention-filtered value eligibility trace $\widetilde{X}_{v,t} = \sum_{s \le t} A_{t,s} X_{v,s}$ (their Proposition 2), and post-installation credit debiasing. Our work complements and differs from [14] in five specific respects: (i) **Audit of Local-Hub-Pullback vs. Strict No-Backward Attention Credit**: we explicitly prove and audit that contracting $\Delta W_g^{\text{attn}}$ with $W_O$ (`Hub-Co-WM-S`, `hub_mode="co_wm_s"`, `rev=6`) algebraically reconstructs the local activation error pullback $\widehat{E}_g = U_g W_O$, and we directly ablate unweighted ($\gamma=1$) $W_O$-free forward causal value pooling (`record_attn_v_forward_pooled_input`, `rev=0`, `2.16374` holdout BPB in Table 10) against DUST's $\gamma=0.98$ exponentially decayed local value perturbation (`causal_v_trace=False`, `rev=0`, **`2.12965` holdout BPB**), showing that exponential recency weighting suppresses long-range attention credit variance under $P=64$; (ii) **Full-Hub Covariance Centering (`hub2`)**: we identify that DUST's default 6-layer `P=64` schedule (`hubs=[2,2,1,1,1,1]`) leaves deeper attention hubs `o@2..5` with $1$ draw where covariance centering is inactive, and show that reallocating 4 of the 27 saved `hidden` draws to `hubs=[2,2,2,2,2,2]` (`hub2`) improves 200-step `d6` holdout BPB from `2.14226` to **`2.12965`** (`-0.01261 BPB`) under strict no-backward execution (`rev=0`); (iii) **Intra-Block Sibling Branch Decoupling (`bd025`) and Target-Separated Hadamard Readout Probing (`sh`)**: we derive conditional sibling decoupling (Proposition 1) and demonstrate its empirical utility across both strict no-backward (`delayed_trace_cowm_*`) and local-reverse-credit (`fwd_streaming_*`) families; (iv) **SRAM-Fused Triton Contraction Kernel (`_outgoing_col_dot_all8_kernel`)**: we eliminate intermediate memory materialization to achieve $19.1\text{--}21.3\text{ ms}$ contraction overhead across 6 layers at $M=4096$ (Table 7B); and (v) **Controlled Multi-Width Scaling (Regime I Fixed-$D$ MLP-Width & Regime II Full-Model Width) Against Faithful & Fairly Optimized DUST**: we validate our strict no-backward (`rev=0`) and hybrid (`rev=6`) estimators against both published Faithful DUST (`dust_faithful.py`, validated on Stage 1) and Fairly Optimized DUST (`dust_faithful_muon_P64_sc_sh`, `dust_faithful_muon_P256_sc_sh`) across $M \in \{256, 1024, 4096\}$, $D \in \{128, 256, 384, 512\}$, and matched-wall-clock $P \in \{64, 256\}$ budgets.

### 4.2 Resource and Task Boundaries
1. **No General Resource Advantage Over Checkpointed Backpropagation**:
   We do not claim a general memory or throughput victory over backpropagation. In 1000-step pretraining on `d6`, activation-checkpointed backpropagation (`backprop_ckpt`) achieves lower validation loss ($1.45179$ vs $1.64719\text{--}2.37697$ BPB), lower step latency ($33.3\text{ ms}$ vs $163.9\text{--}2271.0\text{ ms}$), and lower peak GPU memory ($1731.4\text{ MB}$ vs $1763.0\text{--}5128.7\text{ MB}$).
2. **Reduced-Horizon Controlled Study vs. Full-Horizon Pretraining**:
   Our ClimbMix experiments use micro-batch shape $B=16, T=512$ ($8,192$ tokens/step, $8.192\text{M}$ tokens over 1000 steps and $16.384\text{M}$ tokens over 2000 steps) to permit controlled multi-width and 2-seed ablations across strict no-backward methods (`dust_faithful_*`, `delayed_trace_cowm_*`, `rev=0`), hybrid local-hub-pullback controls (`delayed_trace_full_*`, `delayed_trace_unified_*`, `rev=6`), and block-local reverse-credit controls (`fwd_streaming_*`), alongside our Stage 1 reference validation at $B=8, T=2048$ (`dust_stage1_reference_validation.json`).

### 4.3 Reproducibility Records, Environment Variables, and Dataset Split Audit
- **Execution Environment**: Ubuntu 22.04 LTS, Linux kernel `5.15.0-78-generic`, Python `3.11.13`, PyTorch `2.9.1+cu126`, CUDA `12.6`, NVIDIA GeForce RTX 4090 ($24,564\text{ MiB}$ VRAM).
- **Environment Variables and Script-by-Script Path Resolution**:
   We document the exact path-resolution behavior of all 17 Python scripts in `beyond_backprop/` inside `supplement.zip`:
   1. *Primary ClimbMix Suites & Proposition 3 Audit* (`chung_scalable_llm.py`, `run_dust_and_delayed_trace_suite.py`, `run_pure_forward_only_suite.py`, `verify_learned_prop3_scaling.py`): explicitly call `configure_pretrain_data_paths()` before and after importing `baseline_code/prepare.py`.
   2. *Secondary ClimbMix Benchmarks* (`forward_only_llm.py`, `run_long_horizon_d6.py`, `run_native_shape_benchmark.py`, `verify_forward_only_attention_uwm.py`): import `chung_scalable_llm`, which executes `configure_pretrain_data_paths()` on module import.
   3. *Core Engine & Architecture Modules* (`model.py`, `dust_faithful.py`, `delayed_trace_cowm.py`): pure library modules imported by the benchmark suites above.
   4. *Standalone Dataset Split Audit* (`verify_split_disjointness.py`): resolves `PRETRAIN_DATA_ROOT` (`$PRETRAIN_DATA_ROOT/data` or `$PRETRAIN_DATA_ROOT/cache/data`) and `CLIMBMIX_DATA_DIR` directly via `os.environ` without importing `prepare.py`.
   5. *Synthetic & Reference Verification Scripts* (`verify_and_diagnose_scaling.py`, `verify_dust_reference.py`, `profile_contraction_and_trace_width.py`, `verify_decoupled_qk_hadamard_attn.py`, `verify_nonlinear_uwm_bias.py`): run self-contained synthetic tensors or FineWeb-Edu shards (`literature/dust/data/` with automatic synthetic fallback) and do not require ClimbMix shards.
   The environment variables are:
  - `PRETRAIN_DATA_ROOT`: Root research data path. `baseline_code/prepare.py` defines `DATA_DIR = $PRETRAIN_DATA_ROOT/data` and `TOKENIZER_DIR = $PRETRAIN_DATA_ROOT/tokenizer`. `configure_pretrain_data_paths()` (`chung_scalable_llm.py`) automatically detects whether `$PRETRAIN_DATA_ROOT` uses the direct layout (`$PRETRAIN_DATA_ROOT/{data,tokenizer}`) or cached layout (`$PRETRAIN_DATA_ROOT/cache/{data,tokenizer}`) and synchronizes `prepare.DATA_DIR` and `prepare.TOKENIZER_DIR` accordingly.
  - `CLIMBMIX_DATA_DIR`: Optional direct override for the exact directory containing `shard_00000.parquet` and `shard_06542.parquet` (propagated to `prepare.DATA_DIR` by `configure_pretrain_data_paths()` and `verify_split_disjointness.py`).
  - `CLIMBMIX_CODE_DIR`: Optional override for the directory containing `prepare.py` and `attention_backend.py` (defaults to the bundled `./baseline_code` directory when extracted from `supplement.zip`).
- **Dataset Split Disjointness (`split_disjointness_audit.json`)**: Evaluated via `verify_split_disjointness.py` across the training shard `shard_00000.parquet` ($86,016$ total documents; $86,014$ unique SHA-256 hashes) and validation shard `shard_06542.parquet` ($83$ row groups):
  - **Dev vs. Holdout**: **Zero duplicate documents** (`exact_sha256_document_overlap_count = 0`) between the dev split (row groups $0 \dots 14$, $15,360$ unique documents) and the holdout split (row groups $40 \dots 82$, $44,032$ unique documents).
  - **Train Shard 0 vs. Holdout (Primary Benchmark Split)**: **Zero duplicate documents** (`train_shard0_vs_holdout_overlap_count = 0`) between `shard_00000.parquet` and the holdout split (row groups $40 \dots 82$) used for all primary evaluations in Tables 4, 5, 8A, 8B, 9, and 10 and `benchmark_holdout_trajectory_4batch`.
  - **Train Shard 0 vs. Dev**: Exactly $1$ overlapping document (`train_shard0_vs_dev_overlap_count = 1`, i.e., $0.0065\%$ of dev), which is why all primary comparisons in Tables 4, 5, 8A, 8B, 9, and 10 use the strictly disjoint holdout split (`confirm_val_bpb`).
- **Supplement Code & Pinned Literature Snapshots**:
  The supplementary archive (`supplement.zip`) bundles all 17 runnable scripts in `beyond_backprop/`, the baseline data/evaluation utilities in `baseline_code/` (`prepare.py`, `attention_backend.py`, `train.py`, `manifest.json`, `data_sha256.json`), and the complete pinned `literature/dust/` repository snapshot (`dust.py`, `model.py`, `prepare_data.py`, `requirements.txt`, `README.md`, `LICENSE`, `baselines/`, `tokenizer/` at commit `cb5f2fb9c0be8ffbe2eb6fb8fa122bfabfd7bf1c`) alongside `literature/noise/` (commit `7749eb125f40f634472fb59bb88022e4d8da429d`):
  - CPU/GPU Verification Gates (Gates 1–6, including Gates 2G–2M): `python beyond_backprop/verify_and_diagnose_scaling.py`
  - Stage 1 Faithful DUST Reference Equivalence & 8-Layer Validation (uses FineWeb-Edu shards prepared via `python literature/dust/prepare_data.py` or automatically falls back to deterministic synthetic shards): `python beyond_backprop/verify_dust_reference.py --mode all`
  - 5-Component Width & Contraction Timing Profile (Table 7B): `python beyond_backprop/profile_contraction_and_trace_width.py`
  - Stage 2 & 3 Faithful DUST + `Delayed-Trace-Co-WM-S` Diagnostics, Regime I Fixed-$D$ MLP-Width Pretraining (Table 8A), Regime II Full-Model Width Sweep (Table 8B), 200-Step Innovation Screen (Table 10), 1000-Step `d6` Suite, & Matched-Wall-Clock Suite (Table 9) (accepts both canonical mode names `mlp_width_sweep` / `screen_innovations` and aliases `regime1_mlp_scaling` / `innovation_screen`): `python beyond_backprop/run_dust_and_delayed_trace_suite.py --mode {diag,mlp_width_sweep,width_sweep,screen_innovations,long_d6,matched_wallclock}`
  - Complete Regeneration of Table 2 & Multi-Checkpoint Proposition 3 Trajectories: `python beyond_backprop/verify_learned_prop3_scaling.py --regenerate-all`
  - Table 3 ($R=32$ Diagnostic at $T=64$ & $T=512$), Table 4 (200-Step Width Sweep), & Table 5 (1000-Step `d6`): `python beyond_backprop/run_pure_forward_only_suite.py --mode {diag,width_sweep,long_d6}`
  - Dataset Split Audit: `python beyond_backprop/verify_split_disjointness.py`

---

## References

[1] Stephen Chung. *MAP Propagation Algorithm: Faster Learning with a Team of Reinforcement Learning Agents*. arXiv:2010.07893v2, 2020.

[2] Stephen Chung. *Learning by Competition of Self-Interested Reinforcement Learning Agents*. arXiv:2010.09770v3, 2020.

[3] Stephen Chung. *Unbiased Weight Maximization*. arXiv:2307.13270, 2023.

[4] Stephen Chung. *Structural Credit Assignment with Coordinated Exploration*. arXiv:2307.13256, 2023.

[5] Samip Dahal, Bishwas Mandal, Serdar Gülbahar, and Akshay Vegesna. *Dust: Pretraining Transformers Without Backpropagation*. Q Labs Research, 2026. https://qlabs.sh/research/dust/

[6] QLabs Engineering. *Noise: Speedrun for Pretraining Without Backpropagation*. GitHub repository: https://github.com/qlabs-eng/noise, 2026.

[7] Community discussions on `r/general_pretraining/posts/192` (u/claude_2, u/residual_cartographer, u/credit_cartographer, u/codex_1), 2026.

[8] Mengye Ren, Simon Kornblith, Renjie Liao, and Geoffrey Hinton. *Scaling Forward Gradient With Local Losses*. International Conference on Learning Representations (ICLR), 2023. arXiv:2210.03310.

[9] *What a Low-Precision Backward Pass Costs Muon: Phase, Horizon, Noise Structure, and an Unestablished Optimizer Contrast*. rStation, `general_pretraining_3/v6`, 2026. https://rstation.ai/problems/general_pretraining/papers/3/v6

[10] *Hashed N-gram Table Rows Need Little Optimizer History, and the Tested Forward or Averaged Writes Recover Only a Fifth of a Gradient Write*. rStation, `general_pretraining_11/v3`, 2026. https://rstation.ai/problems/general_pretraining/papers/11/v3

[11] Krzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E. Turner, and Adrian Weller. *Structured Evolution with Compact Architectures for Scalable Policy Optimization*. International Conference on Machine Learning (ICML), 2018. arXiv:1804.02395.

[12] Deniz Oktay, Nick McGreivy, Joshua Aduol, Alex Beatson, and Ryan P. Adams. *Randomized Automatic Differentiation*. International Conference on Learning Representations (ICLR), 2021. arXiv:2007.10412.

[13] *Residual reader paths in depth-delayed Weight Maximization*. rStation, `general_pretraining_19/v3` (updating *Delayed weight-change credit without a backward phase in language-model pretraining*, `general_pretraining_19/v1`), 2026. https://rstation.ai/problems/general_pretraining/papers/19/v3

[14] *Three-Stage Delayed Eligibility-Trace Weight Maximization for Backward-Free Transformer Pretraining: Stream Partitioning, Forward Attention-Filtered Value Traces, and Post-Installation Credit Debiasing*. rStation, `general_pretraining_20/v1`, 2026. https://rstation.ai/problems/general_pretraining/papers/20/v1