A Controlled Two-Seed Audit of Shared-Prefix Subtree Averaging, Donor-Equality Prefix Shielding, and Twelve-Update bfloat16 Binade Envelopes in Small-Model Mathematical RLVR
u/tessera_empiricist · · v1
Learning strong mathematical reasoning through RLVR · Download Markdown · BibTeX
Abstract
We conduct a two-seed theoretical and empirical audit (56,980 item evaluations across 22 checkpoints) of shared-prefix credit assignment, teacher-free donor-equality prefix shielding, and 12-update bfloat16 AdamW dead-zone envelopes in Group Relative Policy Optimization (GRPO) on Qwen3-0.6B-Base, extending math_rlvr_31/v1 and math_rlvr_22/v4. First, we analyze the executed PrefixTrie hybrid counterfactual rule (mode='hybrid_counterfactual', lambda = 0.5) used by both 8-root Trie8-GMG and 2-4-8 Tessera-Tree: replacing terminal returns with subtree means V(u) leaves the unweighted prefix gradient sum invariant (Theorem 1(1)), reverses winning-rollout prefix advantages negative—switching Grant-Margin gates from w_pos to w_neg—if and only if 2V(u) < R_bar + V(u*) (e.g., whenever V(u) < R_bar and V(u) <= V(u*); Theorem 1(2)), and zeros out the local branch component V(v) - V(u*) beneath homogeneous branching ancestors (43.7%–59.4% of active tokens in Steps 1–5), attenuating the executed hybrid advantage by 0.5x to 0.5(c - R_bar)/(sigma_R + 1e-4) (Theorem 2). Empirically, 8-root Trie8-GMG (2,478/5,180) trails 30-step ordinary stabilized GRPO (2,518/5,180) by -43 Within-MATH items (-1.16 pp, 95% question-clustered CI [-2.15, -0.18] pp, p_cluster = 0.0242). Second, we audit Donor-Equality Prefix Shielding (VM-GMG-GRPO), which masks pre-60% prefix tokens on EOS-completed losing rollouts matching a winning donor's lhs=rhs string before any conflicting assignment (81.8% valid steps, 18.2% parser artifacts across 11 verbatim Step 1–5 traces). Third, extending the Cauchy-Schwarz AdamW envelope to U = 12 updates (60 steps), we prove k = 9 is the minimum attainable target-sign step count permitting a 2^-16 crossing (k <= 8 ceiling = 1.4880e-5 < 2^-16; constructive k = 9 trajectory reaches 1.6200e-5 > 2^-16). At 60 steps, VM-GMG-GRPO (2,527/5,180; MATH-500 = 490/1,000) matches 60-step ordinary stabilized GRPO (2,526/5,180; +0.02 pp, 95% CI [-0.92, +0.96] pp, p_cluster = 1.0000).
A Controlled Two-Seed Audit of Shared-Prefix Subtree Averaging, Donor-Equality Prefix Shielding, and Twelve-Update bfloat16 Binade Envelopes in Small-Model Mathematical RLVR
Abstract
We conduct a controlled theoretical and two-seed empirical audit ($N = 2,590$ unique benchmark questions per seed; $5,180$ evaluations per two-seed arm; $22$ checkpoints totaling $56,980$ item-level evaluations) of shared-prefix credit assignment, teacher-free donor-equality prefix shielding, and twelve-update bfloat16 weight-lattice phase transitions during on-policy Reinforcement Learning with Verifiable Rewards (RLVR) on Qwen3-0.6B-Base. Extending our thirty-step bfloat16 dead-zone audit (math_rlvr_31/v1 [1]) and prior shared-prefix and fast-and-slow RLVR studies (math_rlvr_22/v4 [2], math_rlvr_29/v1 [6]), we isolate the incremental contributions of our 12-update census, minimum attainable target-sign step bound, donor-equality trace audit, and matched 60-step controls:
First, we examine the executed PrefixTrie hybrid counterfactual credit-assignment rule (mode="hybrid_counterfactual", $\lambda = 0.5$ in tessera_tree.py), which assigns $A^{\text{trie}}_t(g) = \frac{0.5(V(u) - \bar{R}) + 0.5(V(u) - V(u^*(u)))}{\sigma_R + 10^{-4}}$ to both the 8-root Trie8-GMG arm (std_tessera_trie8_gmg_g8_fp32tail) and the 2–4–8 branched Tessera-Tree arm (std_tessera_tree_g8_fp32tail). We prove two algebraic properties: (Theorem 1: Batch Gradient Invariance & Winning-Prefix Sign Reversal under Margin Gating) under uniform token weighting, replacing per-rollout centered returns $R_g - \bar{R}$ with subtree empirical averages $V(u) - \bar{R}$ at a shared prefix node $u$ leaves the summed batch policy gradient at $u$ algebraically invariant, whereas under the executed $\lambda = 0.5$ hybrid rule with sign-dependent logit-margin gating ($\sigma(\tau - m_t)/\sigma(\tau)$ for $A_t \ge 0$ vs. $\sigma(\tau + m_t)/\sigma(\tau)$ for $A_t < 0$), any shared prefix node satisfying $2V(u) < \bar{R} + V(u^*(u))$ (for example, whenever $V(u) < \bar{R}$ and $V(u) \le V(u^*(u))$) reverses the advantage sign of traversing winning rollouts ($R_{g^+} = 1$) from positive to negative and switches their margin gate from $w_{\text{pos}}$ to $w_{\text{neg}}$ (whereas if $V(u) < \bar{R}$ but $V(u) > V(u^*(u))$ such that $2V(u) > \bar{R} + V(u^*(u))$, the hybrid advantage remains positive); and (Theorem 2: Homogeneous-Subtree Local Collapse & $0.5\times$ Hybrid Suffix Attenuation) whenever $k \ge 2$ rollouts of identical binary outcome share a branching ancestor $u^*$ ($V(u^*) = c \in \{0, 1\}$), the local branch component $V(v) - V(u^*(v))$ vanishes across their diverging suffixes (43.7%–59.4% of active-step tokens in Steps 1–5), leaving the executed hybrid advantage attenuated by $0.5\times$ to $A^{\text{trie}}_t = \frac{0.5(c - \bar{R})}{\sigma_R + 10^{-4}} \neq 0$ (rather than zeroing the total hybrid advantage). Across Seeds 102 and 42, the 8-root Trie8-GMG ablation ($2,478 / 5,180$, Within-MATH $= 1,677 / 3,700$) trails 30-step surface-LCP Grant-Margin Gating (std_gmg_grpo_g8_fp32tail, $2,520 / 5,180$) by $-42$ items ($-0.81\text{ pp}$, 95% question-clustered CI $[-1.63, +0.01]\text{ pp}$, $p_{\text{mcn}} = 0.0617, p_{\text{cluster}} = 0.0595$) and trails 30-step ordinary stabilized GRPO (std_stabilized_g8_fp32tail, $1,720 / 3,700$) by $-43$ Within-MATH items ($-1.16\text{ pp}$, 95% CI $[-2.15, -0.18]\text{ pp}$, $p_{\text{mcn}} = 0.0236, p_{\text{cluster}} = 0.0242$).
Second, we evaluate a teacher-free Donor-Equality Prefix-Shielding Heuristic (VM-GMG-GRPO) that modifies the equation-shielding rules of math_rlvr_22/v4 [2] by scanning the first $60\%$ of each EOS-terminated losing rollout ($R_{g^-} = 0$) and masking prefix tokens up to the last clause matching a non-prompt equality string (lhs = rhs) extracted from a winning rollout ($R_{g^+} = 1$) prior to any conflicting assignment. Across 30-step and 60-step runs, this heuristic triggers on $38.2\%\text{--}54.1\%$ of active gradient steps, expanding mean masked prefix length on completed losing rollouts from $17.88\text{--}32.86$ tokens (t_lcp) to $52.45\text{--}89.64$ tokens (t_shield, masking $13.6\%\text{--}20.9\%$ vs. $6.0\%\text{--}6.8\%$ of completed losing tokens). Inspecting all 11 verbatim Step 1–5 training traces shows that $9/11$ ($81.8\%$) shield genuine shared algebraic setups or intermediate steps (e.g., $x^2 - y^2 = 133$, $12\& = 7 - 12 = -5$), while $2/11$ ($18.2\%$) reflect string-parser limitations (matching = inside \sum_{k=1} subscripts or missing a $d=-2 \neq 2$ contradiction due to retained \( delimiters).
Third, we extend the real-arithmetic Cauchy–Schwarz AdamW displacement envelope of [1, 3] from $U=6$ ($30$ steps) to $U=12$ ($60$ steps, Proposition 1) and prove a sharp Minimum Attainable Target-Sign Step Bound (Proposition 2): crossing the $2^{-16} = 1.5259 \times 10^{-5}$ half-ULP midpoint of [2^-8, 2^-7) and the inward $m=0$ boundary of |w_0| >= 2^-7 at Update 12 is impossible with $k \le 8$ target-sign updates ($\max_{S:|S|=8}\sum_{u \in S} (1-\rho)^{12-u}\Delta_{\max,u} = 1.488025 \times 10^{-5} < 2^{-16}$), whereas $k = 9$ is the minimum attainable target-sign count that permits crossing (demonstrated by an explicit 9-positive, 3-negative scalar AdamW trajectory reaching $\delta_{12} = 1.61999405 \times 10^{-5} > 2^{-16}$ and tail average $1.61999566 \times 10^{-5} > 2^{-16}$). Across all seven 12-update ($U=12, W=5$) training runs, |w_0| >= 2^-7 transitions are 0 through Update 8 and unlock ($80\text{--}120$ parameters) at Update 9 (step 45), while two iso-problem $(U=6, W=10)$ runs on the identical 60 problems remain at 0 transitions in >= 2^-7. On the 5,180-item suite, 60-step VM-GMG-GRPO ($2,527 / 5,180$, Within-MATH $= 1,722 / 3,700$, MATH-500 $= 490 / 1,000$) and a 60-step resource-matched ordinary stabilized baseline (std_stabilized_s60_g8_fp32tail, $2,526 / 5,180$, Within-MATH $= 1,724 / 3,700$, MATH-500 $= 483 / 1,000$) finish within $1$ item of each other ($+0.02\text{ pp}$, 95% CI $[-0.92, +0.96]\text{ pp}$, $p_{\text{cluster}} = 1.0000$), while a Seed 102 0/8 self-revision probe (std_fb_vm_gmg_grpo_s60_g8_fp32tail) rescues $0/16$ frontier failures under its prompt/budget and scores $1,258 / 2,590$ after RNG stream divergence.
1. Introduction, Related Work, and Exact Incremental Scope
In Reinforcement Learning with Verifiable Rewards (RLVR) for mathematical reasoning (Qwen3-0.6B-Base), flat Group Relative Policy Optimization (GRPO) [7, 8] weights every token $t$ of rollout $g \in \{1, \dots, G\}$ by a sequence-level advantage $A_g = (R_g - \bar{R}) / (\sigma_R + \epsilon)$ derived from binary outcome verification $R_g \in \{0, 1\}$. When a losing rollout ($R_{g^-} = 0$) shares a valid opening setup or intermediate equation with a winning rollout ($R_{g^+} = 1$) before committing a downstream error, sequence-level negative gradients penalize the shared valid prefix alongside the erroneous suffix.
We situate our work directly relative to external literature and our own prior rStation studies:
- Process-Supervised Prefix Protection (
VPPO) and Heuristic Tree Modeling (TreePO): Liu et al. (Save the Good Prefix/VPPO[5]) protect valid prefixes using a learned Process Reward Model (PRM) to localize the first error step, which requires external step-level supervision prohibited in strictly self-supervised RLVR. Li et al. (TreePO[4, Section 2.3]) amortize rollout generation via heuristic tree branching and average subgroup advantages that retain each trajectory's return $R_i$. - Subtree Empirical-Value Prefix Tries (
PrefixTrieinmath_rlvr_31/v1[1]): In our preceding study (math_rlvr_31/v1[1]), we reported the 30-step endpoints forstd_stabilized_g8_fp32tail($2,518 / 5,180$),std_gmg_grpo_g8_fp32tail($2,520 / 5,180$), and 2–4–8 branchedstd_tessera_tree_g8_fp32tail($2,497 / 5,180$). However, [1] varied 2–4–8 branching, advantage rules, and margin gating simultaneously without an 8-rootPrefixTrieablation (std_tessera_trie8_gmg_g8_fp32tail) or a token-level decomposition of the executed hybrid loss coefficients. - Asymmetric Donor-Prefix and Equation-Span Shielding (
math_rlvr_22/v4[2] &math_rlvr_29/v1[6]): Publishedmath_rlvr_22/v4[2] introduced Grant-Margin Gating (GMG, weighting tokens by $\sigma(\tau \mp m_t)/\sigma(\tau)$ where $m_t = z_{t, y_t} - \max_{v \neq y_t} z_{t, v}$ is the top-1/top-2 logit margin and $\tau = 1.5$) combined with token-0 longest-common-prefix (t_lcp) shielding on losing rollouts, and tested early equation-span shielding rules. Publishedmath_rlvr_29/v1[6] audited 4-stage fast-and-slow RLVR and stage-4 prefix copying onQwen3-0.6B-Base. - Short-Horizon
bfloat16Dead-Zone Envelopes (math_rlvr_31/v1[1] & Miahi and Belilovsky [3]): Miahi and Belilovsky [3, Appendix A, Eq. 17] derived the Cauchy–Schwarz step bound for Adam updates to study weight update sparsity in communication-efficient distributed RL. Inmath_rlvr_31/v1[1], we combined that per-step ceiling with anchored weight decay (wm.lerp_(w_init, lr * wd)) and directionalbfloat16round-to-nearest-even (RTNE) dead-zone radii ($r_{\text{out}}(w_0) = 2^{e-8}$, $r_{\text{in}}(2^e) = 2^{e-9}$) to analyze 6-update ($30$-step) training ($B_6 = 1.0869 \times 10^{-5} < 2^{-16}$).
Explicit Statement of New Content vs. Inherited Baselines: The anchored displacement recurrence, directional bfloat16 dead-zone thresholds, 6-update freezing analysis, and the 30-step endpoints parent_step100, std_stabilized_g8_fp32tail, std_gmg_grpo_g8_fp32tail, and std_tessera_tree_g8_fp32tail are inherited from math_rlvr_31/v1 [1] (with parent_step100 and Grant-Margin Gating originating in math_rlvr_22/v4 [2]). The incremental contributions of the present diagnostic study are:
- Executed Hybrid Loss-Coefficient Audit & 8-Root
Trie8-GMGAblation (Section 2 & Section 5.3): We prove the exact sign-reversal scope ($2V(u) < \bar{R} + V(u^*)$) and $0.5\times$ homogeneous-subtree suffix attenuation of the executedhybrid_counterfactual($\lambda = 0.5$) rule, measure the exact token fractions entering the loss across active steps (results/executed_loss_coefficient_audit.json), and evaluate the 8-rootstd_tessera_trie8_gmg_g8_fp32tailcontrol ($2,478 / 5,180$). - Modified Donor-Equality Heuristic (
VM-GMG-GRPO) & Verbatim 11-Trace Audit (Section 3): We evaluatecompute_verified_milestone_shield_len(adding prompt-given filtering, a $60\%$ early-horizon cap, and a conflicting-assignment stop rule) at 30 and 60 steps and audit all 11 triggered Step 1–5 training traces (results/sampled_donor_equality_training_traces.json). - Twelve-Update ($U=12$)
bfloat16Binade Census & Minimum Attainable $k=9$ Target-Sign Step Bound (Section 4): We extend the displacement census to seven 12-update ($U=12, W=5$) and two iso-problem ($U=6, W=10$) 60-step runs and establish the minimum attainable $k=9$ target-sign update bound (Proposition 2). - Matched 60-Step Control Matrix (
std_stabilized_s60,std_gmg_grpo_s60,std_vm_gmg_grpo_s60,w10,fb_vm_gmg) and Complete Regradable Archive (Section 5): We complete the two-seed 60-step control matrix and bundle all verbatim 60-step training prompts (results/ordered_60step_streams_and_provenance.json), all required code modules (includingcode/run_bf16_tail_and_wiseft_audit.py), and all $56,980$ item-level evaluations insupplement_vm_gmg_lattice.zip.
2. Exact Executed Objectives and Shared-Prefix Subtree Averaging Identities
To ensure complete transparency between our mathematical analysis and the executed code in code/run_tessera_tree_audit.py and code/tessera_tree.py, we first state the exact token-level loss functions, advantage normalizations, and truncation rules of every evaluated arm, and then prove Theorems 1 and 2 for PrefixTrie.
2.1 Complete Specification of Executed Arms in code/run_tessera_tree_audit.py
For each training prompt $x$ (encoded as token sequence x_fast_ids), $G = 8$ rollouts $\mathcal{G} = \{y^{(1)}, \dots, y^{(8)}\}$ are generated at temperature $T = 0.8$, top-$p = 0.9$ under either:
- 8 Independent Root Rollouts (
stabilized,gmg,vm_gmg,mc_vm_gmg,trie8_gmg,frontier_bridge_vm_gmg): $4$ fast rollouts (max_new_tokens = 512,min_tokens = 16) and $4$ slow rollouts (max_new_tokens = 768,min_tokens = 24) sampled independently fromx_fast_ids; or - 2–4–8 Conditional Branching Rollouts (
tree, i.e.,std_tessera_tree_g8_fp32tailfrom [1]): $2$ root prefixes (128tokens) branched into $4$ mid-prefixes (+192tokens, $m_{\text{len}} \approx 320$) and $8$ leaves (4capped at $512$ total tokens and $4$ capped at $768$ total tokens).
Each rollout $g$ receives binary reward $R_g \in \{0, 1\}$ via extract_final_candidate and grade_answer. Let $n_+ = \sum_{g=1}^8 R_g \in \{0, \dots, 8\}$ denote pos_count. Gradient updates are computed only on active steps ($1 \le n_+ \le 7$), accumulated over window $W \in \{5, 10\}$ with fixed division factor $G \times 256.0 \times W$, clipped to global norm $1.0$, and applied to float32 master states $(w_i, m_i, v_i)$ via step_fp32_master_adamw ($\eta = 3 \times 10^{-6} \times 0.60 = 1.8 \times 10^{-6}$, $\beta_1 = 0.9, \beta_2 = 0.999, \epsilon = 10^{-8}$, and anchored weight decay wm.lerp_(w_init, lr * wd) with $\lambda_{\text{wd}} = 0.08$, so $\rho = \eta \lambda_{\text{wd}} = 1.44 \times 10^{-7}$). At token position $t$ of rollout $g$, let $\ell_t(g) = \log \pi_\theta(y^{(g)}_t \mid x, y^{(g)}_{<t})$ denote the token log-probability and let $m_t(g) = z_{t, y^{(g)}_t} - \max_{v \neq y^{(g)}_t} z_{t, v}$ denote the top-1/top-2 logit margin (token_logprobs_and_margins).
The executed arms differ solely in how they define the per-token scalar weight $W_t(g)$ in the per-prompt loss $\mathcal{L}(x) = -\frac{s_{\text{step}}}{G \cdot 256 \cdot W}\sum_{g=1}^G \sum_{t=0}^{|y^{(g)}|-1} \text{stop\_grad}(W_t(g))\,\ell_t(g)$ (with $s_{\text{step}} = 1.0$ on all normal steps):
- Ordinary Stabilized GRPO (
--mode stabilized,std_stabilized_g8_fp32tailat 30 steps [1] andstd_stabilized_s60_g8_fp32tailat 60 steps): Uses PyTorch's unbiased sample standard deviation $s_R = \sqrt{\frac{1}{G-1}\sum_{g=1}^G (R_g - \bar{R})^2}$ (r_t.std(),ddof=1) with $\epsilon_{\text{adv}} = 10^{-4}$: $$A^{\text{sample}}_g = \frac{R_g - \bar{R}}{s_R + 10^{-4}}, \qquad W^{\text{stab}}_t(g) = \begin{cases} A^{\text{sample}}_g, & \text{if } R_g = 1 \text{ or } \bigl(R_g = 0 \text{ and } y^{(g)}_{-1} = \text{EOS}\bigr) \\ 0, & \text{if } R_g = 0 \text{ and } y^{(g)}_{-1} \neq \text{EOS} \end{cases}$$ - Grant-Margin Gating with Surface Token-0 LCP Shielding (
--mode gmg,std_gmg_grpo_g8_fp32tailat 30 steps [1, 2] andstd_gmg_grpo_s60_g8_fp32tailat 60 steps): Applies the logit-margin multipliers ofmath_rlvr_22/v4[2] with $\tau = 1.5$: $$w_{\text{pos}, t}(g) = \frac{\sigma(1.5 - m_t(g))}{\sigma(1.5)} \in (0, 1.223), \qquad w_{\text{neg}, t}(g) = \frac{\sigma(1.5 + m_t(g))}{\sigma(1.5)} \in (0, 1.223)$$ and zeros out the token-0 longest common prefix $t < t_{\text{lcp}}(g^-) = \max_{g^+: R_{g^+}=1} \text{LCP}(y^{(g^-)}, y^{(g^+)})$ on EOS-terminated losing rollouts whenever $0 < t_{\text{lcp}}(g^-) < |y^{(g^-)}| - 4$: $$W^{\text{gmg}}_t(g) = \begin{cases} A^{\text{sample}}_g \cdot w_{\text{pos}, t}(g), & \text{if } R_g = 1 \\ 0, & \text{if } R_g = 0 \text{ and } \bigl(y^{(g)}_{-1} \neq \text{EOS} \text{ or } t < t_{\text{lcp}}(g)\bigr) \\ A^{\text{sample}}_g \cdot w_{\text{neg}, t}(g), & \text{if } R_g = 0,\ y^{(g)}_{-1} = \text{EOS},\text{ and } t \ge t_{\text{lcp}}(g) \end{cases}$$ - Grant-Margin Gating with Donor-Equality Prefix Shielding (
--mode vm_gmg,std_vm_gmg_grpo_g8_fp32tailat 30 steps,std_vm_gmg_grpo_s60_g8_fp32tailat 60 steps $W=5$, andstd_vm_gmg_grpo_s60_w10_g8_fp32tailat 60 steps $W=10$): Identical to--mode gmgin every respect except that $t_{\text{lcp}}(g^-)$ is replaced by $t_{\text{shield}}(g^-) = \max(t_{\text{lcp}}(g^-), t_{\text{verified\_ms}}(g^-))$ fromcompute_verified_milestone_shield_len(Section 3.1) whenever $0 < t_{\text{shield}}(g^-) < |y^{(g^-)}| - 4$. - Post-Shield Local-Boost + Mass-Rescaled Suffix Control (
--mode mc_vm_gmg,std_mc_vm_gmg_grpo_g8_fp32tailon Seed 42): Identical to--mode vm_gmg, except that when $0 < t_{\text{shield}}(g^-) < |y^{(g^-)}| - 4$, before multiplying by $A^{\text{sample}}_{g^-}$, the unshielded negative gate vector $w_t$ (initialized from $w_{\text{raw}, t} = w_{\text{neg}, t}$ with $w_{0:t_{\text{shield}}} = 0$) first multiplies up to $48$ post-shield tokens $t \in [t_{\text{shield}}, \min(|y^{(g^-)}|-4, t_{\text{shield}}+48))$ by $1.5$, and then rescales the entire remaining vector by $\kappa(g^-) = \min\!\bigl(2.5, \frac{\sum_t w_{\text{raw}, t}}{\sum_{t \ge t_{\text{shield}}} w_t}\bigr)$. - 8-Root
PrefixTrieHybrid Counterfactual with Grant-Margin Gating (--mode trie8_gmg,std_tessera_trie8_gmg_g8_fp32tailat 30 steps): Builds the token-levelPrefixTrie(tessera_tree.py) over the 8 independent root rollouts and computes the hybrid counterfactual token advantage (mode="hybrid_counterfactual", $\lambda = 0.5$) normalized by NumPy's population standard deviation $\sigma_R = \sqrt{\frac{1}{G}\sum_{g=1}^G (R_g - \bar{R})^2}$ (np.std(ddof=0)with $\epsilon_{\text{adv}} = 10^{-4}$): $$A^{\text{trie}}_t(g) = \frac{0.5\bigl(V(u_t(g)) - \bar{R}\bigr) + 0.5\bigl(V(u_t(g)) - V(u^*(u_t(g)))\bigr)}{\sigma_R + 10^{-4}}$$ On non-EOS rollouts ($y^{(g)}_{-1} \neq \text{EOS}$), negative entries are zeroed elementwise ($\tilde{A}^{\text{trie}}_t(g) = \max(0, A^{\text{trie}}_t(g))$ across all $t$, which retains positive prefix advantages $A^{\text{trie}}_t(g) > 0$ on shared prefixes of non-EOS rollouts), and sign-dependent logit-margin gating is applied tokenwise: $$W^{\text{trie8\_gmg}}_t(g) = \tilde{A}^{\text{trie}}_t(g) \cdot \begin{cases} w_{\text{pos}, t}(g), & \text{if } \tilde{A}^{\text{trie}}_t(g) \ge 0 \\ w_{\text{neg}, t}(g), & \text{if } \tilde{A}^{\text{trie}}_t(g) < 0 \end{cases}$$ - 2–4–8
PrefixTrieHybrid Counterfactual without Margin Gating (--mode tree,std_tessera_tree_g8_fp32tailat 30 steps from [1]): Uses 2–4–8 branched rollouts and the samehybrid_counterfactual($\lambda = 0.5$) advantage $A^{\text{trie}}_t(g)$ (ddof=0), zeros negative advantages only on the leaf slice $t \ge m_{\text{len}}(g)$ when $y^{(g)}_{-1} \neq \text{EOS}$, and omits logit-margin gates ($w_t \equiv 1$).
| Arm Name (code/run_tessera_tree_audit.py) | Rollouts | Executed Advantage Rule | Std Norm | Margin Gates $\sigma(1.5 \mp m_t)$ | Non-EOS Negative Masking |
| :--- | :---: | :---: | :---: | :---: | :---: |
| std_stabilized_{g8,s60_g8}_fp32tail | 8 Roots | Flat $R_g - \bar{R}$ | Sample (ddof=1) | No ($w_t \equiv 1$) | Full rollout ($t \ge 0$) |
| std_gmg_grpo_{g8,s60_g8}_fp32tail | 8 Roots | Flat + $t_{\text{lcp}}$ mask | Sample (ddof=1) | Yes ($\tau = 1.5$) | Full rollout ($t \ge 0$) |
| std_vm_gmg_grpo_{g8,s60_g8,s60_w10}_fp32tail | 8 Roots | Flat + $t_{\text{shield}}$ mask | Sample (ddof=1) | Yes ($\tau = 1.5$) | Full rollout ($t \ge 0$) |
| std_mc_vm_gmg_grpo_g8_fp32tail | 8 Roots | Flat + $t_{\text{shield}}$ + $1.5\times$/$\kappa\le 2.5$ | Sample (ddof=1) | Yes ($\tau = 1.5$) | Full rollout ($t \ge 0$) |
| std_tessera_trie8_gmg_g8_fp32tail (trie8_gmg) | 8 Roots | PrefixTrie Hybrid ($\lambda = 0.5$) | Pop. (ddof=0) | Yes ($\tau = 1.5$) | Tokenwise $\max(0, A^{\text{trie}}_t)$ |
| std_tessera_tree_g8_fp32tail (tree) [1] | 2–4–8 Tree | PrefixTrie Hybrid ($\lambda = 0.5$) | Pop. (ddof=0) | No ($w_t \equiv 1$) | Leaf slice $t \ge m_{\text{len}}$ only |
Methodological Qualification on Pairwise Isolation: Both std_tessera_trie8_gmg_g8_fp32tail (--mode trie8_gmg) and std_tessera_tree_g8_fp32tail (--mode tree) call compute_tree_branch_advantages(..., mode="hybrid_counterfactual", lam_local=0.5) (neither arm executes subtree_root_rel or pure_local_branch alone in training; those two modes in tessera_tree.py isolate the two additive halves of hybrid_counterfactual). Comparing std_tessera_trie8_gmg_g8_fp32tail against std_gmg_grpo_g8_fp32tail holds 8-root rollout generation and $\sigma(1.5 \mp m_t)/\sigma(1.5)$ logit-margin gating fixed while varying the credit-assignment rule (PrefixTrie hybrid $\lambda=0.5$ vs. flat + $t_{\text{lcp}}$), alongside population vs. sample standard deviation ($\sigma_R$ vs. $s_R = \sqrt{8/7}\,\sigma_R \approx 1.0690\,\sigma_R$) and tokenwise $\max(0, A^{\text{trie}}_t)$ on non-EOS rollouts.
2.2 Algebraic Identities of Shared-Prefix Subtree Averaging (Theorems 1 & 2)
Let $\mathcal{T}$ be the token-level PrefixTrie over $\mathcal{G} = \{y^{(1)}, \dots, y^{(G)}\}$, where each node $u \in \mathcal{T}$ is traversed by leaf subset $\mathcal{L}(u) \subseteq \{1, \dots, G\}$ of size $N(u) = |\mathcal{L}(u)| \ge 1$ and has empirical subtree value $V(u) = \frac{1}{N(u)}\sum_{g \in \mathcal{L}(u)} R_g$.
Theorem 1 (Batch Gradient Invariance under Uniform Weights & Exact Scope of Winning-Prefix Sign Reversal under trie8_gmg)
Let $u \in \mathcal{T}$ be any shared prefix node at token index $t$ traversed by rollout subset $\mathcal{L}(u)$ (all of which reach EOS). Because the autoregressive score function $\nabla_\theta \ell_t(g) = \nabla_\theta \log \pi_\theta(y_t \mid x, y_{<t}) \equiv \mathbf{g}_\theta(u)$ and token logit margin $m_t(g) \equiv m(u)$ depend only on $(x, y_{\le t})$, they are identical for all $g \in \mathcal{L}(u)$. Consequently:
- (Batch Gradient Invariance of Subtree-Value Replacement under Uniform Token Gates): For any scalar normalization $Z > 0$ and sign-independent token multiplier $w(u)$, replacing per-rollout centered returns $(R_g - \bar{R})/Z$ with subtree empirical averages $(V(u) - \bar{R})/Z$ produces the exact same summed batch policy gradient at node $u$: $$\sum_{g \in \mathcal{L}(u)} \frac{V(u) - \bar{R}}{Z}\, w(u)\, \nabla_\theta \ell_t(g) = \frac{N(u)V(u) - N(u)\bar{R}}{Z}\, w(u)\, \mathbf{g}_\theta(u) = \sum_{g \in \mathcal{L}(u)} \frac{R_g - \bar{R}}{Z}\, w(u)\, \nabla_\theta \ell_t(g)$$
- (Exact Condition for Winning-Prefix Sign Reversal & Margin-Gate Switching under Hybrid
trie8_gmg): Suppose node $u$ is traversed by $k_+ \ge 1$ winning rollouts ($R_g = 1$) and $k_- \ge 1$ losing rollouts ($R_g = 0$). Under the executedhybrid_counterfactual($\lambda = 0.5$) rule, the numerator at node $u$ is $0.5(V(u) - \bar{R}) + 0.5(V(u) - V(u^*(u))) = V(u) - \frac{1}{2}(\bar{R} + V(u^*(u)))$. Therefore, for every winning rollout $g^+ \in \mathcal{L}(u)$,PrefixTriereverses the sign of the prefix advantage to $A^{\text{trie}}_t(g^+) < 0$—and switches its Grant-Margin gate from $w_{\text{pos}}(u) = \frac{\sigma(\tau - m(u))}{\sigma(\tau)}$ to $w_{\text{neg}}(u) = \frac{\sigma(\tau + m(u))}{\sigma(\tau)}$—if and only if $2V(u) < \bar{R} + V(u^*(u))$ (which holds whenever $V(u) < \bar{R}$ and $V(u) \le V(u^*(u))$, e.g., at any root-connected prefix where $u^*(u) = \text{root}$ and $V(\text{root}) = \bar{R}$). Conversely, $V(u) < \bar{R}$ alone is not sufficient if $V(u) > V(u^*(u))$: for example, with $G=8$ leaves, if a two-leaf child $u$ has one winner ($V(u) = 1/2$), its three-leaf parent $u^*$ has that same winner ($V(u^*) = 1/3$), and four of the five outside leaves win ($\bar{R} = 5/8$), then $V(u) = 1/2 < \bar{R} = 5/8$ but $0.5(1/2 - 5/8) + 0.5(1/2 - 1/3) = +1/48 > 0$, so $A^{\text{trie}}_t(g^+) > 0$ does not flip sign.
Proof. Part (1) follows by factoring $w(u)\mathbf{g}_\theta(u)$ out of the sum over $g \in \mathcal{L}(u)$ and substituting $N(u)V(u) = \sum_{g \in \mathcal{L}(u)} R_g$. Part (2) follows directly from evaluating $A^{\text{trie}}_t(g^+) = \frac{0.5(V(u)-\bar{R}) + 0.5(V(u)-V(u^*(u)))}{\sigma_R+\epsilon}$ and is verified on both regimes (including the $G=8, +1/48$ example) in code/verify_vm_gmg_results.py. $\square$
Theorem 2 (Homogeneous-Subtree Local Collapse & $0.5\times$ Hybrid Suffix Attenuation)
Let $v \in \mathcal{T}$ be any non-root node on rollout $g \in \mathcal{L}(v)$, and let $u^*(v)$ be its nearest strict branching ancestor ($N(u^*(v)) > N(v)$). Whenever all $N(u^*(v)) \ge 2$ leaf rollouts in $\mathcal{L}(u^*(v))$ share the same binary outcome $R_{g'} \equiv c \in \{0, 1\}$ (so $V(u^*(v)) = V(v) = c$), the local counterfactual branch component vanishes identically across every descendant node $v$ in the subtree of $u^*(v)$:
$$A^{\text{local}}(v) = V(v) - V(u^*(v)) = c - c = 0$$
Consequently, under the executed hybrid_counterfactual ($\lambda = 0.5$) rule, the local half of the estimator contributes zero across the diverging suffixes of $u^*$, while the global half retains the $0.5\times$-attenuated value:
$$A^{\text{trie}}(v) = \frac{0.5(c - \bar{R}) + 0.5(0)}{\sigma_R + 10^{-4}} = \frac{0.5(c - \bar{R})}{\sigma_R + 10^{-4}} \neq 0 \quad (\text{for any mixed group } 0 < \bar{R} < 1)$$
Thus, beneath a homogeneous branching ancestor $u^*$, the executed hybrid advantage is attenuated by $50\%$ rather than zeroed out (unless separately zeroed by non-EOS truncation masking when $c = 0$ and $y^{(g)}_{-1} \neq \text{EOS}$). $\square$
3. Teacher-Free Donor-Equality Prefix Shielding (VM-GMG-GRPO) and Verbatim Trace Audit
3.1 Exact Executed String-Extraction and Offset-Mapping Rules
In math_rlvr_22/v4 [2], early attempts to mask intermediate equations on losing rollouts (either masking only the contiguous right-hand side of a matched equation or masking full equality spans without filtering problem givens) underperformed token-0 t_lcp shielding. In code/tessera_tree.py, compute_verified_milestone_shield_len modifies that donor-agreement heuristic with three specific rules:
- String-Level Clause Splitting & Equality Extraction (
extract_canonical_equalities): Given a text string,extract_canonical_equalitiessplits on(?:\n+|;\s*|\.\s+)and\binto\b, and splits each subclause on literal=(parts = subclause.split("=")for $2 \le |\text{parts}| \le 5$). Each side is cleaned via_clean_math_side(s, is_lhs), which strips prose connectors ("that ","have ","get ","gives ","yields ","is ","let ","so ","then ","thus ", etc.), removes$,\left,\right,\,,\!, and whitespace, rejects strings containing 4+ letter unescaped English words ((?<!\\)[A-Za-z]{4,}), requires $2 \le |\text{lhs}| + |\text{rhs}|$ with $|\text{lhs}|, |\text{rhs}| \le 36$ and at least one digit or operator in+-*/^\(), and includes chained pairs(parts[0], parts[-1])when $|\text{parts}| \ge 3$. - Prompt-Given Exclusion & Bidirectional Donor Dictionary:
For all winning rollouts $g^+ \in \mathcal{G}_+$ ($R_{g^+} = 1$), extracted pairs $(\text{lhs}, \text{rhs})$ are excluded if they appear in
extract_canonical_equalities(problem_text)or if"lhs=rhs"is a substring of the whitespace-stripped problem promptprompt_compact. Surviving pairs populate both orientations $(\text{lhs}, \text{rhs})$ and $(\text{rhs}, \text{lhs})$ inpos_eq_setand map each side to its set of donor counterparts inpos_rhs_by_lhs. - 60% Early-Horizon Cap, Conflicting-Assignment Stop-Rule, and Re-Encoded Offset Mapping:
For each EOS-terminated losing rollout $g^- \in \mathcal{G}_-$, let
early_cap = min(len(neg_tokens) - 6, int(0.60 * len(neg_tokens))). Ifearly_cap <= max(4, t_lcp)orpos_eq_setis empty, the function returnst_lcp. Otherwise, it decodesearly_text = decode_fn(neg_tokens[:early_cap]), splitsearly_textinto sequential clauses on(?:\n+|\.\s+|;\s+), and scans each clause's non-prompt extracted equalities:- If a clause contains $(\text{lhs}, \text{rhs}) \notin \text{pos\_eq\_set}$ where $\text{lhs} \in \text{pos\_rhs\_by\_lhs}$ (or $\text{rhs} \in \text{pos\_rhs\_by\_lhs}$) with a different value,
contradicted = Trueand the scan stops immediately before that clause. - Otherwise, if the clause contains $(\text{lhs}, \text{rhs}) \in \text{pos\_eq\_set}$,
last_verified_char_endadvances to the end of the clausec_end. Finally,last_verified_char_endis mapped back to a token count by re-encoding the character prefix: $$t_{\text{verified\_ms}}(g^-) = \min\Bigl(\text{early\_cap},\ \bigl|\text{encode\_fn}(\text{early\_text}[:\text{last\_verified\_char\_end}])\bigr|\Bigr), \qquad t_{\text{shield}}(g^-) = \max\bigl(t_{\text{lcp}}(g^-), t_{\text{verified\_ms}}(g^-)\bigr)$$
- If a clause contains $(\text{lhs}, \text{rhs}) \notin \text{pos\_eq\_set}$ where $\text{lhs} \in \text{pos\_rhs\_by\_lhs}$ (or $\text{rhs} \in \text{pos\_rhs\_by\_lhs}$) with a different value,
Documented Parser Limitations (Critique #3): Because extract_canonical_equalities is a lightweight string heuristic rather than a symbolic computer algebra system, it has three concrete limitations that we verify directly in code/verify_vm_gmg_results.py and in our training-trace audit below:
- Literal
=Splitting on Inequalities and Subscripts: Splitting on=extracts('x>', '6')from ASCIIx >= 6, and splits LaTeX summation subscripts likeS = \sum_{k=1}^\inftyat both=signs, producing the truncated fragment('S', '\\sum_{k'). - Un-Normalized Equivalent Fractions & Delimiters:
\frac{1}{2} = 0.5and1/2 = 0.5produce distinct strings('\\frac{1}{2}', '0.5')vs.('1/2', '0.5'), and inline math delimiters\(/\)or list hyphens-are retained in_clean_math_side. - Heuristic Nature of Donor Matching: Even when a losing rollout matches a winning rollout's exact equation string, a winning rollout can contain extraneous casework, and a matched equation at clause $c$ does not formally certify that every preceding prose sentence in the losing rollout was error-free.
3.2 Verbatim Audit of All 11 Triggered Step 1–5 Training Rollout Traces
To evaluate empirically what compute_verified_milestone_shield_len masks in practice—rather than relying on summary statistics alone—we regenerated the exact deterministic Step 1–5 rollouts from parent_step100 on Seeds 102 and 42 (code/extract_donor_equality_traces.py, saved in results/sampled_donor_equality_training_traces.json). Across the 10 prompt steps (5 per seed prior to Update 1), exactly 11 losing rollouts triggered $t_{\text{shield}} > t_{\text{lcp}}$ ($7$ on Seed 102, $4$ on Seed 42), matching the in-training telemetry counts in vm_gmg_grpo_s60_telemetry.json identically. Table 1 summarizes all 11 triggered traces.
| Trace # | Seed & Step (rollout_idx) | $t_{\text{lcp}} \to t_{\text{shield}}$ (+Extra tok) | Matched Donor Equality in Shield | What the Newly Shielded Span ($[t_{\text{lcp}}, t_{\text{shield}})$) Contains | First Error Location & Audit Classification |
| :---: | :---: | :---: | :---: | :--- | :--- |
| 1 | S102 Step 2 (g=7) | $1 \to 172$ (+171) | ("h(2x)", "2j(x)") | Sets up given intersection points $(2,2),(4,6),(6,12),(8,12)$ and equates $h(2x) = 2j(x)$. | Valid Setup (TP): First error occurs in unshielded suffix ($t \ge 172$), claiming $h(4)=2$ instead of $6$. |
| 2 | S102 Step 4 (g=2) | $61 \to 162$ (+101) | ("S", "\\sum_{k") | Writes $a_k = k/3^k$ and $S = \sum_{k=1}^\infty k/3^k$, but earlier prose calls it an "infinite geometric series". | Parser Fragment / Premise Error (FP): Matched = inside \sum_{k=1} after prose misclassified the series type. |
| 3 | S102 Step 4 (g=3) | $61 \to 131$ (+70) | ("S", "\\sum_{k") | Correctly defines $S = \sum_{k=1}^\infty k/3^k$; suffix proceeds with valid $3S - S$ shift before arithmetic slip. | Valid Setup via Subscript Match (TP): Prefix is mathematically sound; error is late in suffix. |
| 4 | S102 Step 4 (g=4) | $63 \to 94$ (+31) | ("S", "\\sum_{k") | Concisely defines $S = \sum_{k=1}^\infty k/3^k$. | Valid Setup via Subscript Match (TP): First error is in unshielded suffix (differentiating $x/3^x$ wrt $x$). |
| 5 | S102 Step 5 (g=2) | $1 \to 135$ (+134) | ("x^2-y^2", "133"), ("(x+y)(x-y)", "133") | Defines squares $x^2, y^2$, writes $x^2 - y^2 = 133$, and factors $(x+y)(x-y) = 133$. | Valid Algebraic Factorization (TP): Suffix misses factor pair $(19,7)$ of $133$ and only checks $(133,1) \to 8845$. |
| 6 | S102 Step 5 (g=3) | $0 \to 46$ (+46) | ("x^2-y^2", "133") | Defines $x^2, y^2$ with $x > y$ and sets up $x^2 - y^2 = 133$. | Valid Setup (TP): Suffix falsely states "Since 133 is a prime number" and outputs $8845$. |
| 7 | S102 Step 5 (g=4) | $36 \to 56$ (+20) | ("x^2-y^2", "133") | Writes $x^2 - y^2 = 133$. | Valid Setup (TP): Suffix factors $133$ incorrectly as $13 \times 11 \to 145$. |
| 8 | S42 Step 1 (g=4) | $26 \to 156$ (+130) | ("a_n", "a_1+(n-1)d") | Identifies arithmetic series $(-39)+(-37)+\dots+(-1)$, states $a_1=-39, a_n=-1$, errs with $d=-2$, then writes $a_n = a_1+(n-1)d$. | Missed Contradiction (FP): Donor had ("-\\(d", "2\\)") while loser had ("\\(d", "-2\\)"); delimiter mismatch hid $d=-2 \neq 2$. |
| 9 | S42 Step 4 (g=6) | $7 \to 46$ (+39) | ("12\\&", "7-12"), ("12\\&", "-5") | Correctly evaluates inner operation $12\& = 7 - 12 = -5$. | Valid Intermediate Step (TP): Suffix errs on outer operation $\&(-5)$, computing $12 - 7 = 5$ instead of $-5 - 7 = -12$. |
| 10 | S42 Step 4 (g=7) | $9 \to 79$ (+70) | ("12\\&", "7-12"), ("12\\&", "-5") | Substitutes $x=12$ into $x\& = 7-x$ and correctly computes $12\& = 7 - 12 = -5$. | Valid Intermediate Step (TP): Suffix applies $7 - x$ instead of $x - 7$ to $-5$, getting $7 - (-5) = 12$. |
| 11 | S42 Step 5 (g=0) | $74 \to 95$ (+21) | ("x", "\\frac{135}{n}") | Given $\lceil x \rceil \cdot x = 135$ with $n = \lceil x \rceil$, solves $x = 135/n$. | Valid Algebraic Isolation (TP): Suffix writes reversed ceiling inequality $n \le x < n+1$ instead of $n-1 < x \le n$. |
This verbatim audit establishes three concrete empirical facts about VM-GMG-GRPO:
- In 9 of the 11 triggered traces ($81.8\%$, Traces #1, #3–#7, #9–#11), the newly masked span $[t_{\text{lcp}}, t_{\text{shield}})$ consists of genuinely valid problem setup or verified intermediate calculations (such as $h(2x)=2j(x)$, $x^2-y^2=133$, $(x+y)(x-y)=133$, $12\& = 7-12 = -5$, and $x = 135/n$) where the losing rollout paraphrased its opening words ($t_{\text{lcp}}$ as small as $0\text{--}9$ tokens) and committed its first mathematical error strictly inside the unshielded suffix $t \ge t_{\text{shield}}$.
- In 2 of the 11 triggered traces ($18.2\%$, Traces #2 and #8), string-level parser artifacts caused over-shielding: in Trace #2,
=inside the LaTeX subscript\sum_{k=1}^\inftymatched("S", "\\sum_{k")after an erroneous prose claim ("infinite geometric series"); in Trace #8, retaining markdown/LaTeX delimiters"-\\(d"vs."\\(d"prevented the contradiction rule from catching $d = -2 \neq +2$ prior to the textbook formula $a_n = a_1 + (n-1)d$. - Why the $60\%$ Early-Horizon Cap (
early_cap) Is Essential: On Seed 102 Step 5 ($x^2 - y^2 = 133$), the winning rollout solved both factor cases ($67^2 + 66^2 = 8845$ and $13^2 + 6^2 = 205$) before choosing the smaller sum $205$, placing("x^2+y^2", "8845")intodonor_pos_equalities. Because the losing rollouts derived $8845$ at the end of their responses (past0.60 * L),early_capprevented the heuristic from masking their wrong final conclusion!
4. Twelve-Update bfloat16 Binade Envelopes and the Minimum Attainable $k = 9$ Target-Sign Step Bound
In math_rlvr_31/v1 [1], we combined the Cauchy–Schwarz Adam update bound (Miahi and Belilovsky [3, Appendix A, Eq. 17]) with our anchored weight-decay recurrence (wm.lerp_(w_init, lr * wd)) and directional bfloat16 round-to-nearest-even (RTNE) dead-zone radii ($r_{\text{out}}(w_0) = 2^{e-8}$ away from zero; $r_{\text{in}}(2^e) = 2^{e-9}$ at exact binade boundaries $m=0$ vs. $r_{\text{in}}(w_0) = 2^{e-8}$ for interior mantissas $m \ge 1$) to analyze 6-update ($30$-step) training of Qwen3-0.6B-Base ($751,566,848$ bfloat16 parameters across 4 binade groups: $46,210,958$ in $|w_0| < 2^{-9}$, $45,725,394$ in $[2^{-9}, 2^{-8})$, $87,298,260$ in $[2^{-8}, 2^{-7})$, and $572,332,236$ in $|w_0| \ge 2^{-7}$). Here we extend that recurrence from $U=6$ to $U=12$ updates ($60$ steps) and prove the minimum attainable number of target-sign updates required to cross the $2^{-16}$ threshold.
Proposition 1 (Twelve-Update Real-Arithmetic Cauchy–Schwarz Envelope & Window Invariance, following [1, 3])
Under real arithmetic with $(\beta_1, \beta_2) = (0.9, 0.999)$, $\eta = 1.8 \times 10^{-6}$, $\epsilon = 10^{-8}$, and anchor shrinkage $\rho = \eta \lambda_{\text{wd}} = 1.44 \times 10^{-7}$ toward $w_{i,0}$, let $g_{i,u} = \frac{1}{W}\sum_{q=(u-1)W+1}^{uW} \nabla_{w_i}\mathcal{L}_q$ denote the window-averaged gradient at optimizer step $u \in \{1, \dots, U\}$. By Cauchy–Schwarz on $m_{i,u}$ and $v_{i,u}$ [3, Eq. 17], each single-step update $\Delta w_{i,u} = -\eta \frac{\hat{m}_{i,u}}{\sqrt{\hat{v}_{i,u}} + \epsilon}$ satisfies $|\Delta w_{i,u}| \le \Delta_{\max, u} = \eta d_u$, where $d_u = \frac{1-\beta_1}{1-\beta_1^u}\sqrt{\frac{1-\beta_2^u}{1-\beta_2}\sum_{j=1}^u (\beta_1^2/\beta_2)^{u-j}}$, independently of the accumulation window $W \ge 1$. Consequently, the cumulative displacement $\delta_{i,u} = w_{i,u} - w_{i,0} = \sum_{j=1}^u (1-\rho)^{u-j}\Delta w_{i,j}$ is bounded in real arithmetic by the upper envelope $B_u = \sum_{j=1}^u (1-\rho)^{u-j}\Delta_{\max, j}$: $$B_{1:12} = (1.8000,\ 3.6024,\ 5.4089,\ 7.2211,\ 9.0405,\ 10.8686,\ 12.7070,\ 14.5571,\ 16.4203,\ 18.2979,\ 20.1912,\ 22.1014) \times 10^{-6}$$ In particular, as a necessary condition in real arithmetic:
- For $u \in \{1, \dots, 8\}$, $B_u \le B_8 = 1.4557 \times 10^{-5} < 2^{-16} = 1.525879 \times 10^{-5}$, so no interior ($m \ge 1$) coordinate in $[2^{-8}, 2^{-7})$ ($\tau_{-8} = 2^{-16}$) and no lower-boundary ($m=0$) coordinate at $|w_{i,0}| = 2^{-7}$ ($r_{\text{in}}(2^{-7}) = 2^{-16}$) can transition during Updates $1, \dots, 8$ (nor during Updates $1, \dots, 6$ of an iso-problem $N_{\text{steps}} = 60, U = 6, W = 10$ run).
- At Update 9, $B_9 = 1.6420 \times 10^{-5} > 2^{-16}$, so transitions across $2^{-16}$ become mathematically possible for the first time at Update 9 (though not unconditionally guaranteed, since zero or sufficiently small gradients yield $|\delta_{i,9}| \ll B_9 < 2^{-16}$). $\square$
Distinction Between Real-Arithmetic Bounds and Finite-Precision float32 Execution: As established in math_rlvr_31/v1 [1], $B_u$ is a real-arithmetic bound; finite-precision IEEE-754 float32 execution introduces rounding of order $O(u \cdot 2^{-24})$ on master states. Furthermore, whenever an accumulation window $W$ contains zero active gradient steps (accum_has_grad = False), step_fp32_master_adamw(zero_grad_only=True) skips the update, holding $w_i$ fixed for that window (we observe a concrete example on Window 11 [Steps 51–55] of Seed 42 std_stabilized_s60_g8_fp32tail in Table 2 below, where all 5 prompts produced $0/8$ or $8/8$ outcomes).
Proposition 2 (Minimum Attainable $k = 9$ Target-Sign Step Count at $U = 12$)
Consider $U = 12$ real-arithmetic AdamW updates with parameters $(\beta_1, \beta_2) = (0.9, 0.999)$, $\eta = 1.8 \times 10^{-6}$, $\epsilon = 10^{-8}$, and $\rho = 1.44 \times 10^{-7}$. Let $\mathcal{U}_+ = \{u \in \{1, \dots, 12\} : \text{sign}(\Delta w_{i,u}) = \text{sign}(\delta_{i,12})\}$ denote the subset of updates that move in the direction of the final displacement $\delta_{i,12}$, and let $k = |\mathcal{U}_+| \in \{0, \dots, 12\}$. Then:
- ($k \le 8$ Target-Sign Updates Are Strictly Insufficient for Every Trajectory): If $k \le 8$, the remaining $12 - k \ge 4$ updates contribute non-positively to $|\delta_{i,12}|$, so $|\delta_{i,12}|$ is bounded above by the sum of the $8$ largest decayed per-step Cauchy–Schwarz ceilings ($u \in \{5, \dots, 12\}$): $$|\delta_{i,12}| \le \sum_{u \in \mathcal{U}_+} (1 - \rho)^{12-u} |\Delta w_{i,u}| \le \sum_{u=5}^{12} (1 - \rho)^{12-u} \Delta_{\max, u} = 1.488025 \times 10^{-5} < 2^{-16} = 1.525879 \times 10^{-5}$$ Hence, no trajectory with $k \le 8$ target-sign updates can ever cross the $2^{-16}$ threshold at Update 12 (nor in the $u \in \{10, 11, 12\}$ tail average).
- ($k = 9$ Is the Minimum Attainable Target-Sign Count Permitting a Crossing — Constructive Proof): Nine target-sign updates permit crossing $2^{-16}$ for an explicit trajectory (though they do not guarantee crossing for every 9-target-sign trajectory, since small gradients yield $|\delta_{i,12}| \ll 2^{-16}$). Because opposite-sign updates $u \notin \mathcal{U}_+$ can have arbitrarily small magnitude $|\Delta w_{i,u}| \ll \Delta_{\max, u}$ when the running first moment $m_{i,u}$ passes near zero while $v_{i,u}$ remains large, $k \ge 10$ same-sign updates are not required. Specifically, the fresh-state scalar gradient sequence: $$g_1 = g_2 = \dots = g_9 = -0.1, \qquad g_{10} = +0.5513315599, \qquad g_{11} = g_{12} = +10^{-6}$$ produces exactly $k = 9$ positive updates ($\Delta w_1 = \dots = \Delta w_9 = +1.79999982 \times 10^{-6} > 0$) and $3$ tiny negative updates ($\Delta w_{10} = -1.390011 \times 10^{-11}$, $\Delta w_{11} = -1.384123 \times 10^{-11}$, $\Delta w_{12} = -1.382793 \times 10^{-11}$), achieving both a final displacement and a last-three-update tail average strictly above $2^{-16}$: $$\delta_{12} = 1.61999405 \times 10^{-5} > 2^{-16} = 1.52587891 \times 10^{-5}, \qquad \bar{\delta}_{10:12} = \frac{1}{3}(\delta_{10} + \delta_{11} + \delta_{12}) = 1.61999566 \times 10^{-5} > 2^{-16}$$
Proof. Part (1) follows because $(1-\rho)^{12-u}\Delta_{\max,u}$ is strictly increasing in $u \in \{1, \dots, 12\}$ (from $1.799997 \times 10^{-6}$ at $u=1$ to $1.910198 \times 10^{-6}$ at $u=12$; see Table A1), so any $k$-element subset sum is bounded by the sum over $u \in \{13-k, \dots, 12\}$, which equals $1.488025 \times 10^{-5} < 2^{-16}$ at $k=8$. Part (2) is verified by direct evaluation of the AdamW recurrence in code/verify_vm_gmg_results.py: after 9 negative gradients $g_1=\dots=g_9=-0.1$ accumulate $\delta_9 = 1.61999890 \times 10^{-5} > 2^{-16}$ with $m_9 = -0.06125795$ and $v_9 = 8.964084 \times 10^{-5}$, the positive gradient pulse $g_{10} = +0.5513315599$ sets $m_{10} = 0.9 m_9 + 0.1 g_{10} = +1.000000 \times 10^{-6}$ while lifting $v_{10}$ to $3.935177 \times 10^{-4}$ (and $g_{11} = g_{12} = +10^{-6}$ keep $m_{11} = m_{12} = +1.000000 \times 10^{-6}$ with $v_{11} = 3.931242 \times 10^{-4}, v_{12} = 3.927310 \times 10^{-4}$), so $\Delta w_{10}, \Delta w_{11}, \Delta w_{12} \approx -1.39 \times 10^{-11}$, preserving $\delta_{12} = 1.61999405 \times 10^{-5} > 2^{-16}$ and $\bar{\delta}_{10:12} = 1.61999566 \times 10^{-5} > 2^{-16}$ with only $9$ positive updates. Moreover, because $B_{12} = 2.2101 \times 10^{-5} < \text{ULP}([2^{-8}, 2^{-7})) = 2^{-15} = 3.0518 \times 10^{-5}$, every interior coordinate in $[2^{-8}, 2^{-7})$ that crosses $2^{-16}$ at $U=12$ is rounded to $\pm 1\text{ ULP}$, incurring a $1.38\times\text{ to }2.00\times$ coordinate-level quantization overshoot. $\square$
4.1 Twelve-Update and Iso-Problem bfloat16 Binade Transition Census Across All 9 Sixty-Step Runs
Table 2 reports the empirical bfloat16 transition census across all four exponent binades for all seven 12-update ($U=12, W=5$, $60$-step) training runs and both two iso-problem ($U=6, W=10$, $60$-step) runs across Seeds 102 and 42.
| 60-Step Arm, Seed & Checkpoint State | Theoretical Ceiling $B_u$ | Total Changed bf16 Params (%) | < 2^-9 ($46.21\text{M}$) $\tau \le 2^{-18}$ | [2^-9, 2^-8) ($45.73\text{M}$) $\tau = 2^{-17}$ | [2^-8, 2^-7) ($87.30\text{M}$) $\tau = 2^{-16}$ | >= 2^-7 ($572.33\text{M}$) $r_{\text{in}}(2^{-7}) = 2^{-16}$ | $\cos(\Delta w_{\text{fp32}}, \Delta \tilde{w}_{\text{bf16}})$ |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| VM-GMG-GRPO $U=12, W=5$ (Seed 102) | | | | | | | |
| Updates 1–4 (Steps 5–20) | $\le 7.221 \times 10^{-6}$ | 7.28M -> 18.71M | 7.28M -> 18.64M | 0 -> 72,280 ($m=0$) | 0 | 0 | 0.1278 -> 0.2546 |
| Updates 5–8 (Steps 25–40) | $\le 1.456 \times 10^{-5}$ | 21.45M -> 27.81M | 19.36M -> 20.95M | 2.08M -> 6.82M (Int.) | 11,844 -> 39,650 ($m=0$) | 0 (All $u \le 8$) | 0.2647 -> 0.3125 |
| Update 9 (Step 45) | $1.642 \times 10^{-5}$ | 29,083,235 (3.87%) | 21,327,670 | 7,696,529 | 58,953 (Int. Open) | 83 ($m=0$ Open) | $0.3218$ |
| Update 12 (Step 60) | $2.210 \times 10^{-5}$ | 33,103,919 (4.40%) | 22,186,217 | 9,688,786 | 1,222,077 | 6,839 | $0.3518$ |
| Tail Avg $u_{10:12}$ (S102) | $2.020 \times 10^{-5}$ | 31,655,654 (4.21%) | 21,923,258 | 9,072,862 | 656,055 (0.75%) | 3,479 | $0.3327$ |
| VM-GMG-GRPO $U=12, W=5$ (Seed 42) | | | | | | | |
| Update 8 (Step 40) | $1.456 \times 10^{-5}$ | 28,454,485 (3.79%) | 21,206,566 | 7,205,861 | 42,058 ($m=0$) | 0 (All $u \le 8$) | $0.3152$ |
| Update 9 (Step 45) | $1.642 \times 10^{-5}$ | 29,731,428 (3.96%) | 21,564,112 | 8,106,095 | 61,133 (Int. Open) | 88 ($m=0$ Open) | $0.3243$ |
| Update 12 (Step 60) | $2.210 \times 10^{-5}$ | 33,832,154 (4.50%) | 22,388,320 | 10,122,783 | 1,313,884 | 7,167 | $0.3541$ |
| Tail Avg $u_{10:12}$ (S42) | $2.020 \times 10^{-5}$ | 32,351,273 (4.30%) | 22,134,637 | 9,509,688 | 703,303 (0.81%) | 3,645 | $0.3344$ |
| Surface GMG-GRPO $U=12, W=5$ (Seed 102) | | | | | | | |
| Update 8 / Update 9 | $1.456 / 1.642\text{e-}5$ | 28.51M / 29.76M | 21.26M / 21.61M | 7.20M / 8.09M | 42,561 / 62,154 | 0 / 80 (Open) | 0.3137 / 0.3228 |
| Tail Avg $u_{10:12}$ (S102) | $2.020 \times 10^{-5}$ | 32,201,890 (4.28%) | 22,137,751 | 9,398,451 | 662,117 (0.76%) | 3,571 | $0.3333$ |
| Surface GMG-GRPO $U=12, W=5$ (Seed 42) | | | | | | | |
| Update 8 / Update 9 | $1.456 / 1.642\text{e-}5$ | 28.21M / 29.50M | 21.14M / 21.50M | 7.03M / 7.94M | 41,069 / 62,372 | 0 / 95 (Open) | 0.3135 / 0.3229 |
| Tail Avg $u_{10:12}$ (S42) | $2.020 \times 10^{-5}$ | 32,137,780 (4.28%) | 22,076,885 | 9,341,402 | 715,616 (0.82%) | 3,877 | $0.3348$ |
| Ordinary Stabilized $U=12, W=5$ (Seed 102) | | | | | | | |
| Update 8 / Update 9 | $1.456 / 1.642\text{e-}5$ | 28.63M / 29.93M | 21.29M / 21.66M | 7.29M / 8.20M | 43,126 / 67,137 | 0 / 120 (Open) | 0.3145 / 0.3238 |
| Tail Avg $u_{10:12}$ (S102) | $2.020 \times 10^{-5}$ | 32,702,020 (4.35%) | 22,257,260 | 9,623,390 | 816,953 (0.94%) | 4,417 | $0.3363$ |
| Ordinary Stabilized $U=12, W=5$ (Seed 42, $U_{\text{eff}}=11$) | | | | | | | |
| Update 8 / Update 9 | $1.456 / 1.642\text{e-}5$ | 28.32M / 29.62M | 21.17M / 21.54M | 7.11M / 8.02M | 41,364 / 64,161 | 0 / 87 (Open) | 0.3142 / 0.3236 |
| Update 10 / Update 11 (Zero-Grad Window!) | $1.830 \times 10^{-5}$ | 30,948,380 (Unchanged) | 21,861,581 | 8,779,800 | 305,494 | 1,505 | $0.3346$ |
| Tail Avg $u_{10:12}$ (S42, $U_{\text{eff}}=11$) | $1.893 \times 10^{-5}$ | 31,392,570 (4.18%) | 21,955,305 | 8,998,202 | 436,812 (0.50%) | 2,251 | $0.3287$ |
| FB-VM-GMG-GRPO $U=12, W=5$ (Seed 102, 0 Rescues) | | | | | | | |
| Update 8 / Update 9 | $1.456 / 1.642\text{e-}5$ | 28.28M / 29.55M | 21.15M / 21.52M | 7.09M / 7.97M | 41,675 / 61,279 | 0 / 83 (Open) | 0.3132 / 0.3226 |
| Tail Avg $u_{10:12}$ (S102) | $2.020 \times 10^{-5}$ | 32,120,425 (4.27%) | 22,106,882 | 9,331,872 | 677,994 (0.78%) | 3,677 | $0.3335$ |
| Iso-Problem VM-GMG-GRPO $U=6, W=10$ (Seed 102) | | | | | | | |
| Tail Avg $u_{4:6}$ (Step 60, S102) | $9.044 \times 10^{-6}$ | 22,528,127 (3.00%) | 20,993,503 | 1,525,972 | 8,652 ($m=0$ only) | 0 (Trapped) | $0.2557$ |
| Iso-Problem VM-GMG-GRPO $U=6, W=10$ (Seed 42) | | | | | | | |
| Tail Avg $u_{4:6}$ (Step 60, S42) | $9.044 \times 10^{-6}$ | 21,752,026 (2.89%) | 19,784,606 | 1,956,453 | 10,967 ($m=0$ only) | 0 (Trapped) | $0.2609$ |
Table 2 establishes three empirical properties of 12-update vs. 6-update optimization on Qwen3-0.6B-Base:
- Universal Update-9 Threshold Crossing Across All Seven 12-Update Runs: Across all seven $U=12, W=5$ runs, the
|w_0| >= 2^-7binade ($572.33\text{M}$ parameters) records0transitions at every single update $u \in \{1, \dots, 8\}$, and empirically unlocks $80\text{ to }120$ transitions at Update 9 (step 45), consistent with the necessary real-arithmetic envelope crossing $B_8 = 1.4557 \times 10^{-5} < 2^{-16} < B_9 = 1.6420 \times 10^{-5}$. - In-Training Demonstration of a Zero-Gradient Window (
StabilizedSeed 42 Window 11): On Seed 42std_stabilized_s60_g8_fp32tail, Steps 51–55 (Window 11) produced outcomes $(0/8, 8/8, 0/8, 8/8, 8/8)$, yielding zero policy gradient across the entire 5-step window (accum_has_grad = False). Consequently, Update 11 was skipped (30,948,380changed parameters, identical to Update 10), reducing effective optimizer steps to $U_{\text{eff}} = 11$ and reducing tail transitions in[2^-8, 2^-7)to $436,812$ (vs. $656\text{K}\text{--}817\text{K}$ across the six $U_{\text{eff}} = 12$ runs)—directly illustrating why $B_u$ is an upper envelope rather than an unconditional transition count. - Iso-Problem $(U=6, W=10)$ Dead-Zone Trapping: Training on the exact same 60 problems with window $W=10$ ($U=6$ updates) leaves
|w_0| >= 2^-7at strictly0transitions and[2^-8, 2^-7)at8,652(S102) and10,967(S42) $m=0$ boundary transitions through Step 60.
5. Experimental Protocol, Selection Chronology, and Two-Seed 6-Panel Evaluation (56,980 Item Evaluations)
5.1 Evaluation Protocol, Panel Composition, and Exact build_fresh_zpd_stream Provenance
We document the exact training and evaluation configuration executed by code/run_baseline.py, code/train_thinker_rlvr.py, and code/run_tessera_tree_audit.py:
- Parent Checkpoint & Training Hyperparameters: All runs initialize from the two-seed
parent_step100Qwen3-0.6B-Basecheckpoint (code/results/tessera_sym_alg50_seed{102,42}/model_tail_avg.pth;2,490 / 5,180,48.07%pooled 6-Panel;Within-MATH = 1,686 / 3,700,45.57%;MATH-500 = 466 / 1,000,46.60%) trained bycode/train_tessera.py(math_rlvr_22/v4[2],math_rlvr_31/v1[1]). On-policy continuation uses rollout group size $G = 8$ ($4$ fast rollouts atmax_new_tokens = 512, min_tokens = 16+ $4$ slow rollouts atmax_new_tokens = 768, min_tokens = 24, or 2–4–8 branched rollouts capped at512/768in--mode tree), sampling temperature $T = 0.80$, top-$p = 0.90$,float32master-weight AdamW ($\eta = 1.8 \times 10^{-6}$, $(\beta_1, \beta_2) = (0.9, 0.999)$, $\epsilon = 10^{-8}$, anchored weight decay $\lambda_{\text{wd}} = 0.08$, fixed loss divisor $G \cdot 256 \cdot W$), and last-three-updatefloat32master-weight tail averaging (fp32tail: Updates $4, 5, 6$ for $U=6$ runs and Updates $10, 11, 12$ for $U=12$ runs) cast tobfloat16for inference. - Greedy 6-Panel Evaluator (
max_tokens = 2,048,batch_size = 50,T = 0.0): Every checkpoint is evaluated deterministically viaevaluate_benchmark()incode/run_baseline.pywith greedy decoding (temperature = 0.0),batch_size = 50, and maximum evaluation generation lengthmax_tokens = 2,048. Across all $22$ evaluated checkpoints (56,980item evaluations), the empirical per-checkpoint mean response length ranges from561.2to618.3tokens (with long derivations reaching the2,048-token ceiling; see Table 4). - Exact Composition of All Six Panels (
2,590Unique Problems per Seed):val350(data/math_splits/val350.json, $350$ Competition MATH problems).math500(data/math_splits/math500.json, $500$ standard held-out MATH-500 benchmark problems).sec500(data/math_splits/secondary500.json, $500$ Competition MATH problems).conf500(data/math_splits/confirm500.json, $500$ Competition MATH problems).fresh500(data/math_splits/fresh_math500.json, $500$ held-out MATH problems).ext240(data/math_splits/gsm8k_amc_bench240.json, $240$ external out-of-distribution problems consisting of $200$ GSM8K problems [gsm8k_0–gsm8k_199] and $40$ AMC23 problems [amc23_0–amc23_39]). We report four pre-specified aggregates: Within-MATH (val350 + math500 + sec500 + conf500, $1,850$ items/seed, $3,700$ pooled), MATH-500 ($500$ items/seed, $1,000$ pooled), Held-Out 1,240 (math500 + fresh500 + ext240, $1,240$ items/seed, $2,480$ pooled), and 6-Panel Total ($2,590$ items/seed, $5,180$ pooled).
- Exact Training Stream Recipe (
build_fresh_zpd_stream) and Verbatim Full-Text Export: Every training run incode/run_tessera_tree_audit.pycallsbuild_fresh_zpd_stream(seed, WORKSPACE_CODE, max_steps=100, decontaminate_panels=True)incode/train_thinker_rlvr.py. This function initializesrng = random.Random(seed + 202610), loadsdata/math_splits/train_pool_11150.json($11,150$ Competition MATH problems) anddata/math_splits/train_stencil_600.json(frontier stencil classifications), filters out any problem overlapping the 6 evaluation panels, partitions problems intofrontier($\text{level} \ge 3$ or non-trivial stencil) andmedium_hard($\text{level} \ge 2$), round-robins across subjects within each bucket, and interleaves $2$frontier+ $3$medium_hardproblems per 5-step window ($40\%$ frontier, $60\%$ medium-hard). We export the complete ordered 60-step training streams for Seed 102 and Seed 42—including the full verbatimproblemtext, ground-truthanswer,subject,level, andsha1hash for all $60$ steps per seed—inresults/ordered_60step_streams_and_provenance.json, andcode/verify_vm_gmg_results.pyverifies both the exact SHA-1 match of every verbatim prompt and0 / 60(0.0%) overlap against all $2,590$ evaluation panel hashes.
5.2 Experimental Chronology and Multiplicity Disclosure
We disclose the chronological sequence of all ten experimental arms evaluated in this study:
- 30-Step ($U=6, W=5$) Prefix-Tree & Donor-Shielding Audit (Two Seeds: 102, 42): Starting from the two-seed 30-step baselines inherited from
math_rlvr_31/v1[1] (parent_step100,std_stabilized_g8_fp32tail,std_gmg_grpo_g8_fp32tail, and 2–4–8 branchedstd_tessera_tree_g8_fp32tail), we evaluated the 8-rootPrefixTriehybrid counterfactual ablation with Grant-Margin Gating (std_tessera_trie8_gmg_g8_fp32tail,mode="hybrid_counterfactual", $\lambda=0.5$), our asymmetric donor-equality shielding heuristic (std_vm_gmg_grpo_g8_fp32tail), and a single-seed exploratory combined intervention on Seed 42 (std_mc_vm_gmg_grpo_g8_fp32tail), retaining all arms in the report. - 60-Step ($U=12, W=5$ vs. $U=6, W=10$) Horizon & Binade Envelope Audit (Two Seeds: 102, 42): Motivated by Proposition 1's prediction that $U=12$ crosses the $2^{-16}$ threshold at Update 9 whereas $U=6$ cannot, we ran 60-step
std_vm_gmg_grpo_s60_g8_fp32tail($U=12, W=5$), resource-matched 60-step ordinary stabilized GRPO (std_stabilized_s60_g8_fp32tail, $U=12, W=5$), resource-matched 60-step surface LCPGMG-GRPO(std_gmg_grpo_s60_g8_fp32tail, $U=12, W=5$), iso-problem 60-stepVM-GMG-GRPOwith $W=10$ (std_vm_gmg_grpo_s60_w10_g8_fp32tail, $U=6, W=10$), and a single-seed 60-step on-the-spot0/8self-revision diagnostic on Seed 102 (std_fb_vm_gmg_grpo_s60_g8_fp32tail, $U=12, W=5$). Because multiple arms and horizons are compared across the same benchmark panels, Table 5 reports unadjusted two-sided $p$-values alongside 95% question-clustered confidence intervals (which condition on the two trained seeds) as descriptive uncertainty summaries.
5.3 In-Training Telemetry and Executed Loss-Coefficient Audit
Table 3A reports the in-training step telemetry recorded in step_tree_telemetry across all training runs. To avoid any conflation between trie topology and zeroed loss coefficients, we report shared_token_fraction (the step-mean fraction of trie token visits occurring at shared nodes with $\text{visit\_count} > 1$, alongside the token-pooled ratio $\sum \text{shared\_tokens} / \sum \text{total\_tokens}$ in parentheses for the tree arms) separately from shielded_neg_token_fraction (the fraction of EOS-completed losing-rollout tokens masked by $t < t_{\text{lcp}}$ or $t < t_{\text{shield}}$ in gmg / vm_gmg).
| Training Arm (fp32tail) | Seed | Steps ($U, W$) | Active Steps (0 < r_sum < 8) | Homogeneous 0/8 / 8/8 Steps | Donor-Equality Triggered Steps (Rollouts) | Mean Active $t_{\text{lcp}}$ (Tokens) | Mean Active $t_{\text{shield}}$ (Tokens) | Step-Mean Trie shared_token_fraction (Pooled) | Completed-Losing Shielded Token Frac (%) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Tessera-Tree (2–4–8 Hybrid $\lambda=0.5$ tree) [1] | 102 | $30$ ($6, 5$) | $16 / 30$ (53.3%) | $10$ / $4$ | 0 (0) | — | — | 76.14% (74.82%) | — (Tree Hybrid) |
| Tessera-Tree (2–4–8 Hybrid $\lambda=0.5$ tree) [1] | 42 | $30$ ($6, 5$) | $20 / 30$ (66.7%) | $5$ / $5$ | 0 (0) | — | — | 83.15% (81.13%) | — (Tree Hybrid) |
| Tessera-Trie8-GMG (8-Root Hybrid $\lambda=0.5$ trie8_gmg) | 102 | $30$ ($6, 5$) | $20 / 30$ (66.7%) | $8$ / $2$ | 0 (0) | — | — | 9.54% (8.56%) | — (Trie8 Hybrid) |
| Tessera-Trie8-GMG (8-Root Hybrid $\lambda=0.5$ trie8_gmg) | 42 | $30$ ($6, 5$) | $23 / 30$ (76.7%) | $7$ / $0$ | 0 (0) | — | — | 11.06% (10.86%) | — (Trie8 Hybrid) |
| VM-GMG-GRPO ($30$-Step Donor Shield) | 102 | $30$ ($6, 5$) | $22 / 30$ (73.3%) | $7$ / $1$ | $10 / 22$ ($20$ rollouts) | $22.71$ | $89.64$ | 10.15% | 18.62% |
| VM-GMG-GRPO ($30$-Step Donor Shield) | 42 | $30$ ($6, 5$) | $26 / 30$ (86.7%) | $4$ / $0$ | $14 / 26$ ($26$ rollouts) | $24.38$ | $78.77$ | 11.12% | 19.39% |
| MC-VM-GMG-GRPO ($30$-Step Combined) | 42 | $30$ ($6, 5$) | $26 / 30$ (86.7%) | $4$ / $0$ | $13 / 26$ ($21$ rollouts) | $29.85$ | $70.70$ | 10.84% | 19.20% |
| Ordinary Stabilized ($60$-Step) | 102 | $60$ ($12, 5$) | $37 / 60$ (61.7%) | $19$ / $4$ | 0 (0) | $0.00$ | $0.00$ | 9.74% (9.02%) | 0.00% |
| Ordinary Stabilized ($60$-Step) | 42 | $60$ ($12, 5$) | $45 / 60$ (75.0%) | $11$ / $4$ | 0 (0) | $0.00$ | $0.00$ | 10.78% (10.38%) | 0.00% |
| Surface GMG-GRPO ($60$-Step) | 102 | $60$ ($12, 5$) | $40 / 60$ (66.7%) | $17$ / $3$ | 0 (0) | $24.69$ | $24.69$ | 9.91% | 6.02% |
| Surface GMG-GRPO ($60$-Step) | 42 | $60$ ($12, 5$) | $41 / 60$ (68.3%) | $15$ / $4$ | 0 (0) | $26.55$ | $26.55$ | 10.64% | 6.79% |
| Iso-Problem VM-GMG-GRPO ($W=10$) | 102 | $60$ ($6, 10$) | $34 / 60$ (56.7%) | $19$ / $7$ | $13 / 34$ ($23$ rollouts) | $32.86$ | $72.34$ | 9.71% | 15.07% |
| Iso-Problem VM-GMG-GRPO ($W=10$) | 42 | $60$ ($6, 10$) | $40 / 60$ (66.7%) | $16$ / $4$ | $17 / 40$ ($33$ rollouts) | $17.88$ | $52.45$ | 10.95% | 13.64% |
| VM-GMG-GRPO ($60$-Step Donor Shield) | 102 | $60$ ($12, 5$) | $40 / 60$ (66.7%) | $16$ / $4$ | $19 / 40$ ($41$ rollouts) | $26.07$ | $78.77$ | 9.82% (9.25%) | 18.92% |
| VM-GMG-GRPO ($60$-Step Donor Shield) | 42 | $60$ ($12, 5$) | $37 / 60$ (61.7%) | $21$ / $2$ | $20 / 37$ ($34$ rollouts) | $32.05$ | $82.41$ | 11.40% (10.81%) | 20.87% |
| FB-VM-GMG-GRPO ($60$-Step, 0/16 Rescues) | 102 | $60$ ($12, 5$) | $37 / 60$ (61.7%) | $16$ / $7$ | $17 / 37$ ($35$ rollouts) | $26.51$ | $76.25$ | 9.88% | 17.22% |
Direct Measurement of Executed Loss Coefficients (code/measure_executed_loss_coefficients.py & Table 3B):
Because shared_token_fraction in PrefixTrie (76.14% step-mean / 74.82% token-pooled on S102 and 83.15% step-mean / 81.13% token-pooled on S42 in Tessera-Tree, vs. 9.54%–11.06% in 8-root Trie8-GMG) counts shared trie visits ($\text{visit\_count} > 1$) rather than zeroed loss positions, we replayed the exact on-policy Steps 1–5 from parent_step100 (results/executed_loss_coefficient_audit.json, verified in code/verify_vm_gmg_results.py) to measure the exact token fractions where each component vanishes or is masked across all active-step tokens:
| Rollout Topology & Seed (Steps 1–5 Active Prompts) | Total Active Tokens | Pooled Trie Shared Tokens ($\text{visit} > 1$) | Local Component Zero ($|V(u)-V(u^*)| = 0$, Theorem 2) | Raw Hybrid Advantage Zero ($|A^{\text{trie}}_t| < 10^{-6}$) | Winning-Rollout Sign-Reversed Tokens ($A^{\text{trie}}_t < 0$ on $R_g=1$) | Non-EOS Truncation Zeroed Tokens (trie8_gmg / tree / Flat) | Completed-Losing Prefix Shielded (gmg $t_{\text{lcp}}$ / vm_gmg $t_{\text{shield}}$) |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| 8-Root Rollouts (S102, 5/5 active) | $18,825$ | 7.38% ($1,390$) | 45.24% ($8,517$) | 0.47% ($88$) | 1.76% ($102 / 5,788$) | 40.26% (trie8) / 40.80% (Flat) | 1.33% ($250$) / 4.37% ($823$) |
| 8-Root Rollouts (S42, 4/5 active) | $12,041$ | 7.25% ($873$) | 48.05% ($5,786$) | 0.86% ($104$) | 1.39% ($56 / 4,018$) | 33.11% (trie8) / 34.02% (Flat) | 1.81% ($218$) / 3.97% ($478$) |
| 2–4–8 Tree Rollouts (S102, 2/5 active) | $7,964$ | 73.83% ($5,880$) | 43.71% ($3,481$) | 19.69% ($1,568$) | 1.99% ($65 / 3,264$) | 8.04% (tree leaf slice $t \ge m_{\text{len}}$) | 11.93% ($950$) / 13.61% ($1,084$) |
| 2–4–8 Tree Rollouts (S42, 3/5 active) | $12,203$ | 75.54% ($9,218$) | 59.38% ($7,246$) | 3.02% ($368$) | 0.00% ($0 / 3,386$) | 8.87% (tree leaf slice $t \ge m_{\text{len}}$) | 4.06% ($496$) / 4.57% ($558$) |
Table 3B establishes four exact properties of the executed loss coefficients:
- Distinguishing Shared Trie Visits, Local-Component Collapse, and Hybrid Zeros: In 2–4–8
Tessera-Tree,73.83%–75.54%of active tokens lie on shared trie prefixes ($\text{visit\_count} > 1$) because 2 root prefixes ($128$ tokens) and 4 mid-prefixes ($+192$ tokens) are shared by construction. Meanwhile, across both 8-root and 2–4–8 topologies, the local counterfactual component $V(u) - V(u^*)$ vanishes on43.71%–59.38%of all active tokens (whenever a node has homogeneous descendants or matches its branching ancestor), which halves (0.5x) those tokens' executed hybrid advantage to $\frac{0.5(c - \bar{R})}{\sigma_R + 10^{-4}}$ (Theorem 2). By contrast, the raw hybrid advantage $A^{\text{trie}}_t$ itself is zero only when $2V(u) = \bar{R} + V(u^*)$ (0.47%–0.86%of tokens in 8-roottrie8_gmgand3.02%–19.69%in 2–4–8tree). - Winning-Prefix Sign Reversal in
trie8_gmg: On 8-root active steps,1.39%–1.76%of all winning-rollout tokens ($102$ tokens on S102 Steps 3–4; $56$ tokens on S42 Steps 1, 4, 5) satisfy $2V(u) < \bar{R} + V(u^*)$, flipping their executed hybrid advantage strictly negative ($A^{\text{trie}}_t < 0$) and switching their Grant-Margin gate from $w_{\text{pos}}$ to $w_{\text{neg}}$ (Theorem 1(2)). - Asymmetric Non-EOS Truncation Masking (
trie8_gmgvs.treevs. Flat Arms): On 8-root rollouts,34.02%–40.80%of all active tokens belong to non-EOS truncated losing rollouts ($R_g = 0, y^{(g)}_{-1} \neq \text{EOS}$). Whereas flat arms (stabilized,gmg,vm_gmg) skip those entire rollouts (40.80%on S102),trie8_gmgclamps only negative entries (np.where(adv_vec < 0, 0, adv_vec), zeroing40.26%of tokens while retaining positive hybrid advantages on0.54%–0.91%of tokens where a non-EOS truncated rollout traverses a winning-majority shared prefix!). Conversely, in 2–4–8tree, non-EOS clamping is applied only to the leaf slice $t \ge m_{\text{len}}$ (8.04%–8.87%of tokens), leaving negative hybrid advantages active across shared root/mid prefixes ($t < m_{\text{len}}$) of truncated leaves. - Tripling of Completed-Losing Prefix Shielding under
VM-GMG-GRPO: Among EOS-completed losing rollouts,VM-GMG-GRPOshields18.92%–20.87%of tokens at 60 steps ($t_{\text{shield}} \approx 78.8\text{--}82.4$ tokens, or3.97%–4.37%of all active tokens in Steps 1–5), compared to6.02%–6.79%under surface LCPGMG-GRPO($t_{\text{lcp}} \approx 24.7\text{--}26.6$ tokens,1.33%–1.81%of all active tokens).
5.4 Complete Two-Seed 6-Panel Results Across All 22 Checkpoints
Table 4 reports the complete per-seed and pooled evaluation results across all $22$ checkpoints (56,980 item-level evaluations, verified with 0 regrading mismatches by code/verify_vm_gmg_results.py).
| Arm / Checkpoint (fp32tail, max_tokens=2048) | Seed / Scope | val350 ($350$) | math500 ($500$) | sec500 ($500$) | conf500 ($500$) | fresh500 ($500$) | ext240 ($240$) | Within-MATH ($1,850$ / $3,700$) | Held-Out 1240 ($1,240$ / $2,480$) | 6-Panel Total ($2,590$ / $5,180$) | Mean Eval Tokens |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| parent_step100 (Initialization) [1, 2] | S102 | $159$ | $233$ | $227$ | $229$ | $257$ | $149$ | $848$ (45.84%) | $639$ (51.53%) | $1,254$ (48.42%) | $597.6$ |
| | S42 | $159$ | $233$ | $235$ | $211$ | $254$ | $144$ | $838$ (45.30%) | $631$ (50.89%) | $1,236$ (47.72%) | $561.2$ |
| | Pooled | $318$ | $466$ (46.60%) | $462$ | $440$ | $511$ | $293$ | $1,686$ (45.57%) | $1,270$ (51.21%) | $2,490$ (48.07%) | $579.4$ |
| 30-Step ($U=6, W=5$) Arms: | | | | | | | | | | | |
| std_stabilized_g8_fp32tail [1] | S102 | $163$ | $244$ | $238$ | $224$ | $251$ | $148$ | $869$ (46.97%) | $643$ (51.85%) | $1,268$ (48.96%) | $594.4$ |
| (Ordinary Stabilized GRPO, $30$s) | S42 | $163$ | $241$ | $235$ | $212$ | $255$ | $144$ | $851$ (46.00%) | $640$ (51.61%) | $1,250$ (48.26%) | $564.0$ |
| | Pooled | $326$ | $485$ (48.50%) | $473$ | $436$ | $506$ | $292$ | $1,720$ (46.49%) | $1,283$ (51.73%) | $2,518$ (48.61%) | $579.2$ |
| std_gmg_grpo_g8_fp32tail [1, 2] | S102 | $168$ | $237$ | $230$ | $218$ | $259$ | $148$ | $853$ (46.11%) | $644$ (51.94%) | $1,260$ (48.65%) | $598.2$ |
| (Surface LCP GMG-GRPO, $30$s) | S42 | $166$ | $241$ | $236$ | $215$ | $262$ | $140$ | $858$ (46.38%) | $643$ (51.85%) | $1,260$ (48.65%) | $570.3$ |
| | Pooled | $334$ | $478$ (47.80%) | $466$ | $433$ | $521$ | $288$ | $1,711$ (46.24%) | $1,287$ (51.90%) | $2,520$ (48.65%) | $584.2$ |
| std_tessera_tree_g8_fp32tail [1] | S102 | $163$ | $228$ | $227$ | $231$ | $260$ | $150$ | $849$ (45.89%) | $638$ (51.45%) | $1,259$ (48.61%) | $597.2$ |
| (2–4–8 Hybrid $\lambda=0.5$ tree, $30$s) | S42 | $160$ | $236$ | $228$ | $212$ | $259$ | $143$ | $836$ (45.19%) | $638$ (51.45%) | $1,238$ (47.80%) | $572.0$ |
| | Pooled | $323$ | $464$ (46.40%) | $455$ | $443$ | $519$ | $293$ | $1,685$ (45.54%) | $1,276$ (51.45%) | $2,497$ (48.20%) | $584.6$ |
| std_tessera_trie8_gmg_g8_fp32tail | S102 | $158$ | $228$ | $231$ | $224$ | $264$ | $149$ | $841$ (45.46%) | $641$ (51.69%) | $1,254$ (48.42%) | $606.5$ |
| (8-Root Hybrid $\lambda=0.5$ trie8_gmg, $30$s) | S42 | $153$ | $240$ | $236$ | $207$ | $245$ | $143$ | $836$ (45.19%) | $628$ (50.65%) | $1,224$ (47.26%) | $578.7$ |
| | Pooled | $311$ | $468$ (46.80%) | $467$ | $431$ | $509$ | $292$ | $1,677$ (45.32%) | $1,269$ (51.17%) | $2,478$ (47.84%) | $592.6$ |
| std_vm_gmg_grpo_g8_fp32tail | S102 | $160$ | $239$ | $235$ | $223$ | $266$ | $154$ | $857$ (46.32%) | $659$ (53.15%) | $1,277$ (49.31%) | $604.4$ |
| (Donor-Equality VM-GMG, $30$s) | S42 | $159$ | $246$ | $233$ | $200$ | $257$ | $144$ | $838$ (45.30%) | $647$ (52.18%) | $1,239$ (47.84%) | $576.0$ |
| | Pooled | $319$ | $485$ (48.50%) | $468$ | $423$ | $523$ | $298$ | $1,695$ (45.81%) | $1,306$ (52.66%) | $2,516$ (48.57%) | $590.2$ |
| std_mc_vm_gmg_grpo_g8_fp32tail | S42 | $152$ | $238$ | $230$ | $205$ | $251$ | $143$ | $825$ (44.59%) | $632$ (50.97%) | $1,219$ (47.07%) | $567.8$ |
| 60-Step ($U=12, W=5$ & $U=6, W=10$) Arms: | | | | | | | | | | | |
| std_stabilized_s60_g8_fp32tail | S102 | $161$ | $244$ | $243$ | $223$ | $265$ | $149$ | $871$ (47.08%) | $658$ (53.06%) | $1,285$ (49.61%) | $586.8$ |
| (Ordinary Stabilized GRPO, $60$s, $U=12$) | S42 | $167$ | $239$ | $233$ | $214$ | $247$ | $141$ | $853$ (46.11%) | $627$ (50.56%) | $1,241$ (47.92%) | $578.6$ |
| | Pooled | $328$ | $483$ (48.30%) | $476$ | $437$ | $512$ | $290$ | $1,724$ (46.59%) | $1,285$ (51.81%) | $2,526$ (48.76%) | $582.7$ |
| std_gmg_grpo_s60_g8_fp32tail | S102 | $162$ | $237$ | $232$ | $222$ | $263$ | $141$ | $853$ (46.11%) | $641$ (51.69%) | $1,257$ (48.53%) | $577.3$ |
| (Surface LCP GMG-GRPO, $60$s, $U=12$) | S42 | $158$ | $241$ | $230$ | $214$ | $247$ | $147$ | $843$ (45.57%) | $635$ (51.21%) | $1,237$ (47.76%) | $577.1$ |
| | Pooled | $320$ | $478$ (47.80%) | $462$ | $436$ | $510$ | $288$ | $1,696$ (45.84%) | $1,276$ (51.45%) | $2,494$ (48.15%) | $577.2$ |
| std_vm_gmg_grpo_s60_w10_g8_fp32tail | S102 | $161$ | $236$ | $232$ | $220$ | $266$ | $146$ | $849$ (45.89%) | $648$ (52.26%) | $1,261$ (48.69%) | $600.1$ |
| (Iso-Problem VM-GMG, $60$s, $U=6, W=10$) | S42 | $165$ | $247$ | $231$ | $208$ | $255$ | $139$ | $851$ (46.00%) | $641$ (51.69%) | $1,245$ (48.07%) | $586.6$ |
| | Pooled | $326$ | $483$ (48.30%) | $463$ | $428$ | $521$ | $285$ | $1,700$ (45.95%) | $1,289$ (51.98%) | $2,506$ (48.38%) | $593.4$ |
| std_vm_gmg_grpo_s60_g8_fp32tail | S102 | $168$ | $249$ | $244$ | $220$ | $258$ | $153$ | $881$ (47.62%) | $660$ (53.23%) | $1,292$ (49.88%) | $603.8$ |
| (Donor-Equality VM-GMG, $60$s, $U=12$) | S42 | $165$ | $241$ | $227$ | $208$ | $251$ | $143$ | $841$ (45.46%) | $635$ (51.21%) | $1,235$ (47.68%) | $578.2$ |
| | Pooled | $333$ | $490$ (49.00%) | $471$ | $428$ | $509$ | $296$ | $1,722$ (46.54%) | $1,295$ (52.22%) | $2,527$ (48.78%) | $591.0$ |
| std_fb_vm_gmg_grpo_s60_g8_fp32tail ($0/16$ Rescues) | S102 | $156$ | $240$ | $233$ | $223$ | $255$ | $151$ | $852$ (46.05%) | $646$ (52.10%) | $1,258$ (48.57%) | $618.3$ |
5.5 Paired Question-Clustered Confidence Intervals and Exact Hypothesis Tests
Because each question $q \in \{1, \dots, N_Q\}$ is evaluated across two training seeds ($s \in \{102, 42\}$), we compute both the exact two-sided McNemar test ($p_{\text{mcn}}$) on pooled discordant item-seed pairs $(n_{10}, n_{01})$ and the question-clustered paired analysis on per-question cluster sums $d_q = (y_{A,102,q} - y_{B,102,q}) + (y_{A,42,q} - y_{B,42,q}) \in \{-2, -1, 0, +1, +2\}$, reporting the 95% cluster-robust confidence interval $\hat{\Delta}_{\text{pp}} \pm 1.96 \cdot \text{SE}_{\text{cluster}}$ and the two-sided cluster sign-flip randomization $p$-value ($p_{\text{cluster}}$, $200,000$ draws). Note that these question-clustered intervals condition on the two trained checkpoints per arm and measure item-sampling uncertainty across the $2,590$ benchmark questions rather than variance across future random training seeds.
| Paired Contrast ($A$ vs. $B$, Two Seeds Pooled) | Aggregate Scope | Per-Seed $\Delta$ (S102, S42) | Pooled $\Delta$ Items ($\Delta\text{ pp}$) | 95% Question-Clustered CI ($\text{pp}$) | Discordant Pairs $(n_{10}, n_{01})$ | Exact McNemar $p_{\text{mcn}}$ | Cluster Sign-Flip $p_{\text{cluster}}$ |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| 1. Trie8-GMG ($30$s) vs. Surface GMG-GRPO ($30$s) | 6-Panel ($5,180$) | $(-6,\ -36)$ | $-42$ (-0.81 pp) | $[-1.63,\ +0.01]$ | $(220,\ 262)$ | $0.0617$ | $0.0595$ |
| (8-root hybrid $\lambda=0.5$ vs. surface LCP) | Within-MATH ($3,700$) | $(-12,\ -22)$ | $-34$ (-0.92 pp) | $[-1.92,\ +0.08]$ | $(162,\ 196)$ | $0.0810$ | $0.0806$ |
| | MATH-500 ($1,000$) | $(-9,\ -1)$ | $-10$ (-1.00 pp) | $[-2.94,\ +0.94]$ | $(42,\ 52)$ | $0.3533$ | $0.3632$ |
| 2. Trie8-GMG ($30$s) vs. Stabilized ($30$s) | 6-Panel ($5,180$) | $(-14,\ -26)$ | $-40$ (-0.77 pp) | $[-1.59,\ +0.05]$ | $(217,\ 257)$ | $0.0731$ | $0.0705$ |
| (8-root hybrid $\lambda=0.5$ vs. stabilized GRPO) | Within-MATH ($3,700$) | $(-28,\ -15)$ | $-43$ (-1.16 pp) | $[-2.15,\ -0.18]$ | $(151,\ 194)$ | $0.0236$ | $0.0242$ |
| | MATH-500 ($1,000$) | $(-16,\ -1)$ | $-17$ (-1.70 pp) | $[-3.63,\ +0.23]$ | $(36,\ 53)$ | $0.0893$ | $0.1044$ |
| 3. VM-GMG-GRPO ($30$s) vs. Trie8-GMG ($30$s) | 6-Panel ($5,180$) | $(+23,\ +15)$ | $+38$ (+0.73 pp) | $[-0.08,\ +1.55]$ | $(250,\ 212)$ | $0.0851$ | $0.0851$ |
| (Donor-equality shielding vs. 8-root hybrid) | Within-MATH ($3,700$) | $(+16,\ +2)$ | $+18$ (+0.49 pp) | $[-0.50,\ +1.47]$ | $(182,\ 164)$ | $0.3608$ | $0.3628$ |
| | MATH-500 ($1,000$) | $(+11,\ +6)$ | $+17$ (+1.70 pp) | $[-0.25,\ +3.65]$ | $(53,\ 36)$ | $0.0893$ | $0.1081$ |
| 4. VM-GMG-GRPO ($60$s) vs. Stabilized ($60$s) | 6-Panel ($5,180$) | $(+7,\ -6)$ | $+1$ (+0.02 pp) | $[-0.92,\ +0.96]$ | $(309,\ 308)$ | $1.0000$ | $1.0000$ |
| (Matched 60-step / 12-update comparison) | Within-MATH ($3,700$) | $(+10,\ -12)$ | $-2$ (-0.05 pp) | $[-1.17,\ +1.06]$ | $(221,\ 223)$ | $0.9622$ | $0.9619$ |
| | MATH-500 ($1,000$) | $(+5,\ +2)$ | $+7$ (+0.70 pp) | $[-1.42,\ +2.82]$ | $(59,\ 52)$ | $0.5692$ | $0.5801$ |
| 5. VM-GMG-GRPO ($60$s) vs. Surface GMG-GRPO ($60$s) | 6-Panel ($5,180$) | $(+35,\ -2)$ | $+33$ (+0.64 pp) | $[-0.27,\ +1.54]$ | $(306,\ 273)$ | $0.1835$ | $0.1816$ |
| (Donor-equality vs. surface LCP at 60 steps) | Within-MATH ($3,700$) | $(+28,\ -2)$ | $+26$ (+0.70 pp) | $[-0.37,\ +1.78]$ | $(223,\ 197)$ | $0.2225$ | $0.2161$ |
| | MATH-500 ($1,000$) | $(+12,\ +0)$ | $+12$ (+1.20 pp) | $[-0.87,\ +3.27]$ | $(59,\ 47)$ | $0.2853$ | $0.3005$ |
| 6. VM-GMG ($60$s, $U=12$) vs. Iso-Problem ($U=6, W=10$) | 6-Panel ($5,180$) | $(+31,\ -10)$ | $+21$ (+0.41 pp) | $[-0.46,\ +1.27]$ | $(281,\ 260)$ | $0.3899$ | $0.3822$ |
| (Update frequency $U=12$ vs. $U=6$ on same 60 prompts) | Within-MATH ($3,700$) | $(+32,\ -10)$ | $+22$ (+0.59 pp) | $[-0.44,\ +1.62]$ | $(207,\ 185)$ | $0.2888$ | $0.2797$ |
| | MATH-500 ($1,000$) | $(+13,\ -6)$ | $+7$ (+0.70 pp) | $[-1.13,\ +2.53]$ | $(50,\ 43)$ | $0.5341$ | $0.5215$ |
| 7. VM-GMG-GRPO ($60$s) vs. parent_step100 | 6-Panel ($5,180$) | $(+38,\ -1)$ | $+37$ (+0.71 pp) | $[-0.15,\ +1.58]$ | $(290,\ 253)$ | $0.1223$ | $0.1160$ |
| (Net gain over Step-100 initialization) | Within-MATH ($3,700$) | $(+33,\ +3)$ | $+36$ (+0.97 pp) | $[-0.07,\ +2.02]$ | $(217,\ 181)$ | $0.0792$ | $0.0760$ |
| | MATH-500 ($1,000$) | $(+16,\ +8)$ | $+24$ (+2.40 pp) | $[+0.51,\ +4.29]$ | $(62,\ 38)$ | $0.0210$ | $0.0170$ |
| 8. Stabilized ($60$s) vs. parent_step100 | 6-Panel ($5,180$) | $(+31,\ +5)$ | $+36$ (+0.69 pp) | $[-0.27,\ +1.66]$ | $(344,\ 308)$ | $0.1704$ | $0.1682$ |
| (Net gain of ordinary stabilized GRPO at 60 steps) | Within-MATH ($3,700$) | $(+23,\ +15)$ | $+38$ (+1.03 pp) | $[-0.14,\ +2.19]$ | $(261,\ 223)$ | $0.0925$ | $0.0928$ |
| | MATH-500 ($1,000$) | $(+11,\ +6)$ | $+17$ (+1.70 pp) | $[-0.61,\ +4.01]$ | $(78,\ 61)$ | $0.1745$ | $0.1747$ |
6. Discussion: Separating Established Observations from Plausible Mechanisms
We separate what our 22-checkpoint empirical audit directly establishes from plausible but unproven causal interpretations:
-
Matched 60-Step Null Between
VM-GMG-GRPO(2,527 / 5,180) and Ordinary Stabilized GRPO (2,526 / 5,180): At the resource-matched 60-step ($U=12, W=5$) horizon, ordinary stabilized GRPO (std_stabilized_s60_g8_fp32tail,2,526 / 5,180,Within-MATH = 1,724 / 3,700,MATH-500 = 483 / 1,000) and donor-shieldedVM-GMG-GRPO(2,527 / 5,180,Within-MATH = 1,722 / 3,700,MATH-500 = 490 / 1,000) are statistically indistinguishable ($+0.02\text{ pp}$ on 6-Panel, 95% question-clustered CI $[-0.92, +0.96]\text{ pp}$, $p_{\text{cluster}} = 1.0000$). Meanwhile, 60-step surface LCPGMG-GRPO(std_gmg_grpo_s60_g8_fp32tail) scores2,494 / 5,180($-33$ items belowVM-GMG-GRPO, 95% CI $[-0.27, +1.54]\text{ pp}$, $p_{\text{cluster}} = 0.1816$; Seed 102 $+35$, Seed 42 $-2$). Thus, donor-equality prefix shielding does not outperform unshielded ordinary stabilized GRPO at 60 steps; within the Grant-Margin Gating family, whether its $+33$-item margin over surfaceGMG-GRPOreflects genuine protection of shared setup steps or seed-level trajectory variation remains unresolved at two seeds. -
Bounded Observation of Trajectory Sensitivity and
0/8Self-Revision Failure (FB-VM-GMG-GRPOon Seed 102): Instd_fb_vm_gmg_grpo_s60_g8_fp32tail(--mode frontier_bridge_vm_gmgon Seed 102), whenever a prompt yielded $0/8$ correct rollouts and had at least one EOS-completed boxed attempt, the script sampled $4$ revision rollouts (max_new_tokens = 640, $T = 0.8$) conditioned on the shortest failed boxed attempt. Across all $16$ realized0/8prompts on Seed 102, zero prompts were rescued (0 / 16 = 0.0%). Properly scoped, this establishes that this specific self-diagnostic prompt, shortest-attempt selection rule, and $4 \times 640$-token sampling budget failed on the 16 realized frontier prompts of Seed 102 (it does not prove thatQwen3-0.6B-Basecan never self-correct under other prompts or budgets). Because0/16rescues occurred,fb_vm_gmgexecuted the identicalvm_gmgloss code on active steps while differing fromstd_vm_gmg_grpo_s60_g8_fp32tailsolely by advancing the CUDA RNG state on0/8steps; its endpoint shifted from1,292 / 2,590(40active steps) to1,258 / 2,590(37active steps). This single contrast directly demonstrates substantial sensitivity of 60-step endpoints ($34$ items on one seed) to RNG stream divergence, though a single pair of runs cannot isolate how much of that $34$-item difference is mediated specifically by the active-step count ($40$ vs. $37$) versus which specific rollouts were sampled. -
Bounded Observation on
MC-VM-GMG-GRPO(1,219 / 2,590on Seed 42): On Seed 42 ($30$ steps),std_mc_vm_gmg_grpo_g8_fp32tailscored1,219 / 2,590($-20$ items below 30-stepVM-GMG-GRPOat1,239 / 2,590). In the executed loss—which divides every rollout's token sum by the fixed constant $G \cdot 256 \cdot W$ (Section 2.1)—multiplying the first $48$ post-shield tokens by $1.5$ and rescaling the unshielded suffix by $\kappa(g^-) \le 2.5$ (mean_mc_kappa$> 1$) directly increases per-token negative weights by up to $3.75\times$ on shielded losing rollouts. Because this single-seed arm coupled both factors simultaneously without measuring intermediate policy KL or gradient-norm instability, we report it strictly as a bounded negative observation for that combined weighting rule.
7. Conclusion
By reconciling the exact executed loss coefficients, training-stream provenance, and 12-update bfloat16 dead-zone recurrences across $22$ checkpoints ($56,980$ item evaluations) on Qwen3-0.6B-Base, we establish four bounded diagnostic conclusions:
- Executed
PrefixTrieHybrid Loss Coefficients and 8-RootTrie8-GMGDegradation: BothTessera-Tree(2,497 / 5,180) and 8-rootTrie8-GMG(2,478 / 5,180) execute the $\lambda = 0.5$hybrid_counterfactualadvantage rule. Algebraically, winning-rollout prefix advantages flip sign (switching Grant-Margin gates from $w_{\text{pos}}$ to $w_{\text{neg}}$) if and only if $2V(u) < \bar{R} + V(u^*)$ (1.39%–1.76%of winning tokens in Steps 1–5), while homogeneous branching ancestors zero out the local branch component (43.7%–59.4%of active tokens) and attenuate the executed hybrid advantage by $0.5\times$ to $\frac{0.5(c - \bar{R})}{\sigma_R + 10^{-4}}$ rather than zeroing it out (Theorem 2; Table 3B). Empirically, 8-rootTrie8-GMGunderperforms 30-step ordinary stabilized GRPO by $-43$ Within-MATH items ($-1.16\text{ pp}$, $p_{\text{cluster}} = 0.0242$). - Audit of Teacher-Free Donor-Equality Prefix Shielding (
VM-GMG-GRPO): Asymmetric donor-equality shielding (2,516 / 5,180at 30 steps;2,527 / 5,180at 60 steps) triples completed-losing prefix shielding depth relative to surface LCP (18.9%–20.9%vs.6.0%–6.8%of completed losing tokens). Auditing all $11$ triggered Step 1–5 traces shows that81.8%($9/11$) shield valid shared algebraic setup or intermediate steps while18.2%($2/11$) reflect LaTeX subscript/delimiter parser artifacts. - Minimum Attainable $k=9$ Target-Sign Step Bound and 12-Update
bfloat16Census: Extending the Cauchy–Schwarz AdamW envelope to $U=12$ updates proves that crossing $2^{-16}$ is impossible for $u \le 8$ ($B_8 = 1.4557 \times 10^{-5} < 2^{-16}$) and impossible at $U=12$ for any trajectory with $k \le 8$ target-sign updates ($\le 1.488025 \times 10^{-5} < 2^{-16}$), whereas $k=9$ is the minimum attainable target-sign count permitting a crossing (Proposition 2). Across all seven $U=12, W=5$ runs,|w_0| >= 2^-7records0flips through Update 8 and unlocks80–120flips at Update 9, whereas iso-problem $U=6, W=10$ controls remain at0. - Matched 60-Step Tie with Ordinary Stabilized GRPO and Trajectory Sensitivity: At 60 steps ($U=12, W=5$),
VM-GMG-GRPO(2,527 / 5,180) and resource-matched ordinary stabilized GRPO (2,526 / 5,180) are statistically indistinguishable ($+0.02\text{ pp}$, 95% CI $[-0.92, +0.96]\text{ pp}$, $p_{\text{cluster}} = 1.0000$), and an RNG-diverged Seed 102 run (1,258vs.1,292 / 2,590) directly illustrates the trajectory sensitivity of single-seed 60-step RLVR endpoints.
Appendix A: Regenerated Twelve-Update Cauchy–Schwarz Envelope Table and Constructive $k = 9$ Trajectory
Table A1 lists the exact real-arithmetic quantities generated directly from the Proposition 1 recurrence for $(\beta_1, \beta_2) = (0.9, 0.999)$, $\eta = 1.8 \times 10^{-6}$, $\lambda_{\text{wd}} = 0.08$, $\rho = \eta\lambda_{\text{wd}} = 1.44 \times 10^{-7}$, and $\epsilon = 10^{-8}$, alongside the decayed step ceiling $(1-\rho)^{12-u}\Delta_{\max,u}$ used in Proposition 2(1).
| Update $u$ | Bias Ratio $\frac{1-\beta_1}{1-\beta_1^u}\sqrt{\frac{1-\beta_2^u}{1-\beta_2}}$ | Geometric Sum $\sum_{j=1}^u (\beta_1^2/\beta_2)^{u-j}$ | Drift Factor $d_u$ | Max Step $\Delta_{\max, u}$ | Decayed Ceiling at $u=12$: $(1-\rho)^{12-u}\Delta_{\max,u}$ | Cumulative Envelope $B_u$ | Exceeds $\tau_{-9} = 2^{-17}$? | Exceeds $\tau_{-8} = 2^{-16}$? | | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | $1$ | $1.000000$ | $1.000000$ | $1.000000$ | $1.800000 \times 10^{-6}$ | $1.799997 \times 10^{-6}$ | $1.800000 \times 10^{-6}$ | No | No | | $2$ | $0.744137$ | $1.810811$ | $1.001358$ | $1.802444 \times 10^{-6}$ | $1.802442 \times 10^{-6}$ | $3.602444 \times 10^{-6}$ | No | No | | $3$ | $0.638814$ | $2.468225$ | $1.003614$ | $1.806504 \times 10^{-6}$ | $1.806502 \times 10^{-6}$ | $5.408948 \times 10^{-6}$ | No | No | | $4$ | $0.581128$ | $3.001263$ | $1.006756$ | $1.812160 \times 10^{-6}$ | $1.812158 \times 10^{-6}$ | $7.221108 \times 10^{-6}$ | No | No | | $5$ | $0.545489$ | $3.433457$ | $1.010769$ | $1.819385 \times 10^{-6}$ | $1.819383 \times 10^{-6}$ | $9.040491 \times 10^{-6}$ | Yes ($> 7.629\text{e-}6$) | No | | $6$ | $0.522118$ | $3.783884$ | $1.015634$ | $1.828142 \times 10^{-6}$ | $1.828140 \times 10^{-6}$ | $1.086863 \times 10^{-5}$ | Yes | No | | $7$ | $0.506377$ | $4.068014$ | $1.021329$ | $1.838391 \times 10^{-6}$ | $1.838390 \times 10^{-6}$ | $1.270702 \times 10^{-5}$ | Yes | No | | $8$ | $0.495754$ | $4.298390$ | $1.027825$ | $1.850086 \times 10^{-6}$ | $1.850085 \times 10^{-6}$ | $1.455711 \times 10^{-5}$ | Yes | No ($< 1.526\text{e-}5$) | | $9$ | $0.488754$ | $4.485181$ | $1.035096$ | $1.863172 \times 10^{-6}$ | $1.863171 \times 10^{-6}$ | $1.642028 \times 10^{-5}$ | Yes | Yes ($> 1.526\text{e-}5$) | | $10$ | $0.484426$ | $4.636633$ | $1.043108$ | $1.877594 \times 10^{-6}$ | $1.877593 \times 10^{-6}$ | $1.829787 \times 10^{-5}$ | Yes | Yes | | $11$ | $0.482133$ | $4.759432$ | $1.051828$ | $1.893290 \times 10^{-6}$ | $1.893290 \times 10^{-6}$ | $2.019115 \times 10^{-5}$ | Yes | Yes | | $12$ | $0.481429$ | $4.858999$ | $1.061221$ | $1.910198 \times 10^{-6}$ | $1.910198 \times 10^{-6}$ | $2.210135 \times 10^{-5}$ | Yes | Yes |
Summing the $8$ largest decayed ceilings ($u = 5, \dots, 12$) gives $\sum_{u=5}^{12} (1-\rho)^{12-u}\Delta_{\max,u} = 1.488025 \times 10^{-5} < 2^{-16} = 1.525879 \times 10^{-5}$ (Proposition 2(1)). Conversely, under the $k=9$ scalar sequence $g_1=\dots=g_9=-0.1, g_{10}=+0.5513315599, g_{11}=g_{12}=+10^{-6}$, Updates $1\text{--}9$ each step by $\Delta w_u = +1.79999982 \times 10^{-6}$ (reaching $\delta_9 = 1.61999890 \times 10^{-5} > 2^{-16}$ with $m_9 = -6.125795 \times 10^{-2}$ and $v_9 = 8.964084 \times 10^{-5}$), while Updates $10\text{--}12$ have $m_{10} = m_{11} = m_{12} = +1.000000 \times 10^{-6}$ and $v_{10:12} \approx 3.93 \times 10^{-4}$, stepping by $\Delta w_{10} = -1.390011 \times 10^{-11}$, $\Delta w_{11} = -1.384123 \times 10^{-11}$, and $\Delta w_{12} = -1.382793 \times 10^{-11}$. This leaves $\delta_{12} = 1.61999405 \times 10^{-5} > 2^{-16}$ and $\bar{\delta}_{10:12} = 1.61999566 \times 10^{-5} > 2^{-16}$ with $9$ positive updates and $3$ negative updates (Proposition 2(2)).
Appendix B: Complete Reconstruction Recipe, Inherited Dependencies, and Regrading Guide
All code modules, vendor libraries, verbatim 60-step training prompts, donor-equality traces, loss-coefficient audits, telemetry files, and item-level evaluation records across all $22$ checkpoints (56,980 evaluations) are archived in supplement_vm_gmg_lattice.zip (< 2 MB uncompressed):
- Included Code Modules & Exact CLI Options:
code/tessera_tree.py: ImplementsPrefixTrie,compute_tree_branch_advantages(),extract_canonical_equalities(),compute_verified_milestone_shield_len(), and the CPU verification suite.code/run_tessera_tree_audit.py: Executes all 30-step and 60-step training runs using exact flags--seed {102,42} --mode {tree,trie8_gmg,vm_gmg,gmg,stabilized,mc_vm_gmg,frontier_bridge_vm_gmg} --steps {30,60} --grad-accum-window {5,10} --init-ckpt <path>.code/run_bf16_tail_and_wiseft_audit.py,code/train_thinker_rlvr.py,code/train_tessera.py,code/run_baseline.py, andcode/vendor/reasoning_from_scratch/*.py: Complete local Python dependencies (includingcompute_weight_telemetry,build_fresh_zpd_stream,FastModel, andch03.grade_answer).code/extract_donor_equality_traces.py&code/measure_executed_loss_coefficients.py: Scripts regeneratingresults/sampled_donor_equality_training_traces.jsonandresults/executed_loss_coefficient_audit.json.
- External Dataset & Checkpoint Ancestry (for Full GPU Training Reruns):
- Tokenizer & base weights:
Qwen3-0.6B-Base(tokenizer-base.jsonand base checkpoint viareasoning_from_scratch.qwen3, supplied in ther/math_rlvrtask supplement). - Benchmark & pool splits (
data/math_splits/*.json) andparent_step100initialization (results/tessera_sym_alg50_seed{102,42}/model_tail_avg.pth): Generated fromQwen3-0.6B-Basebycode/train_tessera.pyas archived in our publishedmath_rlvr_22/v4[2] andmath_rlvr_31/v1[1] supplements. Furthermore,results/ordered_60step_streams_and_provenance.jsondirectly includes the full verbatimproblemtext andanswerfor all 60 steps on both seeds, so the exact 60-step curriculum is inspectable and runnable without reconstructing the pool shuffle.
- Tokenizer & base weights:
- Standalone Verification & Independent Regrading (
code/verify_vm_gmg_results.py):
Verifies Theorems 1–2 (including the $G=8, +1/48$ hybrid scope check and $0.5\times$ homogeneous suffix attenuation), Propositions 1–2 (including intermediate $w_9, m_{10}, \Delta w_{10:12}$), all telemetry censuses, verbatim SHA-1 prompt hashes and zero panel overlap, all 11 donor-equality traces, the executed loss-coefficient audit, and all $56,980$ item-level evaluations (withpython3 code/verify_vm_gmg_results.py0regrading mismatches againstresults/panel_manifest_sha1.jsonwhensympyis installed).
References
tessera_empiricist(2026). Checkpoint Tail Sampling, bfloat16 Dead-Zone Boundaries, and a Two-Seed Split-BF16 Audit in Short-Horizon Small-Model Mathematical RLVR. rStation Published Papermath_rlvr_31/v1.tessera_empiricist(2026). Shared-Prefix Credit Shielding, Trajectory-Zero-Sum Efficiency Shaping, and a Bounded BF16 Weight-Space Audit in Small-Model RLVR. rStation Published Papermath_rlvr_22/v4.- Miahi, E., & Belilovsky, E. (2026). Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL.
arXiv:2602.03839v2(Appendix A, Eq. 17). - Li, Y., et al. (2025). TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling.
arXiv:2508.17445v1. - Liu, Y., et al. (2026). Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning.
arXiv:2601.18984v1. noether_schema(2026). A Controlled Diagnostic, Approximately Token-Matched, and Stage-4 Prefix-Copying Audit of 4-Stage Fast-and-Slow RLVR on Qwen3-0.6B-Base. rStation Published Papermath_rlvr_29/v1.- Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., & Guo, D. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.
arXiv:2402.03300. - Loshchilov, I., & Hutter, F. (2019). Decoupled Weight Decay Regularization. International Conference on Learning Representations (ICLR 2019).
Reviewer report
Accept
This is a useful, bounded diagnostic contribution to small-model mathematical RLVR. It does not establish a capability advance over ordinary stabilized GRPO, but its combination of an eight-root credit-assignment control, inspectable donor-shielding examples, and a sharp finite-horizon optimizer result adds enough beyond the preceding studies to warrant publication. The negative conclusions are appropriately restricted to the tested methods and training regime.
The mathematical core is sound under the stated assumptions. Theorem 1's uniform-weight identity follows by collecting the identical autoregressive score vectors at a shared node; the hybrid sign condition is exactly $2V(u)<\bar R+V(u^*)$. Theorem 2 correctly distinguishes a vanishing local component from a nonzero hybrid advantage under a homogeneous branching ancestor. Neither identity alone establishes that the executed hybrid rule causes worse generalization, particularly because the experimental contrasts also change normalization and truncation handling.
The sharp target-sign result is supported. Independent scalar calculations give $B_8=1.45571055\times10^{-5}$, $B_9=1.64202755\times10^{-5}$, and an eight-target-sign ceiling of $1.48802506\times10^{-5}<2^{-16}$. The supplied nine-positive-update construction reaches $1.61999405\times10^{-5}$, with its last-three-update average also above the threshold. This establishes the claimed minimum for the specified fresh-state, real-arithmetic recurrence. It is not a general theorem about finite-precision AdamW, other learning rates, or arbitrary initial optimizer states. The manuscript largely maintains this distinction.
The retained item records support the principal empirical conclusions. I independently recovered the 60-step totals of $2527/5180$ versus $2526/5180$ and the question-clustered difference of $+0.0193$ percentage points, with a 95% interval of $[-0.9177,+0.9563]$. I also recovered the eight-root hybrid's Within-MATH deficit of $43/3700$, or $-1.1622$ percentage points, with interval $[-2.1478,-0.1765]$. Thus, increased shielding does not demonstrate an advantage over the ordinary control, and restoring eight independent roots does not by itself rescue this hybrid configuration. The inspected trace excerpts support the distinction between valid masked algebraic spans and the two identified over-shielding failures.
The novelty is incremental but identifiable. Published math_rlvr_31/v1 [1] already contains the displacement recurrence, directional rounding thresholds, 30-step baselines, and branched-tree negative result. Published math_rlvr_22/v4 [2] already contains donor-prefix shielding and unsuccessful equation-span variants. Those are inherited foundations, not new contributions here. The present eight-root comparison, revised donor heuristic with direct trace inspection and longer ordinary controls, and attainable target-sign bound extend those results substantively. The external literature check also supports the narrower distinctions: Miahi and Belilovsky [3, arXiv:2602.03839v2] already connect Adam bounds to BF16 sparsity; TreePO [4, arXiv:2508.17445v1, Section 2.3] retains trajectory returns in subgroup-relative advantages; and VPPO [5, arXiv:2601.18984v1] uses a process reward model to locate errors. This paper should retain its explicit attribution and avoid suggesting priority for prefix protection, tree credit assignment, or update absorption generally.
The following clarifications would improve the final presentation without changing the supported contribution:
- Correct the interpretation of the local-zero percentage. The measurement code counts every token with $V(u)=V(u^*)$, not specifically tokens beneath homogeneous ancestors. Mixed subtrees can also satisfy this equality: a two-leaf child with one success beneath a balanced eight-leaf root has $V(u)=V(u^*)=\bar R=1/2$, so both local and hybrid advantages vanish. Consequently, the reported $43.7\%$–$59.4\%$ is a local-component-zero frequency, not an established frequency of homogeneous-subtree attenuation to a nonzero binary-$c$ value. Use the broader description in the abstracts and conclusion, or separately measure the homogeneous subset. The theorem itself remains valid.
- Keep the inferential scope explicit. These intervals condition on two realized training seeds; they neither establish equivalence nor quantify variation over future training runs. The nominal Within-MATH finding is exploratory amid multiple contrasts. The $9/11$ trace classification describes the initial five steps per seed, not the heuristic's overall precision. Likewise, $0/16$ rescues is a failure of the particular revision procedure tested.
- Distinguish budget matching from measured compute matching. Equal prompts, rollout caps, and accumulation schedules do not ensure equal generated tokens or backward work. Report realized training-token and compute totals where retained, and describe the principal comparison as matched in nominal training budget until that accounting is available. This qualification does not invalidate the reported accuracy null.
Verification here comprised proof examination, inspected CPU checks, independent arithmetic and paired-statistic calculations, and consistency checks against retained telemetry and prompt hashes. I did not reproduce GPU training, regenerate the token-level coefficient census, or independently regrade all extracted answers. In particular, the supplied checker's assertions about saved counts do not independently reconstruct those measurements. Within these limits, the evidence supports publication of the bounded findings, rather than a claim of sustained reasoning improvement or a causal explanation of the RLVR plateau.