1. Problem, Scope, and Contribution

A central question in reinforcement learning with verifiable rewards (RLVR) for small language models (Qwen3-0.6B-Base) is whether multi-step deliberation, self-verification, and self-correction can be transferred into a stronger single-pass native reasoning policy without external teacher supervision, or whether apparent slow-to-fast distillation gains are largely explained by single-turn horizon and optimizer stabilization. In the Thinker framework [1, 2], single-turn RLVR is expanded into a four-stage shared-actor episode:

1. Stage 1 (Fast Thinking): Generate an initial candidate derivation $a_{\text{fast}} \sim \pi_\theta(\cdot \mid x_{\text{fast}})$ under a compact token budget ($L_{\text{fast}} = 512$).
2. Stage 2 (Verification): Given $(x_{\text{fast}}, a_{\text{fast}})$, generate a verification critique and binary verdict $\text{\boxed{Yes}}$ or $\text{\boxed{No}}$.
3. Stage 3 (Slow Refinement): Whenever Stage 1 fails ($r(a_{\text{fast}}, y^*) = 0$), condition on the fast attempt and verification critique to generate an extended refinement trajectory $a_{\text{slow}}$.
4. Stage 4 (Summarization): Whenever Stage 3 recovers the correct answer ($r(a_{\text{slow}}, y^*) = 1$), condition on the solved history $h_{\text{teacher}} = [x_{\text{fast}}, a_{\text{slow}}^+]$ to synthesize a self-contained derivation $a_{\text{sum}}$ intended to train the Stage-1 Fast policy.

While Chung et al. [1] demonstrated multi-stage gains on 1.5B and 7B backbones using PPO with generalized advantage estimation (GAE) and a learned value critic (Sec. 5.1 of [1]), their reference codebase (stephen-chung-mh/thinker-task [2]) omits direct supervised updates on summary_buffer (sum_loss) from the default actor backward pass (loss = pg_loss + reg_loss, thinker/actors.py L708 [2]), and subsequent controlled studies on Qwen3-0.6B-Base in r/math_rlvr [3, 4, 12, 13] have repeatedly found that multi-turn revision or contextual solution transfer fails to beat well-configured single-turn RLVR controls.

Summary of Contributions

We present an 11-arm, 2-seed ($N = 5,180$ total evaluations across 6 panels at uniform batch_size = 50 in bfloat16; $N = 5,172$ strictly decontaminated after removing 4 zpd_pool overlap items) diagnostic and matched-dose ablation study starting from our converged Step-100 RLVR checkpoints on Qwen3-0.6B-Base [5, 6]. Throughout, we refer to our policy-gradient estimator accurately as within-prompt standard-deviation-normalized GRPO with constant token-length normalization (GRPO-const256, dividing token log-probability sums by 256 and reward residuals by within-group $\sigma_R$), distinguishing it from external Dr.GRPO [8] (which removes $\sigma_R$ division). Our study establishes three main findings:

1. Diagnostic 1 — Correctness-Conditioned $2\times 3$ Verifier Confusion Matrices and Shared-Actor Boundary Instability (Section 3.1 & Section 4.3): Under within-prompt GRPO-const256, any prompt where all $G=4$ Stage-1 Fast rollouts share the same correctness outcome ($0/4$ or $4/4$) has a constant class-balancing multiplier $w_c \in \{1/(2\bar{p}_{\text{fast}}), 1/(2(1-\bar{p}_{\text{fast}}))\}$ across all $G$ rollouts, which cancels algebraically upon dividing by $\sigma_R$, while any homogeneous verdict group (4/4 Yes or 4/4 No) produces $\sigma_R = 0$ and zero immediate verification gradient. Correctness-conditioned $2\times 3$ verifier confusion matrices on Val-350 ($N = 350$/seed) reveal that the converged parent_step100 checkpoint's zero-shot verifier is not a calibrated critic: it is strongly Yes-biased and Invalid-prone (Fast Right [159]: Yes=113, No=0, Invalid=46 on Seed 102 and Yes=118, No=0, Invalid=41 on Seed 42; Fast Wrong [191]: Yes=149, No=1, Invalid=41 on Seed 102 and Yes=102, No=6, Invalid=83 on Seed 42), so baseline Slow Refinement triggers (19 on S102, 26 on S42) arise almost entirely from missing_fast_box (18 and 22 truncated fast rollouts omitting \boxed{}) rather than explicit \boxed{No} rejection. When Stages 2–4 are trained on the shared actor (thinker_baseline and thinker_unweighted_sumsft), verification exhibits severe seed- and realization-dependent boundary collapse:
   • In thinker_baseline, Seed 102 experiences 14 all-wrong ($0/4$) vs. 2 all-right ($4/4$) Fast steps (and 14 mixed steps) and collapses toward \boxed{No} (340/350 Val-350 triggers, +3/-36, dropping Verify->Slow accuracy from 45.43% to 32.86%), whereas Seed 42 experiences 10 all-wrong ($0/4$) vs. 7 all-right ($4/4$) Fast steps (and 13 mixed steps) and remains near baseline (29/350 triggers, 41.71%).
   • In thinker_unweighted_sumsft, Run 1 collapses toward \boxed{No} on both seeds (350/350 and 233/350 triggers; 220/700 = 31.43% pooled), whereas our fully regradable Seed 102 replication thinker_unweighted_sumsft_rep2 collapses to the opposite homogeneous boundary (346/350 \boxed{Yes} verdicts: 81 on Fast-Right and 265 on Fast-Wrong, with 6-panel Fast accuracy dropping to 636/2,590 = 24.56%).
   • Fast-Context Gradient Isolation (running Stages 2–4 under torch.no_grad() and updating only on $x_{\text{fast}}$) prevents multi-turn verifier collapse (330/700 = 47.14% in bc_thinker_v2, +69 items over thinker_baseline and +110 items over thinker_unweighted_sumsft) by preserving the parent checkpoint's low false-rejection baseline (4–8/350 explicit No verdicts), rather than by creating a discriminative verifier.
2. Diagnostic 2 — Pre-Clip Gradient Spikes in Unweighted Summary SFT vs. Brier-Variance-Gated Distillation Package (Brier-SCD, Section 3.2 & Section 4.4): For a target token $y_t$ with student probability $p_t = \pi_\theta(y_t \mid x_{\text{fast}}, y_{<t})$, unweighted NLL has target-logit derivative $\partial(-\log p_t)/\partial z_{t,y_t} = p_t - 1$ (magnitude $\to 1$ as $p_t \to 0$), whereas multiplying by the detached Bernoulli variance gate $w_t = \mathrm{sg}\big(4 p_t(1-p_t)\big)$ from thinker/actors.py (L201 [2]) yields parameter gradient $2\nabla_\theta (1-p_t)^2$ and target-logit derivative $-4 p_t(1-p_t)^2 \to 0$ as $p_t \to 0$. In thinker_unweighted_sumsft, recorded pre-clip total-loss gradient norms in training.jsonl (prior to 1.0-norm clipping via clip_grad_norm_(..., 1.0)) spike to $\|\nabla_\theta \mathcal{L}\|_2 = 26.80$ (mean $5.14$ on S102; max $23.73$, mean $4.57$ on rep2 S102; max $11.02$, mean $4.17$ on S42) and 6-panel accuracy drops by -313 items (2,177/5,180 = 42.03%, $p < 10^{-15}$). Combining Fast-Context Isolation, Brier-SCD ($\alpha = 0.50$ hard NLL + $0.50$ top-16 teacher KL), fast_anchor_buffer replay, right-censored fast masking, and conservative step scaling (0.35/0.65, wd=0.06) in bc_thinker_v2 bounds pre-clip total-loss gradient norms to max 6.91 (mean 1.86 on S102; max 1.94, mean 0.76 in isolated brier_scd_pure) and recovers +296 items (2,473/5,180 = 47.74%, decontam = 2,466/5,172, $p < 10^{-15}$), though all three 4-stage multi-turn BC-Thinker packages (bc_thinker, bc_thinker_v2, bc_thinker_h1024) plateau at 47.64%–47.74%, below parent_step100 (48.07%).
3. Diagnostic 3 — Matched-Dose Nested Audit of 2-Stage Foresight Prefix-Forking (fork_thinker) vs. Stabilized Single-Turn GRPO-const256 (std_stabilized_g8, Section 3.4 & Section 4.5): To test whether 2-Stage Foresight Prefix-Forking (fork_thinker)—which branches 4 slow continuations (768 tokens) from interior high-entropy prefixes $x_{\text{fast}} + a_{\text{fast}, <t_{\text{fork}}}^-$ ($t_{\text{fork}} \in [32, 288]$ [15]) of failed fast rollouts—genuinely outperforms a matched single-turn control, we evaluate a strictly matched 3-level nested hierarchy where all three levels share the exact same $5,120$-token nominal per-step generation cap ($4\times 512 + 4\times 768$), optimizer schedule (grad_accum=5 yielding 6 updates, lr_scale=0.60, wd=0.08, zero-gradient skip), and right-censored fast masking:
   • Level 0 (std_stabilized_g8, single-turn GRPO-const256 with $4\times 512 + 4\times 768$ root rollouts): Reaches 2,496/5,180 (48.19%, decontam = 2,489/5,172), Within-MATH = 1,701/3,700 (45.97%), and MATH-500 = 481/1,000 (48.10%). This demonstrates that 94.5% (+308 / +326 items) of the raw gap between fork_thinker and unmasked short-horizon std_rlvr_cont (2,188/5,180 = 42.24%) is already achieved by the stabilized single-turn control package.
   • Level 1 (fork_unweighted_nomask, 2-stage prefix forking + unweighted $w_t \equiv 1$ slow-fork and anchor replay SFT without pre-fork prefix masking): Reaches 2,499/5,180 (48.24%, decontam = 2,493/5,172), Within-MATH = 1,687/3,700 (45.59%), MATH-500 = 467/1,000 (46.70%), while reducing measured generated tokens by 15.1% (181,694 vs. 213,987 across 2 seeds) because the 4 768-token slow forks are skipped on $4/4$ Fast-solved steps and complete from interior prefixes.
   • Level 2 (fork_thinker Run 1 and full 2-seed confirmation fork_thinker_rep2 Run 2, adding Brier-SCD gating on slow-fork + anchor replay and explicit pre-fork prefix masking $t < t_{\text{fork}}$):
     • In exploratory Run 1 (fork_thinker), 2-seed pooled accuracy reaches 2,514/5,180 (48.53%, decontam = 2,508/5,172), Within-MATH = 1,723/3,700 (46.57%), and MATH-500 = 487/1,000 (48.70%; +20 items over fork_unweighted_nomask, $W/L = 48/28, p = 0.0286$, CI $[+0.30, +3.80]$ pp; +21 items over parent_step100, $W/L = 55/34, p = 0.0334$, CI $[+0.20, +3.90]$ pp; but only +6 items / +0.60 pp over std_stabilized_g8, $W/L = 45/39, p = 0.5856$, CI $[-1.20, +2.50]$ pp).
     • In our fully regradable, fully token-logged 2-seed confirmation (fork_thinker_rep2 across both Seeds 102 and 42), 2-seed pooled accuracy is 2,481/5,180 (47.90%, decontam = 2,474/5,172), Within-MATH = 1,683/3,700 (45.49%), and MATH-500 = 473/1,000 (47.30%: 241/500 on S102 [+14 items over Level 1 S102, $p = 0.0288$] and 232/500 on S42), with 189,310 measured generated tokens across the 2 seeds (-11.5% vs. 213,987 in std_stabilized_g8) alongside additional non-generation forward/backward passes (40,622 entropy-scoring forward tokens and 23,641 distillation+anchor backward tokens on S42).
   • Overarching Bounded Conclusion: Once single-turn GRPO-const256 is properly stabilized with matched horizon mixtures, right-censored truncation masking, and 5-step gradient accumulation (std_stabilized_g8), 2-stage prefix forking (fork_thinker / fork_thinker_rep2) does not establish a statistically significant accuracy gain over stabilized single-turn RLVR (+0.35 pp in Run 1, -0.29 pp in Run 2; both 95% CIs spanning zero), serving instead as a generation-token-saving alternative (-11.5% to -15.1% generated tokens) whose Brier-SCD + prefix-masking rule modestly improves MATH-500 retention relative to unweighted unmasked forking (487 and 473 vs. 467/1,000).

---

2. Related Work and Distinction

| Prior Work / Study | Multi-Turn / Branching Mechanism | Distillation / Credit Channel | Key Finding at 0.6B Scale & How Our Study Complements It | | :--- | :--- | :--- | :--- | | Thinker (Chung et al., 2025 [1, 2]) | 4-Stage Shared Actor (Fast -> Verify -> Slow -> Summary, $\gamma_{\text{inter}} = 0$) | PPO+GAE with learned critic in [1]; sum_loss omitted in actors.py L708 [2]; OffPolicyRLLoss(mode=1) L201 detached $4p(1-p)$ gate | External Thinker [1] uses PPO with a critic. We show that adapting 4-stage Thinker to critic-free within-prompt GRPO-const256 cancels class-balancing weights on homogeneous Fast groups and suffers severe seed/realization verifier instability (340/350 S102 triggers in thinker_baseline; opposite No/Yes collapses in thinker_unweighted_sumsft). | | Dr.GRPO (Liu et al., 2025 [8]) & GRPO (Shao et al., 2024 [7]) | Single-turn group-relative policy optimization | GRPO divides by within-group $\sigma_R$; Dr.GRPO [8] removes $\sigma_R$ division and length bias | Our estimator (GRPO-const256) uses constant 1/256 token normalization but retains $\sigma_R$ division; Thus, the homogeneous-group scale cancellation analyzed in Section 3.1 is a property of $\sigma_R$-normalized GRPO, not Dr.GRPO [8]. | | MARGE (Gao et al., ICML 2025 [14]) & High-Entropy Minority Tokens (Wang et al., 2025 [15]) | Math-hard-example guided exploration [14]; RLVR update concentration on high-entropy forking tokens [15] | Entropy-weighted or top-entropy-filtered policy gradients | Wang et al. [15] motivate high-entropy tokens as reasoning branch points. We test interior high-entropy prefix forking ($t_{\text{fork}} \in [32, 288]$) against a matched-cap ($5,120$ tokens/step) stabilized single-turn control (std_stabilized_g8), showing near-equivalence (+0.35 pp / -0.29 pp). | | Whole-Attempt Revision (r/math_rlvr Paper 21v1 [3]) & Own-Context Conversion (Paper 12v3 [4]) | Whole-attempt revision rollouts [3] or historical own-model solution prefixes [4] | Root-revision credit [3] or hard-target SFT / mixed-prefix RLVR [4] | Papers 21v1 [3] and 12v3 [4] showed that multi-turn revisions and own-context conditioning fail to beat native single-pass controls. Our 4-stage BC-Thinker plateau (47.64%–47.74% vs. 48.07% parent) independently corroborates this multi-turn conversion bottleneck. | | Monte Carlo Fork Credit (Paper 18v1 [13]) & Concurrent Branched Rollouts (Paper 25v8 [12]) | Prefix value jumps at near-ties [13]; concurrent within-72h counterfactual branched rollouts [12] | Terminal / MC fork-jump credit [13] or branched-pair credit [12] | Consistent with Paper 18v1's null result for fork-jump credit [13], our matched-dose nested ablation shows that once horizon mixture, right-censored masking, and 5-step accumulation are controlled (std_stabilized_g8), prefix forking yields no net gain over stabilized single-turn RLVR. | | Truncation Masking & Prefix Shielding (Papers 20v5, 22v4, 27v1 [11, 5, 6]) | Single-turn GRPO with horizon masking [11] or donor LCP shielding [5] | FP32 master weights / BF16 rollouts and pre-AdamW filtering [6] | Confirming Papers 20v5 [11] and 27v1 [6], our std_stabilized_g8 control recovers +308 items over unmasked std_rlvr_cont, accounting for 94.5% of the raw gap previously attributed to fork_thinker. |

---

3. Diagnostic Framework and Executed Methods

3.1 Within-Group Scale Cancellation and Boundary Degeneracy in $\sigma_R$-Normalized Shared-Actor Verification

In external Thinker (Chung et al. [1], Sec. 5.1), Stage-2 verification is trained via PPO+GAE with a learned critic $V_\phi(s)$, which preserves the cross-prompt scale of the class-balanced verification reward [1, 2]:

$$ R_{\text{verify}}^{(i)} = \begin{cases} w_{\text{yes}} \cdot \mathbb{I}[v_i = \text{Yes}], & \text{if } c_i = 1, \quad w_{\text{yes}} = \frac{1}{2 \bar{p}_{\text{fast}}}, \\[6pt] w_{\text{no}} \cdot \mathbb{I}[v_i = \text{No}], & \text{if } c_i = 0, \quad w_{\text{no}} = \frac{1}{2 (1 - \bar{p}_{\text{fast}})}, \end{cases} $$

where $c_i = \mathbb{I}[\hat{y}(a_{\text{fast}}^{(i)}) = y^*] \in \{0, 1\}$ is Stage-1 Fast correctness, $v_i \in \{\text{Yes}, \text{No}, \text{Invalid}\}$ is the Stage-2 verdict, and $\bar{p}_{\text{fast}} \in [0.15, 0.85]$ is the EMA Stage-1 pass rate. In our critic-free adaptation (thinker_baseline and thinker_unweighted_sumsft), accumulate_stage_drgrpo (code/train_thinker_rlvr.py L949) computes within-prompt standard-deviation-normalized advantages (GRPO-const256, note that unlike external Dr.GRPO [8], $\sigma_R$ division is retained):

$$ \hat{A}_{\text{verify}}^{(i)} = \frac{R_{\text{verify}}^{(i)} - \mu_R}{\sigma_R + 10^{-4}}, \qquad \mu_R = \frac{1}{G}\sum_{j=1}^G R_{\text{verify}}^{(j)}, \qquad \sigma_R = \sqrt{\frac{1}{G-1}\sum_{j=1}^G \big(R_{\text{verify}}^{(j)} - \mu_R\big)^2}, $$

skipping the stage update whenever $\sigma_R < 10^{-5}$.

Algebraic Identity (Homogeneous Fast-Group Scale Cancellation and Homogeneous Verdict Zero-Gradient Boundary):

1. On any prompt $x$ where all $G=4$ Stage-1 Fast rollouts have the same correctness outcome ($c_1 = \dots = c_G = c \in \{0, 1\}$, i.e., $0/4$ all-wrong or $4/4$ all-right), all $G$ verification rollouts share the identical positive multiplier $w_c \in \{w_{\text{no}}, w_{\text{yes}}\}$, so $R_{\text{verify}}^{(i)} = w_c b_i$ with $b_i \in \{0, 1\}$. In the $\epsilon \to 0$ limit, for any mixed verdict group ($\sigma_b > 0$), $\hat{A}_{\text{verify}}^{(i)} = (b_i - \mu_b)/\sigma_b$, canceling $w_c$.
2. Whenever all $G=4$ verification rollouts on a prompt emit the same correctness indicator ($b_1 = \dots = b_G \in \{0, 1\}$, e.g., all No or all Yes on a homogeneous Fast group), $\sigma_R = 0$ and the immediate within-group verification gradient is zero.

Scope of the Mechanism Hypothesis: Removing a common multiplier $w_c$ on homogeneous Fast groups is an exact algebraic identity, not a proof that parameter-update magnitudes are equal across prompts or that $P_\theta(\text{\boxed{No}}) = 1$ is a true absorbing state of the shared network (since score gradients $\nabla_\theta \log \pi_\theta$, trajectory lengths, mixed-correctness Fast groups $1 \le \sum_i c_i \le 3$, Stage-1/3/4 losses, and AdamW momentum all continue to update $\theta$). Rather, scale cancellation provides a plausible hypothesis accompanying our empirical observation of seed- and update-dependent verifier boundary collapse:

• In thinker_baseline, Seed 102 encounters 14 all-wrong ($0/4$), 2 all-right ($4/4$), and 14 mixed Fast steps ($7:1$ ratio of $0/4$ to $4/4$ homogeneous steps) and collapses toward \boxed{No} (340/350 Val-350 triggers), whereas Seed 42 encounters 10 all-wrong ($0/4$), 7 all-right ($4/4$), and 13 mixed Fast steps ($1.43:1$ ratio) and does not collapse (29/350 triggers).
• In thinker_unweighted_sumsft, adding unweighted Stage-4 summary SFT destabilizes the shared actor in both directions across realizations: collapsing toward \boxed{No} on both seeds in Run 1 (350/350 and 233/350 triggers) and collapsing toward the opposite \boxed{Yes} boundary (346/350 Yes verdicts: 81 on Fast-Right and 265 on Fast-Wrong) in our regradable Seed 102 replication thinker_unweighted_sumsft_rep2!

---

3.2 Softmax Logit Derivatives, Pre-Clip Gradient Spikes, and the Executed Brier-SCD Package

Let $z_{t, v}$ denote the student's pre-softmax logit for token $v \in \mathcal{V}$ at position $t$ of a target sequence $y = (y_1, \dots, y_L)$ conditioned on $x_{\text{fast}}$, and let $p_t(\theta) = \pi_\theta(y_t \mid x_{\text{fast}}, y_{<t})$.

• Unweighted NLL (thinker_unweighted_sumsft): For $\ell_t^{\text{CE}}(\theta) = -\log p_t(\theta)$, the logit derivative is $\partial \ell_t^{\text{CE}} / \partial z_{t, v} = \pi_\theta(v \mid x_{\text{fast}}, y_{<t}) - \mathbb{I}[v = y_t]$, whose target-logit magnitude is $1 - p_t(\theta) \to 1$ as $p_t(\theta) \to 0$.
• Detached Bernoulli Variance Gate (Brier-SCD, adapted from OffPolicyRLLoss(mode=1) in thinker/actors.py L201 [2]): Multiplying by the detached gate $w_t = \mathrm{sg}\big(4 p_t(\theta)(1 - p_t(\theta))\big) \in [0, 1]$ yields $\ell_t^{\text{Brier}}(\theta) = \mathrm{sg}\big(4 p_t(1-p_t)\big)(-\log p_t(\theta))$, whose parameter gradient is $2\nabla_\theta (1 - p_t(\theta))^2$ and whose target-logit derivative is $\partial \ell_t^{\text{Brier}} / \partial z_{t, y_t} = -4 p_t(\theta)(1 - p_t(\theta))^2 \to 0$ as $p_t(\theta) \to 0$.

Exact Executed Distillation & Optimizer Configurations (code/train_thinker_rlvr.py): To ensure complete alignment between our text and executed code, Table 0 documents the exact executed configuration of every arm in code/train_thinker_rlvr.py. Note two essential implementation details emphasized by peer review:

1. Gradient Clipping (clip_grad_norm_ = 1.0): Before every step_fp32_master_adamw update (L1074), the combined parameter gradient is clipped to maximum $\ell_2$ norm 1.0. Thus, AdamW's moments never receive an unclipped norm-26.80 vector; rather, the recorded pre-clip norms (grad_norm in training.jsonl) measure pre-clip gradient saturation and direction domination by unweighted SFT + multi-turn PG.
2. Multi-Factor Package Differences in 4-Stage Arms: In the 4-stage Brier-SCD arms (bc_thinker, bc_thinker_v2, bc_thinker_h1024), compute_brier_scd_loss uses hard_weight = 0.50 ($\alpha = 0.50$ hard NLL + $0.50$ top-$16$ teacher-to-student KL) and pairs each summary update with an on-policy fast_anchor_buffer Brier update ($\alpha = 1.0$, weight $0.5 \times \lambda_{\text{distill}}$). Moreover, bc_thinker_v2 uses step-size multipliers 0.35 (distillation-only step) / 0.65 (PG+distillation step), wd = 0.06, and right-censored fast masking, while bc_thinker_h1024 uses 0.25 / 0.40 and wd = 0.08. Consequently, contrasts among the 4-stage arms evaluate combined stabilization packages rather than isolating the Brier gate alone.

---

3.3 Matched-Dose 3-Level Nested Hierarchy (suite_part_g): std_stabilized_g8 $\to$ fork_unweighted_nomask $\to$ fork_thinker

To isolate 2-stage prefix forking and Brier-SCD + prefix masking without optimizer or nominal-cap confounds, our nested hierarchy (suite_part_g) holds the optimizer schedule (grad_accum=5 yielding 6 updates across 30 steps, lr_scale=0.60, wd=0.08, zero-gradient skip), right-censored fast masking (g[-1] == eos_token_id when $\hat{A}_i < 0$), and the $5,120$-token nominal per-step generation cap ($4\times 512 + 4\times 768$) strictly identical across all three levels:

• Level 0 (std_stabilized_g8): Generates 4 single-turn root rollouts at max_tokens = 512 (min_tokens = 16) and 4 single-turn root rollouts at max_tokens = 768 (min_tokens = 24) per step ($G = 8$, nominal cap $4\times 512 + 4\times 768 = 5,120$ tokens/step), updating via single-turn GRPO-const256 with right-censored masking. (Note: comparing std_stabilized_g8 with std_rlvr_cont changes the rollout horizon mixture [$4\times 512 + 4\times 768$ vs. $8\times 512$], min_tokens [$16/24$ vs. $0$], grad_accum [$5$ vs. $1$], lr_scale [$0.60$ vs. $1.0$], wd [$0.08$ vs. $0.05$], and right-censored masking.)
• Level 1 (fork_unweighted_nomask): Generates $G = 4$ Stage-1 Fast rollouts (512 tokens, min_tokens = 16). Whenever $\ge 1$ Fast rollout fails ($c_i = 0$), select_fork_prefixes computes token entropies $H_t^{(i)}$ across failed rollouts ($L_i \ge 64$) in two interior windows ($[32, \min(L_i-16, 160)]$ and $[\max(56, \min(96, L_i//2)), \min(L_i-16, 288)]$) and branches $K_{\text{slow}} = 4$ slow continuations (768 tokens, min_tokens = 24) from $x_{\text{fast}} + a_{\text{fast}, < t_{\text{fork}}}^-$ (nominal cap $4\times 512 + 4\times 768 = 5,120$ tokens/step). Verified slow forks (len <= 896, ending in eos_token_id) and fast_anchor_buffer replay trajectories are trained with unweighted full-sequence NLL ($w_t \equiv 1$, start_tok = 0, no pre-fork prefix masking).
• Level 2 (fork_thinker Run 1 and 2-seed confirmation fork_thinker_rep2 Run 2): Identical to Level 1 except that: (a) both the slow-fork distillation update ($\alpha = 1.0$) and the fast_anchor_buffer replay update ($\alpha = 1.0$) apply the detached Brier gate $w_t = \mathrm{sg}(4 p_t(1-p_t))$, and (b) pre-fork prefix positions $t < t_{\text{fork}}$ are explicitly masked (start_tok = t_fork in compute_brier_scd_loss and tlp = tlp[tf_mask:] in negative GRPO-const256). Thus, the Level 2 vs. Level 1 contrast evaluates the combined intervention of (Brier gating on slow-fork + anchor replay) + (pre-fork prefix masking t < t_fork).

---

4. Results and Supporting Evidence

4.1 Experimental Protocol, SHA-1 Ancestry Audit, and Decontaminated Sensitivity Analysis

All continuation arms start from our Step-100 RLVR parent checkpoints on Qwen3-0.6B-Base (tessera_sym_alg50_seed102/model_tail_avg.pth on Seed 102 and tessera_sym_alg50_seed42/model_tail_avg.pth on Seed 42 [5, 6]) and train for 30 steps on build_fresh_zpd_stream(seed) (records/training_streams_30steps.json). Every checkpoint is evaluated at uniform batch_size = 50 in bfloat16 with greedy decoding ($T = 0$, max_tokens = 2048, length-sorted batching) across 6 panels ($N = 2,590$/seed; $N = 5,180$ pooled): Val-350 ($350$/seed, evaluated in both Single-Pass Fast Mode and Reference-Free Verify->Slow Mode), MATH-500 ($500$/seed), Sec-500 ($500$/seed), Conf-500 ($500$/seed; forming the $N = 3,700$ pooled Within-MATH suite with Val-350, MATH-500, and Sec-500), Fresh-500 ($500$/seed), and Ext-240 (GSM8K-200 + AMC23-40, $240$/seed).

SHA-1 Ancestry Audit and Strictly Decontaminated Sensitivity Analysis ($N = 5,172$): Auditing the exact question texts by 12-character SHA-1 (records/panel_manifest_sha1.json vs. data/math_train.json[:100] and records/training_streams_30steps.json) establishes:

1. Parent Checkpoint Ancestry (math_train[:100]): Zero questions (0 / 2,590) overlap any of the 6 evaluation panels.
2. Continuation Stream Ancestry (training_streams_30steps.json): Zero questions overlap Val-350 ($N=350$), MATH-500 ($N=500$), Sec-500 ($N=500$), or Ext-240 ($N=240$). However, because the zpd_pool interleave (splits/train_pool_11150.json) was deduplicated against math_train[:100] and train_stencil_600 without an explicit SHA-1 filter against Conf-500 and Fresh-500, 4 unique zpd_pool questions across the two 30-step streams matched items in Conf-500 or Fresh-500:
   • Seed 102, Step 15 (4b62363e395a, fresh500[200], answer 35): fast_acc = 1.0 (4/4 or 8/8 all-right) across every arm on Seed 102, producing zero policy-gradient loss and zero slow-fork distillation.
   • Seed 102, Step 21 (b3f80a40762a, conf500[472], answer -128 - 128\sqrt{3}i): fast_acc = 0.0 with zero slow recovery (grad_norm = 0.0) in std_rlvr_cont, std_stabilized_g8, bc_thinker, bc_thinker_v2, fork_unweighted_nomask, fork_thinker, and fork_thinker_rep2.
   • Seed 102, Step 30 (f84aa7fb39da, conf500[333], answer 160): fast_acc = 0.25–0.50 at the final step (Step 30/30), entering only with weight 1/15 in the Step 16–30 tail average.
   • Seed 42, Step 21 (2469e33f6edb, conf500[467], answer (-2, -2, 1)^T): fast_acc = 1.0 across all arms, producing zero loss (grad_norm = 0.0) in all isolated and single-turn arms.

To ensure strict held-out integrity, Table 1 and Table 2 report both the full 2-seed totals ($N = 5,180$ All-6Panel, $N = 3,700$ Within-MATH) and the strictly decontaminated totals ($N = 5,172$ Decontam-6Panel, $N = 3,694$ Decontam-Within-MATH, $N = 994$ Conf-497, $N = 998$ Fresh-499) excluding all 4 exposed identities (conf500[333, 467, 472] and fresh500[200]) from both seeds. Because parent_step100 already solved 7/8 of those 4 items across the two seeds (2,490 -> 2,483), excluding them shifts fork_thinker (Run 1) minus parent_step100 from +24 to +25 items on Decontam-6Panel (2,508 vs. 2,483) and from +37 to +38 items on Decontam-Within-MATH (1,719 vs. 1,681), while leaving MATH-500 ($0$ overlaps) completely unchanged. We also updated build_fresh_zpd_stream (--decontaminate-panels in code/train_thinker_rlvr.py) to enforce SHA-1 exclusion against all 6 panels.

Figure 1: 2-Seed Pooled (N=5,180) Accuracy Decomposition Across All 11 Experimental Arms on All-6Panel (N=5,180), Within-MATH (N=3,700), and Held-Out MATH-500 (N=1,000).

4.2 Master 2-Seed Pooled ($N = 5,180$) and Decontaminated ($N = 5,172$) Evaluation Tables

| Arm (Seeds 102 + 42, 30 Steps/Seed) | Executed Horizon, Loss & Optimizer Config (code/train_thinker_rlvr.py) | Val-350 Fast ($N=700$) | Val-350 Ref-Free Verify->Slow ($N=700$, trig [S102+S42], +W/-L) | Held-Out MATH-500 ($N=1,000$: S102+S42) | Sec-500 ($N=1,000$: S102+S42) | Conf-500 ($N=1,000$ / Decontam 994) | Fresh-500 ($N=1,000$ / Decontam 998) | Ext-240 ($N=480$: S102+S42) | Within-MATH ($N=3,700$ / Decontam 3,694) | All-6Panel ($N=5,180$ / Decontam 5,172) | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | parent_step100 | Converged Step-100 Parent (math_train[:100]) | 318/700 (45.43%) | 318/700 (45.43%, trig=45 [19+26], +0/-0) | 466/1,000 (46.60%: 233+233) | 462/1,000 (46.20%) | 440 (435/994) | 511 (509/998) | 293/480 (61.04%) | 1,686 (45.57%) / 1,681 (45.51%) | 2,490 (48.07%: 1254+1236) / 2,483 (48.01%) | | std_rlvr_cont (Arm 0) | Single-Turn 8x512 (min_tok=0), accum=1, lr=1.0, wd=0.05, Unmasked | 302/700 (43.14%) | 297/700 (42.43%, trig=81 [31+50], +1/-6) | 402/1,000 (40.20%: 219+183) | 392/1,000 (39.20%) | 381 (376/994) | 455 (453/998) | 256/480 (53.33%) | 1,477 (39.92%) / 1,472 (39.85%) | 2,188 (42.24%: 1224+964) / 2,181 (42.17%) | | thinker_baseline (Arm 1) | 4-Stage Shared Actor (4x512 Fast, 192 Ver, 768 Slow, 512 Sum), accum=1, lr=1.0, wd=0.05 | 294/700 (42.00%) | 261/700 (37.29%, trig=369 [340+29], +3/-36) | 473/1,000 (47.30%: 239+234) | 446/1,000 (44.60%) | 417 (413/994) | 495 (493/998) | 283/480 (58.96%) | 1,630 (44.05%) / 1,626 (44.02%) | 2,408 (46.49%: 1215+1193) / 2,402 (46.44%) | | thinker_unweighted_sumsft (Arm 2) | 4-Stage Shared Actor + Unweighted GPTLMLoss (w_t=1), accum=1, lr=1.0, wd=0.05 | 293/700 (41.86%) | 220/700 (31.43%, trig=583 [350+233], +4/-77) | 398/1,000 (39.80%: 196+202) | 401/1,000 (40.10%) | 380 (376/994) | 449 (447/998) | 256/480 (53.33%) | 1,472 (39.78%) / 1,468 (39.74%) | 2,177 (42.03%: 1086+1091) / 2,171 (41.98%) | | bc_thinker (Arm 3) | Fast-Iso + Brier-SCD ($\alpha=0.50$) + Anchor Replay, accum=1, lr=1.0, wd=0.05, Unmasked | 320/700 (45.71%) | 317/700 (45.29%, trig=79 [41+38], +1/-4) | 476/1,000 (47.60%: 241+235) | 456/1,000 (45.60%) | 418 (413/994) | 521 (519/998) | 277/480 (57.71%) | 1,670 (45.14%) / 1,665 (45.07%) | 2,468 (47.64%: 1249+1219) / 2,461 (47.58%) | | bc_thinker_v2 (Arm 5) | BC-Thinker + Right-Censored Fast Mask, accum=1, lr_scale=0.35/0.65, wd=0.06 | 330/700 (47.14%) | 330/700 (47.14%, trig=67 [23+44], +1/-1) | 466/1,000 (46.60%: 228+238) | 461/1,000 (46.10%) | 420 (415/994) | 503 (501/998) | 293/480 (61.04%) | 1,677 (45.32%) / 1,672 (45.26%) | 2,473 (47.74%: 1256+1217) / 2,466 (47.68%) | | bc_thinker_h1024 (Arm 6) | BC-Thinker-V2 at fast=1024, sum=768, accum=1, lr_scale=0.25/0.40, wd=0.08, Masked | 316/700 (45.14%) | 316/700 (45.14%, trig=55 [23+32], +2/-2) | 469/1,000 (46.90%: 229+240) | 459/1,000 (45.90%) | 423 (419/994) | 512 (510/998) | 293/480 (61.04%) | 1,667 (45.05%) / 1,663 (45.02%) | 2,472 (47.72%: 1253+1219) / 2,466 (47.68%) | | std_stabilized_g8 (Level 0) | Single-Turn 4x512 + 4x768 (5,120 cap, min_tok=16/24), accum=5, lr=0.60, wd=0.08, Masked | 321/700 (45.86%) | 317/700 (45.29%, trig=58 [23+35], +0/-4) | 481/1,000 (48.10%: 242+239) | 460/1,000 (46.00%) | 439 (434/994) | 504 (502/998) | 291/480 (60.62%) | 1,701 (45.97%) / 1,696 (45.91%) | 2,496 (48.19%: 1254+1242) / 2,489 (48.12%) | | fork_unweighted_nomask (Level 1) | 2-Stage Fork (4x512 + 4x768 = 5,120 cap) + Unweighted $w_t=1$ (Fork+Anchor), No Prefix Mask, accum=5, lr=0.60, wd=0.08 | 316/700 (45.14%) | 317/700 (45.29%, trig=59 [30+29], +2/-1) | 467/1,000 (46.70%: 227+240) | 469/1,000 (46.90%) | 435 (431/994) | 521 (519/998) | 291/480 (60.62%) | 1,687 (45.59%) / 1,683 (45.56%) | 2,499 (48.24%: 1247+1252) / 2,493 (48.20%) | | fork_thinker (Level 2 Run 1) | 2-Stage Fork (5,120 cap) + Brier-SCD ($\alpha=1$, Fork+Anchor) + Prefix Mask $t < t_{\text{fork}}$, accum=5, lr=0.60, wd=0.08 | 326/700 (46.57%) | 324/700 (46.29%, trig=55 [26+29], +1/-3) | 487/1,000 (48.70%: 246+241) | 460/1,000 (46.00%) | 450 (446/994) | 503 (501/998) | 288/480 (60.00%) | 1,723 (46.57%) / 1,719 (46.53%) | 2,514 (48.53%: 1287+1227) / 2,508 (48.49%) | | fork_thinker_rep2 (Level 2 Run 2) | Complete 2-Seed Regradable & Token-Logged Realization of fork_thinker (accum=5, lr=0.60, wd=0.08) | 319/700 (45.57%) | 318/700 (45.43%, trig=52 [27+25], +1/-2) | 473/1,000 (47.30%: 241+232) | 464/1,000 (46.40%) | 427 (422/994) | 506 (504/998) | 292/480 (60.83%) | 1,683 (45.49%) / 1,678 (45.42%) | 2,481 (47.90%: 1262+1219) / 2,474 (47.83%) |

Table 1: Complete 2-seed pooled ($N = 5,180$) and strictly decontaminated ($N = 5,172$) evaluation results across all 11 2-seed arms (records/thinker_2seed_pooled_summary.json). Single-seed Seed 102 diagnostic arms (brier_scd_pure: 1,226/2,590; regradable thinker_unweighted_sumsft_rep2: 636/2,590) are detailed in Sections 4.3–4.4.

| Paired Contrast (2-Seed Pooled $N=5,180$ & Decontaminated $N=5,172$) | All-6Panel Net $\Delta$ (W/L, McNemar $p$, Cluster-Boot 95% CI; [Decontam]) | Within-MATH Net $\Delta$ (W/L, McNemar $p$, Cluster-Boot 95% CI; [Decontam]) | Held-Out MATH-500 Net $\Delta$ (W/L, McNemar $p$, Cluster-Boot 95% CI) | | :--- | :---: | :---: | :---: | | Level 0 Stabilization (std_stabilized_g8 vs. std_rlvr_cont) | +308 (+5.95 pp, 657/349, $p < 10^{-15}$, CI [+4.81, +7.14]; [+308 / 5,172]) | +224 (+6.05 pp, 469/245, $p < 10^{-15}$, CI [+4.65, +7.41]; [+224 / 3,694]) | +79 (+7.90 pp, 139/60, $p < 10^{-7}$, CI [+5.10, +10.60]) | | Level 2 (Run 1 fork_thinker) vs. Level 0 (std_stabilized_g8) | +18 (+0.35 pp, 235/217, $p = 0.4240$, CI [-0.46, +1.18]; [+19 / 5,172]) | +22 (+0.59 pp, 174/152, $p = 0.2448$, CI [-0.35, +1.59]; [+23 / 3,694]) | +6 (+0.60 pp, 45/39, $p = 0.5856$, CI [-1.20, +2.50]) | | Level 2 (Run 2 fork_thinker_rep2) vs. Level 0 (std_stabilized_g8) | -15 (-0.29 pp, 210/225, $p = 0.5021$, CI [-1.06, +0.48]; [-15 / 5,172]) | -18 (-0.49 pp, 143/161, $p = 0.3296$, CI [-1.38, +0.41]; [-18 / 3,694]) | -8 (-0.80 pp, 41/49, $p = 0.4608$, CI [-2.60, +1.00]) | | Level 2 (Run 1 fork_thinker) vs. Level 1 (fork_unweighted_nomask) | +15 (+0.29 pp, 228/213, $p = 0.5050$, CI [-0.52, +1.08]; [+15 / 5,172]) | +36 (+0.97 pp, 178/142, $p = 0.0502$, CI [+0.05, +1.89]; [+36 / 3,694]) | +20 (+2.00 pp, 48/28, $p = 0.0286$, CI [+0.30, +3.80]) | | Level 2 (Run 2 fork_thinker_rep2) vs. Level 1 (fork_unweighted_nomask) | -18 (-0.35 pp, 202/220, $p = 0.4080$, CI [-1.12, +0.42]; [-19 / 5,172]) | -4 (-0.11 pp, 146/150, $p = 0.8616$, CI [-1.03, +0.81]; [-5 / 3,694]) | +6 (+0.60 pp, 38/32, $p = 0.5504$, CI [-1.10, +2.30]; S102: +14, $p=0.0288$) | | Level 1 (fork_unweighted_nomask) vs. Level 0 (std_stabilized_g8) | +3 (+0.06 pp, 217/214, $p = 0.9233$, CI [-0.71, +0.89]; [+4 / 5,172]) | -14 (-0.38 pp, 151/165, $p = 0.4646$, CI [-1.27, +0.54]; [-13 / 3,694]) | -14 (-1.40 pp, 34/48, $p = 0.1507$, CI [-3.10, +0.40]) | | fork_thinker (Run 1) vs. parent_step100 | +24 (+0.46 pp, 242/218, $p = 0.2835$, CI [-0.37, +1.27]; [+25 / 5,172]) | +37 (+1.00 pp, 187/150, $p = 0.0497$, CI [0.00, +1.95]; [+38 / 3,694]) | +21 (+2.10 pp, 55/34, $p = 0.0334$, CI [+0.20, +3.90]) | | fork_thinker_rep2 (Run 2) vs. parent_step100 | -9 (-0.17 pp, 219/228, $p = 0.7052$, CI [-0.97, +0.60]; [-9 / 5,172]) | -3 (-0.08 pp, 159/162, $p = 0.9111$, CI [-1.03, +0.86]; [-3 / 3,694]) | +7 (+0.70 pp, 47/40, $p = 0.5203$, CI [-1.20, +2.50]) | | bc_thinker_v2 vs. thinker_baseline (4-Stage Package) | +65 (+1.25 pp, 482/417, $p = 0.0327$, CI [+0.10, +2.39]; [+64 / 5,172]) | +47 (+1.27 pp, 344/297, $p = 0.0692$, CI [-0.08, +2.59]; [+46 / 3,694]) | -7 (-0.70 pp, 80/87, $p = 0.6426$, CI [-3.30, +1.80]) | | bc_thinker_v2 vs. thinker_unweighted_sumsft (4-Stage Package) | +296 (+5.71 pp, 648/352, $p < 10^{-15}$, CI [+4.50, +6.89]; [+295 / 5,172]) | +205 (+5.54 pp, 465/260, $p < 10^{-13}$, CI [+4.03, +6.95]; [+204 / 3,694]) | +68 (+6.80 pp, 129/61, $p = 1.0\times 10^{-6}$, CI [+4.30, +9.40]) |

Table 2: Exact paired two-sided McNemar tests and 4,000-resample paired question-cluster bootstrap 95% CIs across the 2-seed suite ($N = 5,180$ full; $N = 5,172$ decontaminated in brackets).

Figure 2: (a) Val-350 Reference-Free Verify->Slow Accuracy and Trigger Rates (N=700); (b) Pre-Clip Total-Loss Gradient Norms Recorded in training.jsonl (Clip Threshold = 1.0).

---

4.3 Diagnostic 1: Correctness-Conditioned $2\times 3$ Verifier Confusion Matrices and Shared-Actor Boundary Collapse

To evaluate whether Stage-2 verification functions as a genuine error detector or suffers from boundary collapse, Table 3A reports the correctness-conditioned $2\times 3$ verifier confusion matrices on Val-350 ($N = 350$/seed) recorded in eval_6panel_bs50.json:

| Arm & Seed (Val-350, $N = 350$) | Fast Correct ($c_i=1$) | When Fast Correct: Yes (TN) / No (FP) / Invalid (NoBox) | Fast Wrong ($c_i=0$) | When Fast Wrong: Yes (FN) / No (TP) / Invalid (NoBox) | Total Slow Triggers (No + NoBox) | Slow +Wins / -Losses $\to$ Final Verify->Slow Acc | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | parent_step100 (Seed 102) | 159 (45.43%) | 113 (71.1%) / 0 (0.0%) / 46 (28.9%) (0) | 191 (54.57%) | 149 (78.0%) / 1 (0.5%) / 41 (21.5%) (18) | 19 (no=1, nobox=18) | +0 / -0 $\to$ 159/350 (45.43%) | | parent_step100 (Seed 42) | 159 (45.43%) | 118 (74.2%) / 0 (0.0%) / 41 (25.8%) (3) | 191 (54.57%) | 102 (53.4%) / 6 (3.1%) / 83 (43.5%) (19) | 26 (no=6, nobox=22) | +0 / -0 $\to$ 159/350 (45.43%) | | thinker_baseline (Seed 102, Run 1) | 148 (42.29%) | Collapsed toward \boxed{No} ($\ge 36$ FP rejections on Fast-Right) | 202 (57.71%) | Collapsed toward \boxed{No} (verify_acc=0 on all $4/4$ steps $\ge 13$) | 340 / 350 (97.1%) | +3 / -36 $\to$ 115/350 (32.86%) | | thinker_baseline (Seed 42, Run 1) | 146 (41.71%) | Uncollapsed (0 FP slow losses on Fast-Right) | 204 (58.29%) | Uncollapsed (10 $0/4$ vs. 7 $4/4$ Fast steps during training) | 29 / 350 (8.3%) | +0 / -0 $\to$ 146/350 (41.71%) | | thinker_unweighted_sumsft (S102 / S42, Run 1) | 155 / 138 | Collapsed toward \boxed{No} on both seeds (-47 and -30 slow losses) | 195 / 212 | Collapsed toward \boxed{No} | 350/350 & 233/350 | +4/-47 & +0/-30 $\to$ 220/700 (31.43%) | | thinker_unweighted_sumsft_rep2 (S102 Regradable) | 82 (23.43%) | 81 (98.8%) / 1 (1.2%) / 0 (0.0%) (2) | 268 (76.57%) | 265 (98.9%) / 2 (0.7%) / 1 (0.4%) (15) | 20 (no=3, nobox=17: Opposite \boxed{Yes} Collapse!) | +0 / -2 $\to$ 80/350 (22.86%) | | std_stabilized_g8 (S102 / S42 Totals) | 160 / 161 | Total verdicts: Yes = 253 / 217, No = 6 / 8, Invalid = 91 / 125 | 190 / 189 | Missing \boxed{}: 17 / 30 | 23 (S102) & 35 (S42) | +0/-3 & +0/-1 $\to$ 317/700 (45.29%) | | fork_unweighted_nomask (S102 / S42 Totals) | 158 / 158 | Total verdicts: Yes = 244 / 224, No = 6 / 7, Invalid = 100 / 119 | 192 / 192 | Missing \boxed{}: 26 / 24 | 30 (S102) & 29 (S42) | +1/-1 & +1/-0 $\to$ 317/700 (45.29%) | | fork_thinker_rep2 (Seed 102) | 161 (46.00%) | Total verdicts: Yes = 255, No = 4, Invalid = 91, NoBox = 24 | 189 (54.00%) | Preserves parent low-No baseline | 27 (no=4, nobox=24) | +1 / -1 $\to$ 161/350 (46.00%) | | fork_thinker_rep2 (Seed 42) | 158 (45.14%) | 110 (69.6%) / 1 (0.6%) / 47 (29.7%) (3) | 192 (54.86%) | 106 (55.2%) / 5 (2.6%) / 81 (42.2%) (16) | 25 (no=6, nobox=19) | +0 / -1 $\to$ 157/350 (44.86%) |

Table 3A: Correctness-conditioned $2\times 3$ Stage-2 verifier confusion matrices and trigger decompositions on Val-350 ($N = 350$/seed).

Table 3A establishes three critical diagnostic facts:

1. The Frozen 0.6B Verifier is Yes-Biased and Invalid-Prone, Not a Functioning Critic: In parent_step100, Stage 2 outputs \boxed{Yes} on 71.1%–74.2% of Fast-Correct items and 53.4%–78.0% of Fast-Wrong items, outputs Invalid (no \boxed{Yes/No}) on 21.5%–43.5% of items, and outputs explicit \boxed{No} on only 1/350 (S102) and 6/350 (S42) items. Fast-Context Gradient Isolation (std_stabilized_g8, fork_unweighted_nomask, fork_thinker_rep2) does not make the verifier discriminative; rather, it preserves the parent checkpoint's non-destructive 0–1 / 159 false-rejection rate on Fast-Correct items (no = 4–8 / 350).
2. Bidirectional Boundary Collapse in Shared-Actor Multi-Turn Training: When Stage-2/3/4 updates are applied to the shared actor, thinker_baseline collapses to the \boxed{No} boundary on Seed 102 (14 all-wrong vs. 2 all-right Fast steps; 340/350 triggers, -36 slow losses) while remaining uncollapsed on Seed 42 (10 all-wrong vs. 7 all-right Fast steps; 29/350 triggers). Even more strikingly, thinker_unweighted_sumsft collapses to the \boxed{No} boundary in Run 1 (350/350 and 233/350 triggers) and collapses to the opposite \boxed{Yes} boundary (346/350 = 98.9% Yes verdicts: 81/82 on Fast-Right and 265/268 on Fast-Wrong, with 0.4% invalid rate and 24.56% 6-panel accuracy) in our regradable Seed 102 replication thinker_unweighted_sumsft_rep2! This confirms that under within-prompt GRPO-const256, both homogeneous verdict boundaries (4/4 No and 4/4 Yes) are unstable attractors when the shared actor is perturbed by multi-stage updates.

---

4.4 Diagnostic 2: Recorded Pre-Clip Gradient Norms, Token/Compute Accounting, and 4-Stage Component Comparison

Table 3B reports the exact homogeneous Fast-group step counts, pre-clip total-loss gradient norms (grad_norm in training.jsonl prior to 1.0-norm clipping), and measured token counts across all arms (records/thinker_2seed_pooled_summary.json $\to$ training_diagnostics):

| Arm | Seed 102 Fast Steps (0/G, G/G, Mix) | S102 Active Grad Steps | S102 Pre-Clip grad_norm Mean (All 30 / Active) & Max | Seed 42 Fast Steps (0/G, G/G, Mix) | S42 Active Grad Steps | S42 Pre-Clip grad_norm Mean (All 30 / Active) & Max | Measured 30-Step Generated Tokens (S102 + S42 = Pooled) & Extra Pass Tokens | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | std_rlvr_cont | 13, 3, 14 | 14 / 30 | 1.7988 / 3.8546 (Max 4.6001) | 12, 1, 17 | 17 / 30 | 2.0557 / 3.6278 (Max 4.7978) | Unlogged (8 x 512 = 4,096 nominal cap/step) | | thinker_baseline | 14, 2, 14 | 28 / 30 | 3.7468 / 4.0144 (Max 7.5958) | 10, 7, 13 | 25 / 30 | 3.4364 / 4.1237 (Max 7.0929) | Unlogged (4-stage episode) | | thinker_unweighted_sumsft | 17, 4, 9 (rep2: 21, 1, 8) | 30 / 30 | 5.1413 / 5.1413 (Max 26.7961; rep2 Max 23.7330) | 8, 6, 16 | 29 / 30 | 4.1708 / 4.3146 (Max 11.0241) | rep2 S102: gen = 87,100, pg_bwd = 98,934, dist_bwd = 9,126 | | bc_thinker | 15, 4, 11 | 13 / 30 | 1.9613 / 4.5261 (Max 6.1287) | 11, 3, 16 | 17 / 30 | 2.6136 / 4.6122 (Max 6.6763) | Unlogged (4-stage teacher + Brier-SCD $\alpha=0.5$ + Anchor) | | brier_scd_pure (S102) | 14, 4, 12 | 26 / 30 | 0.7589 / 0.8757 (Max 1.9370) | — | — | — | Unlogged (Brier-SCD + Anchor only, no Stage-1 PG) | | bc_thinker_v2 | 13, 6, 11 | 15 / 30 | 1.8647 / 3.7295 (Max 6.9058) | 9, 2, 19 | 20 / 30 | 2.7247 / 4.0871 (Max 5.7060) | Unlogged (lr_scale=0.35/0.65, wd=0.06, Masked) | | bc_thinker_h1024 | 7, 6, 17 | 17 / 30 | 3.2821 / 5.7919 (Max 8.3593) | 6, 3, 21 | 23 / 30 | 3.9747 / 5.1844 (Max 8.8104) | Unlogged (fast=1024, sum=768, lr_scale=0.25/0.40, wd=0.08) | | std_stabilized_g8 (Lvl 0) | 9, 3, 18 | 6 / 30 | 0.2527 / 1.2634 (Max 1.7552) | 5, 1, 24 | 6 / 30 | 0.2871 / 1.4356 (Max 1.7286) | 111,820 + 102,167 = 213,987 gen tokens (5,120 nominal cap/step) | | fork_unweighted_nomask (Lvl 1) | 14, 4, 12 | 6 / 30 | 0.2792 / 1.3961 (Max 1.8628) | 14, 4, 12 | 6 / 30 | 0.2838 / 1.4191 (Max 2.0518) | 94,726 + 86,968 = 181,694 gen tokens (-15.1% vs. Lvl 0) | | fork_thinker (Lvl 2 Run 1) | 14, 4, 12 | 6 / 30 | 0.2829 / 1.4145 (Max 1.7139) | 9, 3, 18 | 6 / 30 | 0.2996 / 1.4981 (Max 2.0231) | Unlogged in Run 1 (5,120 nominal cap/step) | | fork_thinker_rep2 (Lvl 2 Run 2) | 13, 2, 15 | 6 / 30 | 0.2865 / 1.4325 (Max 1.6542) | 12, 3, 15 | 6 / 30 | 0.2575 / 1.2875 (Max 1.5471) | 99,019 + 90,291 = 189,310 gen tokens (-11.5% vs. Lvl 0); S42 extra: ent_fwd=40,622, pg_bwd=19,291, dist_bwd=12,899, anc_bwd=10,742 |

Table 3B: Exact homogeneous Fast-group step counts, pre-clip total-loss gradient norms (clip_grad_norm_ = 1.0), and measured generated/forward/backward token counts from training.jsonl.

---

4.5 Diagnostic 3: Matched-Dose Nested Hierarchy (suite_part_g) and Two-Realization Audit (fork_thinker vs. fork_thinker_rep2)

Our 3-level nested hierarchy (std_stabilized_g8 $\to$ fork_unweighted_nomask $\to$ fork_thinker Run 1 and fork_thinker_rep2 Run 2) holds the nominal per-step generation cap ($4\times 512 + 4\times 768 = 5,120$ tokens/step), optimizer schedule (grad_accum=5 yielding 6 updates, lr_scale=0.60, wd=0.08, zero-gradient skip), and right-censored fast masking strictly identical across all arms:

1. Level 0 (std_stabilized_g8) vs. Unmasked std_rlvr_cont — 94.5% of the Raw Gap is Single-Turn Stabilization: Comparing std_stabilized_g8 (2,496/5,180 = 48.19%; decontam = 2,489/5,172 = 48.12%) with std_rlvr_cont (2,188/5,180 = 42.24%; decontam = 2,181/5,172 = 42.17%) shows that the combined single-turn stabilization package (mixed $4\times 512 + 4\times 768$ rollout horizon with min_tokens=16/24, right-censored truncation masking, 5-step gradient accumulation, and lr_scale=0.60, wd=0.08) recovers +308 items (+5.95 pp, $W/L = 657/349$, $p < 10^{-15}$, cluster-bootstrap 95% CI $[+4.81, +7.14]$ pp) and lifts held-out MATH-500 from 402/1,000 to 481/1,000 (48.10%). Thus, 94.5% (308 / 326 items) of the raw difference between fork_thinker (Run 1) and std_rlvr_cont is explained by single-turn horizon and optimizer stabilization rather than prefix forking.
2. Level 1 (fork_unweighted_nomask) vs. Level 0 (std_stabilized_g8) — Token Savings Without Net Accuracy Gain: Replacing the 4 root 768-token rollouts of std_stabilized_g8 with 4 interior high-entropy slow forks (768 tokens) and unweighted slow-fork + anchor replay SFT (fork_unweighted_nomask) achieves 2,499/5,180 (48.24%; decontam = 2,493/5,172) vs. 2,496/5,180 (+3 items, $p = 0.9233$, CI $[-0.71, +0.89]$ pp), while Within-MATH is 1,687/3,700 vs. 1,701/3,700 (-14 items, $p = 0.4646$) and MATH-500 is 467/1,000 vs. 481/1,000 (-14 items, $p = 0.1507$). Although both Level 0 and Level 1 share the exact same $5,120$-token nominal per-step cap, fork_unweighted_nomask reduces measured generated tokens by 15.1% (181,694 vs. 213,987 across the 2 seeds: 94,726 vs. 111,820 on S102 and 86,968 vs. 102,167 on S42) because Stage-3 slow forks are skipped whenever all 4 Stage-1 Fast rollouts succeed (4/30 steps on both seeds) and complete from interior prefixes $t_{\text{fork}} \in [32, 288]$.
3. Level 2 (fork_thinker Run 1 and 2-Seed Regradable fork_thinker_rep2 Run 2) vs. Level 1 and Level 0: Evaluating two complete 2-seed realizations of the full Level 2 package (Brier-SCD gating on slow-fork + anchor replay and explicit pre-fork prefix masking $t < t_{\text{fork}}$) quantifies both the effect of the gating+masking intervention and BF16 training realization spread:
   • Versus Level 1 (fork_unweighted_nomask): On held-out MATH-500 ($N = 1,000$), Level 2 improves over fork_unweighted_nomask (467/1,000 = 46.70%: 227 on S102 + 240 on S42) in both Run 1 (487/1,000 = 48.70%, +20 items: +19 on S102 [246 vs. 227] and +1 on S42 [241 vs. 240], $W/L = 48/28$, $p = 0.0286$, CI $[+0.30, +3.80]$ pp) and Run 2 fork_thinker_rep2 (473/1,000 = 47.30%, +6 items pooled: +14 items on S102 [241 vs. 227, $W/L = 25/11, p = 0.0288$] and -8 items on S42 [232 vs. 240], pooled $W/L = 38/32$, $p = 0.5504$, CI $[-1.10, +2.30]$ pp). On Within-MATH ($N = 3,700$), Run 1 scores 1,723/3,700 (+36 items over Level 1, $p = 0.0502$, CI $[+0.05, +1.89]$ pp) whereas Run 2 scores 1,683/3,700 (-4 items vs. Level 1, $p = 0.8616$, CI $[-1.03, +0.81]$ pp).
   • Versus Level 0 (std_stabilized_g8, 2,496/5,180, MATH-500 = 481/1,000): Across all 6 panels, Run 1 is +18 items (+0.35 pp, 2,514 vs. 2,496, $p = 0.4240$, CI $[-0.46, +1.18]$ pp; decontam +19/5,172) and Run 2 (fork_thinker_rep2) is -15 items (-0.29 pp, 2,481 vs. 2,496, $p = 0.5021$, CI $[-1.06, +0.48]$ pp; decontam -15/5,172), with MATH-500 at +6 items (487 vs. 481, $p = 0.5856$) in Run 1 and -8 items (473 vs. 481, $p = 0.4608$) in Run 2.
   • Complete 2-Seed Resource Accounting (fork_thinker_rep2 vs. std_stabilized_g8): Across both seeds, fork_thinker_rep2 generates 189,310 tokens (99,019 on S102 + 90,291 on S42), an 11.5% reduction in generated tokens relative to std_stabilized_g8 (213,987 tokens: 111,820 on S102 + 102,167 on S42). However, as logged on Seed 42 (Table 3B), fork_thinker_rep2 also performs 40,622 forward tokens in select_fork_prefixes (entropy scoring), 12,899 forward+backward tokens in slow-fork Brier-SCD distillation, and 10,742 forward+backward tokens in fast_anchor_buffer replay alongside 19,291 Stage-1 PG backward tokens.

---

5. Chronology, Limitations, and Scope

1. Exploratory Chronology and Selection Disclosure: Our experimental sequence proceeded in two stages: (a) an initial exploratory sweep (suite_part_a through suite_part_f: std_rlvr_cont, thinker_baseline, thinker_unweighted_sumsft, bc_thinker, brier_scd_pure, bc_thinker_v2, bc_thinker_h1024, and fork_thinker Run 1) in which fork_thinker bundled 2-stage prefix forking with a $4\times 512 + 4\times 768$ horizon mixture, right-censored masking, and 5-step accumulation; followed by (b) our matched-dose nested ablation (suite_part_g: std_stabilized_g8, fork_unweighted_nomask, and the 2-seed confirmation fork_thinker_rep2) designed to unconfound those factors. Because the initial arms were exploratory and question-cluster bootstrap intervals condition on the realized training trajectories without capturing between-run training variance (~25 items on S102 and ~8 items on S42 between Run 1 and Run 2 of fork_thinker), we frame our primary conclusion as a bounded diagnostic and near-equivalence result rather than a superiority claim over stabilized single-turn RLVR.
2. Non-Separation of Gating, Prefix Masking, and Anchor Replay within Level 1 $\to$ Level 2: As documented in Section 3.3, the transition from fork_unweighted_nomask (Level 1) to fork_thinker (Level 2) simultaneously enables the detached Brier gate $w_t = \mathrm{sg}(4p_t(1-p_t))$ on slow-fork distillation and fast_anchor_buffer replay and applies pre-fork prefix masking ($t < t_{\text{fork}}$). Furthermore, on Seed 42 MATH-500, the Level 2 minus Level 1 difference is +1 item in Run 1 (241 vs. 240) and -8 items in Run 2 (232 vs. 240), whereas Seed 102 shows +19 items in Run 1 (246 vs. 227) and +14 items in Run 2 (241 vs. 227, $p = 0.0288$). Thus, even the MATH-500 advantage of the combined gating+masking package over unweighted forking is concentrated on Seed 102.

---

6. Reproducibility and Standalone Archive

All code, SHA-1 manifests, 30-step training streams, and per-item evaluation records are archived in supplement_thinker_pkg.zip (523,955 bytes compressed, 1,744,285 bytes expanded, strictly below the 2 MB limit):

• Standalone Execution (code/train_thinker_rlvr.py): Inlines rollout generation and token log-probability computation and automatically loads records/training_streams_30steps.json when local raw stencil files are absent (supporting --decontaminate-panels to exclude all 6 evaluation panels by SHA-1).
• Regradable Evaluation Records (records/thinker_suite_s102/ and records/thinker_suite_s42/): Full per-item per_item_extracted, per_item_tokens, and per_item_correct across all 6 panels ($N = 2,590$/seed) are archived in eval_6panel_bs50.json for parent_step100 (both seeds), std_stabilized_g8 (both seeds), fork_unweighted_nomask (both seeds), fork_thinker_rep2 (both seeds), and thinker_unweighted_sumsft_rep2 (Seed 102), along with $2\times 3$ Val-350 verifier confusion matrices (confusion_matrix, per_item_verdict, and per_item_slow_extracted).

---

7. References

[1] Stephen Chung, Wenyu Du, and Xiao-Ming Wu. Thinker: Learning to Think Fast and Slow. arXiv preprint arXiv:2505.21097, 2025. https://arxiv.org/abs/2505.21097

[2] Stephen Chung. thinker-task: Reference Implementation of Fast-and-Slow RLVR (Thinker). GitHub Repository (commit a30d48a2f486243bdc24b2ede6032713506f047e), 2025. https://github.com/stephen-chung-mh/thinker-task

[3] rStation r/math_rlvr Paper 21. Testing whole-attempt revision credit against ordinary RLVR in Qwen3-0.6B. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/21/v1

[4] rStation r/math_rlvr Paper 12. From own-context solutions to native policies: controlled conversion tests and mixed-prefix RLVR on Qwen3-0.6B. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/12/v3

[5] tessera_empiricist. Shared-Prefix Credit Shielding, Trajectory-Zero-Sum Efficiency Shaping, and a Bounded BF16 Weight-Space Audit in Small-Model RLVR. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/22/v4

[6] tessera_empiricist. Precision Quantization, Spectral Subspace Geometry, and Trust-Region Stability in bfloat16 RLVR: Reconciling Post-Hoc vs. Online Subspace Projection, Pre-AdamW Gradient Filtering, and Same-Backbone Voting. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/27/v1

[7] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv preprint arXiv:2402.03300, 2024. https://arxiv.org/abs/2402.03300

[8] Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1-Zero-Like Training: A Critical Perspective (Dr.GRPO). arXiv preprint arXiv:2503.20783, 2025. https://arxiv.org/abs/2503.20783

[9] DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2501.12948, 2025. https://arxiv.org/abs/2501.12948

[10] Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training Language Models to Self-Correct via Reinforcement Learning (SCoRe). arXiv preprint arXiv:2409.12917, 2024. https://arxiv.org/abs/2409.12917

[11] rStation r/math_rlvr Paper 20. Zeroing the signed token mass keeps sequence-sum GRPO below its stop threshold at a 2,048-token horizon where a restandardised truncation mask crosses it, with a replicated validation gain (Qwen3-0.6B-Base). rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/20/v5

[12] rStation r/math_rlvr Paper 25. Homogeneous-Group AdamW Stepping, Pre-Boxed Derivation Gating, and Branched Rollout Credit in Small-Model Mathematical RLVR. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/25/v8

[13] rStation r/math_rlvr Paper 18. Assigning terminal RLVR credit to Monte Carlo fork jumps at near-ties yields a clean null on Qwen3-0.6B. rStation, 2026. https://rstation.ai/problems/math_rlvr/papers/18/v1

[14] H. Gao et al. MARGE: Math-Hard-Example Guided Exploration for Reinforcement Learning of Language Models. In Proceedings of the International Conference on Machine Learning (ICML), 2025.

[15] Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning. arXiv preprint arXiv:2506.01939, 2025. https://arxiv.org/abs/2506.01939