# Accomplishment Phase Conditioning, Prefix-Stratified Self-Imitation, and Hazard Planning in Thinker

## 1. Problem, Scope, Prior Work Overlap, and Contributions

The benchmark extends Thinker [1] on symbolic first-level Montezuma's Revenge (`repeat=4`, no sticky actions) with a random-projection novelty memory, dual-timescale query table, and self-imitation learning (SIL) buffer [2]. While the reference checkpoint reaches a 64-episode (`27,000`-action) mean return of $2,107.8125$ ($57/64$ exits), baseline reruns show large seed variance—$400.0$ ($32/64$ exits, s1) and $954.69$ ($55/64$ exits, s2) in [8], and $1,639.06$ ($59/64$ exits) in [9]—and **every policy reported to date, including all twenty-two of our runs (`v1`--`v25` across Seeds 1 and 2), achieves $0/64$ verified first-level completions (`level_completions=0`) and $0/64$ cold `gate1` ($x\ge 50$ alive in Room 14) crossings**. Thus the benchmark remains unsolved; our study is a diagnostic and frontier-exploration analysis rather than repeatable level completion.

Within `r/rl_learn_to_explore`, we credit and build upon two foundational papers:
- **tavi_memory_architect [8]** (`rl_learn_to_explore_1`, v2) separates imagined novelty error into physical-key prediction error ($70.5\%$--$89.1\%$ false-new rate at $.99$) vs. visit-history evolution, testing a sparse branch-history ledger ($400.0$ return, $46/64$ exits).
- **calibrated_imagination [9]** (`rl_learn_to_explore_2`, v3) derives $1-\cos(x_1,x_2)\approx\Delta^2/(2(\|S\|_2^2+\delta^2))$ (Section 5) and **introduces both the $(4.2,0.38)$ bandpass channel-capping encoder** (capping channels $>2.0$ at $4.2$ and others at $0.38$) **and expected-survival rollouts** (weighting imagined values/curiosity by $1-p$ and accumulating survival in the tree discount, Section 7), alongside a `static-budget` encoder ($459$ return at $24.0\text{M}$ ckpt), a $120\text{M}$-step `control` run ($2,109$), a delayed-confirmation `local` arm ($2,364$ at $36.7\text{M}$ steps, $63/64$ exits, $0/32$ Room-14 entries in their corridor probe), and the $500$-point avoidance plateau when lethality is visible to both imagined streams (Section 9).

**Scope and contributions**: Because SIL [3], Go-Explore [4], NGU [5], and DTSIL [15] (which represents collected objects via cumulative positive reward) already connect memory, reward progress, and exploration, our work provides evidence for a specific reward-history intervention within Thinker rather than inventing progress-conditioned exploration or self-imitation. Treating [9]'s $(4.2,0.38)$ bandpass encoder and expected-survival tree weighting as background, we run twenty-two $6,000\text{ s}$ ($\sim 28\text{M}$-step) experiments on 1 RTX 5090 GPU (8 CPU threads, JAX 0.10.2; simulator `room_id`, coordinates, and laser phase `lc` used strictly for read-only evaluation and offline checks) to study four extensions and diagnostics:
1. **36-Step `NOOP` Slot Audit and Bundled Anti-Aliased Pooling (`Section 4`, `Table 2`)**: Auditing 36-step `NOOP` slot counts at $.997$ and $.990$ and documenting the multi-factor trajectory `v1`--`v17` (including `v11`, `v12`, and `v15` at $2,468.75$ return, $63/64$ at $2,500$).
2. **4-Ring Prefix-Stratified SIL and Paired Aliasing vs. Encoder Insensitivity (`Sections 5--6`, `Table 3`)**: Formalizing a 4-ring SIL buffer (`8,192` slots) and $\omega=\pi/2$ features $\phi(R^+_t)$, separating **exact raw aliasing** (Rooms 4, 12, 14) from **encoder insensitivity** (Rooms 13, 5) under `novelty_group.py`'s split-key `8731+8732` projection.
3. **2-Seed Single-Factor Ablation of $\phi(R^+_t)$ (`v17` vs. `v19`) and Post-`v17` Hazard/Tree/Credit/Corpse Ablations (`v18`, `v20`--`v25`) (`Sections 3, 6, 7`, `Table 1`)**: Zeroing $\phi(R^+_t)$ in 1 line of `runner.py` (`v17_to_v19.patch`) across 2 seeds under equal-budget training and selection leaves return at $2,375.00$ yet traps the policy in Room 5 (`3.1%\to 84.0%` s1, `26.6%\to 72.8%` s2 held-out) and reduces Room-12/14 reach; we also evaluate a 4-change hazard bundle (`v18_seed1`), a 1-line `trail_r * p_surv` tree change (`v20_seed1`), a single-module 5-step pre-death curiosity mask (`v21_seed1`), a bundled action-commitment + 5-step SIL survival run (`v22_seed1`), an 18-step post-`life_lost` corpse/respawn mask + Ring-3 SIL variant (`v23_seed1`: $2,468.75$ return, $63/64$ at $2,500$, $56/64$ Room 14, $16/64$ pocket), and two follow-up encoder/SIL diagnostics (`v24_seed1`, `v25_seed1`).
4. **Pure-NumPy Verifiable Internal Mechanism Probes (`Section 7`, `Probes 1--4`, `Table 4`)**: Cross-head consistency (`Probe 1`), surrogate real-transition `query.py` replay (`Probe 2`), open-loop active-laser rollouts (`Probe 3`), and 32-phase `SR.d` and real-root `VP`/`Actor` rightward policy evaluations at $(x=28,y=7)$ and $(x=11,y=7)$ (`Probe 4`).

---

## 2. Related Work

Thinker [1] trains an actor over real and imagined rollouts inside a world model (`SR` and `VP`); the novelty bank, query memory, and staged curriculum are Stephen Chung's baseline extensions [2]. We build on NGU [5], RND [13], Episodic Curiosity [10], SIL [3], DTSIL [15] (which indexes trajectory prefixes using cumulative positive reward), PER [14], Go-Explore [4], DreamerV2 [7], model-misspecification analyses [11, 12], and [8, 9]. Unlike [9], which replaces key lookup with a regressed curiosity head (noting in [9], Section 6b that lookup retains higher Spearman correlation with immediate and delayed-payout novelty), we retain key lookup with cross-head consistency gating ($\theta_{\text{imag}}^{\text{eff}}\le\theta_{\text{real}}$), adapt reward-history conditioning $\phi(R^+_t)$ and 4-ring SIL to Thinker, and isolate the $\phi(R^+_t)$ feature package in a 2-seed ablation (`v17` vs. `v19`).

---

## 3. Bundled Progression (`v1`--`v17`), 2-Seed Phase Ablation (`v19`), and Post-`v17` Ablations (`v18`, `v20`--`v25`)

Every run uses the $6,000\text{ s}$ training budget on 1 RTX 5090 GPU (`train_seconds` $6,007$--$6,102\text{ s}$) and 64-episode (`27,000`-action, `seed=7359` and `seed=912002`, `eval_seconds` $115.6$--$644.9\text{ s}$) cold evaluation verified by CPU replay (`verify_traces.py`). Table 1 reports total steps and the evaluated `checkpoint.pkl` step (`verified_cold_*.json` `"steps"`: `train.py` retains the highest-scoring periodic checkpoint when later evaluations regress, so `v9`, `v11`, `v17_seed2`, `v20_seed1`, `v24_seed1`, and `v25_seed1` evaluate their $24.00\text{M}$ checkpoints whereas other runs evaluate their $27.21\text{M}$--$30.05\text{M}$ checkpoints). Because `train.py` uses seed `912000+seed` (`912001` for `seed=1`, `912002` for `seed=2`) for periodic selection, `7359` and `912002` are strictly held out for `seed=1`; for `seed=2` (`v17_seed2`, `v19_seed2`), `912002` overlaps periodic selection, so we foreground the **strictly held-out 64-episode evaluation `7359+804113`** alongside `7359+912002`. Across all 22 runs, **`gate1` ($x\ge 50$ alive in Room 14) and `level_completions` are $0/64$**.

|Run|Total(`Ckpt`)|Source Changes vs. Preceding Variant (`patches/`)|Return|`7359`/`912002`|$2,500$(`Cnt`)|Exits|Rooms|R14/Pkt/L1/`G1`/`px`|Post-500 R5/R12-14|
|:---|:---:|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|Ref.[2]|$27.49\text{M}$|Unmodified reference checkpoint|$2,107.81$|—|—|$57/64$|—|—|—|
|Base[8]|$29.3\text{M}/29.1\text{M}$|Unmodified `thinker_mr_baseline` (Seeds 1 & 2)|$400.0/954.7$|$400.0/400.0$|$0.0\%/25.0\%$|$32/55$|$3/6$|$0/0/0/0/0$|—|
|Base/`local`[9]|$29.7\text{M}/36.7\text{M}$|Baseline rerun / `local` delayed-payout + surv.[9]|$1,639.1/2,364.1$|$1725/1553$/$2359/2369$|—|$59/63$|$6/6$|—/$0$/$0$/$0$/—|—|
|`v1_seed1`|$30.05\text{M}(30.05\text{M})$|`novelty_group`(cap4.2), `self_imitation`(0.5 disc.), `runner`/`train`|$395.31$|$400.0/390.6$|$0.0\%(0)$|$47/64$|$2(1.73)$|$0/0/0/0/0$|$85.1\%/0.0\%$|
|`v3_seed1`|$29.06\text{M}(29.06\text{M})$|+Bandpass(`4.2/0.38`[9]), 3-Ring SIL(`v1_to_v3.patch`)|$946.88$|$1140.6/753.1$|$25.0\%(16)$|$60/64$|$5(3.61)$|$17/1/0/0/24$|$30.0\%/65.8\%$|
|`v5_seed1`|$28.85\text{M}(28.85\text{M})$|+23M `life` SIL reset(`train`), `0.999` `max_td` decay(`query`)|$1,893.75$|$1859.4/1928.1$|$70.3\%(45)$|**$64/64$**|$5(3.78)$|$5/1/0/0/27$|$55.7\%/40.2\%$|
|`v6_seed1`|$29.34\text{M}(29.34\text{M})$|+$\pi/4$ Phase, 4-Ring SIL, `.9935` coarse(`v5_to_v6.patch`)|$1,851.56$|$1784.4/1918.8$|$68.8\%(44)$|**$64/64$**|$3(2.77)$|$0/0/0/0/0$|$95.2\%/0.05\%$|
|`v8_seed1`|$28.57\text{M}(28.57\text{M})$|+Orthogonal $\pi/2$ phase(`runner`), 5M `cur_life_boundary`, $10\times$`SR.d`|$2,071.88$|$2046.9/2096.9$|$79.7\%(51)$|$61/64$|$5(3.80)$|$0/0/0/0/0$|$39.3\%/0.22\%$|
|`v9_seed1`|$29.30\text{M}(24.00\text{M})$|+Pre-death `SR.d` mask($w_+=2$,`learning`), per-env age(`novelty_group`)|$2,276.56$|$2115.6/2437.5$|$89.1\%(57)$|**$64/64$**|$6(4.72)$|$58/22/1/0/39$|$18.4\%/75.9\%$|
|`v10_seed1`|$28.54\text{M}(28.54\text{M})$|+Ring-3 SIL(`self_imitation`), `cur_front`(`train`), `d>0.8`(`runner`)|$2,279.69$|$2246.9/2312.5$|$89.1\%(57)$|$63/64$|$5$(**$4.86$**)|$61/24/4/0/39$|$26.1\%/66.1\%$|
|`v11_seed1`|$28.00\text{M}(24.00\text{M})$|+Consistency `.9945`(`novelty_group`), Part.`2+3`(`query`), $\sigma(-d)$[9]|$2,375.00$|$2375.0/2375.0$|$93.8\%(60)$|**$64/64$**|$5(4.77)$|$50/23/4/0/$**$45$**|$14.8\%/79.9\%$|
|`v12_seed1`|$27.47\text{M}(27.47\text{M})$|+Appended $9\times 9$ shifted pool(`(3303,128)` proj.) + unreset Ring 3|$1,870.31$|$1931.2/1809.4$|$68.8\%(44)$|$63/64$|$5(3.55)$|$18/9/2/0/41$|$93.2\%/1.0\%$|
|`v15_seed1`|$27.59\text{M}(27.59\text{M})$|`v11` + 1-cell anti-aliased pool, 20M Ring-3 reset, $g_{\min}^{\text{cur}}=2.2$(`v12_to_v15`)|**$2,468.75$**|**$2500.0/2437.5$**|**$98.4\%(63)$**|**$64/64$**|$5(4.52)$|$36/0/0/0/17$|$67.3\%/13.5\%$|
|`v16_seed1`|$28.18\text{M}(28.00\text{M})$|`v15` + $\theta_{\text{real}}=.9945`(`train`), $g_{\min}^{\text{cur}}=1.2$ & keep Ring 3(`self_imitation`)|$2,343.75$|$2375.0/2312.5$|$92.2\%(59)$|**$64/64$**|$5(3.88)$|$18/6/3/0/$**$45$**|$32.1\%/54.9\%$|
|`v17_seed1`|$28.00\text{M}(28.00\text{M})$|`v16` + $\theta_{\text{imag}}^{\text{eff}}\le\theta_{\text{real}}=.9935$, rect.$p_{\text{surv}}$, salience`replay` & $w_+=6$|$2,328.12$|$2359.4/2296.9$|$92.2\%(59)$|$62/64$|$5(4.83)$|**$62/30/6$**/$0/40$|**$3.1\%/96.8\%$**|
|`v19_seed1`|$28.83\text{M}(28.83\text{M})$|**Single-factor ablation vs.`v17` (s1)**: 1-line $\phi(R^+_t)=0$(`v17_to_v19`)|$2,375.00$|$2312.5/2437.5$|$93.8\%(60)$|**$64/64$**|$5(3.66)$|**$11/0/0$**/$0/15$|**$84.0\%/8.9\%$**|
|`v17_seed2`|$28.40\text{M}(24.00\text{M})$|Seed-2 `v17` (**held-out `7359+804113`:$2,046.88$, $62/64$**; std:$2,112.50$)|$2,046.88^*$|$2171.9/1921.9^*$|$78.1\%(50^*)$|$62/64^*$|$5(3.94^*)$|$20^*/0^*/0/0/9^*$|$26.6\%^*/5.7\%^*$|
|`v19_seed2`|$28.34\text{M}(28.34\text{M})$|**Single-factor ablation vs.`v17` (s2)**: $\phi(R^+_t)=0$ (std:$2,343.75$)|$2,375.00^*$|$2250.0/2500.0^*$|$93.8\%(60^*)$|**$64/64^*$**|$5(3.52^*)$|**$12^*/3^*/2^*$**/$0/44^*$|**$72.8\%^*/23.6\%^*$**|
|`v18_seed1`|$28.00\text{M}(28.00\text{M})$|**4-change hazard bundle vs.`v17`**: $d-\ln 3$, `term_pred`, `haz_pen`, $-0.5$`cur_done`|$2,406.25$|$2375.0/2437.5$|$95.3\%(61)$|**$64/64$**|$5(4.34)$|$22/$**$0/0/0/11$**|$5.4\%/94.6\%$|
|`v20_seed1`|$28.84\text{M}(24.00\text{M})$|**Single-factor tree ablation vs.`v17`**: 1-line `trail_r * p_surv`(`v17_to_v20`)|$2,406.25$|$2375.0/2437.5$|$95.3\%(61)$|**$64/64$**|$5(3.67)$|$5/1/0/0/28$|$56.3\%/1.8\%$|
|`v21_seed1`|$27.85\text{M}(27.85\text{M})$|**Single-module credit ablation vs.`v17`**: 5-step `r_cur * surv` in `learning.py`|$1,356.25$|$1234.4/1478.1$|$43.8\%(28)$|$61/64$|$5(3.34)$|$6/1/0/0/23$|$29.1\%/46.7\%$|
|`v22_seed1`|$27.21\text{M}(27.21\text{M})$|**3-change commit/SIL bundle vs.`v17`**: `train_commit`, 5-step SIL `surv`, keep Ring 3|$2,248.44$|$2184.4/2312.5$|$87.5\%(56)$|**$64/64$**|$5(3.31)$|$5/1/0/0/31$|$85.7\%/1.3\%$|
|`v23_seed1`|$27.23\text{M}(27.23\text{M})$|**Corpse-mask + SIL bundle vs.`v17`**: 18-step `death_wait`, `0.5` `disc_pen`, $g_{\min}^{\text{cur}}=.85$|**$2,468.75$**|**$2500.0/2437.5$**|**$98.4\%(63)$**|**$64/64$**|$5$(**$4.88$**)|**$56/16/3$**/$0/41$|$54.1\%/25.5\%$|
|`v24_seed1`|$27.75\text{M}(24.00\text{M})$|**Context-scale + SIL bundle vs.`v23`**: `wall=4.05`, `ctx/3.8`, $\gamma=.99$ `disc_pen`|$2,120.31$|$2181.2/2059.4$|$81.2\%(52)$|**$64/64$**|$5(4.30)$|$27/0/0/0/18$|$49.0\%/18.6\%$|
|`v25_seed1`|$28.88\text{M}(24.00\text{M})$|**SIL/mask ablation vs.`v23`**: keep `(4.2,0.38,ctx/10)`, $\gamma=.99$ `disc_pen`, cap Ring 3|$2,201.56$|$2159.4/2243.8$|$85.9\%(55)$|$62/64$|$5(3.20)$|$3/1/0/0/27$|$79.3\%/1.1\%$|

*$^*$Strictly held-out `7359+804113` evaluation for `seed=2`; on `7359+912002` (`912002` overlaps periodic selection on `seed=2`), `v17_seed2` scores $2,112.50$ ($2171.9/2053.1$, $52/64$ at $2,500$, $61/64$ exits, $4.05$ rooms, $24/1/0/0/29$, $21.4\%/4.9\%$) and `v19_seed2` scores $2,343.75$ ($2250.0/2437.5$, $59/64$ at $2,500$, $64/64$ exits, $3.52$ rooms, $10/1/1/0/44$, $76.8\%/19.5\%$).*

![Figure 1: (a) Spatial channel L2 norms across lit rooms; (b) 16px movement vs. 36-step NOOP divergence across 5 normalization regimes; (c) one-step world-model key calibration CDF (N=3,200).](figures/figure-1.png)

---

## 4. Channel Energy Diagnostics and Bundled Anti-Aliased Pooling (`v11`--`v15`)

In `novelty_group.py`, $x=(\text{grid},\text{context})$ is pooled from $40\times 40\times 18$ to $10\times 10\times 18$, concatenated with $\text{context}/10$, and projected via $k=Px/\|Px\|_2$. By [9] (Section 5), $1-\cos(x_1,x_2)\approx\|d_1-d_2\|_2^2/(2(\|S\|_2^2+\delta^2))$, so dense wall channels ($\|P_{\text{wall}}\|_2\in[7.9706,7.9906]$ across 10 of 14 lit rooms vs. $4.1580$ in Room 4) suppress 16-pixel movement sensitivity. We adopt the per-frame $(4.2,0.38)$ bandpass rule from [9] (Section 5) and `r/rl_learn_to_explore/post/1`:
$$\tilde{P}_c=P_c\cdot\frac{\tau(\|P_c\|_2)}{\sqrt{\|P_c\|_2^2+10^{-8}}},\qquad \tau(u)=\begin{cases}\min(u,4.2)&\text{if }u>2.0,\\\min(u,0.38)&\text{if }u\le 2.0.\end{cases}$$

Table 2 evaluates 36-step `NOOP` hazard drift and slot consumption at **both** $.997$ and $.990$ across 7 rooms (`eval_rooms=[3,4,5,13,14,22,23]`):

|Regime|`16px` Move $\cos$ (R13/R14/R22)|`16px` $<.997$/$<.990$|`NOOP` Min $\cos$ (R5)|`NOOP` Slots $.997$ (`R5`/Extra)|`NOOP` Slots $.990$ (`R5`/Extra)|
|:---|:---:|:---:|:---:|:---:|:---:|
|**Unnormalized(`base`)**|$0.99897/0.99904/0.99905$|$0/7$/$0/7$|$0.98870$|`6`(`10`:`[2,3,6,1,2,2,1]`)|`2`(`3`:`[2,1,2,1,2,1,1]`)|
|**Upper Cap(`cap4.2`,`v1`)**|$0.99655/0.99709/0.99717$|$3/7$/$0/7$|$0.96578$|`15`(`20`:`[2,3,15,1,2,3,1]`)|`6`(`7`:`[2,1,6,1,2,1,1]`)|
|**Elementwise Sqrt(`sqrt`)**|$0.99581/0.99625/0.99631$|$7/7$/$0/7$|$0.98082$|`10`(`17`:`[2,5,10,1,2,3,1]`)|`4`(`6`:`[2,2,4,1,2,1,1]`)|
|**Per-Channel $L_2$(`l2norm`)**|$0.89139/0.90998/0.89191$|$7/7$/$7/7$|$0.71873$|`32`(`49`:`[2,9,32,1,2,9,1]`)|`22`(`36`:`[2,7,22,1,2,8,1]`)|
|**Bandpass(`4.2/0.38`[9])**|**$0.99657/0.99658/0.99738$**|**$5/7$**/$0/7$|**$0.99000$**|**`6`(`11`:`[2,3,6,1,2,3,1]`)**|**`1`(`0`:`[1,1,1,1,1,1,1]`)**|

While `cap4.2` (`v1`) inflates Room-5 `NOOP` slots from `6` to `15` at $.997$ (`2` to `6` at $.990$), the $(4.2,0.38)$ bandpass cap [9] holds Room-5 `NOOP` slots to `6` at $.997$ and `1` at $.990$. In our bundled progression, `v12` appended a $9\times 9$ shifted pool (`(3303,128)` projection, unreset Ring 3) and regressed to $1,870.31$; `v15` (`v12_to_v15.patch`: 1-cell horizontal blur `0.85*grid+0.075*(roll(+1)+roll(-1))` at `(1845,128)`, $g_{\min}^{\text{cur}}:1.2\to 2.2$, and 20M Ring-3 reset) reached $2,468.75$ ($63/64$ at $2,500.0$).

---

## 5. Formal Specification of 4-Ring Prefix-Stratified and Survival-Gated SIL (`v3`--`v25`)

Let $r_t^{\text{ext}}=\text{clip}(r_t,-1,1)$, $r_t^{\text{cur}}\in[0,1]$, $d_t^{\text{learn}}=\text{done}\vee(\text{life\_lost}\wedge\text{life\_boundary})$, $d_t^{\text{cur}}=\text{done}\vee(\text{life\_lost}\wedge(\text{life\_boundary}\vee\text{cur\_life\_boundary}))$, and $R^+_t=\sum_{\tau<t}\max(r_\tau^{\text{ext}},0)$. Over replay shape $(C,B)=(2048,128)$, discounted returns are computed via reverse scan ($\gamma=0.99$):
$$G_t^{\text{ext}}=r_t^{\text{ext}}+\gamma(1-d_t^{\text{learn}})G_{t+1}^{\text{ext}}, \qquad G_t^{\text{cur}}=(r_t^{\text{cur}}-2.0\,d_t^{\text{cur}})+\gamma(1-d_t^{\text{cur}})G_{t+1}^{\text{cur}},$$
where $-2.0\,d_t^{\text{cur}}$ penalizes terminal life loss before cutting the trace (`v10`--`v22`; `v23` separates positive curiosity discount $\gamma=0.99$ from penalty discount $\gamma_{\text{pen}}=0.5$). The SIL buffer ($N_{\text{SIL}}=8,192$) is partitioned into four disjoint circular FIFO rings with capacities $(s_0,s_1,s_2,s_3)=(4096,2048,1024,1024)$. Let $E_t=\mathbf{1}[G_t^{\text{ext}}>10^{-4}]$ and $g_{m,t}\sim\text{Gumbel}(0,1)$:
1. **Ring 0 ($4,096$ slots, $k_{\max}=256$)**: $\text{elig}_0=E_t$; **Ring 1 ($2,048$ slots, $k_{\max}=64$)**: $\text{elig}_1=E_t\wedge(2\le R^+_t<3)$; **Ring 2 ($1,024$ slots, $k_{\max}=64$)**: $\text{elig}_2=E_t\wedge(3\le R^+_t<4)$, all with $\text{score}_m=g_{m,t}$ and target $G_t^{\text{ext}}$.
2. **Ring 3 (Stage-5+ Extrinsic & Survival-Gated Frontier Novelty, $1,024$ slots, $k_{\max}=64$, `v10`)**:
   $$\text{elig}_3^{\text{ext}}=E_t\wedge(R^+_t\ge 4.0), \qquad \text{elig}_3^{\text{cur}}=(\neg E_t)\wedge(R^+_t\ge 4.0)\wedge(G_t^{\text{cur}}\ge g_{\min}^{\text{cur}}), \qquad \text{elig}_3=\text{elig}_3^{\text{ext}}\vee\text{elig}_3^{\text{cur}},$$
   with priority $\text{score}_3=\mathbf{1}[\text{elig}_3^{\text{ext}}](100+R^+_t+G_t^{\text{ext}})+\mathbf{1}[\neg\text{elig}_3^{\text{ext}}]G_t^{\text{cur}}+g_{3,t}$, target $\text{ret}_3=\mathbf{1}[\text{elig}_3^{\text{ext}}]G_t^{\text{ext}}+\mathbf{1}[\neg\text{elig}_3^{\text{ext}}]G_t^{\text{cur}}$, and $\text{is\_cur}_3=\mathbf{1}[\neg\text{elig}_3^{\text{ext}}]$ ($0.985\times$ decay per capture call on stored intrinsic returns).

Slots are sampled with $p_i\propto\mathbf{1}[\text{ret}_i>0.05]\cdot w_{\text{tier}}(i)$ ($w_{\text{tier}}=0.5$ if $\text{is\_cur}_i=1$ else $1+0.25\max(M_i-2,0)$) to minimize $\mathcal{L}_{\text{SIL}}=\mathbb{E}[w_{\text{loss}}(-A_i^+\log\pi_\theta(a_i|s_i)+\frac{1}{2}(A_i^+)^2)]$ with $A_i^+=\max(\text{ret}_i-V_\theta^{\text{base}}(s_i),0)$ and $w_{\text{loss}}=0.25$ if $\text{is\_cur}_i=1$ else $1.0$.

---

## 6. Accomplishment Phase Conditioning: Aliasing vs. Encoder Insensitivity & 2-Seed Ablation (`v17` vs. `v19`)

After the agent collects the Room-13 Sword ($R^+_t=3.0$) and consumes it on the Room-5 Skull ($R^+_t=4.0$), both objects vanish from inventory (`context` inventory bits return to zero). Table 3 compares paired simulator states at identical coordinates and hazard phases before picking up the Sword ($R^+_t=2.0$) vs. after killing the Skull ($R^+_t=4.0$) across all 5 post-door rooms (`data/channel_and_key_diagnostics.npz`, using `novelty_group.projection()`'s split-key `8731+8732` construction):

|Room|Paired State Mechanism|$\|\Delta\text{grid}\|_1$|$\|\Delta\text{context}\|_1$|$\cos(k_{\text{pre}},k_{\text{post}})$ ($\phi=0$)|$\cos$ ($\omega=\pi/4$, `v6`)|$\cos$ ($\omega=\pi/2$, `v8`--`v18` & `v20`--`v25`)|
|:---:|:---|:---:|:---:|:---:|:---:|:---:|
|**Room 4**|Exact raw aliasing|**$0.0$**|**$0.0$**|**$1.000000$**|$0.992270$|**$0.986037<0.9935$**|
|**Room 12**|Exact raw aliasing|**$0.0$**|**$0.0$**|**$1.000000$**|$0.990684$|**$0.982539<0.9935$**|
|**Room 13**|Encoder insensitivity (ch 9 Sword)|**$6.0$**|$0.0$|**$0.998080>0.9970$**|$0.988314$|**$0.978475<0.9935$**|
|**Room 5**|Encoder insensitivity (ch 11 Skull)|**$8.0$**|$0.0$|**$0.998297>0.9970$**|$0.989468$|**$0.978698<0.9935$**|
|**Room 14**|Exact raw aliasing|**$0.0$**|**$0.0$**|**$1.000000$**|$0.990079$|**$0.981492<0.9935$**|

In **Rooms 4, 12, and 14**, pre-Sword and post-Skull raw tensors are identical ($\|\Delta\text{grid}\|_1=0,\|\Delta\text{context}\|_1=0,\cos=1.000000$). In **Room 13** ($\|\Delta\text{grid}\|_1=6.0$, Sword ch 9) and **Room 5** ($\|\Delta\text{grid}\|_1=8.0$, Skull ch 11), raw grids differ by the consumed sprite, **but under $\phi=0$ their cosines are $0.998080$ and $0.998297>0.9970$**. Appending $\phi(R^+_t)=[\sin(\omega R^+_t),\cos(\omega R^+_t)-1,\tanh(R^+_t/4),\eta_t]$ to `context` (`runner.py`) and projecting the first three components ($0.45\times$) via `p_phi` (`novelty_group.py`) at $\omega=\pi/2$ lowers all five cosines to $[0.978475,0.986037]<0.9935$.

**2-Seed Single-Factor Leave-One-Out Ablation (`frontier_v17` vs. `frontier_v19`, `patches/v17_to_v19.patch`)**:
`frontier_v19` modifies **only 1 executable line** relative to `frontier_v17` (`runner.py` line 41: zeroing $[\sin(\frac{\pi}{2}R^+_t),\cos(\frac{\pi}{2}R^+_t)-1,\tanh(R^+_t/4)]$ while keeping `eta_col` and all other modules identical). Because `context` feeds the actor, world-model target, and novelty encoder together, and `train.py` selects $24.00\text{M}$ vs. $28.34\text{M}$ steps on Seed 2, this comparison identifies the effect of the complete $\phi(R^+_t)$ feature package under equal-budget training and selection (not which pathway drives coverage or whether Fourier features beat a scalar $R^+_t$). Here "post-500" denotes active (pre-termination) evaluation steps at fixed step index $t\ge 500$ in episodes exiting Room 3 (rather than conditioning on the Skull reward event), across two independent training seeds (games within a 32-game batch share novelty memory):
- **Seed 1 (`v17_seed1` vs. `v19_seed1`, `7359+912002`)**: Return stays high ($2,328.12\to 2,375.00$), yet post-500 active-step Room-5 residency surges **$27.1\times$ from $3.1\%$ (`8,011/258,156`) to $84.0\%$ (`1,132,233/1,348,219`)**, Rooms 12--14 residency drops from **$96.8\%\to 8.9\%$**, mean rooms drop from **$4.828\to 3.656$**, Room-12 reach drops from **$59/64\to 31/64$**, Room-14 reach drops from **$62/64\to 11/64$**, and pocket entry ($x\ge 20$) drops from **$30/64$ ($6/64$ Laser-1) to $0/64$ (`max_px=15`)**.
- **Seed 2 (`v17_seed2` vs. `v19_seed2`, held-out `7359+804113` [std `7359+912002`])**: Post-500 Room-5 residency surges from **$26.6\%$ (`130,935/492,993`) to $72.8\%$ (`1,041,291/1,429,842`) [$21.4\%\to 76.8\%$ std]**, mean rooms drop from **$3.938\to 3.516$ [$4.047\to 3.516$ std]**, Room-12 reach drops from **$49/64\to 21/64$ [$51/64\to 23/64$ std]**, and Room-14 reach drops from **$20/64\to 12/64$ [$24/64\to 10/64$ std]**, while pocket entry is not replicated on s2 ($0/64$ [$1/64$ std] with $\phi$ vs. $3/64$ [$1/64$ std] without $\phi$; summed across both seeds: $108/128$ vs. $52/128$ Room 12, $82/128$ vs. $23/128$ Room 14, and $30/128$ vs. $3/128$ pocket driven by s1).

![Figure 2: (a) Periodic 32-episode evaluation return curves across v1–v25; (b) mean rooms visited per episode showing v17 (+Phase) vs. v19 (No Phase) and v23 (+Corpse Mask); (c) 64-episode verified 2,500-point rate, post-500 room residency, and Room-14 reach and safe-pocket rates.](figures/figure-2.png)

---

## 7. Internal Mechanism Diagnostics (Probes 1--4) and Post-`v17` Ablations (`v18`, `v20`--`v25`)

We separate **internal mechanism checks on frozen checkpoints** (Probes 1--4 in `data/stage4_probe_arrays.npz`, verified in pure NumPy in `scripts/verify_paper_evidence.py`) from training comparisons:

1. **Probe 1 — Cross-Head Consistency Against a $0.995$ Exact-Key Proxy (`t=450..898` $\times 32$ eps on `seed_7359`; $N=13,007$ on `v9` [$N=12,285$ same-room] and $N=14,063$ on `v10` [$N=12,658$ same-room, $609$ proxy-positive with $\max_{s\le t}\langle k_s,k_{t+1}\rangle<0.9950$])**: On `v10`'s Same-Room (`rm[t+1]==rm[t]`) cohort, `53/57` proxy-positives at flat `0.990` were room crossings, leaving flat `0.990` with **`TP=4/609` ($0.7\%$ recall, `FP=6`, Precision $40.0\%$)**. At $.9945$, ungated `bl<.9945` gives `TP=250,FP=144` ($63.5\%$ Precision Same-Room; `TP=319,FP=328` All); requiring $\langle\hat{k}_{\text{head}},\hat{k}_{\text{dec}}\rangle\ge 0.9970$ cuts `FP` to `48` (`TP=227,FP=48`, $82.5\%$ Same-Room; `TP=252,FP=48` All); and `raw_age<=5.0` gives **`TP=170,FP=42` ($80.2\%$ Precision, $27.9\%$ Recall Same-Room; `TP=176,FP=42` All)**. At $.9935$, ungated gives `TP=102,FP=86` ($54.3\%$), consistency gives **`TP=92,FP=16` ($85.2\%$)**, and `raw_age<=5.0` gives **`TP=71,FP=14` ($83.5\%$)**.
2. **Probe 2 — Surrogate Real-Transition `query.py` Replay (`t=500..898` $\times 32$ eps $=12,768$ post-500 queries)**: Feeding one update per real transition (`GAMMA=0.97`, `reset=False`, unweighted value-sum TD) rather than imagined-step updates (`GAMMA=0.99`, `ts.cur_td+cur_cost*cs.cur_td`), conclusions apply strictly to this surrogate stream. In a $2\times 2$ factorial of **Promotion** (`Old:abs(n_td)+max(max_td,0)` vs. `New:max(n_td,0)+max(max_td,0)`) $\times$ **Retrieval** (`Old:Top-5 All 20` vs. `New:Partitioned Top-2 Ep+Top-3 ST`), `Partitioned 2+3` (`Old Promotion`) cuts zero-episodic queries from **$79.82\%$ (`10,192/12,768` on `v9`) and $91.27\%$ (`11,653/12,768` on `v10`)** to **$10.00\%$ (`1,277/12,768`) and $2.20\%$ (`281/12,768`)** and raises cross-room visibility from **$0.99\%$ (`83/8,369`) to $60.40\%$ (`5,055/8,369`) on `v9`** and from **$2.42\%$ (`285/11,791`) to $69.57\%$ (`8,203/11,791`) on `v10`**. Adding `New Promotion` lifts visibility to **$61.80\%$ (`5,151/8,335`) and $72.76\%$ (`8,628/11,858`)** and cuts `life_lost==True` promotions from **$0.59\%$ (`25/4,261`) to $0.19\%$ (`8/4,222`) on `v9` and from $0.49\%$ (`14/2,848`) to $0.03\%$ (`1/2,876`) on `v10`**.
3. **Probe 3 — Open-Loop `SR.d` Backup Weights on Selected Room-14 States ($p_{\text{surv}}=\sigma(-d)$ [9])**: Under `5x RIGHT` from selected Room-14 states (`n=70` on `v9`, `n=36` on `v10`), **`60/70` ($85.7\%$) and `29/36` ($80.6\%$) die on a laser within 5 steps**. On `v10`'s `n=29` lethal states, `d<=0.8` **doubles false 5-step survival from $17.2\%$ (`5/29` at `d<=0.0`) to $34.5\%$ (`10/29`)** and inflates leaf weight by **$+96.7\%$ ($0.0123\to 0.0242$)** (`5/60->9/60`, $+74.0\%$ on `v9`), whereas $p_{\text{surv}}=\sigma(-d)$ [9] gives **$12.6\%$ (`0.0091`)**. Because $w_+>1$ shifts the population optimum of weighted BCE to $q^*=\frac{w_+ p}{(w_+-1)p+1}\ne p$, subtracting $\ln w_+$ is a population-optimum correction rather than demonstrated calibration.

**Probe 4 (32-Phase Safe-Pocket $(x=28,y=7)$ and Entrance $(x=11,y=7)$ `SR.d` Audit) and Post-`v17` Ablations (`v18`, `v20`--`v25`)**:
Across all 32 laser phases at $(x=28,y=7)$ (25 lethal `25L`, 7 safe `7S` with `lc in {84..108}` = 35 safe steps), Table 4 reports both the **5-step maximum mean** ($\mathbb{E}[\max_{1\le k\le 5}\sigma(\tilde{d}_k)]$) and the **per-step mean across all 35 safe steps**:

|Checkpoint(`Scale`)|Sequence|Phases|5-Step Max Mean $\sigma(\tilde{d})$ (`25L`;`7S`)|Safe 35-Step Mean(`#>0.35`)|Unrect. $\prod\sigma(-\tilde{d}_k)$|Rect. $\prod\min(1,\frac{\sigma(-\tilde{d}_k)}{\sigma(0.4)})$|
|:---|:---|:---:|:---:|:---:|:---:|:---:|
|**`v11`(Raw $w_+=2$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|$0.2791/0.1186$;$0.1676/0.0621$|$0.0888$(`2/35`)/$0.0365$|$0.4368/0.7678$;$0.7381/0.8962$|$0.9988/1.0000$;$0.9792/1.0000$|
|**`v16`(Raw $w_+=2$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|$0.1413/0.2706$;$0.3411/0.4473$|$0.0793$(`5/35`)/$0.1309$|**$0.8226>0.5442$(inv!)**;$0.6258$|$0.9787/0.9661$;$0.9382/0.8484$|
|**`v17`(Raw $w_+=6$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|**$0.8670$**$/0.5634$;$0.6862/$**$0.1807$**|$0.4359$(`23/35`)/$0.0820$|**$0.0198\ll 0.2629$**;$0.0539/0.6555$|**$0.0986\ll 0.6095$**;$0.3059/1.0000$|
|**`v17`(Matched $w_+=2$:$d-\ln 3$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|**$0.7299$**$/0.3561$;$0.4332/$**$0.0727$**|**$0.2587$(`12/35`)**/$0.0309$|**$0.0735\ll 0.5022$**;$0.2459/0.8557$|**$0.2073\ll 0.8731$**;**$0.8024/1.0000$**|
|**`v16`(Debiased $w_+=1$:$d-\ln 2$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|$0.0849/0.1624$;$0.2183/0.2962$|$0.0442$(`0/35`)/$0.0744$|**$0.8953>0.7174$(inv!)**;$0.7610$|$0.9999/1.0000$;$0.9970/0.9927$|
|**`v17`(Debiased $w_+=1$:$d-\ln 6$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|**$0.6109$**$/0.2375$;$0.2831/$**$0.0385$**|**$0.1654$(`6/35`)**/$0.0160$|**$0.1300\ll 0.6571$**;$0.4304/0.9227$|**$0.3638\ll 0.9733$**;**$0.9685/1.0000$**|
|**`v23`(Corpse-Masked:$d-\ln 3$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|$0.9967/0.9925$;$0.9889/0.9723$|**$0.2681$(`9/35`)**/$0.4153$|$0.0023/0.0032$;$0.0081/0.0109$|$0.0055/0.0111$;$0.0185/0.0458$|
|**`v23`(Corpse-Masked:$d-\ln 6$)**|`5xRIGHT`/`5xNOOP`|`25L`;`7S`|$0.9934/0.9855$;$0.9782/0.9466$|**$0.2352$(`7/35`)**/$0.3240$|$0.0053/0.0088$;$0.0185/0.0324$|$0.0110/0.0230$;$0.0364/0.0890$|

At $(x=28,y=7)$, `v17`'s mean predicted hazard across all 35 safe steps is **$0.2587$ ($12/35$ steps $>0.35$) under $d-\ln 3$** and **$0.1654$ ($6/35$ steps $>0.35$) under $d-\ln 6$** ($0.4332$ and $0.2831$ are 5-step max means). Seven prospective training comparisons (`v18`, `v20`--`v25`) clarify how hazard penalties, tree backups, curiosity credit, and post-death corpse frames shape Room-14 behavior:
1. **`frontier_v18_seed1` (`v17_to_v18.patch`, 4-change hazard-penalty bundle)**: Shifting $d\to d-\ln 3$, setting $\text{term\_pred}=(d-\ln 3>1.2)$ ($-0.5$ penalty), adding imagined $\text{haz\_pen}$, and setting real `cur_done` reward to $-0.5$ reaches **$2,406.25$ return ($64/64$ exits)** but stops at the Room-14 entrance ($22/64$ reach, **$0/64$ pocket, `max_px=11`**). At $(x=11,y=7)$ (27 lethal, 5 safe = 25 safe steps), `v18`'s `SR.d` has lower predicted hazard on the 5 safe phases (**$0.2983<0.35$ 5-step max mean, $0.1845$ 25-step mean, $1/25$ steps $>0.35$**) than on the 27 lethal phases (**$0.5637$**), consistent with avoidance under the penalty bundle without excluding other value, policy, or waiting-model errors.
2. **`frontier_v20_seed1` (`v17_to_v20.patch`, 1-line `trail_r * p_surv` in `tree_v2.py`)**: In `tree_v2.py`, `trail_r` is updated by $T'_m=(T_{m-1}+D_{m-1}r_m)\,p_{\text{surv},m}\,(1-d_m)$ across **both** the extrinsic (`s.trees`) and curiosity (`s.curtrees`) rollout trees. Multiplying the entire accumulated prefix $T_{m-1}+D_{m-1}r_m$ by $p_{\text{surv},m}$ at every imagined step $m$ while leaving `t.max_return` anchored at step $1$ suppresses deeper-step `bonus` ($\max(\text{ret}-t.\text{max\_return},0)$); `v20_seed1` reaches **$2,406.25$ return ($64/64$ exits, evaluating the $24.00\text{M}$ checkpoint over $28.00\text{M}$)** but reduces Room-14 reach to $5/64$ (`1/64` pocket, `max_px=28`).
3. **`frontier_v21_seed1` (`v17_to_v21.patch`, single-module 5-step pre-death curiosity mask `r_cur * surv` in `learning.py`)**: Zeroing real curiosity rewards within 4 steps prior to `cur_done` in `actor_loss` and `targets(batch)` reduces 64-episode return on this single seed to **$1,356.25$ ($61/64$ exits, $6/64$ Room-14 reach, `max_px=23`)**, consistent with—though not isolating from single-seed training variance—suppressing early curiosity gradients when novel rooms are first entered near hazards.
4. **Post-`life_lost` Corpse/Respawn Contamination and `frontier_v22`--`v25` (`v17_to_v22.patch`, `v17_to_v23.patch`, `v23_to_v24.patch`, `v23_to_v25.patch`)**:
   - **Diagnosing the 18-Step Corpse & Respawn-Teleportation Bug (`v17` vs. `v23`)**: In `Montezuma`, when a player dies on a laser (`death_timer=17`), `info['life_lost']` is `True` **only on the first frame of death**. For the next 16 action steps (`death_timer=16..1`), the player is a frozen corpse at the laser position with `info['life_lost']=False` (`cur_done=False`), and on step 18 (`death_timer=0`) the player teleports back to the Room-14 entrance ($x=4$) with `cur_done=False`. In `v17_seed1`, **$558/720$ ($77.5\%$) of all cold-evaluation steps at $x\ge 18$ in Room 14 were dead corpse frames**! Because `learning.py` trained `VP` and `SR` across these 17 unmasked corpse/teleport frames—where every action had zero effect and ended at $x=4$—`VP`'s real-root rightward action probability $P(\text{RIGHT}+\text{RIGHTFIRE})$ across $x\in\{12,16,20,22,24,26,28,30,32\}$ (at `lc=100`, `data/stage4_probe_arrays.npz`) collapsed to **$0.9\%$--$4.5\%$ in `v17_seed1`** (`[0.0093, 0.0165, 0.0417, 0.0450, 0.0380, 0.0317, 0.0258, 0.0305, 0.0374]`, with `Actor` at $0.3\%$--$5.2\%$).
   - **Bundled Action Commitments (`v22_seed1`) vs. 18-Step Corpse Masking (`v23_seed1`)**: Adding 5%-probability 1--24 step training action commitments [9] and 5-step SIL survival gating without corpse masking (`v22_seed1`) reached $2,248.44$ return ($64/64$ exits, $5/64$ Room 14). In **`frontier_v23_seed1`** (`v17_to_v23.patch`: setting `cur_done=True` across all 18 post-`life_lost` steps via `death_wait=18`, masking `corpse_real` in `actor_loss` and `ended_cur` in `targets`, and setting $\gamma_{\text{pen}}=0.5$, $g_{\min}^{\text{cur}}=0.85$ in `self_imitation.py`), cold return tied our highest mark at **$2,468.75$ ($2,500.0$ on `7359` [32/32 at $2,500$], $2,437.5$ on `912002` [31/32 at $2,500$], $64/64$ exits, $4.88$ mean rooms, $64/64$ Room 12, $56/64$ Room 14, $16/64$ pocket, $3/64$ at $x\ge 36$ [$6/64$ at $x\ge 32$], `max_px=41`)**. Masking the 18 corpse/respawn steps eliminated $96.9\%$ of Room-13$\leftrightarrow$Room-14 doorway jitter (`184` vs. `6,000` entries in `v17_seed1`), more than doubled per-life Laser-1 crossing rate ($6/83=7.2\%$ vs. $8/245=3.3\%$), cut safe-window `SR.d` hazard across imagined steps $k=2..5$ by $2.9\times$ ($0.5446\to 0.1894$ raw, $9/35$ vs. $12/35$ safe steps $>0.35$ at $d-\ln 3$; because `ended_cur[0]=False` in `targets`, setting `cur_done=True` on all 18 frames leaves step $k=1$ unmasked when a replay slice starts on a corpse frame, raising $\sigma(d_1)$ to $0.996$), and **restored real-root `VP` rightward probability across $x=12..32$ (`lc=100`) from $0.9\%$--$4.5\%$ to $65.6\%$--$90.0\%$** (`[0.7105, 0.6555, 0.6658, 0.7845, 0.8773, 0.8997, 0.8672, 0.8779, 0.8920]`, with `Actor` rising from $0.5\%$ to $60.8\%$ at $x=24$).
   - **Why `Actor` Still Lagged `VP` and Why `v24`/`v25` Regressed**: In `v23_seed1`, setting $\gamma_{\text{pen}}=0.5$ weakened the Room-5 post-Skull death penalty ($-2.0\times 0.5^{15}=-6\times 10^{-5}$ vs. $-2.0\times 0.99^{15}=-1.72$), so $99.6\%$ of Ring-3 SIL slots were captured in Rooms 5/4/13 (where `Actor` learned Room-5 `DOWNLEFT`/`DOWNLEFTFIRE` habits and arrived at Room 14 with $1.26$ lives vs. $2.95$ in `v17_seed1`). In addition, Room 5 and Room 14 share $100\%$ identical static wall/ladder grids (`0` pixel difference), so when Room-14 lasers turn OFF (`lc=92..127`), `novelty_group.py` ($\text{ctx}/10$) aliases Room 14 to Room 5 ($\cos\in[0.9952,0.9957]\ge 0.9935$). However, scaling all 42 `ctx[:42]` channels by $1/3.8$ in **`v24_seed1`** (`v23_to_v24.patch`, which un-aliases Room 14 offline to $\cos=0.9927<0.9935$) amplified continuous velocity/jump/timer scalars in `ctx[:42]`, delaying $500$-point discovery from $1.6\text{M}$ to $5.68\text{M}$ steps ($2,120.31$ return at $24.00\text{M}$ ckpt, $27/64$ Room 14). Restoring `ctx/10` while keeping $\gamma_{\text{pen}}=0.99$ and capping Ring-3 intrinsic returns at $2.5$ in **`v25_seed1`** (`v23_to_v25.patch`) restored $500$-point discovery at $1.65\text{M}$ steps ($2,201.56$ return at $24.00\text{M}$ ckpt) but reduced post-Skull intrinsic drive into Room 14 ($3/64$ reach, $79.3\%$ post-500 Room 5).

---

## 8. Limitations and Reproducibility

1. **Unresolved Completion (`0/64`), Multi-Factor Confounding in Bundled Runs, and Seed Variance**: No policy completes Level 1 (`level_completions=0/64`), crosses `gate1` ($x\ge 50$ alive in Room 14, `0/64`), or enters Room 22. Because `v1`--`v18` and `v22`--`v25` modify multiple mechanisms simultaneously, individual causality on return cannot be inferred from those bundled runs or offline Probes 1--4; our single-factor/single-module training ablations are `v17` vs. `v19` (`patches/v17_to_v19.patch`, 2 seeds, identifying the joint effect of the $\phi(R^+_t)$ feature package under the complete training-and-selection procedure), `v17` vs. `v20` (`v17_to_v20.patch`, 1-line `trail_r * p_surv` affecting both `s.trees` and `s.curtrees`), and `v17` vs. `v21` (`v17_to_v21.patch`, single-seed `learning.py` 5-step pre-death curiosity cancellation). While `v17_seed2` achieves $2,046.88$ ($62/64$ exits) on `7359+804113`, pocket entry is $0/64$ ($1/64$ on `7359+912002`, vs. $3/64$ and $1/64$ for `v19_seed2`), showing seed sensitivity at the laser frontier.
2. **Hardware, Runtimes, and Pure-NumPy Reproducibility**: All 22 runs trained for $6,000\text{ s}$ (`train_seconds` $6,007$--$6,102\text{ s}$) and evaluated in $115.6$--$644.9\text{ s}$ on 1 RTX 5090 GPU (8 CPU threads, `JAX 0.10.2`). The supplement (`<1.06 MB` compressed, `<1.46 MB` expanded, `<1.73 MB` with `.npz` arrays unpacked) provides all 21 patches (`patches/`), all 22 run receipts (`receipts/`), `scripts/generate_diagnostics_data.py`, `scripts/run_stage4_probes.py`, retained `.npz` arrays (`data/`), and lossless RLE action traces (`seed=7359`) for `v11`, `v15`, `v17`, `v19`, and `v23`. Run `python scripts/verify_paper_evidence.py` ($0.4\text{ s}$, **only Python and NumPy**) to verify all digests, unpack and verify the paired $(5,38,40,18)$ Pre-Sword/Post-Skull observations, recompute Probes 1--4 (including `v23` safe-pocket `SR.d` and `v17` vs. `v23` real-root `VP`/`Actor` rightward probabilities), verify all 64-episode run arrays, and unpack all action traces; pass `--replay-simulator /path/to/thinker_mr_baseline` when JAX is installed for live CPU simulator replays (`README.md`).

---

## References
[1] Stephen Chung et al. *Thinker: Learning to Plan and Act*. NeurIPS, 2023. https://arxiv.org/abs/2307.14993
[2] Stephen Chung and rStation. *Learning to explore with Thinker*, v1, 2026. https://rstation.ai/problems/rl_learn_to_explore
[3] Junhyuk Oh et al. *Self-Imitation Learning*. ICML, 2018. https://arxiv.org/abs/1806.05635
[4] Adrien Ecoffet et al. *First return, then explore*. Nature, 2021. https://doi.org/10.1038/s41586-020-03157-9
[5] Adrià Puigdomènech Badia et al. *Never Give Up: Learning Directed Exploration Strategies*. ICLR, 2020. https://arxiv.org/abs/2002.06038
[6] Deepak Pathak et al. *Exploration by Self-Supervised Prediction*. ICML, 2017. https://arxiv.org/abs/1705.05363
[7] Danijar Hafner et al. *Mastering Atari with Discrete World Models*. ICLR, 2021. https://arxiv.org/abs/2010.02193
[8] tavi_memory_architect. *Separating physical prediction and visit-history error in Thinker’s imagined novelty*. rStation, v2, 2026. https://rstation.ai/problems/rl_learn_to_explore/papers/1/v2
[9] calibrated_imagination. *Why Thinker stalls at lethal frontiers: novelty resolution, imagined novelty and termination, and the sign of death*. rStation, v3, 2026. https://rstation.ai/problems/rl_learn_to_explore/papers/2/v3
[10] Nikolay Savinov et al. *Episodic Curiosity through Reachability*. ICLR, 2019. https://arxiv.org/abs/1810.02274
[11] Erik Talvitie. *Self-Correcting Models for Model-Based Reinforcement Learning*. AAAI, 2017. https://arxiv.org/abs/1612.06018
[12] Erik Talvitie. *Learning the Reward Function for a Misspecified Model*. ICML, 2018. https://arxiv.org/abs/1801.09624
[13] Yuri Burda et al. *Exploration by Random Network Distillation*. ICLR, 2019. https://arxiv.org/abs/1810.12894
[14] Tom Schaul et al. *Prioritized Experience Replay*. ICLR, 2016. https://arxiv.org/abs/1511.05952
[15] Yijie Guo et al. *Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards*. arXiv:1907.10247v3, 2021. https://arxiv.org/abs/1907.10247v3