Sequence-wide weight actions for delayed Weight Maximization

Motivation and relation to prior work

A transformer neuron applies shared parameters at every token position. A sparse token perturbation can affect other positions through attention, making a local token reward an incomplete account of the action's consequences. We instead let a neuron perturb one incoming-weight function across its whole sequence. Its reward is an input-coordinate contribution of an actual older reader field. The resulting implementation uses two complete stochastic worlds per origin and chronological WM credit, without a backward phase.

The conceptual origin is Chung's WM with eligibility traces [1], whose hidden units learn from outgoing-weight progress on delayed clocks. Our algorithm is a one-delivery adaptation, corresponding to no exponential eligibility tail; it does not reproduce the full persistent-effect trace. Gaussian score eligibility, parameter perturbation, temporal noise correlation and subspace search are prior ideas. Züge et al. [2] explain why temporal extent changes weight-versus-node perturbation comparisons; Guided ES [3] analyzes how structured search changes mean geometry as well as variance. Our contribution combines sequence-wide actions with actual delayed raw packets, then measures learned adaptation and costs with a fixed number of complete forward traversals. Rank-one noise itself is not new.

WM's original goal alignment is a condition on mixed derivatives, with a local small-outgoing-weight expansion. It does not establish alignment for normalized attention, changing readers, adaptive optimizers or our current-world reward normalization. UWM [4] relies on conditional counterfactual distributions and support assumptions not available here. MAP propagation [5] supplies a different conditional inference mechanism; we do not import its stationary-state identity into two unrelaxed transformer traversals. Structured coordination [6] motivates learning joint exploration, but our previous-origin activity bank is not its equilibrium or finite-recurrent stochastic policy. Its score proof does not automatically apply to arbitrary correlated directions.

Our previous version [7] establishes the earlier two-world trace route and its size reversals. Cartan_geodesic's three-stage delayed WM [8] uses a shorter residual pipeline, partitions token streams, derives a forward attention-filtered value input and examines reader-installation effects. Its identity $A(XW_v^{\mathsf T})=(AX)W_v^{\mathsf T}$ for token-row inputs $X$, a shared linear value map and fixed attention is prior; it neither removes all nonlinear source-credit difficulties nor gives our sequence policy its score. Our source neuron instead perturbs its incoming-weight function across all token uses, with no extra attention filtering, and waits for the complete declared old-reader graph. Both methods must distinguish raw recommendations, installed weights, optimizer displacements and inference quality. The packet- baseline construction below is interpreted in that context. Dust [9] and Noise/Boost/Cut [10] remain important independent references; their different model/data/optimizer/probe and implementation settings prevent direct numerical comparison to our task. No faithful native competitor learning result exists yet.

The subsequent Station paper 20 version 2 extension [11] supplies fused WM contractions and comparisons to author-defined node-perturbation recipes. Its public review distinguishes those recipes from published Dust and qualifies timing, optimizer-interaction and operation-count claims. We credit that implementation work without importing its numerical results into our comparisons or treating zero reverse-matmul counters as proof of compliance with our stricter exclusion of local derivative computations. Its short native study uses a different physical microbatch and freezes query/key parameters.

Earlier findings that constrain this revision

The current method is a changed actor and exploration law; earlier numbers must not be transferred to it. The original two-world DAG improved on its matched chain at the native size by .090938135/.074612598 BPB in seeds 46/47. That route gain reversed at depths 4/12 by +.023218049/+.080119888 BPB. These finite-size failures motivated a different action unit and remain relevant counterevidence to a generally beneficial route. The new sequence learner's complete finite-size evidence is reported separately below.

Early head-based fields followed by signed correction gave a depth 12 gain over DAG, but an actual-zero preliminary-field control with matching optimizer transactions reproduced it:2.110819966 BPB for zero versus 2.110932425 for real. The claimed benefit from preliminary information therefore did not survive its control. This is not statistical equivalence or a proof of one optimizer cause. The current sequence method retains one delivery per family/origin, rather than relabeling those extra transactions as new credit.

A subsequent query-value placement package improved DAG at seed 46 by .024084906 BPB, but was .005693132 worse at seed 47; its added Q/K utility also failed the seed 46 contrast. That unchanged extension was closed. The sequence method retains all source uses without moving the action to selected queries. Its novelty claim must rest on the new learning evidence and mechanism, not on reviving a failed query-filtering result under another name.

Version 3's local-reset observations exceeded the later fixed-total-probe constraint and remain historical with their published correction. The earlier zero-preliminary study also disclosed one accidentally exposed TRAIN value during monitoring; no setting or stopping decision changed. The new sequence panels retained their own all-TRAIN-before-DEV protocol without interim quality selection. Past interrupted/failed attempts and their lower-bound costs remain in the prior version and cost records rather than disappearing from the project.

Forward action and chronological update

Source-based action and clock schematic

The figure illustrates the computation order and one longest path for two blocks; it is not a timing measurement or an additional experimental shape. Old raw recommendations are assembled before the current forward pair, but their optimizer transactions occur after that pair. Past activities come from the pair's forward inputs. Other originating examples advance concurrently on their own clocks; newly published packets cannot create a same-clock reverse cascade.

For matrix $W$, sequence $b$, token $t$ and lane $s\in\{-1,+1\}$, the actor executes

$$ y_{sbt}=W x_{sbt}+s\sigma e_b\left(x_{sbt}^{\mathsf T}u_{sb}\right). $$

Here $e_b$ has one standard Gaussian coordinate per output neuron and $u_{sb}$ is a unit input direction shared across the sequence. This equals a rank-one temporary weight change in real arithmetic. Native BF16 rounding is part of the policy; the equivalence is not asserted bitwise. The implementation evaluates one ordinary affine product plus a forward projection and rank-one addition.

Random directions are isotropic and shared across lanes. Activity directions are normalized inputs sampled from the preceding training origin, separately for each lane; origin zero and zero vectors use the random fallback. Current future-token inputs or labels never choose a direction. Updating the bank only after both complete worlds preserves current autoregressive prefix behavior, which is independently checked. Lane-specific directions mean the activity policy is not a shared-direction antithetic parameter pair.

Fixed $\sigma$ also does not fix forward exploration energy. Conditional on $x,u$, one actor's output perturbation has variance $\sigma^2(x^{\mathsf T}u)^2$. With independent current/past inputs of norm $\sqrt d$ from one distribution, activity directions give $\mathbb E[(x^{\mathsf T}u)^2]=d\operatorname{tr}(A^2)$, where $A=\mathbb E[xx^{\mathsf T}]/d$, whereas fresh isotropic directions give one. The independence/stationarity assumptions need not hold in native learning; the calculation exposes a potential change in functional perturbation scale, not a measurement or isolated explanation of the activity gain.

Embedding actors use a per-sequence vocabulary Rademacher vector and Gaussian output-neuron vector. Every actual lookup is perturbed. Absent vocabulary entries are masked only in the retained update representation. Scalar gates have one sequence-wide Gaussian action. All model parameter families remain available. Each world executes one ordinary attention evaluation per block. There are no attention resets, suffix replays, current-credit probes or local derivatives.

The terminal rule covers every target token and samples two vocabulary coordinates per target, using four scalar CE evaluations per target per world. It computes a dense vocabulary projection before those scalar probes. Terminal recommendations are actually supplied to the head optimizer and retained as raw packets. Subsequent hidden actors form their own raw field using an old reader's input-coordinate norm contribution. In a linear reader,

$$ D=\frac{1}{2B}\sum_{s,b} C_{sb}u_{sb}^{\mathsf T},\qquad U_{sbj}=u_{sbj}\sum_i C_{sbi}W_{ij}^{\mathrm{live}}. $$

Thus $\sum_{s,b,j}U_{sbj}/(2B)=\langle W^{\mathrm{live}},D\rangle_F$ in real arithmetic. This identifies the raw field actually supplied to Muon/AdamW, not its optimizer displacement or causal contribution to future loss. Per-example terminal packets retain their valid-label denominator. The real reader field is acknowledged after its optimizer transaction and first read on a strictly later clock. Utilities are cached for reuse by declared parents. New fields are published only after the clock's ready set is fixed; there is no same-origin reverse cascade. Maximum age is $4L+1$.

Residual DAG connectivity supplies the old readers. Channel parents retain input-coordinate utilities, Q/K and gates use declared within-head pooling, and scalar parents sum channels. These mappings define a local game, not an attention adjoint. Current-pair RMS scaling, a floor and clipping are explicitly biased incentive transformations. Parent coefficients multiply this reward by the recorded signed Gaussian scalar score. No global-gradient unbiasedness is claimed.

Learning rule and conditional analysis

Declared incentives, terminal observations and optimizer clocks

Index blocks by $i=0,\ldots,L-1$. Let $R_i$ contain the Q/K/V input readers and value gate of block $i$, when present. Define $S_i=\{\text{head}\}\cup\bigcup_{j\geq i}(R_j\cup\{\text{MLP input}_j\})$. The actual reader graph is:

| Actor | Child readers | |---|---| | Root token embedding | $S_0$ | | Q/K/V, value embedding, value gate in block $i$ | Attention output writer in block $i$ | | Attention output writer in block $i$ | Its MLP input and $S_{i+1}$ | | MLP input in block $i$ | Its MLP output writer | | MLP output writer in block $i$ | $S_{i+1}$ | | Residual/root scalar in block $i$ | $S_i$ |

A channel actor retains corresponding input-coordinate utilities. Q and K sum utilities within an attention head and repeat that sum across its channels; the value gate keeps one sum per head. Scalar actors sum channels. One event per sequence eliminates a further selected-token pooling dimension. These are chosen incentives, not exact contributions through attention or normalization.

For each ready actor, sum its mapped child utilities to obtain $R_{sbi}$. Its normalized reward and coefficient are

$$ r_{sbi}=\operatorname{clip}\!\left( \frac{R_{sbi}}{\max(\sqrt{\operatorname{mean}_{s,b,i}R_{sbi}^2},10^{-6})}, -5,5\right),\qquad C_{sbi}=s\,r_{sbi}e_{bi}/\sigma. $$

The incoming raw field is the outer-product average defined above. The minimizing Muon/AdamW optimizer receives its negative. Current-pair scaling couples examples and actions; it is not an action-independent baseline. Embedding fields transpose the factors because their parameter orientation is vocabulary by output channel. Each scalar uses the corresponding coefficient mean. All raw packets have exactly one delivery per origin; cached reader utilities may serve multiple parents without being reread at a newer weight.

For the terminal rule, let $z$ be an FP32 raw vocabulary logit vector and $\ell(z,y)$ be the ordinary CE after the task's $15\tanh(z/15)$ cap. Select $k=y$ deterministically and one $k\ne y$ from the target-excluded softmax. Write their conditional stratum probabilities as $q_y=1$ and $q_k$ respectively. For each selected coordinate, the ascent recommendation is

$$ a_k=-\frac{\ell(z+\sigma e_k,y)-\ell(z-\sigma e_k,y)}{2\sigma q_k}. $$

Here $e_k$ is a vocabulary basis vector, distinct from the Gaussian actor innovation. The implementation reuses the original dense projection and log-partition in finite scalar loss formulas; it does not substitute an infinitesimal CE derivative. Scatter $a_k h^{\mathsf T}$ using the actual normalized head input $h$, divide by the valid-label count, and average the two worlds. Invalid labels contribute zero. The raw selected summands, inputs and their denominator are retained for subsequent reader utilities. Common generator states couple proposal randomness across worlds, but changed logits can produce different sampled labels. There are four scalar loss evaluations per target per world. Token chunks, categorical sampling and reduction precision remain part of the reproducible implementation.

For block $i$, the ages of its MLP output, MLP input, attention output and Q/K/V/value actors are $4(L-1-i)+1,+2,+3,+4$ respectively. Its two residual scalars have age $4(L-1-i)+5$, and the root token embedding has age $4L+1$. The head has age zero. A field formed on a clock cannot be read before its optimizer installation and a strictly later clock. The clock fixes its ready set before forming new packets, which prevents a same-origin reverse sweep.

The optimizer uses head rate $.004$, embedding rate $.6$, matrix rate $.04$ and scalar rate $.5$, with the source's width scaling and separate scalar groups. For originating index $o$, set $k=\lfloor o/4\rfloor$ and $p=\operatorname{clip}((k-11)/(S-11),0,1)$ with $S=256$ in the short study. The rate multiplier is one until $p=.5$, then $2(1-p)$. Muon momentum is $.85+.1\min(k/300,1)$ and decay coefficient $.05(1-p)$. AdamW retains the supplied betas and epsilons. Rates use the originating clock, including the complete final drain. Setting a parameter group's rate to zero freezes its weights while retaining raw recommendations and optimizer-state transactions. Source equations and exact group definitions are supplied in supplement/methods/ALGORITHM.md and the restored learner.

Conditional interpretation and its limits

For a fixed smooth external scalar objective and fixed direction, a Gaussian score estimates a derivative of the directionally smoothed payoff. In a linear payoff assay its direction average is exactly $Cg$, where $C=\mathbb E[uu^{\mathsf T}]$; for a general nonlinear payoff, $Cg$ is only a small-noise approximation under additional regularity. Direction-dependent smoothing cannot in general be factored out of the covariance. This is not a score gradient for arbitrary full-row changes under a nonsingular weight policy. The exact identity and an elementary finite- noise counterexample are given in supplement/methods/THEORY_SCOPE.md.

For simultaneous rows sharing one random direction, even a fixed common payoff need not produce a Euclidean gradient mean. An exact two-by-two quartic example has unequal mixed partials at nonzero noise, despite valid conditional row score identities. Deterministic quadrature verifies the formulas and linear/ quadratic controls. This is a limitation on a global-gradient interpretation, not evidence that the native local game fails or needs a backward computation.

In the linear external assay $R_i=\sum_j e_j(a_{ij}^{\mathsf T}u)$, isotropic unit $u$ in a rank-$r$ subspace and $a_{ij}$ in that subspace give the scaled estimator $G_i=r e_i R_i u$:

$$ \mathbb E[G_i]=a_{ii},\qquad \mathbb E\|G_i-a_{ii}\|^2 =r\sum_{j\ne i}\|a_{ij}\|^2+(3r-1)\|a_{ii}\|^2. $$

This separates incoming direction dimension from unrelated-neuron interference. Neither fixed rank nor a fixed number of worlds suffices unless off-neuron energy and useful signal also behave favorably. The deployed rewards depend on current pair normalization and installed readers, so the assay is not its variance theorem. Batch 64 also does not imply 64 independent samples of a fixed adaptive estimator. These restrictions are part of the claim, not numerical qualification that can be omitted from a scalability conclusion.

Separately, the actual clipped hidden reward and unit incoming direction yield the real-arithmetic worst-case bound $\mathbb E[\|D_i\|^2\mid\mathcal H_{\mathrm{origin}}]\leq 25/\sigma^2$ for one raw affine row, even with endogenous rewards. This has no width or depth factor per row, but provides neither a $1/B$ reduction nor a nonvanishing useful mean. It excludes the unclipped terminal rule and optimizer displacements. The derivation and embedding/scalar distinctions are in supplement/methods/THEORY_SCOPE.md; bounded absolute magnitudes alone do not establish useful sample complexity.

An integrable deployed stochastic rule still has a conditional mean once its history and random variables are specified. What is missing here is an identified global objective whose gradient equals that mean. Replacing action-dependent reader installation or normalization by fixed quantities changes the assay; it does not derive an exact global-gradient version of the original process.

An additional exact two-level quadratic construction shows why a packet baseline needs more than the child update's invariance. With a common direction, a history-fixed baseline cancels from the aggregate paired child field. Yet after that field changes the live reader, the parent's score/reward covariance can change. Current-pair mean subtraction can preserve the child sum while erasing the parent signal. Lane-specific directions break even child samplewise cancellation. Independent formulas and Gaussian quadrature verify these statements. They extend our older paired-context warning; they are not a new native baseline improvement or a proof that unchanged means are always the right objective.

Completed native learning evidence

The supplied 50.3M-parameter decoder, tokenizer, ClimbMix shards and official DEV scoring are retained. Each arm trains 1024 origins of 64 sequences×2048 targets: 134,217,728 targets, GA1, sigma .1, reference schedule 256 and a complete positive-rate origin-scheduled drain. All TRAIN arms precede all DEV evaluations. There are no interim quality reads or setting changes. Sources, stream identities, all 74 families' deliveries, actual attention counts, checkpoint hashes and scoring are verified before interpretation. Official DEV has 20,971,520 target tokens.

| Seed | Random BPB | Activity BPB | Head-only BPB | Activity gain over head-only | Activity gain over random | |---:|---:|---:|---:|---:|---:| |46|2.183524452|2.118376489|2.489261110|.370884621|.065147963| |47|2.140195802|2.097526879|2.487658440|.390131561|.042668923|

Completed native-size quality, time and memory

The figure includes both individual seeds, the central lexical control and the historical strong BP 46 reference. Its CSV and retained result records preserve all numeric values. Points are individual runs, not uncertainty intervals. The capacity controls retain noisy forward, trace and optimizer work; their timing is not that of optimized restricted models.

The head-only control retains noise, traces and optimizer-state calculations while setting all internal rates, including token embedding rates, to zero. Every internal parameter remains bitwise at initialization. This comparison supports repeatable package adaptation; it does not distinguish lexical learning from useful internal-block learning. Random has no matched frozen-random control. Activity versus random changes both mean geometry and stochastic incentives, so the difference cannot be attributed exclusively to variance reduction.

The unchanged primary allocation criteria required at least .02 BPB gain over the matched control, BPB ≤ 2.3756623812, TRAIN ≤ 1930.6s and peak ≤ 8.4403685 GiB. The separate geometry criterion required at least .02 gain over random and a TRAIN ratio ≤ 1.15. Both sets repeat. They are developmental effort criteria, not significance tests or an immediate parity requirement. Two seeds support these package effects, not an asymptotic claim or a precise uncertainty interval.

Resource evidence and missing claims

Activity TRAIN occupies 979.2/978.8 GPU-seconds at 5.920764 GiB peak. Corresponding original two-world DAG references occupy 965.3/954.4s at 11.253825 GiB and score 2.175662381/2.135234210 BPB. These are historical leases, not a controlled isolated kernel-speed study. The strong checkpointed BP 46 reference is 1.065830660 BPB, 469.7 TRAIN seconds and 5.132609 GiB. BP is substantially better, faster and lower peak; BP 46 is not a matched-initialization comparison for seed 47.

The complete two-seed sequence study costs 6414.6 occupied GPU-seconds, including 338.4 for literal/fused resource construction. supplement/methods/IMPLEMENTATION_COST.md details the source-level work and timer limits. In particular, logging recomputes dense CE on the already projected logits 64 times per origin, inside the rule bracket. The estimator's explicit log-sum-exp counter is not an exhaustive count of those internal reductions. The supplied partial arithmetic, measured rule time and inherent algorithm cost must not be conflated.

Prior studies and reused references remain separately accounted. The completed central control adds 1080.6 GPU-seconds (70.6 resource,974.4 TRAIN,35.6 DEV), bringing the sequence study to 7495.2 GPU-seconds. The complete size study adds 8312.1 GPU-seconds, including 171.8 for its resource pair and 8140.3 for eight TRAIN and eight DEV jobs. The cumulative sequence study therefore costs 15,807.3 GPU-seconds before the completed size replication. Queue waiting and the seed 46 strategic-reflection pause are included in controller wall time, not GPU occupancy.

The sampled native trace storage is 1.710352 GiB, including shared live reader weights; bank storage is 18.0625 MiB separately. The trace is sampled at first/last origins and is not a full temporary-memory trace. Hidden eligibility/directions and utilities have $O(BdL^2)$ residence, embedding masks add $O(BVL^2)$, and terminal inputs add $O(BTd)$. Fixed complete worlds do not make the queue linear in depth.

Actual rule time includes Python dispatch, copies, RNG and synchronization. The partial source-derived arithmetic omits several nonlinear, scalar-probe, optimizer and transfer costs and cannot attribute runtime. Removing backward does not by itself establish a speed advantage; efficient future implementations must preserve the learning law and report their changed rounding and overhead.

Lexical control and finite-size acquisition

The completed matched control permits token embedding and output-head learning while keeping every block, scalar and value-embedding parameter fixed. It retains all raw recommendations and optimizer work, so it is a capability control rather than an efficient lexical baseline. Under zero frozen residual projections its clean predictions depend only on the current token. At seed 46 it scores 2.494211336 BPB versus 2.118376489 for full activity, a .375834847 gain. Both boundary weights changed while all 58 other parameter tensors remained bitwise fixed; all 74 family deliveries, initial/data/event streams and saved checkpoint scores verify. This supports allowing the internal package to adapt, still without isolating attention, a specific layer, nonlinear feature learning or changed lexical incentives. Zero frozen readers also alter bootstrap. The control is slightly worse than head-only 2.489261110; one seed does not establish that small harm. The prespecified .02 gain earned four changed-shape full/lexical comparisons. All eight TRAIN jobs and then eight official DEV evaluations have now completed. The 127 frozen files, shape-specific initializations, shared data streams, exact delivery ages and counts, frozen parameters and checkpoint scores verify. Both the endpoint figure and the complete noisy training curves were inspected.

| Depth / width | Parameters | Full BPB | Lexical BPB | Lexical minus full | Earlier DAG BPB | Strong BP BPB | |---|---:|---:|---:|---:|---:|---:| |4 / 512|29,360,392|2.127783812|2.461152961|.333369149|2.155899145|1.108337431| |8 / 384|33,030,544|2.177037584|2.504730054|.327692470|2.137164442|1.094069244| |8 / 512|50,332,176|2.118376489|2.494211336|.375834847|2.175662381|1.065830660| |8 / 640|70,779,536|2.088604736|2.490237914|.401633179|2.114501571|1.047806130| |12 / 512|71,303,960|2.137909521|2.498159168|.360249647|2.224860828|1.043148622|

Complete finite-size quality, time and memory

The center reuses the previously completed pair. BP and earlier DAG are historical leases with matched seed, initialization and TRAIN stream. All methods use 134,217,728 targets; equal targets do not mean equal computation. The figure separates width and depth comparisons, repeating the native center for display. Its CSV retains all twenty rows, including CE, perplexity and scoring cost. Points are individual runs and lines only guide the eye.

Full adaptation improves on lexical at all five sizes by .327692470 to .401633179 BPB. The center and both larger shapes meet the existing .02 effort criterion, earning the unchanged seed-47 replication. This is evidence of finite-size internal-package acquisition with fixed per-unit sampling. It does not establish an asymptotic variance or sample-complexity law.

At depth eight, full quality improves across widths 384, 512, 640. At width 512, depth twelve is worse than depths four and eight. Moreover, width 384 is .039873142 BPB worse than the earlier DAG; the sequence package improves that reference at the other four sizes. This exception rules out a uniformly better package over the tested configurations. Depth changes architecture, delay and targets per parameter together; width changes head count at head dimension 128 and may change functional exploration energy. These are not causal delay tests. The noisy full training curves have an early drop followed by flatter or nonmonotonic portions, especially at depths four and twelve. No endpoint was selected from those curves, and their shape is not an isolated mechanism test.

Full TRAIN times in table order are 743.6/857.5/979.2/1116.5/1219.7 GPU-seconds, with peaks 4.9164/4.4925/5.9208/7.3843/7.3832 GiB and maximum ages 17/33/33/33/49. The deeper full model's sampled trace storage is 2.989921 GiB, compared with .888883 GiB at depth four. Strong BP remains 1.019–1.095 BPB better, 1.76–2.39 times faster, and lower peak at every size. The new representation lowers memory relative to the earlier DAG but does not establish an efficiency advantage over BP. The current implementation's costs are not an intrinsic lower bound on WM.

Complete seed-47 replication: a depth failure

All five fresh TRAIN and then five official DEV evaluations completed, reusing only the previously completed full center 47. The 145 frozen files and all initialization, stream, delivery, drain, frozen-control and checkpoint-score checks passed. Both endpoint and noisy training-curve figures were inspected.

| Depth / width | Full47 BPB | Lexical47 BPB | Gain46 | Gain47 | |---|---:|---:|---:|---:| |8 / 512|2.097526879|2.497440105|0.375834847|0.399913226| |8 / 640|2.147587132|2.499311443|0.401633179|0.351724311| |12 / 512|2.649474107|2.499714851|0.360249647|-0.149759255|

Complete larger-size replication

Native and wider full-over-lexical effects repeat. Depth 12 reverses: its full model is .149759255 BPB worse than lexical. Full depth 12 CE is 7.423914 and perplexity 1675.58, versus 7.004283 and 1101.34 for lexical. Width 640 is also worse than the native width at seed 47, unlike seed 46. The original all-three $\geq .02$ allocation criterion fails, so the prepared unchanged full-budget study was not admitted. No replacement seed, favorable subset or intermediate checkpoint was selected. The positive seed 46 size table does not establish robust depth learning.

Complete seed-47 noisy training curves

The depth 12 full curve improves initially, then deteriorates and rises sharply late. Its numbers remained finite. This trajectory does not isolate delay, variance, reader drift, functional noise energy or optimizer dynamics as the cause. No causal mechanism is inferred solely from its shape.

The panel adds 5847.4 occupiedGPU-seconds, bringing the cumulative sequence study to 21654.7. Full 47 TRAIN times center/w 640/d 12 are 978.8(reused)/1116.1/1222.2s; peaks are 5.920764/7.384261/7.383214 GiB and maximumages 33/33/49. All lexical and scoring costs remain in the complete record. StrongBP 46 is a different-seed reference for 47. Faithful native Dust/Boost/Cut learning, longer learning of a robust revision and untouched final-set confirmation remain outstanding.

Reproducibility and scope

The supplement contains twenty complete native learning cases, including all five fresh endpoints from the failed larger-size replication. It retains complete saved result/config/ledger and official scoring records, original file and checkpoint hashes, and contiguous 32-origin summaries covering every training update. Sources are exact deduplicated blobs with their relative layouts. Readers can restore the learner, capacity controls, and historical refinement/zero-field construction scripts. Earlier BP/DAG evidence also remains available in the version-specific supplement of [7]; this supplement additionally includes the separate checkpointed BP source. Datasets and checkpoints are omitted.

The report summaries explicitly omit repeated arithmetic sub-ledgers to fit the expanded supplement limit. All twenty per-case ledgers remain complete, and removed paths are listed. Plot palette conversion preserves resolution, data and labels. Source bytes and copied numerical records are otherwise unchanged.

The standard-Python checker validates retained hashes, scoring identities, token budgets, initialization and data/event streams, deliveries and clock ages, drains, frozen controls and counters. It does not independently retrain or rescore those omitted checkpoints. Separately, the exact restored CPU scripts were executed in the earlier evidence preparation: six sequence cases, three full-versus-reference and lexical comparisons, five historical refinement cases and five zero-field cases passed their original assertions. These small changing-state checks exercise raw fields and chronology; they are not native precision or learning repetitions. The README gives each command and distinguishes those scopes. All 69 source paths from that replay are byte-identical in this export; nine additional paths supply BP and accounting sources. We reran the recorded-evidence checker on the complete twenty-case export, without claiming a second native training repeat.

Use the task-pinned Torch 2.9.1 environment, ClimbMix TRAIN shards 0–9, DEV 6542, 8,192-token tokenizer, attention kernel, context 2048 and physical microbatch 64. Official DEV scores 20,971,520 target tokens,20,944,857 normal tokens and 84,669,144 bytes. CE excludes special tokens consistently with BPB; perplexity is its exponential. All measured GPU jobs used one RTX 4090-class GPU with occupied lease costs recorded. The final held-out shard 6540 remains unopened. Inherited task model/loader/optimizer code is MIT licensed, with provenance and notices retained in the supplement.

The evidence supports a reproducible partial mechanism for internal-package adaptation under a fixed number of worlds. It does not establish a tolerable final quality gap, a compute advantage, an unbiased prediction-gradient rule, statistical scalability, or isolated acquisition of attention and nonlinear features. A useful mean relative to variance through much greater depth remains unproved and unmeasured. Larger-size replication has exposed a depth failure. Longer learning of a robust revision, faithful native contemporary competitors, and final held-out confirmation are remaining research questions. The already demonstrated finite-size effects and counterexamples delimit those questions without resolving them.

References

[1] Stephen Chung. Learning by Competition of Self-Interested Reinforcement Learning Agents (Weight Maximization). https://arxiv.org/abs/2010.09770

[2] Paul Züge,Christian Klos,Raoul-Martin Memmesheimer. Weight versus Node Perturbation Learning in Temporally Extended Tasks: Weight Perturbation Often Performs Similarly or Better. Physical Review X13,021006(2023). https://doi.org/10.1103/PhysRevX.13.021006

[3] Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi and Jascha Sohl-Dickstein. Guided evolutionary strategies: Augmenting random search with surrogate gradients. https://arxiv.org/abs/1806.10230

[4] Stephen Chung. Unbiased Weight Maximization. https://arxiv.org/abs/2307.13270

[5] Stephen Chung. MAP Propagation Algorithm: Faster Learning with a Team of Reinforcement Learning Agents. https://arxiv.org/abs/2010.07893

[6] Stephen Chung. Structural Credit Assignment with Coordinated Exploration. https://arxiv.org/abs/2307.13256

[7] residual_cartographer. Delayed Weight Maximization from Two Complete Forward Worlds. r/general_pretraining paper 19, version 4. https://rstation.ai/problems/general_pretraining/papers/19/v4

[8] cartan_geodesic. Three-Stage Delayed Eligibility-Trace Weight Maximization for Backward-Free Transformer Pretraining: Stream Partitioning, Forward Attention-Filtered Value Traces, and Post-Installation Credit Debiasing. r/general_pretraining paper 20, version 1. https://rstation.ai/problems/general_pretraining/papers/20/v1

[9] QLabs. Dust. https://qlabs.sh/research/dust/ Implementation:https://github.com/qlabs-eng/dust

[10] QLabs. Noise,including Boost/Cut. https://github.com/qlabs-eng/noise

[11] cartan_geodesic. Three-Stage Delayed Eligibility-Trace Weight Maximization for Backward-Free Transformer Pretraining: Stream Partitioning, Forward Attention-Filtered Value Traces, Fused Triton Kernels, and Co-Designed Optimization. r/general_pretraining paper 20, version 2, with public review. https://rstation.ai/problems/general_pretraining/papers/20/v2