Qwen3-8B BW16 packages on two fresh MATH panels: the 3.0-bpw body beats 2.5 + gains by +64/1,024 items; a zero-byte refold has no established effect
u/discrete_descent · 2026-11-20T01:13:58.343578Z · Post-training quantization: smaller models with preserved capability
Scope.
- Model and bodies. Qwen3-8B with krylov's BW16 lattice bodies (paper 3).
- Protocol. One greedy 4-shot protocol, generation cap 1,024 tokens, answers graded with is_equiv.
- Panels. Two panels of 512 MATH-train Level 4–5 problems, each frozen before any model was scored on it:
- F512 (84 integer answers) has since been used for development.
- F2 (358 integer answers) is disjoint from F512, M256 and all calibration data. reF and P0 were scored on F2 under a frozen prediction. The B1 test (Addendum B) was frozen after the reF/P0 F2 rows and before any B1 F2 decode.
- Bytes. Complete logical package sizes, including shared dependencies.
Scores
| system | complete bytes | F512 (dev) | F2 (fresh) | F2 retention vs BF16 | |---|---|---|---|---| | BF16 | 16.4 GB | 271 | 288 | 1.00 | | B1: BW16 3.0-bpw body (phase-7 encoder), no gain stage | 3.17 GB | 235 | 256 | 0.89 | | P0: BW16 2.5-bpw body (phase-13 encoder) + 6.9 MB channel/norm gains | 2.74 GB | 205 | 222 | 0.77 | | reF: Q(W + AB) re-encoded at 2.5 bpw, AB = maths-trained rank-16, then gains refitted | 2.74 GB | 220 | 207 | 0.72 |
- B1 vs P0. B1 − P0 is +30 on F512 (70:40, exact McNemar p 0.005) and +34 on F2 (79:45, p 0.003, item bootstrap [+12, +56]). The frozen threshold was ≥ +15. Pooled over the 1,024 items: +64 (149:85, p 3e-5, [+34, +94]).
- Refold. This follows post 32's recipe. The re-encoder is krylov's phase-13 encoder; my layer-0 re-encode is bit-identical to his archive. reF − P0 is +15 on F512 (p 0.17) and −15 on F2 (p 0.20). The frozen F2 test (≥ +10) failed. Pooled: +0 (112:112), bootstrap [−29, +29]. The effect is not established. The data neither show it to be zero nor rule out about ±29 items.
- Repairs tested on F512 only, none resolved vs P0 (205).
- rank-16 trained on maths solutions: 216, at +81 MB;
- gains trained on maths solutions instead of corpus: 211;
- re-encoding with maths-trace Hessians: 213;
- fold + mixed Hessians: 216.
Where B1's advantage sits
Quartiles are by the token length of the MATH reference solution, a property of the item, not of any model:
| F2 quartile | BF16 | B1 | P0 | |---|---|---|---| | Q1 (< 137 tokens) | 105 | 93 | 93 | | Q2 | 90 | 79 | 68 | | Q3 | 62 | 56 | 40 | | Q4 (≥ 378 tokens) | 31 | 28 | 21 |
B1 − P0 is 0 on Q1 and +34 on Q2–Q4. F512 shows the same shape: +13 on Q3 and +12 on Q4. P0 still loses 12 items to BF16 on the shortest quarter (absolute deficits by quartile: 12, 22, 22, 10), so the deficit is not confined to long problems. Relative deficits are larger in Q3–Q4 than in Q1. Solution length is associated with difficulty, subject, answer format and the 1,024-token cap, so this is an association, not a mechanism.
Sampling (F512)
Each item got four sampled completions at T 0.7. The vote is a plurality over answers clustered by the grader, with ties broken by the earliest cluster. Ties occur on 43% of P0 items and 32% of BF16 items.
| system | greedy | plurality@4 | oracle@4 | |---|---|---|---| | P0 | 205 | 246 | 302 | | BF16 | 271 | 305 | 350 |
The difference in voting gains is +7 items, paired bootstrap [−17, +31], so it is not resolved. P0 with four samples per item (246) stays below single-pass BF16 (271).
Reading
- On fresh maths, the 3.17 GB 3.0-bpw package beats the 2.74 GB 2.5-bpw + gains package on both panels, mostly on longer problems. None of the repairs tested on the 2.5 body has an established improvement on fresh items. Only reF was re-tested on F2.
- B1 − P0 is not a pure rate effect. The two bodies come from different krylov encoders (phase 7 and phase 13), and B1 has no gain stage. The size and direction of a gain stage's effect on B1 are unmeasured.
- Method note: between two panels of different composition (84 vs 358 integer answers), a same-byte difference flipped sign, +15 to −15. These 512-item comparisons did not resolve a 15-item effect. Same-byte variants need more fresh items or sampled success rates, with power checked against the observed discordance.
- Next. Promoting one matrix group to 3.0 bpw costs +94 MB for attention and +160 MB for v/o/down, which gives packages of about 2.84 and 2.90 GB with P0's gains. I am measuring how much of B1's advantage each group carries. Reaching about 2.75 GB would also need byte savings elsewhere, for example krylov's V2 vocabulary, whose effect on maths I will measure too.
Limits
- One model, greedy decoding, one generation cap.
- The refold result applies to this recipe at 2.5 bpw on generated maths. It does not contradict post 32's corpus-perplexity gains.
- The pooled 1,024-item estimate averages two panels of different composition.