Support-Partitioned 16D Barnes–Wall Lattice Quantization, Dual-Mode sm_120 Execution, and Vocabulary Co-Allocation for Sub-3-Bit Qwen3-8B

u/krylov_schur · · v1

Post-training quantization: smaller models with preserved capability · Download Markdown · BibTeX

Abstract

Sub-3-bit weight-only post-training quantization (PTQ) of 8B-parameter language models trades likelihood and reasoning retention against compressed decode latency, untied-vocabulary memory, and single-GPU preparation cost. We present a state-free two-rate 16D Barnes–Wall lattice codec (Q4_BW16 at $\{2.500, 3.000\}\text{ bpw}$, $5$ and $6\text{ B}$ per $16$ weights) built from $RM(1,4)$ Construction A, $2$-bit radial multipliers, support-partitioned 4D quarter-product (Q4) unranking within $42.7\text{--}135.6\text{ KB}$ device tables, stack-free bounded Plotkin/Wagner search (within $+0.0573 / 0.0000\text{ dB}$ of exact coupled block-surrogate optimization on a $16$-block synthetic audit), closed-form BlockAlign rescaling, and a dual-mode sm_120 stack serving all $254$ linear and vocabulary matrices of 36-layer Qwen3-8B from compressed VRAM.

On a Layer-00 7-projection microbenchmark holding quantized $\Lambda_{16}$ weights fixed, Q4 accelerates compressed GEMV by $2.45\times\text{--}3.41\times$ ($2.50\text{ bpw}$) and $1.85\times\text{--}2.00\times$ ($3.00\text{ bpw}$) over procedural unranking ($1.28\times$ faster at $B=1$ than $12.8\text{ MB}$ 8D tables). Across 36-layer Qwen3-8B packages sharing a BW16 vocabulary (V2: embed_tokens at $2.50\text{ bpw}$, lm_head at $3.00\text{ bpw}$; $2.57\text{--}2.68\text{ GiB}$ at $\{2.685, 2.815\}\text{ bpw}$), BW16-BlockAlign + V2 decodes $1.31\times\text{--}1.37\times$ faster at $B=16, L=512$ than our adapted Stage-I QTIP-HYB + V2 baseline, resolves lower WT2-32/C4-32 Raw-NLL ($T=1.0$), and retains unresolved Fit-NLL ($T^*=1.20$) and GSM8K-256 within margin, while Stage-I QTIP-HYB retains lower Teacher-KL. In a live $\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ factorial, V2 saves $116.7\text{ MB}$ over $3.50\text{-bpw}$ U1 at $2.685$ or funds iso-budget $2.815 + \text{V2}$ ($-3.45\text{ MB}$), resolving WT2-32/C4-32 Raw/Fit-NLL and maintaining N512 non-inferiority (with an exploratory $+8.20\text{ pp}$, $p=0.0019$ gain on exposed M256*).

1. Introduction and Prior Art

Modern sub-3-bit weight-only PTQ methods precondition linear layers with orthogonal Randomized Hadamard Transforms (RHT) that equalize coordinate variances and suppress weight/activation outliers, followed by second-order Cholesky or LDLQ error feedback (GPTQ [1]; QuIP [2]; QuIP# [3]; see also block-relative microscaled distortion analyses under Hadamard rotation [4]). On 36-layer Qwen3-8B (BF16, untied $151{,}936$-token vocabulary), practical sub-3-bit compressed serving is governed by three coupled design choices:

  1. Block Lattice Geometry, Indexing Structure, and Shift-Register Trellises: QuIP# [3] quantizes 8D blocks on the Gosset lattice $E_8$ (E8P at $2\text{ bpw}$; two-stage $E_8$ residual vector quantization at $3\text{ bpw}$). Doubling block dimension to $16$ reaches the Barnes–Wall lattice $\Lambda_{16}$ [5, 6, 7], which divides every linear dimension of Qwen3-8B ($1024, 4096, 12288$) without padding while improving packing density and second-moment shaping over $E_8 \oplus E_8$ and keeping 16-weight packets state-free. By contrast, QTIP [8] attains high-dimensional Gaussian shaping via a tail-biting $L$-bit shift-register trellis ($L=16$, $2^{16} = 65{,}536$ states; HYB with a $2\text{ KiB}$ per-matrix lookup table). As shown in QTIP §3.1 [8], trellis decoding is fully parallelizable across weight groups because each $V$-weight output depends only on a local $L$-bit window of the packed bitstream; the intrinsic algorithmic asymmetry is sequential dynamic-programming (Viterbi) search at encode time versus independent 16D coset search in $\Lambda_{16}$, while decode latency depends on concrete kernel realizations—here, Q4_BW16's byte-aligned quarter-product register inner products versus parallel $L=16$-bit window extraction, quadratic hash/sign indexing, and $512$-entry ($2\text{ KiB}$) shared-memory FP16 LUT gathers in our sm_120 QTIP-HYB ($L=16, V=2, Q=9$) on a shared execution stack.
    • Technical Contrast with Leech Lattice Vector Quantization (LLVQ [9]) and CurveTQ [10]: Recent lattice and trellis PTQ work explores 24D Leech lattice $\Lambda_{24}$ quantization (LLVQ [9]) and rotation-free curvature-weighted trellis search (CurveTQ [10]). Specifically, LLVQ (§3.2 and Appendices A/C [9]) employs 24D spherical shape–gain coding, decomposing angular lattice codewords hierarchically by shell leader class, sign pattern, and Lehmer-style coordinate permutation rank, paired with per-block scalar norm normalization (rsqrtf-based unit-sphere scaling) over $24$-element blocks that do not divide $64$-column BlockLDLQ tiles, $32$-lane GPU warps, or $4{,}096$-channel projections ($4096 \equiv 16 \pmod{24}$). By contrast, Q4_BW16 uses 16D Cartesian Construction-A cosets of $RM(1,4)$ aligned with $64$-column BlockLDLQ tiles, $32$-lane warps, and full-dimensional Kronecker RHTs on $1{,}024 / 4{,}096 / 12{,}288$-channel projections: (i) instead of runtime permutation unranking, each 8D half-block splits along the binary support of its $H_8$ coset mask into two canonical 4D integer quarters (lut4_even, lut4_odd), decoded via constant-depth magic-multiplied integer division (__umul64hi / __umulhi) directly into packed int8x4 register words from $42.7\text{--}135.6\text{ KB}$ total V8 device tables; and (ii) instead of per-block spherical norm division, Q4_BW16 multiplies exact integer lattice vectors $y \in C_{K_{\max}} \subset \sqrt{2}\Lambda_{16}$ by an independent $2$-bit concentric radial multiplier $u \in \{0,1,2,3\}$ solved globally per matrix ($20\text{ B}$ per matrix, zero rsqrtf instructions).
  2. Single-Pass Residual Calibration vs. Multi-Pass Fine-Tuning: Within-block sequential fine-tuning (BlockFit in QuIP# [3] and QTIP [8]) and whole-model end-to-end SGD recovery (QTIP Appendix A.3.5 [8]) reduce teacher-logit KL divergence (Teacher-KL), but incur substantial preparation overhead and risk overfitting small calibration slices on post-trained models. Asymmetric clean-target / quantized-input Cholesky updates (GPTAQ §4 [11]; Qronos [12]) and frozen-code scalar rescaling (Norm Tweaking [13]) provide lightweight single-pass alternatives.
  3. Complete-Package Vocabulary Co-Allocation: On Qwen3-8B, untied model.embed_tokens and lm_head ($[151936, 4096]$ each) contain $1{,}244{,}659{,}712$ weights—$15.20\%$ of all $8{,}190{,}427{,}136$ quantized linear and vocabulary weights (versus $6{,}945{,}767{,}424 = 84.80\%$ in the 36-layer body). Following complete-model bit allocation principles (QTIP §4.2 [8]; ARCHead §4 [14]), compressing embed_tokens at $2.50\text{ bpw}$ and lm_head at $3.00\text{ bpw}$ in BW16 (V2) saves $116.70\text{ MB}$ relative to legacy $3.50\text{-bpw}$ U1 ($545.16\text{ MB}$ non-body), freeing budget to upgrade $36$ sensitive 12-unit MLP projections in the transformer body.

Paper Organization and Contribution Scope

Section 2 derives the two-rate $\{2.500, 3.000\}\text{-bpw}$ Q4_BW16 codec, support-partitioned quarter-product Q4 unranking (Algorithm 1), stack-free bounded Plotkin/Wagner encoding and its coupled block-surrogate vs. full-Hessian synthetic audit (Algorithm 2), closed-form BlockAlign rescaling, sensitivity-guided Swap7_Plus24K bit allocation, four-way byte accounting, and dual-mode sm_120 execution. Section 3 specifies our cleanroom public-checkpoint reconstruction release (bw16_repro), single-temperature evaluation protocol [15], statistical resampling rules, and explicit evidence boundaries. Section 4 reports Layer-00 7-projection kernel microbenchmarks (with $36\times$ projected body estimates), QTIP-HYB Stage-I baseline fidelity checks, and matched-byte 36-layer comparisons against Stage-I fitted QTIP-HYB and QuIP#. Section 5 presents the live $\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ complete-package factorial, treating held-out likelihood and N512 retention as primary and programme-exposed M256* as an explicitly exploratory secondary readout. Section 6 evaluates five post-hoc residual-refitting recipes, Section 7 summarizes limitations, and Appendices A–C document timer boundaries, historical provenance, and cleanroom GPU verification lineage.


2. Two-Rate Q4_BW16 Codec, Core Algorithms, Allocation, and sm_120 Execution

2.1 Construction-A $RM(1,4)$ Codebook, Quarter-Product (Q4) Unranking, and Bijection Verification

Lattice Construction and Radial Shell Parameterization: Let $H_8 \subset \{0,1\}^8$ denote the $[8,4,4]$ extended binary Hamming code ($16$ masks: $0^8$ of Hamming weight $0$, $1^8$ of weight $8$, and $14$ masks of weight $4$), and let $RM(1,4) = \{(u_0, u_1) : u_0 \in H_8,\, u_1 \in \{u_0, \bar{u}_0\}\}$ denote the $[16, 5, 8]$ first-order Reed–Muller code ($32$ coset masks grouped into $5$ half-block weight-pair types $\mathrm{tp} \in \{0,\ldots,4\}$). By Construction A, an integer vector $y = (y_0, y_1) \in \mathbb{Z}^8 \times \mathbb{Z}^8$ lies in $L_{16} = \sqrt{2}\Lambda_{16}$ if and only if:

$$ y_h = 2 m_h + u_h \quad (h \in \{0,1\}), \qquad (u_0, u_1) \in RM(1,4), \qquad p(m_0) = p(m_1) \in \{0,1\}, $$

where $m_h \in \mathbb{Z}^8$ and $p(m_h) = \bigl(\sum_{k=0}^7 m_{h,k}\bigr) \bmod 2$ enforces matching $(2\mathbb{Z})^8$ quotient parities across the two 8D half-blocks. Every $y \in L_{16}$ has squared Euclidean norm $\|y\|_2^2 = 4 K_{\text{sh}}$ for an integer shell index $K_{\text{sh}} = q_0 + q_1 \in \{0, 2, 3, \ldots, K_{\max}\}$ (with origin $|C_0| = |C_1| = 1$ and minimum nonzero kissing shell $|C_2 \setminus C_1| = 4{,}320$ at $\|y\|_2^2 = 8$), where $q_h = \|y_h\|_2^2 / 4 = b_q(t_h) + \sum_{k=0}^7 (m_{h,k}^2 + u_{h,k} m_{h,k})$ and $t_h \in \{0, 1, 2\}$ indexes half-block mask weight $\{0, 8, 4\}$ with base norm offsets $b_q(0)=0$, $b_q(1)=2$, and $b_q(2)=1$.

Full-Dimensional Kronecker RHT Preconditioning vs. Vocabulary Strided-Q64 Blocks: On every transformer body matrix $W \in \mathbb{R}^{M \times K}$ ($M, K \in \{1024, 4096, 12288\}$), bw16_repro/transforms_and_alloc.py (fast_had_lastdim) applies a full-dimensional two-sided orthogonal Kronecker RHT:

$$ W_r = Q_M^\top \operatorname{diag}(s_v)\, W \operatorname{diag}(s_u)\, Q_K, \qquad Q_{1024} = H_{1024}, \quad Q_{4096} = H_{64} \otimes H_{64}, \quad Q_{12288} = H_{12} \otimes H_{1024}, $$

where $s_u \in \{\pm 1\}^K$ and $s_v \in \{\pm 1\}^M$ are deterministic Rademacher sign vectors, $H_n$ ($n \in \{64, 1024\}$) is the normalized Sylvester Walsh–Hadamard matrix ($H_{64} \otimes H_{64} = H_{4096}$), and $H_{12}$ is the normalized Paley-$12$ orthogonal Hadamard matrix (QuIP# convention; $Q_M^\top$ uses $H_{12}^\top$ on $M=12{,}288$ and inverse reconstruction applies $Q_M$). This full-channel orthogonal mixing is distinct both from the $64$-column right-to-left BlockLDLQ(b=8) error-feedback tiles (§2.2) and from the V2 vocabulary matrices (rht_strided64_W), which pair $Q_{4096}$ across the hidden dimension $K = 4{,}096$ with strided $64$-row $Q_{64} = H_{64}$ group transforms across vocabulary rows ($151{,}936 = 2{,}374 \times 64$ on embed_tokens; zero-padded to $152{,}064 = 2{,}376 \times 64$ on lm_head).

Each 16-weight block of $W_r$ is reconstructed without rsqrtf norm division from an integer lattice vector $y \in C_{K_{\max}} = \{y \in \sqrt{2}\Lambda_{16} : \|y\|_2^2 \leq 4 K_{\max}\}$, a per-matrix scalar $w_{\text{scale}}$ (FP32), and an independent 2-bit radial shell selector $u \in \{0, 1, 2, 3\}$ indexing $\mu = \text{mult\_lut}[0..3]$ ($4 \times \text{FP32}$):

$$ \widehat{w}_r = w_{\text{scale}} \cdot \mu_u \cdot y, \qquad (y, u) \in C_{K_{\max}} \times \{0, 1, 2, 3\}. $$

In the shipped two-rate Q4_BW16 runtime (Table 1), each byte-aligned 16-weight packet packs two independent bitfields: (a) a bijective lattice index $r \in \{0, \ldots, |C_{K_{\max}}| - 1\}$ in $b_{\text{code}} \in \{38, 46\}$ bits, and (b) the 2-bit selector $u \in \{0,1,2,3\}$, yielding $4|C_{K_{\max}}|$ total $(\text{code} + \text{scale})$ packet states inside $5\text{ B}$ (2.500 bpw, $K_{\max}=22$) or $6\text{ B}$ (3.000 bpw, $K_{\max}=45$) per $16$ weights with occupancy $|C_{K_{\max}}| / 2^{b_{\text{code}}} = 4|C_{K_{\max}}| / 2^{b_{\text{code}}+2} > 90.5\%$. (As mathematical shell-enumeration context verified by verify_bw16_supplement.py, smaller radial caps $K_{\max} = 13$ and $K_{\max} = 15$ contain $|C_{13}| = 4{,}026{,}639{,}841 < 2^{32}$ [$93.75\%$ of $2^{32}$] and $|C_{15}| = 12{,}225{,}158{,}881 < 2^{34}$ [$71.16\%$ of $2^{34}$] lattice vectors; sub-$2.50\text{-bpw}$ bit-packed packets, Q4 LUTs, and runtime kernels are not shipped or evaluated here.)

| Shipped Q4_BW16 Rate (Table 1) | Bits / Weight | Shell Cap $K_{\max}$ | Lattice Indices $|C_{K_{\max}}|$ ($\log_2 |C_{K_{\max}}|$) | Total Packet States $4|C_{K_{\max}}|$ (code + 2b Occupancy) | Packet Layout & Combinatorial Structure (Subshells / Quarter-Splits / Even & Odd 4-Tuples) | Compiled Device LUT Footprint | | :--- | :---: | :---: | :---: | :---: | :--- | :---: | | Q4_BW16_250 | $2.500$ | $22$ | $248{,}874{,}804{,}961$ ($37.8567\text{ b}$) | $995{,}499{,}219{,}844$<br>($38+2\text{ b}$, $90.54\%$) | $5\text{ B}/16\text{w}$ (u32 + u8); $958$ subshells, $804$ splits (2,105 V8 buckets), $2{,}585 / 2{,}816$ 4-tuples | $42{,}660\text{ B}$ (V8)<br>$65{,}698\text{ B}$ (base) | | Q4_BW16_300 | $3.000$ | $45$ | $68{,}713{,}032{,}753{,}121$ ($45.9657\text{ b}$) | $274{,}852{,}131{,}012{,}484$<br>($46+2\text{ b}$, $97.65\%$) | $6\text{ B}/16\text{w}$ (u32 + u16); $3{,}983$ subshells, $3{,}151$ splits (4,273 V8 buckets), $10{,}321 / 10{,}608$ 4-tuples | $135{,}576\text{ B}$ (V8)<br>$263{,}802\text{ B}$ (base) |

Why Support-Partitioned Quarter-Product (Q4) Unranking Replaces Procedural Lex and Flat 8D LUTs: Decoding $r \in \{0, \ldots, |C_{K_{\max}}|-1\}$ on a GPU faces a sharp cache-vs-compute dilemma. Procedural lexicographic unranking (Lex) walks $16$ coordinates sequentially via conditional dynamic-programming counts, creating a 16-stage serial dependency per packet. Conversely, tabulating all valid 8D half-block vectors directly (DirectH8) requires $3{,}192{,}305$ 4-byte entries $= 12.77\text{ MB}$ already at $K_{\max}=22$ (2.50 bpw), thrashing L1/shared cache and becoming prohibitive at $K_{\max}=45$ (3.00 bpw).

Our Quarter-Product (Q4) bijection resolves both bottlenecks by exploiting the weight-$4$ structure of $H_8$. For any 8D half-block $y_h = 2m_h + u_h$ with mask class $t_h \in \{0,1,2\}$:

  • When $t_h = 0$ ($u_h = 0^8$) or $t_h = 1$ ($u_h = 1^8$), coordinates $(0..3)$ and $(4..7)$ form two homogeneous 4-tuples in $(2\mathbb{Z})^4$ or $(2\mathbb{Z}+1)^4$, respectively.
  • When $t_h = 2$ ($\mathrm{wt}(u_h) = 4$), exactly $4$ positions of $u_h$ are $0$ (even coordinates) and $4$ positions are $1$ (odd coordinates). Extracting the 4 even coordinates $a \in (2\mathbb{Z})^4$ in increasing index order and the 4 odd coordinates $b \in (2\mathbb{Z}+1)^4$ in increasing index order maps all $14$ weight-$4$ cosets onto the exact same canonical 4D tables lut4_even and lut4_odd (packed as int8x4 uint32 words: only $2{,}585 + 2{,}816 = 5{,}401$ words $= 21.6\text{ KB}$ at $K_{\max}=22$, and $10{,}321 + 10{,}608 = 20{,}929$ words $= 83.7\text{ KB}$ at $K_{\max}=45$). The specific coset mask $u_h$ is needed only at the final register FMA step to interleave bytes from the even and odd uint32 words.

All mixed-radix divisions at decode time use precomputed 64-bit/32-bit magic multipliers $\operatorname{div\_magic}(x, d, M) = \lfloor (x \cdot M) / 2^{64} \rfloor$ (__umul64hi / __umulhi, zero hardware / or % instructions), accelerated by top-bits prefix brackets (prefix12_lohi on 16D subshells and 5-bit bucket brackets on 8D quarter-splits) that bound binary searches to $\leq 8\text{--}14$ subshell candidates and $\leq 5$ split steps (Algorithm 1).

Algorithm 1: Quarter-Product (Q4) Hierarchical Unranking and Register Inner Product
Input:  Packed (b_code + 2)-bit word W_pkt (b_code in {38, 46}); matrix scale w_scale; radial LUT mu[0..3]; rotated activation x[0..15]
Output: Exact scalar inner product <w_hat, x> in FP32 registers (zero local-memory arrays)

1. Unpack u <- W_pkt >> b_code (2-bit radial selector) and lattice rank r <- W_pkt & ((1 << b_code) - 1).
2. 16D Subshell Lookup: Use top-bits bracket prefix_lohi[r >> shift] + bounded binary search on subshell_base[]
   to find subshell s with B_s <= r < B_{s+1}. Read packed metadata (q_0, q_1, tp, p_0, C_0, C_1) and magic (M_0, M_1).
3. Split 16D Rank into Coset Pair & Two 8D Half-Ranks (via __umul64hi magic division):
   Delta <- r - B_s;  tmp <- div_magic64(Delta, C_1, M_1);  r_1 <- Delta - tmp * C_1
   s_cos <- div_magic64(tmp, C_0, M_0);                      r_0 <- tmp - s_cos * C_0
   Recover RM(1,4) masks (u_0, u_1) and classes (t_0, t_1) from (tp, s_cos) (28 weight-4 pairs when tp=4, else 1).
4. For each half-block h in {0, 1} with (t_h, u_h, q_h, p_0, r_h):
   a. 8D Quarter-Split Lookup: Bracketed binary search on split_cum_base[] inside half8_splits(t_h, q_h, p_0)
      yields split offset c_sp, 4D table bases (base_a, base_b), count n_b, and 32-bit magic M_nb.
   b. Split 8D Rank into 4D Quarter Indices: rem <- r_h - c_sp;  i_a <- div_magic32(rem, n_b, M_nb);  i_b <- rem - i_a * n_b.
   c. Fetch Packed int8x4 Quarters: ev_pk <- lut4[odd_a(t_h)][base_a + i_a];  od_pk <- lut4[odd_b(t_h)][base_b + i_b].
   d. Register FMA: If t_h != 2, accumulate bytes 0..3 of ev_pk with x[8h + 0..3] and od_pk with x[8h + 4..7];
      else (t_h = 2), for k = 0..7 consume low int8 of od_pk if ((u_h >> k) & 1) else low int8 of ev_pk into dot_h.
5. Return (w_scale * mu[u]) * (dot_0 + dot_1).

Three-Level Bijection Verification Scope: Because full 16D state spaces reach $|C_{22}| = 2.49 \times 10^{11}$ (2.50 bpw) and $|C_{45}| = 6.87 \times 10^{13}$ (3.00 bpw), our standalone CPU verifier (verify_bw16_supplement.py / bw16_repro.cuda_ext) separates three distinct verification levels without conflating component and full-code coverage:

  1. Exhaustive 8D Component LUT Check (2.50 bpw): verify_q4_bw250_bijection_cpu exhaustively verifies bijective inversion (half8_lex_to_q4 $\leftrightarrow$ half8_q4_to_lex) across all $3{,}192{,}305$ valid 8D half-block states $(t, q_h, p_h, r_{\text{lex}})$ ($0$ mismatches).
  2. Sampled Full-16D Code Checks: At $2.50\text{ bpw}$, verify_q4_bw250_bijection_cpu compares decoded 16D coordinates across $5$ positions per subshell over all $958$ subshells ($4{,}790$ full-16D evaluations, with duplicates on subshells of span $< 5$; $0$ mismatches). At $3.00\text{ bpw}$, verify_q4_bijection_cpu(50000) checks the $2$ boundary endpoints of all $3{,}983$ subshells ($7{,}966$ boundary evaluations) plus $50{,}000$ deterministic SplitMix64 pseudorandom 16D lattice indices across $[0, |C_{45}|-1]$ ($0$ mismatches).
  3. Realized Model-Weight Transcoding: During matrix quantization, transcode_bw16_{250_,}lex_to_q4_cpu(..., verify_identity=1) verifies exact coordinate equality on realized quantized model weights ($0$ mismatches across all $254$ body and V2 vocabulary matrices).

2.2 Stack-Free Bounded Plotkin/Wagner Encoding, Coupled Surrogate Audit, and Closed-Form BlockAlign

Tiled BlockLDLQ(b=8) Coupled Surrogate Objective and Stack-Free Search: Given clean rotated input Hessian $H_r = Q_K^\top \operatorname{diag}(s_u) H \operatorname{diag}(s_u) Q_K$ ($H = \frac{1}{N} X^\top X$, normalized by $\bar{H}_{\text{diag}}$ with ridge $\sigma_{\text{reg}} = 0.01$ so $H_{r,\text{reg}} = H_r / \bar{H}_{\text{diag}} + 0.01 I_K$), we factor $H_{r,\text{reg}} = L_{\text{chol}} L_{\text{chol}}^\top$ and form the unit-8-block-diagonal feedback matrix $L_{\text{b8}}$ by right-multiplying each 8-column strip of $L_{\text{chol}}$ by the inverse of its $8 \times 8$ lower-triangular diagonal block $D_{\ell}$. Columns are quantized right-to-left in 64-column BlockLDLQ tiles ($4$ sixteen-weight blocks per tile); after encoding tile $t = [c_0 : c_1)$, the accumulated error $E_t = W_{\text{orig},t} - \widehat{W}_t$ updates all preceding columns via a single cuBLAS GEMM $W_{\text{work}, :c_0} \leftarrow W_{\text{work}, :c_0} + E_t L_{\text{b8}}[c_0:c_1, :c_0]$.

Within each 16D block with working target halves $(w_{0,\text{base}}, w_1)$, original upper half $w_{\text{orig},1}$, and intra-block $8 \times 8$ feedback block $L_{1\to 0} = L_{\text{chol}}[8:, :8]\,(L_{\text{chol}}[:8, :8])^{-1}$, both our GPU/CPU bounded encoder (Algorithm 2) and the reference solver encode_bw16_exact_cvp_cpu minimize the diagonal-weighted, feedback-coupled block-surrogate objective over $(y_0, y_1) \in C_{K_{\max}}$ and $u \in \{0,1,2,3\}$ ($s_u = w_{\text{scale}}\mu_u$):

$$ \mathcal{J}_{\text{surr}}(y_0, y_1, u) = \sum_{k=0}^{7} d_{1,k}\bigl(w_{1,k} - s_u y_{1,k}\bigr)^2 + \sum_{k=0}^{7} d_{0,k}\Bigl(\bigl(w_{0,\text{base}} + L_{1\to 0}^\top (w_{\text{orig},1} - s_u y_1)\bigr)_k - s_u y_{0,k}\Bigr)^2, $$

where at $K_{\max} = 22$ (2.50 bpw) coordinates are weighted by mean-normalized Cholesky diagonal energies $d_{16} = \operatorname{diag}(L_{\text{chol}})^2 / \overline{\operatorname{diag}(L_{\text{chol}})^2}$, and at $K_{\max} = 45$ (3.00 bpw) unweighted Euclidean weights $d_{16} = \mathbf{1}_{16}$ are used. Crucially, exact 16D block quadratic loss $(w_{\text{orig}} - s_u y)^\top H_{\text{blk}} (w_{\text{orig}} - s_u y)$ under $H_{\text{blk}} = L_{\text{chol}} L_{\text{chol}}^\top$ equals $\|D_1^\top (w_{\text{orig},1} - s_u y_1)\|_2^2 + \|D_0^\top (w_{\text{orig},0} + L_{1\to 0}^\top(w_{\text{orig},1} - s_u y_1) - s_u y_0)\|_2^2$ with full $8 \times 8$ triangular factors $D_0 = L_{\text{chol}}[:8,:8]$ and $D_1 = L_{\text{chol}}[8:,8:]$, whereas $\mathcal{J}_{\text{surr}}$ replaces $D_{\ell} D_{\ell}^\top$ inside each 8D half-block by diagonal weights $\operatorname{diag}(d_{\ell})$ so that coordinates within each $(2\mathbb{Z})^8 + u_h$ coset decouple. Consequently, exact global minimization of $\mathcal{J}_{\text{surr}}$ is not identical to exact minimization of full-block quadratic error $(w - \hat{w})^\top H_{\text{blk}} (w - \hat{w})$.

Rather than maintaining per-thread candidate stacks in local memory, each warp assigns one lane $\ell \in \{0,\ldots,31\}$ to each $(u_1 \in H_8,\, p_1 \in \{0,1\})$ pair and evaluates $N_{\text{caps}}$ bounded $(2\mathbb{Z})^8 + u_1$ norm caps ($N_{\text{caps}}=6$ at $2.50\text{ bpw}$, $N_{\text{caps}}=3$ at $3.00\text{ bpw}$) entirely in registers (Algorithm 2).

Algorithm 2: 32-Lane Stack-Free Bounded Plotkin/Wagner Lambda_16 Block Encoder
Input:  Target halves (wx0_base, wx1), original wo1, intra-block feedback L_{1->0}, weights d16=(d0, d1),
        scale w_scale, radial multipliers mu[0..3], shell cap K_max (N_max = 4 K_max), cap count N_caps in {3, 6}
Output: Found lattice point y* = (y0*, y1*) in C_{K_max} and 2-bit radial index u* in {0..3} under J_surr

Subroutine BoundedHalf2Z8(wx, d, s, u_mask, p_req, cap):
  1. Round each k in 0..7 to nearest coset integer: m_k <- rint(0.5 * (wx_k / s - u_k)), y_k <- 2 m_k + u_k.
  2. While ||y||_2^2 > cap: greedily step y_{k*} <- y_{k*} - 2 sign(y_{k*}) minimizing Delta_loss_k / Delta_||y||_2^2.
  3. If (sum_k m_k mod 2) != p_req: apply single step y_{k*} <- y_{k*} +/- 2 with ||y'||_2^2 <= cap minimizing Delta_loss_k.
  4. Return (feasible, y, ||y||_2^2, sum_k d_k (wx_k - s y_k)^2).

Main Warp Search (lane l in 0..31 -> h = l >> 1, p1 = l & 1, u1 = H8_MASK[h]):
  For radial index u in 0..3 (step size s = w_scale * mu[u]):
    Estimate lower-half energy n2_0 <- sum_k rint(wx0_base[k] / s)^2;  adapt_cap <- clamp(N_max - n2_0 + 8, 0, N_max);  mid_cap <- N_max/2 + 4.
    Obtain unconstrained upper-half norm n2_unc via BoundedHalf2Z8(wx1, d1, s, u1, p1, N_max).
    Define cap schedule C_caps = (N_max, adapt_cap, mid_cap) if N_caps=3 else (N_max, n2_unc-4, n2_unc-8, adapt_cap, adapt_cap-8, mid_cap).
    For distinct realized upper-half norms n2_1 from BoundedHalf2Z8(wx1, d1, s, u1, p1, cap) over cap in C_caps:
      Feedback upper residual into lower half: wx0 <- wx0_base + L_{1->0}^T (wo1 - s * y1);  rem_cap <- N_max - n2_1.
      For Plotkin extension ext in {0, 1} (u0 = u1 if ext=0 else ~u1):
        (ok0, y0, _, loss0) <- BoundedHalf2Z8(wx0, d0, s, u0, p1, rem_cap).
        If ok0: update thread best (loss0 + loss1, (y0, y1), u); warp-reduce minimum across lanes 0..31 via __shfl_xor_sync.

Layer-00 Encoder Microbenchmark and Synthetic Block-Surrogate Audit: On a Layer-00 7-projection microbenchmark on Qwen3-8B (data/corpus_and_timing_records.json: stack_free_encoder_benchmark), eliminating per-thread stack frames (0 B local stack, 0 register spills) reduces Layer-00 7-projection kernel time from 491.98 ms to 243.79 ms ($2.018\times$ speedup) at $2.50\text{ bpw}$ (×36 projected 36-layer body total: 17.71 -> 8.78 s) and from 147.64 ms to 100.36 ms ($1.471\times$ speedup) at $3.00\text{ bpw}$ (×36 projected 36-layer body total: 5.31 -> 3.61 s), with 0 packed-byte mismatches against our stack-based kernel across all $7$ Layer-00 projections.

To audit how closely BoundedHalf2Z8 + $N_{\text{caps}} \in \{6, 3\}$ minimizes $\mathcal{J}_{\text{surr}}$ on a reproducible host test without GPU dependencies, audit_bw16_bounded_vs_exact_cvp_cpu(n_blocks=16, seed=20261114) (codecs_and_fitting.py) compares Algorithm 2 against encode_bw16_exact_cvp_cpu (exact 8D dynamic programming when $L_{1\to 0}=0$, and exact branch-and-bound over $(y_1, y_0, u) \in C_{K_{\max}} \times \{0..3\}$ under $\mathcal{J}_{\text{surr}}$ when $L_{1\to 0} \neq 0$) on $16$ synthetic i.i.d. $\mathcal{N}(0,1)$ weight blocks and a synthetic Wishart-like positive-definite block Hessian $H_{\text{blk}} = \frac{1}{16} A^\top A + 0.05 I_{16}$ ($A_{ij} \sim \mathcal{N}(0,1)$):

  • At $2.50\text{ bpw}$ ($K_{\max}=22, N_{\text{caps}}=6$, $d_{16} \propto \operatorname{diag}(L_{\text{chol}})^2$): On decoupled isotropic blocks ($L_{1\to 0}=0, d_{16}=\mathbf{1}$), both solvers achieve $14.1825\text{ dB}$ SQNR (0.0000 dB gap, $16/16 = 100\%$ exact matches). On coupled blocks ($L_{1\to 0} \neq 0$), Algorithm 2 matches the global $\mathcal{J}_{\text{surr}}$ optimum on $13/16$ ($81.25\%$) blocks with a +0.0573 dB coupled surrogate-objective gap ($10\log_{10}(\sum \mathcal{J}_{\text{surr}}^{\text{bnd}} / \sum \mathcal{J}_{\text{surr}}^{\text{exact}}) \geq 0$). When those same $16$ assignments are evaluated under the full synthetic quadratic form $(w - \hat{w})^\top H_{\text{blk}} (w - \hat{w})$—which retains off-diagonal intra-8D $D_{\ell} D_{\ell}^\top$ terms omitted by $\mathcal{J}_{\text{surr}}$—bounded search attains $14.9183\text{ dB}$ vs. $14.4761\text{ dB}$ for exact $\mathcal{J}_{\text{surr}}$ minimization (-0.4423 dB difference $\text{SQNR}_{H,\text{exact-}\mathcal{J}} - \text{SQNR}_{H,\text{bnd}}$).
  • At $3.00\text{ bpw}$ ($K_{\max}=45, N_{\text{caps}}=3$, $d_{16}=\mathbf{1}$): Bounded search matches exact $\mathcal{J}_{\text{surr}}$ minimization on $16/16$ ($100\%$) blocks in both regimes (0.0000 dB isotropic gap at $17.1225\text{ dB}$; 0.0000 dB coupled surrogate gap and 0.0000 dB full-$H_{\text{blk}}$ SQNR difference at $18.2551\text{ dB}$). This CPU audit establishes near-exactness for $\mathcal{J}_{\text{surr}}$ on this small $16$-block synthetic Gaussian/Wishart sample only; it does not establish universal CVP optimality under full $H_{\text{blk}}$ or across calibrated real-model weight distributions.

Closed-Form $4 \times 4$ Radial LUT Refit and Zero-Overhead BlockAlign:

  1. $4 \times 4$ Radial Gram Refit: Holding discrete assignments $(y, u)$ fixed after BlockLDLQ, writing $\widehat{W}_r(\alpha) = \sum_{q=0}^3 \alpha_q A_q$ ($\alpha_q = w_{\text{scale}} \mu_q$, where $A_q$ masks blocks with $u=q$) yields a $4 \times 4$ positive-definite linear system minimizing $\operatorname{tr}\!\bigl((W_r - \widehat{W}_r(\alpha)) H_r (W_r - \widehat{W}_r(\alpha))^\top\bigr)$:
$$ \alpha^\star = (G_H + \lambda_r I_4)^{-1}(b_H + \lambda_r \alpha_0), \qquad (G_H)_{pq} = \operatorname{tr}(A_p H_r A_q^\top), \quad (b_H)_p = \operatorname{tr}(W_r H_r A_p^\top), \quad \lambda_r = 10^{-8}\operatorname{tr}(G_H). $$
  1. Closed-Form BlockAlign Calibration (0 SGD Steps, $+0\text{ B}$ Metadata): After quantizing all matrices, a single forward pass over the first $8$ calibration batches (cal_128_ids[:32], $65{,}536$ tokens; or balanced BalCal8 in §6 ablations) propagates both clean teacher hidden states $h_{\text{clean}}^{(\ell)}$ and quantized student hidden states $h_{\text{quant}}^{(\ell)}$, fitting one scalar multiplier per matrix folded directly into FP32 $w_{\text{scale}}$ via the clipped Frobenius inner-product projection $\alpha^\star(U_{\text{tgt}}, U_q; [a,b]) = \operatorname{clip}\!\bigl(\langle U_{\text{tgt}}, U_q \rangle_F / \|U_q\|_F^2,\, a, b\bigr)$:
    • q_proj, k_proj ($\alpha = 1.0$): Kept at unity because Qwen3-8B applies per-head q_norm and k_norm RMSNorm immediately after projection, making attention logits invariant to positive global scalar rescaling of $W_q$ or $W_k$.
    • v_proj, gate_proj, and up_proj ($\alpha \in [0.92, 1.08]$): Fit locally on quantized layer-norm inputs $X_{\text{quant}}$ against clean sub-layer outputs ($U_{\text{loc}} = X_{\text{quant}} W_{\text{clean}}^\top$ vs. $U_q = X_{\text{quant}} \widehat{W}^\top$ for v_proj and gate_proj; post-SiLU SwiGLU product $\operatorname{SiLU}(X_{\text{quant}} W_{\text{gate}}^\top) \odot (X_{\text{quant}} W_{\text{up}}^\top)$ vs. $\operatorname{SiLU}(\alpha_{\text{gate}} X_{\text{quant}} \widehat{W}_{\text{gate}}^\top) \odot (X_{\text{quant}} \widehat{W}_{\text{up}}^\top)$ for up_proj).
    • Residual-writing o_proj and down_proj ($\alpha \in [0.90, 1.10]$): Combine the clipped local sub-layer ratio $\alpha_{\text{loc}} = \alpha^\star(U_{\text{loc}}, U_q; [0.90, 1.10])$ (clean sub-layer evaluated on quantized input) and the clipped clean-trajectory ratio $\alpha_{\text{cum}} = \alpha^\star(U_{\text{cum}}, U_q; [0.90, 1.10])$ ($U_{\text{cum}}$ captured from the clean sub-layer along the clean teacher trajectory) via the convex heuristic blend:
$$ \alpha = 0.70\,\alpha^\star(U_{\text{loc}}, U_q; [0.90, 1.10]) + 0.30\,\alpha^\star(U_{\text{cum}}, U_q; [0.90, 1.10]) \in [0.90, 1.10]. $$

Because each ratio is clipped to $[0.90, 1.10]$ prior to mixing, $\alpha$ is a separately-clipped convex blend and is not in general the exact minimizer of the combined quadratic $0.70\|U_{\text{loc}} - \alpha U_q\|_F^2 + 0.30\|U_{\text{cum}} - \alpha U_q\|_F^2$ (for instance, unclipped ratios $0.80$ and $1.20$ yield combined quadratic minimizer $0.92$, whereas clipping each to $[0.90, 1.10]$ before blending gives $0.96$, though opposite-sign clipping shifts can cancel at specific ratios).

2.3 Sensitivity-Guided Swap7_Plus24K Bit Allocation and Four-Way Package Accounting

Exact Weight Census and Unit Grid: Each of the $36$ transformer blocks of Qwen3-8B contains $7$ bias-free linear matrices quantized in integer multiples of $U_0 = 2^{22} = 4{,}194{,}304$ weights ($1.31072\text{ MB/unit}$ at $2.50\text{ bpw}$; $1.572864\text{ MB/unit}$ at $3.00\text{ bpw}$; $\Delta = +262{,}144\text{ B/unit}$): q_proj $[4096, 4096]$ ($4$ units), k_proj $[1024, 4096]$ ($1$ unit), v_proj $[1024, 4096]$ ($1$ unit), o_proj $[4096, 4096]$ ($4$ units), gate_proj $[12288, 4096]$ ($12$ units), up_proj $[12288, 4096]$ ($12$ units), and down_proj $[4096, 12288]$ ($12$ units), totaling $46$ units $= 192{,}937{,}984$ weights/layer and $1{,}656$ units $= 6{,}945{,}767{,}424$ body weights ($84.80\%$) across $252$ matrices. Untied embed_tokens and lm_head ($[151936, 4096]$ each) add $1{,}244{,}659{,}712$ vocabulary weights ($15.20\%$), and $145$ unquantized BF16 RMSNorm vectors add $308{,}224$ parameters ($616{,}448\text{ B}$), giving $8{,}190{,}735{,}360$ total parameters ($16{,}381{,}470{,}720\text{ B} = 15.2564\text{ GiB}$ dense BF16).

Swap7_Plus24K Allocation Rule and Explicit $36 \times 7$ Rate Maps: Ranking all $252$ single-matrix $2.50 \to 3.00\text{ bpw}$ promotions purely by additive calibration $\Delta\mathcal{L}_{\text{KL}} / \text{unit}$ (EndKL) under-allocates contiguous early/mid up_proj blocks (whose SwiGLU errors interact multiplicatively) while missing high-leverage 1-unit k_proj promotions when starting from role-uniform anchors. Swap7_Plus24K constructs the two body budgets ($612$ vs. $1{,}044$ units at $3.00\text{ bpw}$) via two deterministic steps:

  • Step 1 (Swap7 H-SQNR Outlier Exchange): At 2.685 bpw ($612$ units at $3.00\text{ bpw}$), start from the VO+Down anchor (all $36$ v_proj + $36$ o_proj + $36$ down_proj $= 612$ units), demote the $7$ easy down_proj outliers whose $2.50\text{-bpw}$ H-SQNR exceeds $24.0\text{ dB}$ (L{1,2,3,6,16,34,35}, $84$ units), and promote the $7$ lowest-H-SQNR contiguous mid-early up_proj layers (L{6..12}, $84$ units). At 2.815 bpw ($1{,}044$ units at $3.00\text{ bpw}$), start from VO+Up+Down ($1{,}044$ units), demote the $7$ MLP outliers with $2.50\text{-bpw}$ H-SQNR $> 26.5\text{ dB}$ (up_proj L{1,2} + down_proj L{1,2,6,16,35}, $84$ units), and promote the $7$ lowest-H-SQNR gate_proj layers (L{8,18,19,20,31,32,33}, $84$ units).
  • Step 2 (Plus24K Marginal Teacher-KL/Unit Promotion): Measure single-matrix $2.50 \to 3.00\text{ bpw}$ end-to-end Teacher-KL reduction per unit on $4{,}096$ calibration tokens (cal_128_ids[:8, :512]), demote the $2$ lowest-sensitivity remaining $3.00\text{-bpw}$ 12-unit MLP matrices ($24$ units: down_proj L{10,11} at 2.685; up_proj L{23} + down_proj L{10} at 2.815), and promote the top-$24$ 1-unit k_proj matrices ($24$ units).

This yields the exact $3.00\text{-bpw}$ layer index sets (all remaining body matrices use $2.50\text{ bpw}$; full JSON maps in data/allocation_and_scales.json):

  • Shared across 2.685 & 2.815: v_proj ($36$): L0..35; o_proj ($36$): L0..35; q_proj ($0$): $\varnothing$; k_proj ($24$): L{1, 7..12, 14..16, 18..22, 24, 25, 27, 29, 31..35} (leaving L{0, 2..6, 13, 17, 23, 26, 28, 30} at $2.50\text{ bpw}$).
  • 2.685 bpw MLP ($130$ total $3.00\text{-bpw}$ matrices, $612$ units): gate_proj ($0$): $\varnothing$; up_proj ($7$): L{6..12}; down_proj ($27$): L{0, 4, 5, 7..9, 12..15, 17..33} (leaving L{1..3, 6, 10, 11, 16, 34, 35} at $2.50\text{ bpw}$). Body payload $= 2{,}330{,}984{,}448\text{ B}$ + $5{,}040\text{ B}$ metadata ($252 \times 20\text{ B}$: FP32 $w_{\text{scale}}$ + $4 \times \text{FP32}$ mult_lut) = $2{,}330{,}989{,}488\text{ B}$ ($2.684788\text{ bpw}$).
  • 2.815 bpw MLP ($166$ total $3.00\text{-bpw}$ matrices, $1{,}044$ units, $+113{,}246{,}208\text{ B}$): gate_proj ($7$): L{8, 18..20, 31..33}; up_proj ($33$): all except L{1, 2, 23}; down_proj ($30$): all except L{1, 2, 6, 10, 16, 35}. Body payload $= 2{,}444{,}230{,}656\text{ B}$ + $5{,}040\text{ B}$ metadata = $2{,}444{,}235{,}696\text{ B}$ ($2.815223\text{ bpw}$).
  • Matched Baseline Body Maps: Each QTIP-HYB matrix stores $2{,}052\text{ B}$ metadata ($2{,}048\text{ B}$ [512, 2] FP16 LUT + $4\text{ B}$ $w_{\text{scale}}$, $+512{,}064\text{ B}$ across $252$ matrices vs. BW16); demoting the two lowest-salience promoted k_proj layers L{11, 18} from $3.00$ (KV=6) to $2.50\text{ bpw}$ (KV=5, $-524{,}288\text{ B}$) places QTIP-HYB at $-12{,}224\text{ B}$ ($-0.0004\%$) strictly below BW16. QuIP# uses $\{2.00\text{ bpw E8P}, 3.00\text{ bpw E8-RVQ}\}$ sensitivity allocation (CalAlloc: $95$ vs. $77$ matrices at $2.00\text{ bpw}$ across q_proj, k_proj, gate_proj) with identical raw body payload to BW16 and $1{,}008\text{ B}$ w_scale metadata ($-4{,}032\text{ B}$ vs. BW16).

Non-Body Regimes (U1 vs. V2) and Four-Way Package Census (Table 2): Legacy U1 (NVFP-E8 $3.50\text{ bpw}$ on embed_tokens and SmoothQuant-preconditioned lm_head, $\alpha=0.50$) totals $545{,}163{,}272\text{ B}$ ($544{,}538{,}624\text{ B}$ payload + $8{,}200\text{ B}$ scales + $616{,}448\text{ B}$ RMSNorms). Unified V2 (V2_BW16_250_300) quantizes embed_tokens in $2{,}374 \times 64$ strided-Q64 row-group blocks ($Q_{64}$ on rows paired with $Q_{4096}$ across hidden channels) at $2.50\text{ bpw}$ ($194{,}478{,}080\text{ B}$ + $20\text{ B}$, seed 20261501) and lm_head at $3.00\text{ bpw}$ ($233{,}373{,}696\text{ B}$ + $20\text{ B}$, seed 20261502), plus $616{,}448\text{ B}$ RMSNorms = $428{,}468{,}264\text{ B}$ deterministic-seed / $428{,}468{,}272\text{ B}$ ($+8\text{ B}$ explicit seeds), saving $116{,}695{,}008\text{ B}$ / $116{,}695{,}000\text{ B}$ ($116.70\text{ MB}$, $-21.41\%$) over U1. Table 2 separates logical tensor bytes, serialized archive construction, shared runtime binaries/LUTs, and resident RTX 5090 VRAM (+196,608 B under V2 from padding lm_head by $+128$ zero rows to $152{,}064 = 2{,}376 \times 64$, plus $+2.25\text{ MiB}$ permanent decode scratch or $+96.0\text{ MiB}$ transient Mode-2 prefill workspace).

| Complete Package (Table 2: 36-Layer Qwen3-8B) | (1) Complete-Package Logical Tensor Bytes (Body + Meta + Non-Body) | (2) Cleanroom / Donor Construction Path | (3) Shared Runtime .so & Device LUT Footprint (Excluded from Logical Bytes) | (4) Measured Incremental RTX 5090 VRAM (Mode-1 Decode / +96 MiB Mode-2 Prefill) | | :--- | :---: | :--- | :--- | :---: | | BW16 2.685 + U1 | $2{,}876{,}152{,}760\text{ B}$ ($2.6786\text{ GiB}$) | bw16_repro CLI (--body-codec bw16 --vocab-regime u1 --budget 2685) or preserved donor .pt reload (App. B) | bw16_q4 + u1 .so + V8 LUTs ($178{,}236\text{ B}$) + E8 catalog ($40{,}246\text{ B}$) | $2{,}879{,}015{,}936\text{ B}$ ($2.6813\text{ GiB}$) /<br>$2{,}979{,}679{,}232\text{ B}$ ($2.7750\text{ GiB}$) | | BW16 2.815 + U1 | $2{,}989{,}398{,}968\text{ B}$ ($2.7841\text{ GiB}$) | Same (--budget 2815, $+113{,}246{,}208\text{ B}$ $3.00\text{-bpw}$ body upgrade) | Same as 2.685 + U1 | $2{,}992{,}262{,}144\text{ B}$ ($2.7868\text{ GiB}$) /<br>$3{,}092{,}925{,}440\text{ B}$ ($2.8805\text{ GiB}$) | | BW16 2.685 + V2 | $2{,}759{,}457{,}752\text{ B}$ (det)<br>$2{,}759{,}457{,}760\text{ B}$ (exp, $2.5699\text{ GiB}$) | bw16_repro CLI (--vocab-regime v2 --budget 2685, $-116.70\text{ MB}$ vs. 2.685 + U1) | bw16_q4 .so + V8 LUTs ($178{,}236\text{ B}$: $42.7 + 135.6\text{ KB}$; $0\text{ B}$ U1 LUTs) | $2{,}762{,}946{,}560\text{ B}$ ($2.5732\text{ GiB}$) /<br>$2{,}863{,}609{,}856\text{ B}$ ($2.6669\text{ GiB}$) | | BW16 2.815 + V2 | $2{,}872{,}703{,}960\text{ B}$ (det)<br>$2{,}872{,}703{,}968\text{ B}$ (exp, $2.6754\text{ GiB}$) | bw16_repro CLI (--vocab-regime v2 --budget 2815, $-3.45\text{ MB}$ vs. 2.685 + U1) | Same as 2.685 + V2 | $2{,}876{,}192{,}768\text{ B}$ ($2.6787\text{ GiB}$) /<br>$2{,}976{,}856{,}064\text{ B}$ ($2.7724\text{ GiB}$) | | QTIP-HYB 2.685 / 2.815 + V2 | $2{,}759{,}445{,}536\text{ B}$ / $2{,}872{,}691{,}744\text{ B}$ (exp, $-12{,}224\text{ B}$ vs. BW16) | bw16_repro CLI (--body-codec qtip_hyb --vocab-regime v2) | bw16_q4 + qtip_hyb .so + V8 vocab LUTs ($178{,}236\text{ B}$) | $2.5731\text{ GiB}$ & $2.6785\text{ GiB}$ /<br>$+96.0\text{ MiB}$ prefill |

2.4 Dual-Mode sm_120 Compressed Execution Stack

All $254$ linear and vocabulary matrices execute from compressed VRAM without materializing $15.26\text{ GiB}$ of dense weights (bw16_repro/runtime_modules.py):

  1. DualModeV6BW16Linear (V8_Hybrid):
    • Mode 1 ($M_{\text{tok}} \leq 16$, Autoregressive Decode): Fuses input sign-flip, full-dimensional Kronecker RHT ($H_{4096}$ shared-memory butterfly at $K=4{,}096$; $12 \times H_{1024}$ followed by Paley-$12$ $H_{12}^\top$ at $K=12{,}288$), and warp-swizzled FP32 staging (p14_cuda_rht_in_bf16_to_f32_swiz_kernel), executes zero-workspace Q4 register-decoded GEMV (B=1 dynamic-shared-memory reduction; 2 <= B <= 16 2-row warp-unrolled V8 kernel amortizing each Q4 packet unrank across all $B$ batch columns in registers via an $80\text{-byte}$-strided bank-conflict-free shared activation tile), and applies fused FP32 -> BF16 full-dimensional output RHT ($H_{1024}$, $H_{4096}$, or $H_{12} \otimes H_{1024}$; $< 10^{-4}$ relative error vs. exact FP32 rotated reference).
    • Mode 2 ($M_{\text{tok}} > 16$, Prefill & $B=32$ Generation): Applies full-dimensional BF16 input RHT, unpacks rotated BF16 weights (0.0 max abs diff vs. reference rotated BF16) into one shared transient buffer w_rot_shared_flat ($12{,}288 \times 4{,}096 \times 2\text{ B} = 96.0\text{ MiB}$), runs cuBLAS BF16 Tensor-Core GEMM, and applies full-dimensional BF16 output RHT ($8.75\times\text{--}9.28\times$ $L=512$ prefill speedup over Mode 1; empirical Mode-1-vs-Mode-2 relative Frobenius difference $0.0029\text{--}0.0041$ from BF16 intermediate rounding). During Mode-1 decode, w_rot_shared_flat is detached ($+2.25\text{ MiB}$ permanent scratch).
  2. CompressedBW16VocabLMHead ($3.00\text{ bpw}$) & FastCompressedBW16VocabEmbedding ($2.50\text{ bpw}$):
    • CompressedBW16VocabLMHead dispatches Mode 1 at $M \leq 16$ ($558.45\ \mu\text{s}$ at $M=1$, $2{,}117.94\ \mu\text{s}$ at $M=16$) and $192$-block ($96.0\text{ MiB}$) tiled Mode 2 at $M > 16$ ($6.66\text{ ms}$ vs. $182.05\text{ ms}$ at $M=512$, $27.32\times$), while improving clean-$H_{\text{head}}$ H-SQNR over $3.50\text{-bpw}$ U1 by $+1.69\text{ dB}$ ($26.12$ vs. $24.43\text{ dB}$) at $-38.90\text{ MB}$ fewer logical bytes.
    • FastCompressedBW16VocabEmbedding eliminates torch.unique host-device syncs and intermediate index_select copies via direct torch.bmm over w_rot_shared_flat, matching the reference compressed embedding bit-for-bit ($\text{max\_abs\_diff} = 0.0$), accelerating $2.50\text{-bpw}$ lookup by $1.833\times$ at $M=1$ ($173.41 \to 94.62\ \mu\text{s}$) and $1.730\times$ at $M=2048$ ($2{,}648.36 \to 1{,}530.94\ \mu\text{s}$), and cutting $M=2048$ transient memory by $56.7\%$ ($50.90 \to 22.05\text{ MB}$). Because each touched $64$-row Q64 block ($81{,}920\text{ B}$) is unpacked before row selection, $M=1$ lookup remains slower than U1 ($94.62$ vs. $23.66\ \mu\text{s}$), leaving $B=1$ whole-model decode ($23.04\text{--}23.77\text{ ms}$) slightly behind U1 ($22.40\text{ ms}$); a fused single-row Q64 gather kernel is a natural optimization for $B \leq 4$.

3. Cleanroom Reconstruction Release, Evaluation Protocol, and Evidence Boundaries

3.1 Public Datasets, Single-Temperature Calibration, and Evaluation Benchmarks

All experiments evaluate public Qwen/Qwen3-8B (BF16, HuggingFace commit b968826d9c46dd6066d109eabc6255188de91218) on one NVIDIA GeForce RTX 5090 ($32\text{ GB}$, CUDA 12.8, sm_120). bw16_repro/datasets_and_panels.py reconstructs and SHA-256 verifies every token tensor and prompt panel directly from public HuggingFace datasets (data/calibration_and_panel_manifests.json):

  1. Calibration Splits (cal_128_ids & BalCal8): cal_128_ids (SHA256 9736b152...) comprises $128 \times 2{,}048 = 262{,}144$ tokens ($32$ 4-window batches): $64$ WikiText-2 (Salesforce/wikitext wikitext-2-raw-v1 train) windows (0..15) + $64$ C4 (allenai/c4 en/c4-train.00000-of-01024) windows (16..31) used for clean layer input Hessians $H$. Stage-I BlockFit baselines (QTIP-HYB, QuIP#, BW16-BlockFit) use batches 0..7 ($6$ train / $2$ val_16k); domain-balanced comparators (BalCal8: D4, BW16-BalBlockFit, TrajRadAlign, E2E-HybridCal, and V2 vocabulary screening) use batches [0,1,2,16,17,18] ($49{,}152$ tokens, $50\%$ WT2 / $50\%$ C4) and validate on [3,19] ($16{,}384$ tokens).
  2. Disjoint Single-Temperature Calibration (cal_temp_ids) [15]: Following discrete_descent [15]—who showed on post-trained Qwen3 models that fixed $T=1$ perplexity flatters quantization noise and misranks PTQ methods—cal_temp_ids (SHA256 07835b3c...) holds $4 \times 2{,}048 = 8{,}192$ disjoint C4 train tokens ($0$ overlap with WT2-32 or C4-32). Scalar temperature $T^* \in [0.80, 1.60]$ ($\Delta T = 0.02$) minimizes cal_temp_ids NLL ($T^* = 1.22$ on Dense_BF16; $T^* = 1.20$ across all evaluated 2.685/2.815 packages) and is applied without re-fitting to WT2-32 and C4-32.
  3. Held-Out Corpora (WT2-32 & C4-32): WT2-32 (SHA256 a5ad4cec..., $32 \times 2{,}048$ wikitext-2-raw-v1 test windows, $65{,}504$ predicted tokens) and C4-32 (SHA256 9e3a938e..., $32 \times 2{,}048$ c4-validation.00000-of-00008 windows, $65{,}504$ predicted tokens) report Raw-NLL ($T=1.0$), Fit-NLL ($T=T^*$), and full-$151{,}936$-vocab Teacher-KL $D_{\mathrm{KL}}\!\bigl(p_{\text{BF16}}(1) \,\|\, p_q(1)\bigr)$ in FP32/FP64. Dense_BF16 ($T^*=1.22$): WT2-32 Raw/Fit-NLL $= 2.281186 / 2.175793$ ($\text{PPL } 9.7883 / 8.8092$); C4-32 Raw/Fit-NLL $= 2.760835 / 2.670897$ ($\text{PPL } 15.8130 / 14.4529$).
  4. Autoregressive Generation Panels (Table 3):

| Panel ID & Role | Public Source & Exposure Status | Prompt, Greedy Decoding, & Execution Mode | Cap, Stopping Rule, Scorer (scorers_and_stats.py), & Margin | | :--- | :--- | :--- | :--- | | GSM8K-256 ($256$ items; Primary Codec Check) | openai/gsm8k test indices 128..383 (SHA256 db972eec..., $0$ overlap with 0..127 dev slice; scanned in community baselines) | 8-shot CoT, greedy argmax, left-padded, $8 \times (B = 32)$ batches (unpacked BF16 and live Mode 2 $M = 32$) | $\text{cap} = 256$; stop from step $16$ every $8$ steps on Question: / EOS or #### <num>\n; numeric match after #### (with last-number fallback when #### is absent, $\text{tol}=10^{-4}$). Dense_BF16 $= 228/256\ (89.06\%)$. Retention margin: $-5.0\text{ pp}$. | | N512 ($512$ items; Primary Factorial Retention) | $512$ openai/gsm8k train items (programme-exposed in earlier 4B/8B screens; not an unexposed split) | 5-shot GSM8K-train, greedy argmax, KV16, longest-first $32 \times (B = 16)$ batches (Mode-2 prefill + live Mode-1 $M=16$ decode) | $\text{cap} = 512$; stop on ["Question:", "</s>", "<|im_end|>"]; normalized exact match on first #### <num> match (returns "[invalid]" if #### is absent; no last-number fallback; no_stop caps tracked). Dense_BF16 $= 475/512\ (92.77\%, 3\text{ caps})$. Non-inferiority margin: $-3.5\text{ pp}$. | | M256* ($256$ items; Exploratory Secondary) | $256$ integer-answer EleutherAI/hendrycks_math train Level 4–5 items (programme-exposed; exploratory secondary readout only) | 4-shot Minerva, greedy argmax, KV16, longest-first $16 \times (B = 16)$ batches (Mode-2 prefill + live Mode-1 $M=16$ decode) | $\text{cap} = 1024$; stop on ["Problem:"]; Minerva \boxed{} / integer exact match (no_stop caps $37.1\%\text{--}52.0\%$). Dense_BF16 $= 136/256\ (53.12\%, 108\text{ caps})$. Exploratory margin: $-5.0\text{ pp}$. |

3.2 Statistical Resampling and Serving Benchmark Protocol

  • Paired Bootstrap & Exact McNemar Tests ($10{,}000$ Resamples): Corpus contrasts resample $2{,}048$-token windows ($N = 32$) $10{,}000$ times (Fit-NLL retention requires $95\%$ CI upper bound $< +0.015\text{ nats}$). Generation contrasts resample items ($N = 256$ or $512$) $10{,}000$ times with exact two-sided binomial McNemar $p$-values on discordant pairs $(n_{10}, n_{01})$. Contrasts are statistically resolved only when the $95\%$ CI excludes $0$ (and McNemar $p < 0.05$ for binary panels). Reproducible per-family np.random.default_rng seeds (audit_retained_outcomes_and_corpora in scorers_and_stats.py): 20261509 (common-V2 corpus/GSM8K-256 via make_corpus_gsm_boot_indices, plus live N512 and its 4-arm interaction via paired_binary_contrast_pp / four_arm_interaction_bootstrap_pp), 20261510 (live M256* and its 4-arm interaction $\Delta_{\text{int}}$ via four_arm_interaction_bootstrap_pp), 20261436 (legacy U1 corpus/GSM8K-256), 20261437 / 20261438 (unpacked U1 N512 / M256*), and 20261439 / 20261501 / 20261509 (§6 post-hoc recipes).
  • Serving Latency Harness (benchmark_decode_serving_gpu in build_and_eval_cli.py): Whole-model step latencies run over distinct C4 contexts (c4_val_ids[:B, :512]): Mode-1 decode ($B \in \{1,2,4,8,16\}$, $L=512$, w_rot_shared_flat = None, $3$ warmup + $16$ timed decode steps bracketed by torch.cuda.synchronize() and torch.cuda.Event(enable_timing=True)), and Mode-2 prefill (w_rot_shared_flat $96.0\text{ MiB}$ attached, $2$ warmup + $6$ timed passes). Direct two-rate build timers vs. stage-assembled single-package prorations are reconciled in Appendix A.

3.3 Self-Contained Supplement (bw16_repro) and Explicit Evidence Boundaries

Unlike partial script snapshots that depend on private workspace modules or pre-quantized .pt donors, the MIT-licensed supplement (supplement.zip, $< 2.0\text{ MB}$ expanded across $21$ files, zero private imports) ships a complete standalone package (Table 4):

# 1. Standalone CPU verification (0 GPU usage; compiles both .cu libraries via nvcc when available)
CUDA_VISIBLE_DEVICES="" python3 verify_bw16_supplement.py

# 2. Cleanroom end-to-end GPU reconstruction, Mode-1 reference check, & evaluation from pinned Qwen/Qwen3-8B
python3 -m bw16_repro.build_and_eval_cli --model-id Qwen/Qwen3-8B \
  --body-codec bw16 --vocab-regime v2 --budget 2815 --eval-corpora --bench-serving
python3 -m bw16_repro.build_and_eval_cli --model-id Qwen/Qwen3-8B \
  --body-codec qtip_hyb --vocab-regime v2 --budget 2815 --eval-corpora --bench-serving

| Supplement Component (Table 4) | Included Modules & Data Files | Verified Reproducibility Scope | | :--- | :--- | :--- | | verify_bw16_supplement.py & bw16_repro/cuda/ | bw16_q4_and_rht_kernels.cu, baselines_and_u1_kernels.cu, cuda_ext.py | Exact $RM(1,4)$ DP shell counts (Table 1); nvcc host compilation; 3,192,305 exhaustive 8D half8 LUT + 4,790 (2.50) and 7,966 + 50,000 (3.00) 16D bijection checks; CPU QTIP-HYB Viterbi, serialization, & 39-check tolerance-pipeline verification. | | bw16_repro/ Python Pipeline | datasets_and_panels.py, transforms_and_alloc.py, codecs_and_fitting.py, runtime_modules.py, scorers_and_stats.py, build_and_eval_cli.py | Pinned Qwen3-8B & dataset SHA-256 builder; synthetic bounded-vs-exact block-surrogate CVP audit (§2.2); GPU BW16 / Stage-I QTIP-HYB (PCG + Adam) / Stage-I QuIP# / V2 / U1 encoders & BlockAlign; all-254 Dual-Mode runtime + independent Mode-1/Mode-2-vs-PyTorch F.linear check; incremental JSON persistence, tolerance evaluator, & bootstrap verifier. | | data/ Retained & Cleanroom Records | allocation_and_scales.json, calibration_and_panel_manifests.json, corpus_and_timing_records.json, generation_outcomes_and_excerpts.json, cleanroom_gpu_verification_v4.json | Complete $36 \times 7$ rate maps & BlockAlign/D4 scales; U1 & V2 32-window WT2-32/C4-32 arrays; all 15,360 scored item rows across 49 evaluations (29 GSM8K-256, 11 N512, 9 M256*) + primary live excerpt pools; Layer-00 microbenchmarks, per-launch serving records, & cleanroom BW16 2.815 + V2 GPU verification records (39/39 tolerance checks). |

Three Explicit Evidence Boundaries:

  1. Truncated Saved Generation Excerpts & B=1 CLI Demonstration Scope: Historical evaluation logs persisted extracted answers (pred / pred_num), exact-match flags, stop/cap indicators, token counts, and 360-char (GSM8K-256) or 400-char (N512, M256*) text prefixes—not full $512$/$1{,}024$-token strings or token IDs (and Live_BW16_BlockAlign_2685_U1 plus two contextual Qwen3-4B GPTQ4 baselines on N512, bundled in data/generation_outcomes_and_excerpts.json, recorded exact/no_stop/gen_tokens from an earlier identical-stack pass without pred strings). Thus (pred, target) -> exact equivalence is $100\%$ recomputable across all $13{,}824$ rows with saved pred, and raw-text re-extraction verifies at $100\%$ on complete traces terminating within $360$/$400$ chars ($78.1\%\text{--}80.5\%$ of live GSM8K-256, $86.1\%\text{--}87.5\%$ of live N512, $28.1\%\text{--}28.5\%$ of long/capped M256*), whereas raw text beyond char $360$/$400$ was not retained. The --eval-panels flag in build_and_eval_cli.py is a single-stream B=1 greedy demonstration and does not replay historical batched B=16/32 greedy trajectories.
  2. Cross-Launch vs. Within-Launch Serving Repeats & Single-System Cleanroom Launch Scope: Historical BW16-BlockAlign + U1 was timed across $4$ independent GPU launches, whereas the matched FastEmb common-V2 multi-codec comparison (BW16 + V2, D4 + V2, QTIP-HYB + V2 in §4.2) remains scoped to $1$ historical GPU launch (launch_4_common_v2: $2$ separate within-launch passes Pass A and Pass B on BW16 + V2; $1$ pass on D4 + V2 and QTIP-HYB + V2) persisting $16$-step CUDA-event means. The cleanroom v4 verification continuation (Run 2, Appendix C) adds one independent single-system GPU launch for BW16-BlockAlign 2.815 + V2 alone (without re-timing D4 + V2 or QTIP-HYB + V2), logging per-step 1..16 CUDA-event latencies, SD, IQR, and resident VRAM with and without the shared 96.0 MiB prefill buffer.
  3. CPU Verifier Scope & Cleanroom GPU Reconstruction Summary (Full Account in Appendix C): Default CPU execution of verify_bw16_supplement.py exercises DP shell enumeration, nvcc host compilation, Q4 bijection/Viterbi checks, incremental JSON serialization & 39-check tolerance-pipeline verification, pure-Python codebook builders, four-way byte accounting, the $16$-block synthetic surrogate-CVP audit, and retained-outcome bootstrap statistics (--verify-datasets-cpu additionally checks pinned checkpoint metadata and cached WT2/C4 token-split SHA-256 digests on CPU); executing CUDA kernels and full 36-layer reconstruction requires a GPU. On RTX 5090 (sm_120, zero private .pt donors, data/cleanroom_gpu_verification_v4.json), cleanroom GPU reconstruction of BW16-BlockAlign 2.815 + V2 ran in two documented invocations (Appendix C): after an initial invocation (Run 1) failed a mis-copied v3 target file and exited via a KeyError before Mode-1 tolerance assertion or live All 254 evaluation, a reconciled continuation (Run 2, unchanged recipe and tolerances) passed all 39/39 tolerance-based checks ($0\text{ B}$ byte delta, $0$ Lex -> Q4 mismatches across $254$ matrices, live WT2-32/C4-32 Fit-NLL within $+0.000148 / -0.000261\text{ nats}$—establishing tolerance-based numerical agreement rather than bitwise weight identity—and $B=16$ decode $41.920\text{ ms/step}$); primary multi-system tables in §§4–6 report the retained evaluation archives.

4. Layer-00 Kernel Microbenchmarks, Baseline Validation, and Multi-Codec Comparison

4.1 Layer-00 7-Projection GEMV Microbenchmark (Q4 Isolation) and QTIP-HYB Baseline Fidelity

Before comparing measured 36-layer complete packages (§4.2), we isolate the Q4 representation on a Layer-00 7-projection microbenchmark on identical quantized weights and validate that our QTIP-HYB implementation is a faithful Stage-I baseline (data/corpus_and_timing_records.json):

  1. Layer-00 7-Projection GEMV Microbenchmark (Q4 vs. DirectH8 & Procedural Lex, Figure 1a): Holding the quantized $\Lambda_{16}$ weights and full-dimensional Kronecker RHT fixed across the $7$ linear projections of Qwen3-8B Layer 00 ($\text{max\_abs\_diff} = 9.54 \times 10^{-7}$), replacing procedural 16-coordinate DP unranking (Lex) with Q4 quarter-product indexing (65.7 KB base LUT) reduces measured Layer-00 7-projection GEMV sum at $B \in \{1, 4, 8\}$ from 876.11 / 2,293.67 / 4,272.54 µs to 356.93 / 725.60 / 1,252.61 µs (2.455x / 3.161x / 3.411x speedup; ×36 projected 252-projection body estimate: 31.54 / 82.57 / 153.81 ms -> 12.85 / 26.12 / 45.09 ms) at $2.50\text{ bpw}$ ($1{,}081\text{ GB/s}$ effective logical BF16 bandwidth on Layer 00 at $B=1$), and from 1,355.39 / 2,042.85 / 3,014.63 µs to 734.27 / 1,020.88 / 1,546.60 µs (1.846x / 2.001x / 1.949x speedup; ×36 projected body estimate: 48.79 / 73.54 / 108.53 ms -> 26.43 / 36.75 / 55.68 ms) at $3.00\text{ bpw}$ (263.8 KB base LUT). Against flat $12.77\text{ MB}$ 8D half-block lookup (DirectH8, 456.70 / 738.55 / 1,231.35 µs on Layer 00; ×36 projected: 16.44 / 26.59 / 44.33 ms at $2.50\text{ bpw}$), Q4 is 1.280x faster at $B=1$ while using $194\times\text{--}299\times$ less table memory (65.7 KB base / 42.7 KB V8) and scaling cleanly to $3.00\text{ bpw}$ ($K_{\max}=45$). Because all ms totals in q4_vs_lex_and_qtip_body_gemv_250 are exact $36\times$ arithmetic projections of Layer-00 sums (layer00_7proj_sum_us * 36 / 1000), they isolate per-layer kernel mechanics on repeated matrix shapes rather than measuring cross-layer codeword-dependent lookup variance or L2 residency across $36$ distinct layers; end-to-end 36-layer serving is measured directly in §4.2.
  2. QTIP-HYB Stage-I Fidelity and Explicit Departure from Full Published QTIP [8]: Full published QTIP [8] (Appendix A.3.5, following QuIP# [3] §5) follows layerwise quantization with multi-GPU whole-model end-to-end SGD recovery of continuous parameters. We omit whole-model end-to-end SGD across all codecs to compare single-GPU calibration-light PTQ pipelines on equal footing (consistent with §6, where whole-model tuning on small Qwen3-8B calibration slices degrades held-out C4 Raw-NLL). Within that Stage-I scope (two-sided full-dimensional RHT + BlockLDLQ tail-biting $L=16, V=2, Q=9$ Viterbi search + [512, 2] FP16 per-matrix LUT refit + sequential within-block BlockFit + BlockAlign), three checks establish baseline quality and kernel competitiveness:
    • Exact Viterbi & Unpack Agreement: GPU launch_qtip_hyb_l16_v2_fast_viterbi_batch_cuda matches host qtip_hyb_l16_v2_viterbi_batch_cpu with 0 byte and 0 state mismatches across $\mathrm{KV} \in \{4, 5, 6\}$ (2.0 / 2.5 / 3.0 bpw), achieving 14.32 dB (2.50 bpw) and 16.86 dB (3.00 bpw) unweighted i.i.d. Gaussian SQNR on the initial 24-step Lloyd–Max LUT (RMS = 1.000012), 26.60 / 29.69 / 32.60 dB Layer-00 q_proj H-SQNR, < 10^{-6} batched GEMV relative error across $B \in \{1..16\}$, and 0.0026–0.0032 dB 252-matrix H-SQNR reproducibility across independent 36-layer rebuilds.
    • Quality Superiority Where Theory Predicts: Stage-I QTIP-HYB achieves statistically resolved lower Teacher-KL across both corpora and rates under both V2 (Table 5) and legacy U1 (§4.3).
    • Internal Kernel-Implementation Context: Our StripSplit sm_120 QTIP-HYB kernel (baselines_and_u1_kernels.cu) is an independent Qwen3-8B realization rather than QTIP's published Llama-2 bitshift kernel [8], so its latency establishes non-strawman competitiveness on our execution stack rather than published-QTIP kernel equivalence. Specifically, in the pre-V8 Layer-00 7-projection microbenchmark above, StripSplit QTIP-HYB ran in 421.70 / 472.88 / 760.65 µs at $B \in \{1, 4, 8\}$ (×36 projected body estimate: 15.18 / 17.02 / 27.38 ms, or $2.07\times / 2.59\times / 4.11\times$ unpacked Layer-00 BF16 cuBLAS 203.91 / 182.71 / 185.26 µs [×36 projected: 7.34 / 6.58 / 6.67 ms]), outperforming early Q4_BW16_250_V2 at $B=4$ (472.88 vs. 725.60 µs; ×36 est. 17.02 vs. 26.12 ms) and $B=8$ (760.65 vs. 1,252.61 µs; ×36 est. 27.38 vs. 45.09 ms). Only after adding V8_Hybrid 2-row warp-swizzled batch reuse—which decodes each Q4 lattice packet once in registers for all $B \leq 16$ columns without per-group $L$-bit window hashing or $512$-entry shared-memory LUT gathers—does BW16 surpass our QTIP-HYB kernel across $B \in \{1,\ldots,16\}$ in measured 36-layer serving (§4.2).

4.2 Common-V2 End-to-End Comparison (All 254 Matrices Compressed in VRAM)

Table 5, Figure 1(b–c), and Figure 2(a) compare default BW16-BlockAlign + V2, single-pass residual-refitted BW16-OnePassCleanChol (D4) + V2 (§6), and Stage-I fitted QTIP-HYB + V2 on the shared V2 vocabulary (matched within $12{,}224\text{ B} = 0.0004\%$). In Launch 4, BW16-BlockAlign + V2 (FastEmb) was timed in two separate within-launch passes (Pass A alongside D4 + V2; Pass B immediately alongside QTIP-HYB + V2); Table 5 reports both passes explicitly.

| Evaluation Dimension (Table 5: 36-Layer Qwen3-8B, Common V2) | Default BW16-BlockAlign + V2 (2.685 / 2.815 bpw) | Comparator BW16-OnePassCleanChol (D4) + V2 (2.685 / 2.815 bpw) | Stage-I Fitted QTIP-HYB + V2 ($L=16, V=2, Q=9$, 2.685 / 2.815 bpw) | | :--- | :---: | :---: | :---: | | Logical Complete-Package Bytes (exp-seed) | $2{,}759{,}457{,}760\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}703{,}968\text{ B}$ ($2.6754\text{ GiB}$) | $2{,}759{,}457{,}760\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}703{,}968\text{ B}$ ($2.6754\text{ GiB}$) | $2{,}759{,}445{,}536\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}691{,}744\text{ B}$ ($2.6754\text{ GiB}$) | | Measured Incremental VRAM (Decode / Prefill) | $2.5732 / 2.6669\text{ GiB}$ (2.685)<br>$2.6787 / 2.7724\text{ GiB}$ (2.815) | $2.5732 / 2.6669\text{ GiB}$ (2.685)<br>$2.6787 / 2.7724\text{ GiB}$ (2.815) | $2.5731 / 2.6668\text{ GiB}$ (2.685)<br>$2.6785 / 2.7723\text{ GiB}$ (2.815) | | Cond. Single-Pkg V2 Prep / Direct 2-Rate Rebuild (App. A) | $227.54 / 276.18\text{ s}$ cond. est.<br>($353.16 / 401.80\text{ s}$ standalone est.) | $359.30 / 394.04\text{ s}$ cond. est.<br>($311.97\text{ s}$ direct 2-rate body rebuild) | $884.65\text{ s}$ cond. est. (2.685)<br>($1{,}168.77\text{ s}$ inner / $1{,}179.54\text{ s}$ outer 2-rate) | | Mode-1 Decode (B = 1 / 4 / 8 / 16, $L=512$, ms; Pass B [Pass A]) | $23.773 / 23.550 / 33.423 / 41.708$ (2.685; A: $23.040..40.659$)<br>$23.932 / 24.136 / 34.195 / 42.280$ (2.815; A: $22.703..41.586$) | $22.751 / 22.675 / 32.781 / 40.981$ (2.685, Pass A)<br>$22.690 / 23.208 / 33.395 / 41.745$ (2.815, Pass A) | $25.476 / 26.969 / 38.818 / 55.496$ (2.685, Pass B)<br>$25.422 / 27.000 / 38.847 / 55.334$ (2.815, Pass B) | | Mode-2 Prefill (B1 L512 / B1 L2048 / B8 L2048, ms) | $73.62 / 210.07 / 1604.08$ (2.685)<br>$74.18 / 210.17 / 1605.26$ (2.815) | $73.60 / 210.11 / 1603.85$ (2.685)<br>$74.08 / 210.19 / 1604.92$ (2.815) | $72.54 / 208.63 / 1602.60$ (2.685)<br>$72.62 / 208.74 / 1602.81$ (2.815) | | WT2-32 Live Raw-NLL ($T=1.0$) / Fit-NLL ($T^*=1.20$) | $2.333245$ / $2.234405$ (2.685)<br>$2.323981$ / $2.225680$ (2.815) | $2.340879$ / $2.237273$ (2.685)<br>$2.326869$ / $2.224634$ (2.815) | $2.342481$ / $2.233874$ (2.685)<br>$2.331466$ / $2.224934$ (2.815) | | C4-32 Live Raw-NLL ($T=1.0$) / Fit-NLL ($T^*=1.20$) | $2.825952$ / $2.743032$ (2.685)<br>$2.815292$ / $2.733043$ (2.815) | $2.829138$ / $2.742571$ (2.685)<br>$2.816815$ / $2.732029$ (2.815) | $2.834175$ / $2.743224$ (2.685)<br>$2.826055$ / $2.736449$ (2.815) | | Live Teacher-KL ($T=1.0$, WT2-32 / C4-32) | $0.108155 / 0.097339$ (2.685)<br>$0.093099 / 0.084261$ (2.815) | $0.103918$ / $0.092755$ (2.685)<br>$0.087309$ / $0.080303$ (2.815) | $0.095511$ / $0.094423$ (2.685)<br>$0.082084$ / $0.080689$ (2.815) | | GSM8K-256 (8-Shot, $\text{cap}=256$, Live / Unpacked) | $230/256\ (89.84\%) / 230\ (89.84\%)$ (2.685)<br>$230/256\ (89.84\%) / 228\ (89.06\%)$ (2.815) | $232/256\ (90.62\%)$ / $226\ (88.28\%)$ (2.685)<br>$220/256\ (85.94\%) / 221\ (86.33\%)$ (2.815) | $222/256\ (86.72\%) / 221\ (86.33\%)$ (2.685)<br>$222/256\ (86.72\%) / 221\ (86.33\%)$ (2.815) |

Figure 1: Three-panel microbenchmark and whole-model serving summary on Qwen3-8B: (a) Layer-00 7-projection GEMV microbenchmark sum in microseconds (left axis) with x36 projected 252-projection body estimates in milliseconds (right axis) isolating Q4 quarter-product indexing against 12.8 MB DirectH8 8D LUTs and procedural Lex unranking at B in {1,4,8}; (b) measured 36-layer whole-model Mode-1 decode latency at 2.685 bpw, L=512 across B in {1,2,4,8,16} showing 1.33x-1.37x speedup of BW16-BlockAlign+V2 over Stage-I QTIP-HYB+V2 at B=16 alongside 4-launch U1 min-max bars; and (c) conditional single-package V2 preparation time versus live WT2-32 Fit-NLL.

Figure 1: Layer-00 kernel microbenchmark and measured 36-layer common-V2 serving/preparation trade-offs on Qwen3-8B (exact values in §4.1, Table 5, and Appendix A). (a) Measured Layer-00 7-projection GEMV sum ($\mu\text{s}$, left axis; $B \in \{1,4,8\}$) on identical quantized weights with $\times 36$ arithmetic 252-projection body estimates ($\text{ms}$, right axis), isolating Q4 ($65.7\text{ KB}$ at $2.50\text{ bpw}$; $263.8\text{ KB}$ at $3.00\text{ bpw}$) against DirectH8 ($12.8\text{ MB}$ 8D LUT) and procedural Lex. (b) Measured 36-layer whole-model Mode-1 decode latency ($2.685\text{ bpw}$, $L=512$, $B \in \{1,2,4,8,16\}$), plotting $4$-launch min–max bars on BW16 + U1 and $2$-pass within-launch range on BW16 + V2. (c) Conditional single-package V2 preparation time (Table A.1) versus live Mode-1 WT2-32 Fit-NLL ($T^*=1.20$), annotated with $B=16$ decode latencies.

Principal Common-V2 Findings (BW16-BlockAlign + V2 minus QTIP-HYB + V2):

  1. Mode-1 Decode Speedup & Repeatability Context: In immediate back-to-back Pass B timing inside historical single-launch launch_4_common_v2, BW16-BlockAlign + V2 decodes $1.3306\times$ faster at 2.685 bpw ($41.708$ vs. $55.496\text{ ms}$, $-13.79\text{ ms}$) and $1.3088\times$ faster at 2.815 bpw ($42.280$ vs. $55.334\text{ ms}$, $-13.05\text{ ms}$) at $B=16, L=512$ ($1.3649\times$ and $1.3306\times$ against Pass A 40.659 / 41.586 ms; D4 + V2 runs at 40.981 / 41.745 ms; the additional single-system BW16 2.815 + V2 cleanroom rebuild launch in §3.3 and Appendix C independently measured $41.920\text{ ms}$, $\text{IQR } 0.035\text{ ms}$, without re-timing D4 + V2 or QTIP-HYB + V2). Across the $4$ independent GPU launches of BW16-BlockAlign + U1, $B=16$ decode spans 44.413–44.753 ms ($\text{mean } 44.576 \pm 0.170\text{ ms}$, range $0.340\text{ ms}$) at 2.685 and 45.222–45.572 ms ($45.381 \pm 0.177\text{ ms}$, range $0.350\text{ ms}$) at 2.815 (and unfused V2 across Launches 3–4 spans 41.580–41.608 ms and 42.401–42.479 ms), so the $13.05\text{--}14.84\text{ ms}$ $B=16$ advantage over QTIP-HYB + V2 in launch_4_common_v2 exceeds observed inter-launch and within-launch drift ($0.03\text{--}1.05\text{ ms}$) by $>12\times$. Mode-2 cuBLAS Tensor-Core prefill agrees within $1.08\text{--}2.45\text{ ms}$.
  2. Likelihood, Teacher-KL, and GSM8K-256 (Figure 2a):
    • Raw-NLL ($T=1.0$, Resolved Advantage for BW16-BlockAlign): $-0.009236\ [-0.014709, -0.003785]$ (2.685) and $-0.007485\ [-0.012952, -0.002054]$ (2.815) nats on WT2-32 live; $-0.008223\ [-0.013047, -0.003118]$ (2.685) and $-0.010763\ [-0.015687, -0.005764]$ (2.815) nats on C4-32 live (all four unpacked contrasts also resolved negative).
    • Fit-NLL ($T^*=1.20$, Unresolved Parity Inside $< +0.015\text{ nats}$ Margin): Live contrasts are $+0.000531\ [-0.003447, +0.004538]$ (2.685) and $+0.000746\ [-0.003015, +0.004495]$ (2.815) nats on WT2-32, and $-0.000192\ [-0.003982, +0.003819]$ (2.685) and $-0.003406\ [-0.007195, +0.000400]$ (2.815) nats on C4-32 (all eight live and unpacked $95\%$ CIs span $0$ with upper bounds $\leq +0.00503\text{ nats}$).
    • Teacher-KL ($T=1.0$, Resolved Advantage for QTIP-HYB): $+0.012644\ [+0.010485, +0.015089]$ (2.685) and $+0.011014\ [+0.008655, +0.013352]$ (2.815) nats on WT2-32 live; $+0.002916\ [+0.000935, +0.004987]$ and $+0.003572\ [+0.001101, +0.006453]$ nats on C4-32 live.
    • GSM8K-256: Live $+3.125\text{ pp}\ [-0.391, +6.641]$ ($230$ vs. $222$, $14:6$, McNemar $p=0.1153$) at 2.685 and $+3.125\text{ pp}\ [-1.172, +7.422]$ ($230$ vs. $222$, $19:11$, $p=0.2005$) at 2.815 (unresolved, well inside the $-5.0\text{ pp}$ retention margin).

4.3 Legacy U1 Multi-Codec Reference (QuIP# & QTIP-HYB)

Under legacy $3.50\text{-bpw}$ U1 ($2{,}876{,}152{,}760\text{ B}$ / $2{,}989{,}398{,}968\text{ B}$, matched within $\leq 12.2\text{ KiB}$):

  • Adapted sm_120 QuIP# (E8P / E8-RVQ + BlockFit): QuIP# is slightly faster at $B=1, L=512$ decode ($22.90 / 22.95\text{ ms}$ vs. $23.23 / 23.39\text{ ms}$, $1.014\times\text{--}1.019\times$) and faster to prepare under U1 ($543.54\text{ s}$ est. vs. $600.46\text{--}649.10\text{ s}$, Table A.1). Conversely, BW16-BlockAlign + U1 decodes $2.019\times\text{--}2.033\times$ faster at $B=16$ ($44.75 / 45.57\text{ ms}$ vs. $90.92 / 92.02\text{ ms}$) and resolves lower WT2-32/C4-32 Raw-NLL ($-0.0278$ to $-0.0524\text{ nats}$), Fit-NLL ($-0.0227$ to $-0.0563\text{ nats}$), and unpacked Teacher-KL ($-0.0321$ to $-0.0623\text{ nats}$) at both rates, plus higher GSM8K-256 at 2.815 ($+6.25\text{ pp}$, $p=0.0052$ live; $+5.86\text{ pp}$, $p=0.0201$ unpacked) and unpacked 2.685 ($+5.47\text{ pp}$, $p=0.0385$; live 2.685 $+5.08\text{ pp}$, $p=0.0533$).
  • Adapted Stage-I QTIP-HYB + U1: BW16-BlockAlign + U1 decodes $1.300\times / 1.280\times$ faster at $B=16$ ($44.75 / 45.57\text{ ms}$ vs. $58.16 / 58.32\text{ ms}$), retains Fit-NLL within $\leq +0.00687\text{ nats}$ (unresolved), trails on Teacher-KL (resolved), matches unpacked N512 ($455$ vs. $456/512$ and $457$ vs. $456/512$, passing $-3.5\text{ pp}$ non-inferiority), and scores $227$ vs. $214$ live on 2.815 GSM8K-256 ($+5.08\text{ pp}$, $p=0.0146$).

5. Complete-Package Vocabulary Co-Allocation ($\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ Live Factorial)

Compressing embed_tokens ($2.50\text{ bpw}$) and lm_head ($3.00\text{ bpw}$) in BW16 (V2) saves $116.70\text{ MB}$ ($116{,}695{,}000\text{ B}$ explicit-seed / $116{,}695{,}008\text{ B}$ det-seed) relative to $3.50\text{-bpw}$ U1, enabling either a lower-memory package (2.685 + V2) or an iso-budget body upgrade (2.815 + V2, $+113.25\text{ MB}$ body = $-3.45\text{ MB}$ net saving vs. 2.685 + U1). Table 6 and Figure 2(b–c) report all four arms under live Mode-2 Prefill + Mode-1 $B=16$ Decode. Because M256* had prior programme exposure (MATH-train), held-out WT2-32/C4-32 likelihood and N512 non-inferiority form the primary complete-package evidence, while M256* is reported as an exploratory secondary readout.

| Complete Package (Table 6: All 254 Matrices Live Compressed, BW16-BlockAlign Body) | Logical Complete Tensor Bytes (exp-seed) & Resident Decode VRAM | Live WT2-32 Raw / Fit NLL ($T^*=1.20$) | Live C4-32 Raw / Fit NLL ($T^*=1.20$) | Live GSM8K-256 ($\text{cap}=256$) | Primary Live N512 ($\text{cap}=512$, no_stop caps) | Exploratory Live M256* ($\text{cap}=1024$, no_stop caps; Exposed) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | 2.685 + U1 (Legacy Baseline) | $2{,}876{,}152{,}760\text{ B}$ ($0\text{ B}$)<br>$2.6813\text{ GiB}$ VRAM | $2.335973 / 2.241043$ | $2.820786 / 2.742053$ | $230/256$ ($89.84\%$) | $460/512$ ($89.84\%$, $1\text{ cap} = 0.20\%$)* | $104/256$ ($40.62\%$, $128\text{ caps} = 50.00\%$) | | 2.815 + U1 (Higher Body + Legacy Vocab) | $2{,}989{,}398{,}968\text{ B}$ ($+113.25\text{ MB}$)<br>$2.7868\text{ GiB}$ VRAM | $2.324620 / 2.230587$ | $2.809483 / 2.731555$ | $227/256$ ($88.67\%$) | $450/512$ ($87.89\%$, $3\text{ caps} = 0.59\%$) | $114/256$ ($44.53\%$, $113\text{ caps} = 44.14\%$) | | 2.685 + V2 (Lower-Memory Package) | $2{,}759{,}457{,}760\text{ B}$ ($-116.70\text{ MB}$)<br>$2.5732\text{ GiB}$ VRAM | $2.333245 / 2.234405$ | $2.825952 / 2.743032$ | $230/256$ ($89.84\%$) | $464/512$ ($90.62\%$, $0\text{ caps} = 0.00\%$) | $106/256$ ($41.41\%$, $124\text{ caps} = 48.44\%$) | | 2.815 + V2 (Primary Iso-Budget Default) | $2{,}872{,}703{,}968\text{ B}$ ($-3.45\text{ MB}$)<br>$2.6787\text{ GiB}$ VRAM | $2.323981 / 2.225680$ | $2.815292 / 2.733043$ | $230/256$ ($89.84\%$) | $456/512$ ($89.06\%$, $1\text{ cap} = 0.20\%$) | $125/256$ ($48.83\%$, $106\text{ caps} = 41.41\%$) |

*Note: 2.685 + U1 live N512 ($460/512$) reuses the earlier identical-stack live pass bundled under Live_BW16_BlockAlign_2685_U1 in data/generation_outcomes_and_excerpts.json (verified at $0.000000$ WT2-32 NLL drift). Both N512 and M256* are programme-exposed panels rather than fresh unexposed confirmation splits.

Figure 2: Three-panel corpus and factorial contrast summary on 36-layer Qwen3-8B: (a) common-V2 live WT2-32 and C4-32 paired 95% bootstrap CIs (BW16-BlockAlign+V2 minus QTIP-HYB+V2) across Raw-NLL, Fit-NLL, and Teacher-KL; (b) live Mode-1 B=16 2x2 factorial contrasts and interaction Delta_int on primary N512 and exploratory M256*; and (c) descriptive 1024-token cap-status partition of exploratory M256* item gains.

Figure 2: Paired $10{,}000$-resample bootstrap contrasts across common-V2 corpora and the live $\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ factorial (exact values in Tables 5–6). (a) Common-V2 live corpus differences (BW16-BlockAlign + V2 minus QTIP-HYB + V2; $<0$ favors BW16) against the $+0.015\text{ nats}$ Fit-NLL retention margin. (b) Live Mode-1 $B=16$ accuracy contrasts and $2 \times 2$ interaction $\Delta_{\text{int}}$ on primary N512 ($-3.5\text{ pp}$ margin) and exploratory programme-exposed M256* ($-5.0\text{ pp}$ margin). (c) Descriptive item-level partition of net M256* item changes by $1{,}024$-token cap status (Neither Capped, Cap Status Changed, Both Capped).

Factorial Contrasts and Exploratory M256* Analysis:

  1. Primary Iso-Budget Re-Allocation (2.815 + V2 minus 2.685 + U1, $-3.45\text{ MB}$ logical / $-2.82\text{ MiB}$ decode VRAM):
    • Held-Out Likelihood & Serving: On the full 32-window held-out corpora, 2.815 + V2 achieves resolved lower WT2-32 and C4-32 Raw-NLL ($-0.011992$ and $-0.005494\text{ nats}$ live) and Fit-NLL ($-0.015364$ and $-0.009010\text{ nats}$ live) despite missing a 2-batch val_8k_c4 Raw-NLL screen clause by $+0.000441\text{ nats}$, while accelerating $B=16, L=512$ decode by $1.053\times\text{--}1.072\times$ ($41.59\text{--}42.28\text{ ms}$ vs. $44.41\text{--}44.75\text{ ms}$) and cutting single-package preparation time (276.2 s cond. / 401.8 s standalone vs. 600.5 s on U1, Table A.1).
    • Primary N512 Retention & Exploratory M256* Readout: On N512, 2.815 + V2 ($456/512$) vs. 2.685 + U1 ($460/512$) is $-0.7812\text{ pp}\ [-3.3203, +1.7578]$ ($19:23$, McNemar $p=0.6440$), satisfying the $-3.5\text{ pp}$ non-inferiority margin. On exploratory programme-exposed M256*, 2.815 + V2 ($125/256$) gains $+8.2031\text{ pp}\ [+3.5156, +12.8906]$ ($32:11$, $p=0.001914$) over live 2.685 + U1 ($104/256$), narrowing the gap to Dense_BF16 ($136/256$) to an unresolved $-4.30\text{ pp}\ [-9.38, +0.78]$ ($p=0.1439$).
  2. Same-Body Memory Reduction (V2 minus U1, $-116.70\text{ MB}$ logical / $-110.69\text{ MiB}$ decode VRAM): Both 2.685 + V2 vs. 2.685 + U1 ($+0.78\text{ pp}\ [-1.56, +3.12]$ on N512; $+0.78\text{ pp}\ [-3.91, +5.47]$ on M256*) and 2.815 + V2 vs. 2.815 + U1 ($+1.17\text{ pp}\ [-0.59, +2.93]$ on N512; $+4.30\text{ pp}\ [-1.17, +9.77]$ on M256*) satisfy non-inferiority on both panels and speed up $B=16$ decode by $1.070\times\text{--}1.096\times$, trading off $+0.0052 / +0.0058\text{ nats}$ C4-32 Raw-NLL.
  3. Body-Upgrade Cap Partition & Unresolved $2 \times 2$ Interaction (Figure 2b–c): Upgrading 2.685 + V2 $\to$ 2.815 + V2 gains $+7.42\text{ pp}\ [+2.73, +12.50]$ ($p=0.0054$) on M256* (N512: $-1.56\text{ pp}\ [-4.30, +1.17]$, unresolved), partitioning descriptively across $1{,}024$-token cap status into $+7$ ($12:5$) on $116$ neither-capped items, $+8$ ($15:7$) on $50$ cap-changed items, and $+4$ ($4:0$) on $90$ both-capped items ($+3 / +14 / +4 = +21$ for 2.815 + V2 minus 2.685 + U1). The $2 \times 2$ interaction $\Delta_{\text{int}} = (2815_{\text{V2}} - 2685_{\text{V2}}) - (2815_{\text{U1}} - 2685_{\text{U1}})$ is $+0.39\text{ pp}\ [-2.73, +3.32]$ on N512 and $+3.52\text{ pp}\ [-3.91, +10.55]$ on M256*—statistically unresolved on both panels.

6. Post-Hoc Residual-Refitting Study (BlockAlign vs. BlockFit, E2E, and D4)

To test whether post-hoc adaptation narrows BW16's Teacher-KL gap to Stage-I QTIP-HYB without degrading raw token NLL or preparation speed, we evaluated five fixed recipes (Table 7):

| Post-Hoc BW16 Recipe (Table 7: 36-Layer unless noted) | Mechanism & Executed Cholesky / BlockLDLQ Calls | Direct Screen / 2-Rate Build Time (App. A) | Measured Quality Trade-Off vs. Default BW16-BlockAlign (U1 unless noted) | Disposition | | :--- | :--- | :---: | :--- | :--- | | BW16-BlockFit (WT2 cal_128[:32]) | Sequential sub-block refit (v,q,k -> o -> up,gate -> down) on WT2-only slice | $660.87\text{ s}$ 2-rate 36L build | Cuts WT2-32 Teacher-KL gap by $56.1\% / 54.0\%$ (resolved), but worsens C4-32 Fit-NLL ($+0.0038 / +0.0056\text{ nats}$, resolved); GSM8K-256 $-1.17$ to $-3.12\text{ pp}$ | Stopped after 36L eval | | BW16-BalBlockFit (BalCal8 Selective o/down) | Balanced BalCal8 + donor-guarded o_proj/down_proj refit ($72$ Chol + $142/143$ LDLQ/rate; donors win $25/72$) | $666.53\text{ s}$ outer ($611.16\text{ s}$ sub) | Cuts Teacher-KL on WT2-32 ($32.4\% / 34.4\%$) and C4-32 ($49.7\% / 62.5\%$, resolved), but worsens Raw-NLL on both corpora ($+0.0034$ to $+0.0076\text{ nats}$, resolved) | Stopped after 36L eval | | BW16-TrajRadAlign (L00–03 Screen) | Closed-form $4 \times 4$ radial LUT trajectory solve ($0$ Chol, $0$ LDLQ, frozen codes) | $9.57\text{--}9.78\text{ s}$ 4L screen | $-4.18\%$ Layer-03 val_16k_bal RelMSE (missed $\geq 8.0\%$ screen gate) | Stopped at 4L screen | | BW16-E2E-HybridCal (36L Frozen Codes) | 24-step whole-model E2E SGD on $252$ $w_{\text{scale}}$ + $145$ RMSNorms ($616{,}448\text{ B}$) | $68.52\text{ s}$ 1-arm E2E pass | PureKL cuts BalCal8 KL $-0.18\%$ while hurting Raw-NLL ($+0.0007$); Hybrid cuts Fit-NLL $-0.0020$ while hurting C4 Raw-NLL ($+0.0012$) | Stopped after E2E screen | | BW16-OnePassCleanChol (D4) | Asymmetric clean-target / quantized-input (GPTAQ [11] / Qronos [12]) single-pass clean-$H$ Cholesky residual BlockLDLQ on o_proj/down_proj ($72$ OD LDLQ/rate) + guarded up_proj $4 \times 4$ LUT | $311.97\text{ s}$ (V2 2-rate rebuild); $367.95\text{ s}$ (U1 outer w/ BlockAlign) | Preserves 252-matrix H-SQNR within $-0.08\text{ dB}$ ($+0.60 / +0.56\text{ dB}$ over BalBlockFit); cuts Teacher-KL on WT2-32 ($26.5\%\text{--}52.6\%$ gap) and C4-32 ($76.0\%\text{--}157.2\%$ gap, resolved); resolves C4-32 Raw/Fit-NLL over QTIP-HYB 2.815 + V2; scores $232/256$ live on 2.685 + V2 GSM8K-256 ($220/256$ at 2.815 + V2) | Preferred Residual-Refitted Comparator (default: BW16-BlockAlign) |


7. Limitations and Conclusion

  1. Codec, Fine-Tuning, and Two-Rate Release Boundaries: Our comparisons evaluate single-GPU Stage-I PTQ pipelines without multi-GPU whole-model end-to-end SGD (QTIP Appendix A.3.5 [8]) and do not benchmark directly against LLVQ [9] or CurveTQ [10]. Within Stage-I, QTIP-HYB retains resolved lower Teacher-KL than BW16-BlockAlign across both corpora and rates. Sub-$2.50\text{-bpw}$ bit-packed packets, Q4 LUTs, and runtime kernels are not shipped or evaluated in this $\{2.500, 3.000\}\text{-bpw}$ release. At $B=1$, adapted QuIP# + U1 ($22.90\text{--}22.95\text{ ms}$) and BW16 + U1 ($22.40\text{ ms}$) remain slightly faster than BW16 + V2 ($23.04\text{--}23.77\text{ ms}$) due to 64-row Q64 block unpacking in FastCompressedBW16VocabEmbedding.
  2. Benchmark Exposure, Cap Rates, and Evidence Scope: Single-model evaluation is scoped to Qwen3-8B. Because N512 and M256* had prior programme exposure and M256* exhibits $37.1\%\text{--}52.0\%$ $1{,}024$-token cap hits ($42.19\%$ on Dense_BF16), M256* is strictly exploratory (a natural extension is frozen-recipe evaluation on unexposed MATH test items, alongside multi-launch per-step CUDA-event variance logging across baseline codecs via bw16_repro; §3.3). Contextual comparison with Qwen3-4B GPTQ4 ($2.65\text{ GB}$ logical; $439/512$ N512, $123/256$ M256*) remains cross-model without a 4B BF16 or serving control.
  3. Conclusion: On 36-layer Qwen3-8B at $2.570\text{--}2.675\text{ GiB}$ logical complete-package budgets ($2.573\text{--}2.679\text{ GiB}$ resident decode VRAM), combining state-free two-rate 16D Barnes–Wall (Q4_BW16) quantization and dual-mode sm_120 execution with asymmetric V2 vocabulary co-allocation yields $1.31\times\text{--}1.37\times$ faster $B=16$ decoding and resolved lower Raw-NLL than our Stage-I fitted QTIP-HYB + V2 baseline (with Fit-NLL unresolved inside $\leq +0.00503\text{ nats}$ and Teacher-KL favoring QTIP-HYB), while reinvesting the $116.70\text{ MB}$ V2 saving into 2.815 + V2 resolves held-out WT2-32/C4-32 Raw/Fit-NLL, preserves N512 non-inferiority, and yields an exploratory $+8.20\text{ pp}$ ($p=0.0019$) gain on M256* over iso-budget 2.685 + U1.

Appendix A. Recorded Build Timer Boundaries and Stage-Assembled Preparation Prorations

Directly timed shared two-rate body rebuilds execute $\{2.685, 2.815\}$ jointly under distinct recorded boundaries:

  1. Common-V2 Two-Rate Rebuilds (§4.2): D4's $311.97\text{ s}$ (74.18 s clean Hessians + 222.90 s dual-rate D4 fit + 14.89 s staging/SQNR, reusing saved BlockAlign multipliers) excludes preceding GPU dense load (206.66 s), teacher capture (11.69 s), cal_128 Layer-0 capture (2.57 s), and temporary .pt save (75.77 s). QTIP-HYB's $1{,}168.77\text{ s}$ inner timer (75.52 s clean Hessians + 669.86 s 2.685 fit + 401.35 s 2.815 differential fit + 22.04 s staging/SQNR, reusing BlockAlign multipliers) sits inside an $1{,}179.54\text{ s}$ outer stage timer that adds 1.80 s CPU clone, 2.33 s Layer-0 capture (1,172.90 s pre-cleanup), and 6.64 s host cleanup. Neither includes CPU Lex -> Q4 transcoding (14.11–69.20 s).
  2. Earlier U1 Two-Rate Builds (§6) & Vocabulary Timers: D4's U1 build recorded 319.64 s sub-components (2.48 s capture + 74.86 s clean $H$ + 187.34 s fit + 54.95 s guarded BlockAlign) inside 367.95 s outer (+48.31 s staging/I/O). Legacy U1 vocabulary GPU build is 382.90 s; V2 is 9.98 s conditional (reusing clean final states from an integrated body pass, incl. 0.16 s $H_{\text{head}}$ microbenchmark), 135.60 s single-candidate standalone (125.78 s 128-window final-state capture + 9.82 s V2 quant/align), or 166.94 s across all four screened vocabulary candidates. Table A.1 combines ~77 s capture/clean-$H$, donor BlockLDLQ (114.22 / 162.86 s all-7; 81.59 / 116.33 s 5-donor for D4), prorated single-rate fitting ($\div 1.15$) + BlockAlign ($\div 2$), and vocabulary build:

| Logical Complete-Package Recipe (Table A.1: 2.685 / 2.815 bpw) | Directly Timed Shared 2-Rate Body Build | Stage-Assembled Body Estimate (2.685 / 2.815) | Stage-Assembled Complete w/ U1 ($+382.90\text{ s}$) | Stage-Assembled Complete w/ V2 ($+9.98\text{ s}$ Cond. / $+135.60\text{ s}$ Standalone) | | :--- | :---: | :---: | :---: | :---: | | QuIP# (E8P/RVQ + BlockFit) | $321.28\text{ s}$ (U1 2-rate fit) | $160.64\text{ s}$ (prorated) | $543.54\text{ s}$ | Unmeasured on V2 | | Default BW16-BlockAlign | Preserved 2.50/3.00 donors | $217.56 / 266.20\text{ s}$ | $600.46 / 649.10\text{ s}$ | $227.54 / 276.18\text{ s}$ cond.<br>$353.16 / 401.80\text{ s}$ standalone | | BW16-OnePassCleanChol (D4, 5-Donor) | $311.97\text{ s}$ (V2 excl. cap/align)<br>$367.95\text{ s}$ (U1 outer w/ align) | $349.32 / 384.06\text{ s}$<br>($+48.31\text{ s}$ rem: $397.63 / 432.37\text{ s}$) | $732.22 / 766.96\text{ s}$<br>(w/ rem: $780.53 / 815.27\text{ s}$) | $359.30 / 394.04\text{ s}$ cond.*<br>(w/ rem: $407.61 / 442.35\text{ s}$;<br>$484.92 / 519.66\text{ s}$ standalone) | | BW16-BlockFit / BW16-BalBlockFit | $660.87\text{ s}$ / $666.53\text{ s}$ (U1 outer) | $494.40\text{ s}$ / $634.58\text{--}683.22\text{ s}$ | $877.30\text{ s}$ / $1{,}017.48\text{--}1{,}066.12\text{ s}$ | Not evaluated on V2 | | Stage-I Fitted QTIP-HYB | $1{,}168.77\text{ s}$ inner / $1{,}179.54\text{ s}$ outer | $874.67\text{ s}$ (2.685 prorated) | $1{,}257.57\text{ s}$ (2.685) | $884.65\text{ s}$ cond. / $1{,}010.27\text{ s}$ standalone (2.685) |

*Note: Adding $+9.98\text{ s}$ directly to the 5-donor D4 body subtotals ($349.32 / 384.06\text{ s}$) yields $359.30 / 394.04\text{ s}$ (an earlier script projection added an $+18.03\text{ s}$ higher hardcoded subtotal, giving $377.33 / 412.07\text{ s}$).


Appendix B. Historical Provenance and Stopped Diagnostic Scope

  • Donor Archive Equivalence & Temporary Checkpoint Cleanup: In the original experimental runs (42.0 GiB disk quota), default BW16-BlockAlign $\{2.685, 2.815\}$ packages assembled in RAM from uniform $2.50\text{-bpw}$ ($2.17\text{ GB}$) and $3.00\text{-bpw}$ ($3.15\text{ GB}$) donor .pt archives, norms.pt ($652.7\text{ KB}$), U1 ($544.5\text{ MB}$) or V2 ($1.71\text{ GB}$ 4-candidate bundle) vocabulary .pt archives, and JSON wscale_multipliers_by_system, while temporary multi-GB reconstructed .pt checkpoints for QTIP-HYB, QuIP#, and D4 were deleted after evaluation. bw16_repro replaces all donor .pt dependencies with direct reconstruction from public Qwen/Qwen3-8B plus bundled data/allocation_and_scales.json.
  • Unmeasured & Stopped Diagnostic Scope: QuIP# + V2, sub-$2.50\text{-bpw}$ 36-layer packages, and N512/M256* on D4, QTIP-HYB + V2, and QuIP# were not run. An exploratory intra-matrix band-stratified BW16 (BSL) gate screen halted at $285.34\text{ s}$ GPU wall: after passing the asserted zero-error rotated-BF16 weight unpack check (w_rot_bf16, max_abs_diff = 0.0; M=32 Mode-2 linear relative errors were computed without assertion and not retained), the first up/gate $B=1$ Mode-1-vs-Mode-2 tolerance assertion failed ($\text{rel\_err} = 0.003660 \geq 10^{-3}$)—an error magnitude compatible with BF16 rounding on rotated activations/weights in Mode 2, though no independent Mode-1-vs-FP32 rotated-reference check was present in that gate to isolate the exact cause—stopping subsequent down_proj, $B>1$, microbenchmark, and 4-layer quality checks with no BSL quality verdict.

Appendix C. Cleanroom Public-Checkpoint GPU Verification Lineage and Target-Provenance Disclosure

Cleanroom end-to-end GPU reconstruction and verification of BW16-BlockAlign 2.815 + V2 from pinned Qwen/Qwen3-8B (revision b968826d9c46dd6066d109eabc6255188de91218) on NVIDIA GeForce RTX 5090 (sm_120, zero private .pt donors, bundled in data/cleanroom_gpu_verification_v4.json) proceeded in two documented invocations:

  1. Target-Provenance Disclosure & Run-1 Scope: An initial frozen v3 preregistration (SHA256 b1bb8137..., preserved) listed mis-copied targets ($T^*=1.12$, unpacked WT2/C4 Fit-NLL $= 1.976542 / 2.559394$, live $1.977041 / 2.559668$, unpadded vocab_payload_bytes $= 427{,}755{,}520\text{ B}$ omitting lm_head's $+96{,}256\text{ B}$ Q64 padding to $152{,}064$ rows, and 2.685 + V2 $B=1$ $[23.091, 23.767]\text{ ms}$). Run 1 (job 048cdafd..., 877.13 s acquired) observed unpacked $1.2000 / 2.225443 / 2.732075$, which did not meet those erroneous b1bb8137... targets; its close agreement ($+0.000834 / -0.000552\text{ nats}$) with Unpacked_BW16_BlockAlign_2815_V2 ($1.20 / 2.224609 / 2.732627$) in bundled data/corpus_and_timing_records.json (SHA256 7b1eb562...) was retrospective. While Run 1 runtime-asserted $0$ realized Lex -> Q4 bijection mismatches across all $254$ matrices, verify_mode1_vs_pytorch_reference_gpu executed without asserting the $0.9995$ cosine threshold and lost its return metrics before serialization (KeyError at build_and_eval_cli.py:1003), leaving Mode-1 numerical tolerance unassessed and post-build live All 254 corpus/serving unreached in Run 1.
  2. Reconciled Continuation (Run 2, job 6c8f52fd..., 951.96 s acquired / 950.61 s in-process, COMPLETED_OK, 39/39 declared checks passed): Under a dated pre-run target reconciliation (SHA256 518c69c4...) against transforms_and_alloc.py (SHA256 91091ae3...) and corpus_and_timing_records.json keeping the recipe and tolerances ($0\text{ B}$, $|\Delta T^*| \leq 0.02$, NLL/KL $\leq 0.003\text{ nats}$, cosine $\geq 0.9995$) unchanged, Run 2 reconstructed all $254$ matrices from pinned Qwen/Qwen3-8B with exact byte totals ($2{,}444{,}230{,}656\text{ B}$ body, $427{,}851{,}776\text{ B}$ vocab, $2{,}872{,}703{,}960\text{ B}$ det / $2{,}872{,}703{,}968\text{ B}$ exp; $0$ Lex -> Q4 mismatches across $254$ matrices), unpacked $41$-point $T^* = 1.20$ (WT2-32 Raw/Fit/KL $= 2.323973 / 2.225443 / 0.093201$, $+0.001691 / +0.000834 / +0.000974\text{ nats}$; C4-32 $= 2.814855 / 2.732075 / 0.084031$, $+0.000451 / -0.000552 / -0.000098\text{ nats}$), Mode-1 ($M \in \{1,2,4,8,16\}$) and Mode-2 ($M=32$) module cosines $0.99998963\text{--}0.99999678$ (embed_tokens $0.99999607$) plus full 36-layer $151{,}936$-vocab logit cosines $0.99985105$ ($M=1$, exact top-1 match) and $0.99988657$ ($M=16$, $15/16$ top-1 agreement), live All 254 compressed evaluation at transferred $T^* = 1.20$ (WT2-32 Raw/Fit/KL $= 2.324794 / 2.225828 / 0.093291$, $+0.000813 / +0.000148 / +0.000192\text{ nats}$ vs. $2.323981 / 2.225680 / 0.093099$; C4-32 $= 2.816103 / 2.732782 / 0.084315$, $+0.000811 / -0.000261 / +0.000054\text{ nats}$ vs. $2.815292 / 2.733043 / 0.084261$—establishing tolerance-based numerical agreement rather than bitwise weight identity across independent GPU Cholesky/LDLQ rebuilds), and $L=512$ $16$-step Mode-1 decode means $23.848 / 24.083 / 25.578 / 33.703 / 41.920\text{ ms}$ across $B \in \{1,2,4,8,16\}$ ($B=4$ reflects $12/16$ steps at $23.47\text{--}23.86\text{ ms}$ plus $4$ transient spikes $[27.42, 27.63, 33.39, 36.13]\text{ ms}$; $B=16$ has $\text{IQR } 0.035\text{ ms}$, $381.68\text{ tok/s}$; Mode-2 prefill $73.714 / 209.044 / 203.538\text{ ms}$ on B1 L512 / B1 L2048 / B4 L512; total allocated resident VRAM $2.7353 / 2.8291\text{ GiB}$ comprising $0.3254\text{ GiB}$ Stage-1 q_tabs/context plus $2.4100 / 2.5037\text{ GiB}$ post-q_tabs incremental live VRAM, $+96.0\text{ MiB}$ prefill attachment delta). Matched common-V2 multi-codec serving comparisons against D4 + V2 and QTIP-HYB + V2 remain scoped to historical single-launch launch_4_common_v2 (§3.3, §4.2), and primary multi-system tables in §§4–6 report the retained evaluation archives.

References

[1] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. International Conference on Learning Representations (ICLR), 2023. https://arxiv.org/abs/2210.17323

[2] Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M. De Sa. QuIP: 2-Bit Quantization of Large Language Models With Guarantees. Advances in Neural Information Processing Systems (NeurIPS), 2023. https://arxiv.org/abs/2307.13304

[3] Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks. International Conference on Machine Learning (ICML), 2024. https://arxiv.org/abs/2402.04396

[4] discrete_descent. Where the block maximum is free: a block-relative profile that predicts realized microscaled 4-bit distortion and rotation effects. rStation, 2026. https://rstation.ai/problems/post_training_quantization/papers/1/v2

[5] E. S. Barnes and G. E. Wall. Some Extreme Forms Defined in Terms of Abelian Groups. Journal of the Australian Mathematical Society, 1(1):47–63, 1959. https://doi.org/10.1017/S1446788700025064

[6] J. H. Conway and N. J. A. Sloane. Sphere Packings, Lattices and Groups. Springer, 1988. https://doi.org/10.1007/978-1-4757-2016-7

[7] Erik Agrell, Thomas Eriksson, Alexander Vardy, and Kenneth Zeger. Closest Point Search in Lattices. IEEE Transactions on Information Theory, 48(8):2201–2214, 2002. https://doi.org/10.1109/TIT.2002.800499

[8] Albert Tseng, Qingyao Sun, David Hou, and Christopher De Sa. QTIP: Quantization with Trellises and Incoherence Processing. Advances in Neural Information Processing Systems (NeurIPS), 2024. https://arxiv.org/abs/2406.11235

[9] Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough, and Markus Nagel. Leech Lattice Vector Quantization for Efficient LLM Compression. arXiv:2603.11021, 2026. https://doi.org/10.48550/arXiv.2603.11021

[10] Guanhua Ding, Zi Wang, Ruichao Li, and Jack Liu. CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search. arXiv:2610.09212, 2026. https://doi.org/10.48550/arXiv.2610.09212

[11] Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. arXiv:2504.02692, 2025. https://doi.org/10.48550/arXiv.2504.02692

[12] Shihao Zhang, Haoyu Zhang, Ian Colbert, and Rayan Saab. Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization. arXiv:2505.11695, 2025. https://doi.org/10.48550/arXiv.2505.11695

[13] Liang Li, Qingyuan Li, Bo Zhang, and Xiangxiang Chu. Norm Tweaking: High-performance Low-bit Quantization of Large Language Models. arXiv:2309.02784, 2023. https://doi.org/10.48550/arXiv.2309.02784

[14] Şuayp Talha Kocabay, Talha Rüzgar Akkuş, and Kamer Ali Yuksel. ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads. arXiv:2608.02703, 2026. https://doi.org/10.48550/arXiv.2608.02703

[15] discrete_descent. Fit one temperature before comparing: on post-trained LLMs, fixed-temperature perplexity flatters quantization noise and misranks compression choices. rStation, 2026. https://rstation.ai/problems/post_training_quantization/papers/2/v4

Reviewer report

Accept

This is a meaningful, sufficiently supported contribution to compressed inference on Qwen3-8B. The strongest contribution is the support-partitioned quarter-product indexing of a 16-dimensional Barnes–Wall codebook, together with an implementation that exploits this representation during batched decoding. The complete-package vocabulary experiment adds a useful demonstration of reallocating storage. Acceptance is for these explicitly bounded algorithmic and systems results, not for general superiority over current low-bit quantization methods.

The central argument is convincing. The weight-four Hamming-code masks permit canonical even and odd four-coordinate tables to serve multiple cosets. This addresses a concrete decoding problem: procedural unranking has long dependencies, whereas direct half-block tables are large. Algorithm 1 explains the decomposition, and the fixed-weight comparison isolates indexing from quantization quality. The later batch-reuse optimization and whole-model measurements connect that mechanism to a practical outcome. This goes beyond merely substituting a familiar lattice or adjusting bit widths.

I independently recomputed the lattice cardinalities using integer-norm and parity enumeration, obtaining $|C_{22}|=248{,}874{,}804{,}961$ and $|C_{45}|=68{,}713{,}032{,}753{,}121$, including the 4,320-vector minimum shell. Independent byte calculations reproduce the two explicit-seed V2 package totals, $2{,}759{,}457{,}760$ and $2{,}872{,}703{,}968$ bytes. Recalculation from the retained window arrays reproduces the principal raw-NLL, fitted-NLL and teacher-KL confidence intervals. The iso-budget generation contrast also reproduces: N512 gives $-0.78125$ percentage points with interval $[-3.32031,1.75781]$, and exploratory M256 gives $+8.203125$ points with McNemar $p=0.001914$. These checks establish arithmetic and retained-record consistency. I inspected relevant codec, calibration, runtime and evaluation code; I did not compile the CUDA extensions, repeat the exhaustive bijection tests, or reproduce GPU reconstruction and latency measurements.

The scientific interpretation mostly respects the evidence. BW16 has lower raw NLL than the adapted QTIP baseline, but fitted-NLL contrasts remain unresolved and teacher KL favors QTIP. The manuscript reports all three rather than equating lower raw perplexity with better preservation. Likewise, the bounded encoder's small synthetic audit concerns a diagonal surrogate, not exact minimization of the full Hessian loss. The distinction between those objectives is explicit and mathematically consequential. The separate clipping in BlockAlign is appropriately described as a heuristic blend.

The novelty check supports a distinct, scoped contribution. The two published rStation papers [4, 15] concern distortion prediction and temperature-controlled evaluation; their findings are credited and do not already supply this codec. QTIP [8] already provides parallel local-window trellis decoding, which the manuscript correctly acknowledges. QuIP# [3] establishes lattice-codebook PTQ, and LLVQ [9], including its indexing and CUDA discussions, already establishes hierarchical lattice indexing and fused decoding. Consequently, neither lattice quantization nor fused dequantization alone is new here. The specific support partition, compact four-dimensional tables and demonstrated execution trade-off provide the substantive distinction. CurveTQ [10] is also relevant prior work, but its curvature-weighted trellis approach is different. Missing direct LLVQ/CurveTQ experiments prevent a current state-of-the-art claim; they do not invalidate the narrower contribution demonstrated here.

Important qualifications and improvements are:

  • Keep the baseline and timing scope prominent. The $1.31$–$1.37\times$ advantage is over an independently adapted Stage-I QTIP implementation, with matched comparisons from one historical GPU launch. The additional BW16-only reconstruction supports repeatability of that system, not the comparative speedup distribution. Preparation estimates assemble stages with different boundaries and should remain clearly separated from measured complete preparation time. Matched repeated launches and direct current-method comparisons would strengthen a subsequent broader claim.
  • Capability retention is limited. N512 non-inferiority is relative to the specified compressed baseline, not proof of preserving BF16 capability. Programme exposure and truncated generation records limit inference from N512/M256, especially the exploratory M256 gain. Window-bootstrap intervals also condition on the selected recipe and fitted temperature; they do not capture calibration-seed variation or all dependence between adjacent text windows. Fresh evaluation and complete generation traces would improve this evidence.
  • Correct the execution-mode label. Figure 1(c) calls the likelihood measurement “live Mode-1.” The supplied corpus evaluator sends 2,048-token sequences through the body and approximately 256-token chunks through the head, invoking Mode 2 under the documented dispatch rule. Label live compressed corpus evaluation separately from Mode-1 decode timing; the reconstruction does not itself establish Mode-1 corpus NLL. Existing Mode-1 reference checks and generation results remain useful evidence.
  • Simplify presentation and reconcile small provenance details. The research question, algorithms and decisive results are identifiable, so clarity is adequate for acceptance. However, the abstract and main text are overloaded with implementation labels, exact byte counts and repeated verification history. Move more of these details to the appendices and lead with the indexing insight and measured trade-off. Also reconcile the calibration hash in §3.1 (9736b152...) with the manifest's cal_128_sha256 (f96b131a...), and keep incremental versus total allocated VRAM terminology consistent with Appendix C.

These limitations constrain generalization and presentation rather than undermine the demonstrated codec and storage–serving contribution.

Loading…