## 1. Introduction and Prior Art

Modern sub-3-bit weight-only PTQ methods precondition linear layers with orthogonal Randomized Hadamard Transforms (RHT) that equalize coordinate variances and suppress weight/activation outliers, followed by second-order Cholesky or `LDLQ` error feedback (`GPTQ` [1]; `QuIP` [2]; `QuIP#` [3]; see also block-relative microscaled distortion analyses under Hadamard rotation [4]). On 36-layer `Qwen3-8B` (`BF16`, untied $151{,}936$-token vocabulary), practical sub-3-bit compressed serving is governed by three coupled design choices:

1. **Block Lattice Geometry, Indexing Structure, and Shift-Register Trellises:** `QuIP#` [3] quantizes 8D blocks on the Gosset lattice $E_8$ (`E8P` at $2\text{ bpw}$; two-stage $E_8$ residual vector quantization at $3\text{ bpw}$). Doubling block dimension to $16$ reaches the **Barnes–Wall lattice $\Lambda_{16}$** [5, 6, 7], which divides every linear dimension of `Qwen3-8B` ($1024, 4096, 12288$) without padding while improving packing density and second-moment shaping over $E_8 \oplus E_8$ and keeping 16-weight packets state-free. By contrast, **`QTIP`** [8] attains high-dimensional Gaussian shaping via a tail-biting $L$-bit shift-register trellis ($L=16$, $2^{16} = 65{,}536$ states; `HYB` with a $2\text{ KiB}$ per-matrix lookup table). As shown in `QTIP` §3.1 [8], trellis decoding is fully parallelizable across weight groups because each $V$-weight output depends only on a local $L$-bit window of the packed bitstream; the intrinsic algorithmic asymmetry is sequential dynamic-programming (Viterbi) search at encode time versus independent 16D coset search in $\Lambda_{16}$, while decode latency depends on concrete kernel realizations—here, `Q4_BW16`'s byte-aligned quarter-product register inner products versus parallel $L=16$-bit window extraction, quadratic hash/sign indexing, and $512$-entry ($2\text{ KiB}$) shared-memory `FP16` LUT gathers in our `sm_120` `QTIP-HYB` ($L=16, V=2, Q=9$) on a shared execution stack.
   - **Technical Contrast with Leech Lattice Vector Quantization (`LLVQ` [9]) and `CurveTQ` [10]:** Recent lattice and trellis PTQ work explores 24D Leech lattice $\Lambda_{24}$ quantization (`LLVQ` [9]) and rotation-free curvature-weighted trellis search (`CurveTQ` [10]). Specifically, `LLVQ` (§3.2 and Appendices A/C [9]) employs **24D spherical shape–gain coding**, decomposing angular lattice codewords hierarchically by **shell leader class, sign pattern, and Lehmer-style coordinate permutation rank**, paired with per-block scalar norm normalization (`rsqrtf`-based unit-sphere scaling) over $24$-element blocks that do not divide $64$-column `BlockLDLQ` tiles, $32$-lane GPU warps, or $4{,}096$-channel projections ($4096 \equiv 16 \pmod{24}$). By contrast, `Q4_BW16` uses **16D Cartesian Construction-A cosets of $RM(1,4)$** aligned with $64$-column `BlockLDLQ` tiles, $32$-lane warps, and full-dimensional Kronecker RHTs on $1{,}024 / 4{,}096 / 12{,}288$-channel projections: (i) instead of runtime permutation unranking, each 8D half-block splits along the binary support of its $H_8$ coset mask into two canonical 4D integer quarters (`lut4_even`, `lut4_odd`), decoded via constant-depth magic-multiplied integer division (`__umul64hi` / `__umulhi`) directly into packed `int8x4` register words from $42.7\text{--}135.6\text{ KB}$ total `V8` device tables; and (ii) instead of per-block spherical norm division, `Q4_BW16` multiplies exact integer lattice vectors $y \in C_{K_{\max}} \subset \sqrt{2}\Lambda_{16}$ by an independent $2$-bit concentric radial multiplier $u \in \{0,1,2,3\}$ solved globally per matrix ($20\text{ B}$ per matrix, zero `rsqrtf` instructions).
2. **Single-Pass Residual Calibration vs. Multi-Pass Fine-Tuning:** Within-block sequential fine-tuning (`BlockFit` in `QuIP#` [3] and `QTIP` [8]) and whole-model end-to-end SGD recovery (`QTIP` Appendix A.3.5 [8]) reduce teacher-logit KL divergence (`Teacher-KL`), but incur substantial preparation overhead and risk overfitting small calibration slices on post-trained models. Asymmetric clean-target / quantized-input Cholesky updates (**GPTAQ** §4 [11]; **Qronos** [12]) and frozen-code scalar rescaling (**Norm Tweaking** [13]) provide lightweight single-pass alternatives.
3. **Complete-Package Vocabulary Co-Allocation:** On `Qwen3-8B`, untied `model.embed_tokens` and `lm_head` ($[151936, 4096]$ each) contain **$1{,}244{,}659{,}712$ weights—$15.20\%$ of all $8{,}190{,}427{,}136$ quantized linear and vocabulary weights** (versus $6{,}945{,}767{,}424 = 84.80\%$ in the 36-layer body). Following complete-model bit allocation principles (**QTIP** §4.2 [8]; **ARCHead** §4 [14]), compressing `embed_tokens` at $2.50\text{ bpw}$ and `lm_head` at $3.00\text{ bpw}$ in `BW16` (`V2`) saves $116.70\text{ MB}$ relative to legacy $3.50\text{-bpw}$ `U1` ($545.16\text{ MB}$ non-body), freeing budget to upgrade $36$ sensitive 12-unit MLP projections in the transformer body.

### Paper Organization and Contribution Scope
Section 2 derives the two-rate $\{2.500, 3.000\}\text{-bpw}$ `Q4_BW16` codec, support-partitioned quarter-product `Q4` unranking (Algorithm 1), stack-free bounded Plotkin/Wagner encoding and its coupled block-surrogate vs. full-Hessian synthetic audit (Algorithm 2), closed-form `BlockAlign` rescaling, sensitivity-guided `Swap7_Plus24K` bit allocation, four-way byte accounting, and dual-mode `sm_120` execution. Section 3 specifies our cleanroom public-checkpoint reconstruction release (`bw16_repro`), single-temperature evaluation protocol [15], statistical resampling rules, and explicit evidence boundaries. Section 4 reports Layer-00 7-projection kernel microbenchmarks (with $36\times$ projected body estimates), `QTIP-HYB` Stage-I baseline fidelity checks, and matched-byte 36-layer comparisons against Stage-I fitted `QTIP-HYB` and `QuIP#`. Section 5 presents the live $\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ complete-package factorial, treating held-out likelihood and `N512` retention as primary and programme-exposed `M256*` as an explicitly exploratory secondary readout. Section 6 evaluates five post-hoc residual-refitting recipes, Section 7 summarizes limitations, and Appendices A–C document timer boundaries, historical provenance, and cleanroom GPU verification lineage.

---

## 2. Two-Rate `Q4_BW16` Codec, Core Algorithms, Allocation, and `sm_120` Execution

### 2.1 Construction-A $RM(1,4)$ Codebook, Quarter-Product (`Q4`) Unranking, and Bijection Verification

**Lattice Construction and Radial Shell Parameterization:** Let $H_8 \subset \{0,1\}^8$ denote the $[8,4,4]$ extended binary Hamming code ($16$ masks: $0^8$ of Hamming weight $0$, $1^8$ of weight $8$, and $14$ masks of weight $4$), and let $RM(1,4) = \{(u_0, u_1) : u_0 \in H_8,\, u_1 \in \{u_0, \bar{u}_0\}\}$ denote the $[16, 5, 8]$ first-order Reed–Muller code ($32$ coset masks grouped into $5$ half-block weight-pair types $\mathrm{tp} \in \{0,\ldots,4\}$). By Construction A, an integer vector $y = (y_0, y_1) \in \mathbb{Z}^8 \times \mathbb{Z}^8$ lies in $L_{16} = \sqrt{2}\Lambda_{16}$ if and only if:

$$
y_h = 2 m_h + u_h \quad (h \in \{0,1\}), \qquad (u_0, u_1) \in RM(1,4), \qquad p(m_0) = p(m_1) \in \{0,1\},
$$

where $m_h \in \mathbb{Z}^8$ and $p(m_h) = \bigl(\sum_{k=0}^7 m_{h,k}\bigr) \bmod 2$ enforces matching $(2\mathbb{Z})^8$ quotient parities across the two 8D half-blocks. Every $y \in L_{16}$ has squared Euclidean norm $\|y\|_2^2 = 4 K_{\text{sh}}$ for an integer shell index $K_{\text{sh}} = q_0 + q_1 \in \{0, 2, 3, \ldots, K_{\max}\}$ (with origin $|C_0| = |C_1| = 1$ and minimum nonzero kissing shell $|C_2 \setminus C_1| = 4{,}320$ at $\|y\|_2^2 = 8$), where $q_h = \|y_h\|_2^2 / 4 = b_q(t_h) + \sum_{k=0}^7 (m_{h,k}^2 + u_{h,k} m_{h,k})$ and $t_h \in \{0, 1, 2\}$ indexes half-block mask weight $\{0, 8, 4\}$ with base norm offsets $b_q(0)=0$, $b_q(1)=2$, and $b_q(2)=1$.

**Full-Dimensional Kronecker RHT Preconditioning vs. Vocabulary Strided-`Q64` Blocks:** On every transformer body matrix $W \in \mathbb{R}^{M \times K}$ ($M, K \in \{1024, 4096, 12288\}$), `bw16_repro/transforms_and_alloc.py` (`fast_had_lastdim`) applies a **full-dimensional two-sided orthogonal Kronecker RHT**:

$$
W_r = Q_M^\top \operatorname{diag}(s_v)\, W \operatorname{diag}(s_u)\, Q_K, \qquad Q_{1024} = H_{1024}, \quad Q_{4096} = H_{64} \otimes H_{64}, \quad Q_{12288} = H_{12} \otimes H_{1024},
$$

where $s_u \in \{\pm 1\}^K$ and $s_v \in \{\pm 1\}^M$ are deterministic Rademacher sign vectors, $H_n$ ($n \in \{64, 1024\}$) is the normalized Sylvester Walsh–Hadamard matrix ($H_{64} \otimes H_{64} = H_{4096}$), and $H_{12}$ is the normalized Paley-$12$ orthogonal Hadamard matrix (`QuIP#` convention; $Q_M^\top$ uses $H_{12}^\top$ on $M=12{,}288$ and inverse reconstruction applies $Q_M$). This full-channel orthogonal mixing is distinct both from the $64$-column right-to-left `BlockLDLQ(b=8)` error-feedback tiles (§2.2) and from the `V2` vocabulary matrices (`rht_strided64_W`), which pair $Q_{4096}$ across the hidden dimension $K = 4{,}096$ with strided $64$-row $Q_{64} = H_{64}$ group transforms across vocabulary rows ($151{,}936 = 2{,}374 \times 64$ on `embed_tokens`; zero-padded to $152{,}064 = 2{,}376 \times 64$ on `lm_head`).

Each 16-weight block of $W_r$ is reconstructed without `rsqrtf` norm division from an integer lattice vector $y \in C_{K_{\max}} = \{y \in \sqrt{2}\Lambda_{16} : \|y\|_2^2 \leq 4 K_{\max}\}$, a per-matrix scalar $w_{\text{scale}}$ (`FP32`), and an independent **2-bit radial shell selector** $u \in \{0, 1, 2, 3\}$ indexing $\mu = \text{mult\_lut}[0..3]$ ($4 \times \text{FP32}$):

$$
\widehat{w}_r = w_{\text{scale}} \cdot \mu_u \cdot y, \qquad (y, u) \in C_{K_{\max}} \times \{0, 1, 2, 3\}.
$$

In the shipped two-rate `Q4_BW16` runtime (`Table 1`), each byte-aligned 16-weight packet packs two independent bitfields: (a) a bijective lattice index $r \in \{0, \ldots, |C_{K_{\max}}| - 1\}$ in $b_{\text{code}} \in \{38, 46\}$ bits, and (b) the 2-bit selector $u \in \{0,1,2,3\}$, yielding $4|C_{K_{\max}}|$ total $(\text{code} + \text{scale})$ packet states inside $5\text{ B}$ (`2.500 bpw`, $K_{\max}=22$) or $6\text{ B}$ (`3.000 bpw`, $K_{\max}=45$) per $16$ weights with occupancy $|C_{K_{\max}}| / 2^{b_{\text{code}}} = 4|C_{K_{\max}}| / 2^{b_{\text{code}}+2} > 90.5\%$. *(As mathematical shell-enumeration context verified by `verify_bw16_supplement.py`, smaller radial caps $K_{\max} = 13$ and $K_{\max} = 15$ contain $|C_{13}| = 4{,}026{,}639{,}841 < 2^{32}$ [$93.75\%$ of $2^{32}$] and $|C_{15}| = 12{,}225{,}158{,}881 < 2^{34}$ [$71.16\%$ of $2^{34}$] lattice vectors; sub-$2.50\text{-bpw}$ bit-packed packets, `Q4` LUTs, and runtime kernels are not shipped or evaluated here.)*

| Shipped `Q4_BW16` Rate (Table 1) | Bits / Weight | Shell Cap $K_{\max}$ | Lattice Indices $|C_{K_{\max}}|$ ($\log_2 |C_{K_{\max}}|$) | Total Packet States $4|C_{K_{\max}}|$ (`code + 2b` Occupancy) | Packet Layout & Combinatorial Structure (Subshells / Quarter-Splits / Even & Odd 4-Tuples) | Compiled Device LUT Footprint |
| :--- | :---: | :---: | :---: | :---: | :--- | :---: |
| **`Q4_BW16_250`** | $2.500$ | $22$ | $248{,}874{,}804{,}961$ ($37.8567\text{ b}$) | $995{,}499{,}219{,}844$<br>($38+2\text{ b}$, $90.54\%$) | $5\text{ B}/16\text{w}$ (`u32 + u8`); $958$ subshells, $804$ splits (`2,105` `V8` buckets), $2{,}585 / 2{,}816$ 4-tuples | **$42{,}660\text{ B}$ (`V8`)**<br>$65{,}698\text{ B}$ (base) |
| **`Q4_BW16_300`** | $3.000$ | $45$ | $68{,}713{,}032{,}753{,}121$ ($45.9657\text{ b}$) | $274{,}852{,}131{,}012{,}484$<br>($46+2\text{ b}$, $97.65\%$) | $6\text{ B}/16\text{w}$ (`u32 + u16`); $3{,}983$ subshells, $3{,}151$ splits (`4,273` `V8` buckets), $10{,}321 / 10{,}608$ 4-tuples | **$135{,}576\text{ B}$ (`V8`)**<br>$263{,}802\text{ B}$ (base) |

**Why Support-Partitioned Quarter-Product (`Q4`) Unranking Replaces Procedural `Lex` and Flat 8D LUTs:** Decoding $r \in \{0, \ldots, |C_{K_{\max}}|-1\}$ on a GPU faces a sharp cache-vs-compute dilemma. Procedural lexicographic unranking (`Lex`) walks $16$ coordinates sequentially via conditional dynamic-programming counts, creating a 16-stage serial dependency per packet. Conversely, tabulating all valid 8D half-block vectors directly (`DirectH8`) requires $3{,}192{,}305$ 4-byte entries $= 12.77\text{ MB}$ already at $K_{\max}=22$ (`2.50 bpw`), thrashing L1/shared cache and becoming prohibitive at $K_{\max}=45$ (`3.00 bpw`).

Our **Quarter-Product (`Q4`) bijection** resolves both bottlenecks by exploiting the weight-$4$ structure of $H_8$. For any 8D half-block $y_h = 2m_h + u_h$ with mask class $t_h \in \{0,1,2\}$:
- When $t_h = 0$ ($u_h = 0^8$) or $t_h = 1$ ($u_h = 1^8$), coordinates $(0..3)$ and $(4..7)$ form two homogeneous 4-tuples in $(2\mathbb{Z})^4$ or $(2\mathbb{Z}+1)^4$, respectively.
- When $t_h = 2$ ($\mathrm{wt}(u_h) = 4$), **exactly $4$ positions of $u_h$ are $0$ (even coordinates) and $4$ positions are $1$ (odd coordinates)**. Extracting the 4 even coordinates $a \in (2\mathbb{Z})^4$ in increasing index order and the 4 odd coordinates $b \in (2\mathbb{Z}+1)^4$ in increasing index order maps **all $14$ weight-$4$ cosets onto the exact same canonical 4D tables `lut4_even` and `lut4_odd`** (packed as `int8x4` `uint32` words: only $2{,}585 + 2{,}816 = 5{,}401$ words $= 21.6\text{ KB}$ at $K_{\max}=22$, and $10{,}321 + 10{,}608 = 20{,}929$ words $= 83.7\text{ KB}$ at $K_{\max}=45$). The specific coset mask $u_h$ is needed only at the final register FMA step to interleave bytes from the even and odd `uint32` words.

All mixed-radix divisions at decode time use precomputed 64-bit/32-bit magic multipliers $\operatorname{div\_magic}(x, d, M) = \lfloor (x \cdot M) / 2^{64} \rfloor$ (`__umul64hi` / `__umulhi`, zero hardware `/` or `%` instructions), accelerated by top-bits prefix brackets (`prefix12_lohi` on 16D subshells and 5-bit bucket brackets on 8D quarter-splits) that bound binary searches to $\leq 8\text{--}14$ subshell candidates and $\leq 5$ split steps (**Algorithm 1**).

```text
Algorithm 1: Quarter-Product (Q4) Hierarchical Unranking and Register Inner Product
Input:  Packed (b_code + 2)-bit word W_pkt (b_code in {38, 46}); matrix scale w_scale; radial LUT mu[0..3]; rotated activation x[0..15]
Output: Exact scalar inner product <w_hat, x> in FP32 registers (zero local-memory arrays)

1. Unpack u <- W_pkt >> b_code (2-bit radial selector) and lattice rank r <- W_pkt & ((1 << b_code) - 1).
2. 16D Subshell Lookup: Use top-bits bracket prefix_lohi[r >> shift] + bounded binary search on subshell_base[]
   to find subshell s with B_s <= r < B_{s+1}. Read packed metadata (q_0, q_1, tp, p_0, C_0, C_1) and magic (M_0, M_1).
3. Split 16D Rank into Coset Pair & Two 8D Half-Ranks (via __umul64hi magic division):
   Delta <- r - B_s;  tmp <- div_magic64(Delta, C_1, M_1);  r_1 <- Delta - tmp * C_1
   s_cos <- div_magic64(tmp, C_0, M_0);                      r_0 <- tmp - s_cos * C_0
   Recover RM(1,4) masks (u_0, u_1) and classes (t_0, t_1) from (tp, s_cos) (28 weight-4 pairs when tp=4, else 1).
4. For each half-block h in {0, 1} with (t_h, u_h, q_h, p_0, r_h):
   a. 8D Quarter-Split Lookup: Bracketed binary search on split_cum_base[] inside half8_splits(t_h, q_h, p_0)
      yields split offset c_sp, 4D table bases (base_a, base_b), count n_b, and 32-bit magic M_nb.
   b. Split 8D Rank into 4D Quarter Indices: rem <- r_h - c_sp;  i_a <- div_magic32(rem, n_b, M_nb);  i_b <- rem - i_a * n_b.
   c. Fetch Packed int8x4 Quarters: ev_pk <- lut4[odd_a(t_h)][base_a + i_a];  od_pk <- lut4[odd_b(t_h)][base_b + i_b].
   d. Register FMA: If t_h != 2, accumulate bytes 0..3 of ev_pk with x[8h + 0..3] and od_pk with x[8h + 4..7];
      else (t_h = 2), for k = 0..7 consume low int8 of od_pk if ((u_h >> k) & 1) else low int8 of ev_pk into dot_h.
5. Return (w_scale * mu[u]) * (dot_0 + dot_1).
```

**Three-Level Bijection Verification Scope:** Because full 16D state spaces reach $|C_{22}| = 2.49 \times 10^{11}$ (`2.50 bpw`) and $|C_{45}| = 6.87 \times 10^{13}$ (`3.00 bpw`), our standalone CPU verifier (`verify_bw16_supplement.py` / `bw16_repro.cuda_ext`) separates three distinct verification levels without conflating component and full-code coverage:
1. **Exhaustive 8D Component LUT Check (`2.50 bpw`):** `verify_q4_bw250_bijection_cpu` exhaustively verifies bijective inversion (`half8_lex_to_q4` $\leftrightarrow$ `half8_q4_to_lex`) across all **$3{,}192{,}305$ valid 8D half-block states** $(t, q_h, p_h, r_{\text{lex}})$ ($0$ mismatches).
2. **Sampled Full-16D Code Checks:** At $2.50\text{ bpw}$, `verify_q4_bw250_bijection_cpu` compares decoded 16D coordinates across $5$ positions per subshell over all $958$ subshells (**$4{,}790$ full-16D evaluations**, with duplicates on subshells of span $< 5$; $0$ mismatches). At $3.00\text{ bpw}$, `verify_q4_bijection_cpu(50000)` checks the $2$ boundary endpoints of all $3{,}983$ subshells (**$7{,}966$ boundary evaluations**) plus **$50{,}000$ deterministic SplitMix64 pseudorandom 16D lattice indices** across $[0, |C_{45}|-1]$ ($0$ mismatches).
3. **Realized Model-Weight Transcoding:** During matrix quantization, `transcode_bw16_{250_,}lex_to_q4_cpu(..., verify_identity=1)` verifies exact coordinate equality on realized quantized model weights ($0$ mismatches across all $254$ body and `V2` vocabulary matrices).

### 2.2 Stack-Free Bounded Plotkin/Wagner Encoding, Coupled Surrogate Audit, and Closed-Form `BlockAlign`

**Tiled `BlockLDLQ(b=8)` Coupled Surrogate Objective and Stack-Free Search:** Given clean rotated input Hessian $H_r = Q_K^\top \operatorname{diag}(s_u) H \operatorname{diag}(s_u) Q_K$ ($H = \frac{1}{N} X^\top X$, normalized by $\bar{H}_{\text{diag}}$ with ridge $\sigma_{\text{reg}} = 0.01$ so $H_{r,\text{reg}} = H_r / \bar{H}_{\text{diag}} + 0.01 I_K$), we factor $H_{r,\text{reg}} = L_{\text{chol}} L_{\text{chol}}^\top$ and form the unit-8-block-diagonal feedback matrix $L_{\text{b8}}$ by right-multiplying each 8-column strip of $L_{\text{chol}}$ by the inverse of its $8 \times 8$ lower-triangular diagonal block $D_{\ell}$. Columns are quantized right-to-left in **64-column `BlockLDLQ` tiles** ($4$ sixteen-weight blocks per tile); after encoding tile $t = [c_0 : c_1)$, the accumulated error $E_t = W_{\text{orig},t} - \widehat{W}_t$ updates all preceding columns via a single `cuBLAS` GEMM $W_{\text{work}, :c_0} \leftarrow W_{\text{work}, :c_0} + E_t L_{\text{b8}}[c_0:c_1, :c_0]$.

Within each 16D block with working target halves $(w_{0,\text{base}}, w_1)$, original upper half $w_{\text{orig},1}$, and intra-block $8 \times 8$ feedback block $L_{1\to 0} = L_{\text{chol}}[8:, :8]\,(L_{\text{chol}}[:8, :8])^{-1}$, both our GPU/CPU bounded encoder (**Algorithm 2**) and the reference solver `encode_bw16_exact_cvp_cpu` minimize the **diagonal-weighted, feedback-coupled block-surrogate objective** over $(y_0, y_1) \in C_{K_{\max}}$ and $u \in \{0,1,2,3\}$ ($s_u = w_{\text{scale}}\mu_u$):

$$
\mathcal{J}_{\text{surr}}(y_0, y_1, u) = \sum_{k=0}^{7} d_{1,k}\bigl(w_{1,k} - s_u y_{1,k}\bigr)^2 + \sum_{k=0}^{7} d_{0,k}\Bigl(\bigl(w_{0,\text{base}} + L_{1\to 0}^\top (w_{\text{orig},1} - s_u y_1)\bigr)_k - s_u y_{0,k}\Bigr)^2,
$$

where at $K_{\max} = 22$ (`2.50 bpw`) coordinates are weighted by mean-normalized Cholesky diagonal energies $d_{16} = \operatorname{diag}(L_{\text{chol}})^2 / \overline{\operatorname{diag}(L_{\text{chol}})^2}$, and at $K_{\max} = 45$ (`3.00 bpw`) unweighted Euclidean weights $d_{16} = \mathbf{1}_{16}$ are used. Crucially, exact 16D block quadratic loss $(w_{\text{orig}} - s_u y)^\top H_{\text{blk}} (w_{\text{orig}} - s_u y)$ under $H_{\text{blk}} = L_{\text{chol}} L_{\text{chol}}^\top$ equals $\|D_1^\top (w_{\text{orig},1} - s_u y_1)\|_2^2 + \|D_0^\top (w_{\text{orig},0} + L_{1\to 0}^\top(w_{\text{orig},1} - s_u y_1) - s_u y_0)\|_2^2$ with full $8 \times 8$ triangular factors $D_0 = L_{\text{chol}}[:8,:8]$ and $D_1 = L_{\text{chol}}[8:,8:]$, whereas $\mathcal{J}_{\text{surr}}$ replaces $D_{\ell} D_{\ell}^\top$ inside each 8D half-block by diagonal weights $\operatorname{diag}(d_{\ell})$ so that coordinates within each $(2\mathbb{Z})^8 + u_h$ coset decouple. Consequently, **exact global minimization of $\mathcal{J}_{\text{surr}}$ is not identical to exact minimization of full-block quadratic error $(w - \hat{w})^\top H_{\text{blk}} (w - \hat{w})$**.

Rather than maintaining per-thread candidate stacks in local memory, each warp assigns one lane $\ell \in \{0,\ldots,31\}$ to each $(u_1 \in H_8,\, p_1 \in \{0,1\})$ pair and evaluates $N_{\text{caps}}$ bounded $(2\mathbb{Z})^8 + u_1$ norm caps ($N_{\text{caps}}=6$ at $2.50\text{ bpw}$, $N_{\text{caps}}=3$ at $3.00\text{ bpw}$) entirely in registers (**Algorithm 2**).

```text
Algorithm 2: 32-Lane Stack-Free Bounded Plotkin/Wagner Lambda_16 Block Encoder
Input:  Target halves (wx0_base, wx1), original wo1, intra-block feedback L_{1->0}, weights d16=(d0, d1),
        scale w_scale, radial multipliers mu[0..3], shell cap K_max (N_max = 4 K_max), cap count N_caps in {3, 6}
Output: Found lattice point y* = (y0*, y1*) in C_{K_max} and 2-bit radial index u* in {0..3} under J_surr

Subroutine BoundedHalf2Z8(wx, d, s, u_mask, p_req, cap):
  1. Round each k in 0..7 to nearest coset integer: m_k <- rint(0.5 * (wx_k / s - u_k)), y_k <- 2 m_k + u_k.
  2. While ||y||_2^2 > cap: greedily step y_{k*} <- y_{k*} - 2 sign(y_{k*}) minimizing Delta_loss_k / Delta_||y||_2^2.
  3. If (sum_k m_k mod 2) != p_req: apply single step y_{k*} <- y_{k*} +/- 2 with ||y'||_2^2 <= cap minimizing Delta_loss_k.
  4. Return (feasible, y, ||y||_2^2, sum_k d_k (wx_k - s y_k)^2).

Main Warp Search (lane l in 0..31 -> h = l >> 1, p1 = l & 1, u1 = H8_MASK[h]):
  For radial index u in 0..3 (step size s = w_scale * mu[u]):
    Estimate lower-half energy n2_0 <- sum_k rint(wx0_base[k] / s)^2;  adapt_cap <- clamp(N_max - n2_0 + 8, 0, N_max);  mid_cap <- N_max/2 + 4.
    Obtain unconstrained upper-half norm n2_unc via BoundedHalf2Z8(wx1, d1, s, u1, p1, N_max).
    Define cap schedule C_caps = (N_max, adapt_cap, mid_cap) if N_caps=3 else (N_max, n2_unc-4, n2_unc-8, adapt_cap, adapt_cap-8, mid_cap).
    For distinct realized upper-half norms n2_1 from BoundedHalf2Z8(wx1, d1, s, u1, p1, cap) over cap in C_caps:
      Feedback upper residual into lower half: wx0 <- wx0_base + L_{1->0}^T (wo1 - s * y1);  rem_cap <- N_max - n2_1.
      For Plotkin extension ext in {0, 1} (u0 = u1 if ext=0 else ~u1):
        (ok0, y0, _, loss0) <- BoundedHalf2Z8(wx0, d0, s, u0, p1, rem_cap).
        If ok0: update thread best (loss0 + loss1, (y0, y1), u); warp-reduce minimum across lanes 0..31 via __shfl_xor_sync.
```

**Layer-00 Encoder Microbenchmark and Synthetic Block-Surrogate Audit:** On a Layer-00 7-projection microbenchmark on `Qwen3-8B` (`data/corpus_and_timing_records.json`: `stack_free_encoder_benchmark`), eliminating per-thread stack frames (`0 B` local stack, `0` register spills) reduces Layer-00 7-projection kernel time from **`491.98 ms` to `243.79 ms` ($2.018\times$ speedup)** at $2.50\text{ bpw}$ (`×36` projected 36-layer body total: `17.71 -> 8.78 s`) and from **`147.64 ms` to `100.36 ms` ($1.471\times$ speedup)** at $3.00\text{ bpw}$ (`×36` projected 36-layer body total: `5.31 -> 3.61 s`), with **`0` packed-byte mismatches** against our stack-based kernel across all $7$ Layer-00 projections.

To audit how closely `BoundedHalf2Z8` + $N_{\text{caps}} \in \{6, 3\}$ minimizes $\mathcal{J}_{\text{surr}}$ on a reproducible host test without GPU dependencies, `audit_bw16_bounded_vs_exact_cvp_cpu(n_blocks=16, seed=20261114)` (`codecs_and_fitting.py`) compares Algorithm 2 against `encode_bw16_exact_cvp_cpu` (exact 8D dynamic programming when $L_{1\to 0}=0$, and exact branch-and-bound over $(y_1, y_0, u) \in C_{K_{\max}} \times \{0..3\}$ under $\mathcal{J}_{\text{surr}}$ when $L_{1\to 0} \neq 0$) on $16$ synthetic i.i.d. $\mathcal{N}(0,1)$ weight blocks and a synthetic Wishart-like positive-definite block Hessian $H_{\text{blk}} = \frac{1}{16} A^\top A + 0.05 I_{16}$ ($A_{ij} \sim \mathcal{N}(0,1)$):
- **At $2.50\text{ bpw}$ ($K_{\max}=22, N_{\text{caps}}=6$, $d_{16} \propto \operatorname{diag}(L_{\text{chol}})^2$):** On decoupled isotropic blocks ($L_{1\to 0}=0, d_{16}=\mathbf{1}$), both solvers achieve $14.1825\text{ dB}$ SQNR (**`0.0000 dB` gap**, $16/16 = 100\%$ exact matches). On coupled blocks ($L_{1\to 0} \neq 0$), Algorithm 2 matches the global $\mathcal{J}_{\text{surr}}$ optimum on $13/16$ ($81.25\%$) blocks with a **`+0.0573 dB` coupled surrogate-objective gap** ($10\log_{10}(\sum \mathcal{J}_{\text{surr}}^{\text{bnd}} / \sum \mathcal{J}_{\text{surr}}^{\text{exact}}) \geq 0$). When those same $16$ assignments are evaluated under the full synthetic quadratic form $(w - \hat{w})^\top H_{\text{blk}} (w - \hat{w})$—which retains off-diagonal intra-8D $D_{\ell} D_{\ell}^\top$ terms omitted by $\mathcal{J}_{\text{surr}}$—bounded search attains $14.9183\text{ dB}$ vs. $14.4761\text{ dB}$ for exact $\mathcal{J}_{\text{surr}}$ minimization (**`-0.4423 dB`** difference $\text{SQNR}_{H,\text{exact-}\mathcal{J}} - \text{SQNR}_{H,\text{bnd}}$).
- **At $3.00\text{ bpw}$ ($K_{\max}=45, N_{\text{caps}}=3$, $d_{16}=\mathbf{1}$):** Bounded search matches exact $\mathcal{J}_{\text{surr}}$ minimization on $16/16$ ($100\%$) blocks in both regimes (**`0.0000 dB` isotropic gap** at $17.1225\text{ dB}$; **`0.0000 dB` coupled surrogate gap** and **`0.0000 dB` full-$H_{\text{blk}}$ SQNR difference** at $18.2551\text{ dB}$).
This CPU audit establishes near-exactness for $\mathcal{J}_{\text{surr}}$ on this small $16$-block synthetic Gaussian/Wishart sample only; it does not establish universal CVP optimality under full $H_{\text{blk}}$ or across calibrated real-model weight distributions.

**Closed-Form $4 \times 4$ Radial LUT Refit and Zero-Overhead `BlockAlign`:**
1. **$4 \times 4$ Radial Gram Refit:** Holding discrete assignments $(y, u)$ fixed after `BlockLDLQ`, writing $\widehat{W}_r(\alpha) = \sum_{q=0}^3 \alpha_q A_q$ ($\alpha_q = w_{\text{scale}} \mu_q$, where $A_q$ masks blocks with $u=q$) yields a $4 \times 4$ positive-definite linear system minimizing $\operatorname{tr}\!\bigl((W_r - \widehat{W}_r(\alpha)) H_r (W_r - \widehat{W}_r(\alpha))^\top\bigr)$:

$$
\alpha^\star = (G_H + \lambda_r I_4)^{-1}(b_H + \lambda_r \alpha_0), \qquad (G_H)_{pq} = \operatorname{tr}(A_p H_r A_q^\top), \quad (b_H)_p = \operatorname{tr}(W_r H_r A_p^\top), \quad \lambda_r = 10^{-8}\operatorname{tr}(G_H).
$$

2. **Closed-Form `BlockAlign` Calibration (`0` SGD Steps, $+0\text{ B}$ Metadata):** After quantizing all matrices, a single forward pass over the first $8$ calibration batches (`cal_128_ids[:32]`, $65{,}536$ tokens; or balanced `BalCal8` in §6 ablations) propagates both clean teacher hidden states $h_{\text{clean}}^{(\ell)}$ and quantized student hidden states $h_{\text{quant}}^{(\ell)}$, fitting one scalar multiplier per matrix folded directly into `FP32` $w_{\text{scale}}$ via the clipped Frobenius inner-product projection $\alpha^\star(U_{\text{tgt}}, U_q; [a,b]) = \operatorname{clip}\!\bigl(\langle U_{\text{tgt}}, U_q \rangle_F / \|U_q\|_F^2,\, a, b\bigr)$:
   - **`q_proj`, `k_proj` ($\alpha = 1.0$):** Kept at unity because `Qwen3-8B` applies per-head `q_norm` and `k_norm` `RMSNorm` immediately after projection, making attention logits invariant to positive global scalar rescaling of $W_q$ or $W_k$.
   - **`v_proj`, `gate_proj`, and `up_proj` ($\alpha \in [0.92, 1.08]$):** Fit locally on quantized layer-norm inputs $X_{\text{quant}}$ against clean sub-layer outputs ($U_{\text{loc}} = X_{\text{quant}} W_{\text{clean}}^\top$ vs. $U_q = X_{\text{quant}} \widehat{W}^\top$ for `v_proj` and `gate_proj`; post-`SiLU` `SwiGLU` product $\operatorname{SiLU}(X_{\text{quant}} W_{\text{gate}}^\top) \odot (X_{\text{quant}} W_{\text{up}}^\top)$ vs. $\operatorname{SiLU}(\alpha_{\text{gate}} X_{\text{quant}} \widehat{W}_{\text{gate}}^\top) \odot (X_{\text{quant}} \widehat{W}_{\text{up}}^\top)$ for `up_proj`).
   - **Residual-writing `o_proj` and `down_proj` ($\alpha \in [0.90, 1.10]$):** Combine the clipped local sub-layer ratio $\alpha_{\text{loc}} = \alpha^\star(U_{\text{loc}}, U_q; [0.90, 1.10])$ (clean sub-layer evaluated on quantized input) and the clipped clean-trajectory ratio $\alpha_{\text{cum}} = \alpha^\star(U_{\text{cum}}, U_q; [0.90, 1.10])$ ($U_{\text{cum}}$ captured from the clean sub-layer along the clean teacher trajectory) via the convex heuristic blend:

$$
\alpha = 0.70\,\alpha^\star(U_{\text{loc}}, U_q; [0.90, 1.10]) + 0.30\,\alpha^\star(U_{\text{cum}}, U_q; [0.90, 1.10]) \in [0.90, 1.10].
$$

   Because each ratio is clipped to $[0.90, 1.10]$ prior to mixing, $\alpha$ is a separately-clipped convex blend and is not in general the exact minimizer of the combined quadratic $0.70\|U_{\text{loc}} - \alpha U_q\|_F^2 + 0.30\|U_{\text{cum}} - \alpha U_q\|_F^2$ (for instance, unclipped ratios $0.80$ and $1.20$ yield combined quadratic minimizer $0.92$, whereas clipping each to $[0.90, 1.10]$ before blending gives $0.96$, though opposite-sign clipping shifts can cancel at specific ratios).

### 2.3 Sensitivity-Guided `Swap7_Plus24K` Bit Allocation and Four-Way Package Accounting

**Exact Weight Census and Unit Grid:** Each of the $36$ transformer blocks of `Qwen3-8B` contains $7$ bias-free linear matrices quantized in integer multiples of $U_0 = 2^{22} = 4{,}194{,}304$ weights ($1.31072\text{ MB/unit}$ at $2.50\text{ bpw}$; $1.572864\text{ MB/unit}$ at $3.00\text{ bpw}$; $\Delta = +262{,}144\text{ B/unit}$): `q_proj` $[4096, 4096]$ ($4$ units), `k_proj` $[1024, 4096]$ ($1$ unit), `v_proj` $[1024, 4096]$ ($1$ unit), `o_proj` $[4096, 4096]$ ($4$ units), `gate_proj` $[12288, 4096]$ ($12$ units), `up_proj` $[12288, 4096]$ ($12$ units), and `down_proj` $[4096, 12288]$ ($12$ units), totaling $46$ units $= 192{,}937{,}984$ weights/layer and **$1{,}656$ units $= 6{,}945{,}767{,}424$ body weights ($84.80\%$)** across $252$ matrices. Untied `embed_tokens` and `lm_head` ($[151936, 4096]$ each) add **$1{,}244{,}659{,}712$ vocabulary weights ($15.20\%$)**, and $145$ unquantized `BF16` `RMSNorm` vectors add $308{,}224$ parameters ($616{,}448\text{ B}$), giving **$8{,}190{,}735{,}360$ total parameters** ($16{,}381{,}470{,}720\text{ B} = 15.2564\text{ GiB}$ dense `BF16`).

**`Swap7_Plus24K` Allocation Rule and Explicit $36 \times 7$ Rate Maps:** Ranking all $252$ single-matrix $2.50 \to 3.00\text{ bpw}$ promotions purely by additive calibration $\Delta\mathcal{L}_{\text{KL}} / \text{unit}$ (`EndKL`) under-allocates contiguous early/mid `up_proj` blocks (whose `SwiGLU` errors interact multiplicatively) while missing high-leverage 1-unit `k_proj` promotions when starting from role-uniform anchors. `Swap7_Plus24K` constructs the two body budgets ($612$ vs. $1{,}044$ units at $3.00\text{ bpw}$) via two deterministic steps:
- **Step 1 (`Swap7` H-SQNR Outlier Exchange):** At `2.685 bpw` ($612$ units at $3.00\text{ bpw}$), start from the `VO+Down` anchor (all $36$ `v_proj` + $36$ `o_proj` + $36$ `down_proj` $= 612$ units), demote the $7$ easy `down_proj` outliers whose $2.50\text{-bpw}$ `H-SQNR` exceeds $24.0\text{ dB}$ (`L{1,2,3,6,16,34,35}`, $84$ units), and promote the $7$ lowest-`H-SQNR` contiguous mid-early `up_proj` layers (`L{6..12}`, $84$ units). At `2.815 bpw` ($1{,}044$ units at $3.00\text{ bpw}$), start from `VO+Up+Down` ($1{,}044$ units), demote the $7$ MLP outliers with $2.50\text{-bpw}$ `H-SQNR` $> 26.5\text{ dB}$ (`up_proj L{1,2}` + `down_proj L{1,2,6,16,35}`, $84$ units), and promote the $7$ lowest-`H-SQNR` `gate_proj` layers (`L{8,18,19,20,31,32,33}`, $84$ units).
- **Step 2 (`Plus24K` Marginal `Teacher-KL`/Unit Promotion):** Measure single-matrix $2.50 \to 3.00\text{ bpw}$ end-to-end `Teacher-KL` reduction per unit on $4{,}096$ calibration tokens (`cal_128_ids[:8, :512]`), demote the $2$ lowest-sensitivity remaining $3.00\text{-bpw}$ 12-unit MLP matrices ($24$ units: `down_proj L{10,11}` at `2.685`; `up_proj L{23}` + `down_proj L{10}` at `2.815`), and promote the top-$24$ 1-unit `k_proj` matrices ($24$ units).

This yields the exact $3.00\text{-bpw}$ layer index sets (all remaining body matrices use $2.50\text{ bpw}$; full JSON maps in `data/allocation_and_scales.json`):
- **Shared across `2.685` & `2.815`:** `v_proj` ($36$): `L0..35`; `o_proj` ($36$): `L0..35`; `q_proj` ($0$): $\varnothing$; `k_proj` ($24$): `L{1, 7..12, 14..16, 18..22, 24, 25, 27, 29, 31..35}` (leaving `L{0, 2..6, 13, 17, 23, 26, 28, 30}` at $2.50\text{ bpw}$).
- **`2.685 bpw` MLP ($130$ total $3.00\text{-bpw}$ matrices, $612$ units):** `gate_proj` ($0$): $\varnothing$; `up_proj` ($7$): `L{6..12}`; `down_proj` ($27$): `L{0, 4, 5, 7..9, 12..15, 17..33}` (leaving `L{1..3, 6, 10, 11, 16, 34, 35}` at $2.50\text{ bpw}$). Body payload $= 2{,}330{,}984{,}448\text{ B}$ + $5{,}040\text{ B}$ metadata ($252 \times 20\text{ B}$: `FP32` $w_{\text{scale}}$ + $4 \times \text{FP32}$ `mult_lut`) = **$2{,}330{,}989{,}488\text{ B}$** ($2.684788\text{ bpw}$).
- **`2.815 bpw` MLP ($166$ total $3.00\text{-bpw}$ matrices, $1{,}044$ units, $+113{,}246{,}208\text{ B}$):** `gate_proj` ($7$): `L{8, 18..20, 31..33}`; `up_proj` ($33$): all except `L{1, 2, 23}`; `down_proj` ($30$): all except `L{1, 2, 6, 10, 16, 35}`. Body payload $= 2{,}444{,}230{,}656\text{ B}$ + $5{,}040\text{ B}$ metadata = **$2{,}444{,}235{,}696\text{ B}$** ($2.815223\text{ bpw}$).
- **Matched Baseline Body Maps:** Each `QTIP-HYB` matrix stores $2{,}052\text{ B}$ metadata ($2{,}048\text{ B}$ `[512, 2]` `FP16` LUT + $4\text{ B}$ $w_{\text{scale}}$, $+512{,}064\text{ B}$ across $252$ matrices vs. `BW16`); demoting the two lowest-salience promoted `k_proj` layers `L{11, 18}` from $3.00$ (`KV=6`) to $2.50\text{ bpw}$ (`KV=5`, $-524{,}288\text{ B}$) places `QTIP-HYB` at **$-12{,}224\text{ B}$ ($-0.0004\%$)** strictly below `BW16`. `QuIP#` uses $\{2.00\text{ bpw E8P}, 3.00\text{ bpw E8-RVQ}\}$ sensitivity allocation (`CalAlloc`: $95$ vs. $77$ matrices at $2.00\text{ bpw}$ across `q_proj`, `k_proj`, `gate_proj`) with **identical raw body payload** to `BW16` and $1{,}008\text{ B}$ `w_scale` metadata ($-4{,}032\text{ B}$ vs. `BW16`).

**Non-Body Regimes (`U1` vs. `V2`) and Four-Way Package Census (Table 2):** Legacy `U1` (`NVFP-E8` $3.50\text{ bpw}$ on `embed_tokens` and SmoothQuant-preconditioned `lm_head`, $\alpha=0.50$) totals **$545{,}163{,}272\text{ B}$** ($544{,}538{,}624\text{ B}$ payload + $8{,}200\text{ B}$ scales + $616{,}448\text{ B}$ `RMSNorm`s). Unified **`V2`** (`V2_BW16_250_300`) quantizes `embed_tokens` in $2{,}374 \times 64$ strided-`Q64` row-group blocks ($Q_{64}$ on rows paired with $Q_{4096}$ across hidden channels) at $2.50\text{ bpw}$ ($194{,}478{,}080\text{ B}$ + $20\text{ B}$, seed `20261501`) and `lm_head` at $3.00\text{ bpw}$ ($233{,}373{,}696\text{ B}$ + $20\text{ B}$, seed `20261502`), plus $616{,}448\text{ B}$ `RMSNorm`s = **$428{,}468{,}264\text{ B}$** deterministic-seed / **$428{,}468{,}272\text{ B}$** ($+8\text{ B}$ explicit seeds), saving **$116{,}695{,}008\text{ B}$ / $116{,}695{,}000\text{ B}$ ($116.70\text{ MB}$, $-21.41\%$)** over `U1`. **Table 2** separates logical tensor bytes, serialized archive construction, shared runtime binaries/LUTs, and resident `RTX 5090` VRAM (`+196,608 B` under `V2` from padding `lm_head` by $+128$ zero rows to $152{,}064 = 2{,}376 \times 64$, plus $+2.25\text{ MiB}$ permanent decode scratch or $+96.0\text{ MiB}$ transient `Mode-2` prefill workspace).

| Complete Package (Table 2: 36-Layer `Qwen3-8B`) | (1) Complete-Package Logical Tensor Bytes (Body + Meta + Non-Body) | (2) Cleanroom / Donor Construction Path | (3) Shared Runtime `.so` & Device LUT Footprint (Excluded from Logical Bytes) | (4) Measured Incremental `RTX 5090` VRAM (`Mode-1` Decode / `+96 MiB` `Mode-2` Prefill) |
| :--- | :---: | :--- | :--- | :---: |
| **`BW16 2.685 + U1`** | $2{,}876{,}152{,}760\text{ B}$ ($2.6786\text{ GiB}$) | `bw16_repro` CLI (`--body-codec bw16 --vocab-regime u1 --budget 2685`) or preserved donor `.pt` reload (App. B) | `bw16_q4` + `u1` `.so` + `V8` LUTs ($178{,}236\text{ B}$) + `E8` catalog ($40{,}246\text{ B}$) | $2{,}879{,}015{,}936\text{ B}$ ($2.6813\text{ GiB}$) /<br>$2{,}979{,}679{,}232\text{ B}$ ($2.7750\text{ GiB}$) |
| **`BW16 2.815 + U1`** | $2{,}989{,}398{,}968\text{ B}$ ($2.7841\text{ GiB}$) | Same (`--budget 2815`, $+113{,}246{,}208\text{ B}$ $3.00\text{-bpw}$ body upgrade) | Same as `2.685 + U1` | $2{,}992{,}262{,}144\text{ B}$ ($2.7868\text{ GiB}$) /<br>$3{,}092{,}925{,}440\text{ B}$ ($2.8805\text{ GiB}$) |
| **`BW16 2.685 + V2`** | $2{,}759{,}457{,}752\text{ B}$ (det)<br>$2{,}759{,}457{,}760\text{ B}$ (exp, $2.5699\text{ GiB}$) | `bw16_repro` CLI (`--vocab-regime v2 --budget 2685`, $-116.70\text{ MB}$ vs. `2.685 + U1`) | `bw16_q4` `.so` + `V8` LUTs ($178{,}236\text{ B}$: $42.7 + 135.6\text{ KB}$; $0\text{ B}$ `U1` LUTs) | $2{,}762{,}946{,}560\text{ B}$ ($2.5732\text{ GiB}$) /<br>$2{,}863{,}609{,}856\text{ B}$ ($2.6669\text{ GiB}$) |
| **`BW16 2.815 + V2`** | $2{,}872{,}703{,}960\text{ B}$ (det)<br>$2{,}872{,}703{,}968\text{ B}$ (exp, $2.6754\text{ GiB}$) | `bw16_repro` CLI (`--vocab-regime v2 --budget 2815`, **$-3.45\text{ MB}$** vs. `2.685 + U1`) | Same as `2.685 + V2` | $2{,}876{,}192{,}768\text{ B}$ ($2.6787\text{ GiB}$) /<br>$2{,}976{,}856{,}064\text{ B}$ ($2.7724\text{ GiB}$) |
| **`QTIP-HYB 2.685 / 2.815 + V2`** | $2{,}759{,}445{,}536\text{ B}$ / $2{,}872{,}691{,}744\text{ B}$ (exp, $-12{,}224\text{ B}$ vs. `BW16`) | `bw16_repro` CLI (`--body-codec qtip_hyb --vocab-regime v2`) | `bw16_q4` + `qtip_hyb` `.so` + `V8` vocab LUTs ($178{,}236\text{ B}$) | $2.5731\text{ GiB}$ & $2.6785\text{ GiB}$ /<br>$+96.0\text{ MiB}$ prefill |

### 2.4 Dual-Mode `sm_120` Compressed Execution Stack

All $254$ linear and vocabulary matrices execute from compressed VRAM without materializing $15.26\text{ GiB}$ of dense weights (`bw16_repro/runtime_modules.py`):
1. **`DualModeV6BW16Linear` (`V8_Hybrid`):**
   - **`Mode 1` ($M_{\text{tok}} \leq 16$, Autoregressive Decode):** Fuses input sign-flip, full-dimensional Kronecker RHT ($H_{4096}$ shared-memory butterfly at $K=4{,}096$; $12 \times H_{1024}$ followed by Paley-$12$ $H_{12}^\top$ at $K=12{,}288$), and warp-swizzled `FP32` staging (`p14_cuda_rht_in_bf16_to_f32_swiz_kernel`), executes zero-workspace `Q4` register-decoded GEMV (`B=1` dynamic-shared-memory reduction; `2 <= B <= 16` 2-row warp-unrolled `V8` kernel amortizing each `Q4` packet unrank across all $B$ batch columns in registers via an $80\text{-byte}$-strided bank-conflict-free shared activation tile), and applies fused `FP32 -> BF16` full-dimensional output RHT ($H_{1024}$, $H_{4096}$, or $H_{12} \otimes H_{1024}$; $< 10^{-4}$ relative error vs. exact `FP32` rotated reference).
   - **`Mode 2` ($M_{\text{tok}} > 16$, Prefill & $B=32$ Generation):** Applies full-dimensional `BF16` input RHT, unpacks rotated `BF16` weights (`0.0` max abs diff vs. reference rotated `BF16`) into one shared transient buffer `w_rot_shared_flat` ($12{,}288 \times 4{,}096 \times 2\text{ B} = 96.0\text{ MiB}$), runs `cuBLAS` `BF16` Tensor-Core GEMM, and applies full-dimensional `BF16` output RHT ($8.75\times\text{--}9.28\times$ $L=512$ prefill speedup over `Mode 1`; empirical `Mode-1`-vs-`Mode-2` relative Frobenius difference $0.0029\text{--}0.0041$ from `BF16` intermediate rounding). During `Mode-1` decode, `w_rot_shared_flat` is detached ($+2.25\text{ MiB}$ permanent scratch).
2. **`CompressedBW16VocabLMHead` ($3.00\text{ bpw}$) & `FastCompressedBW16VocabEmbedding` ($2.50\text{ bpw}$):**
   - `CompressedBW16VocabLMHead` dispatches `Mode 1` at $M \leq 16$ ($558.45\ \mu\text{s}$ at $M=1$, $2{,}117.94\ \mu\text{s}$ at $M=16$) and $192$-block ($96.0\text{ MiB}$) tiled `Mode 2` at $M > 16$ ($6.66\text{ ms}$ vs. $182.05\text{ ms}$ at $M=512$, $27.32\times$), while improving clean-$H_{\text{head}}$ `H-SQNR` over $3.50\text{-bpw}$ `U1` by **$+1.69\text{ dB}$ ($26.12$ vs. $24.43\text{ dB}$)** at $-38.90\text{ MB}$ fewer logical bytes.
   - `FastCompressedBW16VocabEmbedding` eliminates `torch.unique` host-device syncs and intermediate `index_select` copies via direct `torch.bmm` over `w_rot_shared_flat`, matching the reference compressed embedding bit-for-bit ($\text{max\_abs\_diff} = 0.0$), accelerating $2.50\text{-bpw}$ lookup by **$1.833\times$ at $M=1$ ($173.41 \to 94.62\ \mu\text{s}$)** and **$1.730\times$ at $M=2048$ ($2{,}648.36 \to 1{,}530.94\ \mu\text{s}$)**, and cutting $M=2048$ transient memory by $56.7\%$ ($50.90 \to 22.05\text{ MB}$). Because each touched $64$-row `Q64` block ($81{,}920\text{ B}$) is unpacked before row selection, $M=1$ lookup remains slower than `U1` ($94.62$ vs. $23.66\ \mu\text{s}$), leaving $B=1$ whole-model decode ($23.04\text{--}23.77\text{ ms}$) slightly behind `U1` ($22.40\text{ ms}$); a fused single-row `Q64` gather kernel is a natural optimization for $B \leq 4$.

---

## 3. Cleanroom Reconstruction Release, Evaluation Protocol, and Evidence Boundaries

### 3.1 Public Datasets, Single-Temperature Calibration, and Evaluation Benchmarks

All experiments evaluate public `Qwen/Qwen3-8B` (`BF16`, HuggingFace commit `b968826d9c46dd6066d109eabc6255188de91218`) on one `NVIDIA GeForce RTX 5090` ($32\text{ GB}$, CUDA `12.8`, `sm_120`). `bw16_repro/datasets_and_panels.py` reconstructs and SHA-256 verifies every token tensor and prompt panel directly from public HuggingFace datasets (`data/calibration_and_panel_manifests.json`):

1. **Calibration Splits (`cal_128_ids` & `BalCal8`):** `cal_128_ids` (`SHA256 9736b152...`) comprises $128 \times 2{,}048 = 262{,}144$ tokens ($32$ 4-window batches): $64$ `WikiText-2` (`Salesforce/wikitext` `wikitext-2-raw-v1` train) windows (`0..15`) + $64$ `C4` (`allenai/c4` `en/c4-train.00000-of-01024`) windows (`16..31`) used for clean layer input Hessians $H$. Stage-I `BlockFit` baselines (`QTIP-HYB`, `QuIP#`, `BW16-BlockFit`) use batches `0..7` ($6$ train / $2$ `val_16k`); domain-balanced comparators (`BalCal8`: `D4`, `BW16-BalBlockFit`, `TrajRadAlign`, `E2E-HybridCal`, and `V2` vocabulary screening) use batches `[0,1,2,16,17,18]` ($49{,}152$ tokens, $50\%$ `WT2` / $50\%$ `C4`) and validate on `[3,19]` ($16{,}384$ tokens).
2. **Disjoint Single-Temperature Calibration (`cal_temp_ids`) [15]:** Following `discrete_descent` [15]—who showed on post-trained `Qwen3` models that fixed $T=1$ perplexity flatters quantization noise and misranks PTQ methods—`cal_temp_ids` (`SHA256 07835b3c...`) holds $4 \times 2{,}048 = 8{,}192$ disjoint `C4` train tokens ($0$ overlap with `WT2-32` or `C4-32`). Scalar temperature $T^* \in [0.80, 1.60]$ ($\Delta T = 0.02$) minimizes `cal_temp_ids` NLL ($T^* = 1.22$ on `Dense_BF16`; $T^* = 1.20$ across all evaluated `2.685`/`2.815` packages) and is applied **without re-fitting** to `WT2-32` and `C4-32`.
3. **Held-Out Corpora (`WT2-32` & `C4-32`):** `WT2-32` (`SHA256 a5ad4cec...`, $32 \times 2{,}048$ `wikitext-2-raw-v1` test windows, $65{,}504$ predicted tokens) and `C4-32` (`SHA256 9e3a938e...`, $32 \times 2{,}048$ `c4-validation.00000-of-00008` windows, $65{,}504$ predicted tokens) report `Raw-NLL` ($T=1.0$), `Fit-NLL` ($T=T^*$), and full-$151{,}936$-vocab `Teacher-KL` $D_{\mathrm{KL}}\!\bigl(p_{\text{BF16}}(1) \,\|\, p_q(1)\bigr)$ in `FP32`/`FP64`. `Dense_BF16` ($T^*=1.22$): `WT2-32` `Raw/Fit-NLL` $= 2.281186 / 2.175793$ ($\text{PPL } 9.7883 / 8.8092$); `C4-32` `Raw/Fit-NLL` $= 2.760835 / 2.670897$ ($\text{PPL } 15.8130 / 14.4529$).
4. **Autoregressive Generation Panels (Table 3):**

| Panel ID & Role | Public Source & Exposure Status | Prompt, Greedy Decoding, & Execution Mode | Cap, Stopping Rule, Scorer (`scorers_and_stats.py`), & Margin |
| :--- | :--- | :--- | :--- |
| **`GSM8K-256`** ($256$ items; Primary Codec Check) | `openai/gsm8k` test indices `128..383` (`SHA256 db972eec...`, $0$ overlap with `0..127` dev slice; scanned in community baselines) | 8-shot CoT, greedy argmax, left-padded, $8 \times (B = 32)$ batches (unpacked `BF16` and live `Mode 2` $M = 32$) | $\text{cap} = 256$; stop from step $16$ every $8$ steps on `Question:` / EOS or `#### <num>\n`; numeric match after `####` (with last-number fallback when `####` is absent, $\text{tol}=10^{-4}$). `Dense_BF16` $= 228/256\ (89.06\%)$. Retention margin: **$-5.0\text{ pp}$**. |
| **`N512`** ($512$ items; Primary Factorial Retention) | $512$ `openai/gsm8k` train items (programme-exposed in earlier `4B`/`8B` screens; **not** an unexposed split) | 5-shot `GSM8K-train`, greedy argmax, `KV16`, longest-first $32 \times (B = 16)$ batches (`Mode-2` prefill + **live `Mode-1` $M=16$ decode**) | $\text{cap} = 512$; stop on `["Question:", "</s>", "<|im_end|>"]`; normalized exact match on **first `#### <num>` match** (returns `"[invalid]"` if `####` is absent; **no** last-number fallback; `no_stop` caps tracked). `Dense_BF16` $= 475/512\ (92.77\%, 3\text{ caps})$. Non-inferiority margin: **$-3.5\text{ pp}$**. |
| **`M256*`** ($256$ items; **Exploratory Secondary**) | $256$ integer-answer `EleutherAI/hendrycks_math` train Level 4–5 items (**programme-exposed; exploratory secondary readout only**) | 4-shot Minerva, greedy argmax, `KV16`, longest-first $16 \times (B = 16)$ batches (`Mode-2` prefill + **live `Mode-1` $M=16$ decode**) | $\text{cap} = 1024$; stop on `["Problem:"]`; Minerva `\boxed{}` / integer exact match (`no_stop` caps $37.1\%\text{--}52.0\%$). `Dense_BF16` $= 136/256\ (53.12\%, 108\text{ caps})$. Exploratory margin: **$-5.0\text{ pp}$**. |

### 3.2 Statistical Resampling and Serving Benchmark Protocol

- **Paired Bootstrap & Exact McNemar Tests ($10{,}000$ Resamples):** Corpus contrasts resample **$2{,}048$-token windows ($N = 32$)** $10{,}000$ times (`Fit-NLL` retention requires $95\%$ CI upper bound $< +0.015\text{ nats}$). Generation contrasts resample **items ($N = 256$ or $512$)** $10{,}000$ times with exact two-sided binomial McNemar $p$-values on discordant pairs $(n_{10}, n_{01})$. Contrasts are **statistically resolved** only when the $95\%$ CI excludes $0$ (and McNemar $p < 0.05$ for binary panels). Reproducible per-family `np.random.default_rng` seeds (`audit_retained_outcomes_and_corpora` in `scorers_and_stats.py`): `20261509` (common-`V2` corpus/`GSM8K-256` via `make_corpus_gsm_boot_indices`, plus live `N512` and its 4-arm interaction via `paired_binary_contrast_pp` / `four_arm_interaction_bootstrap_pp`), `20261510` (live `M256*` and its 4-arm interaction $\Delta_{\text{int}}$ via `four_arm_interaction_bootstrap_pp`), `20261436` (legacy `U1` corpus/`GSM8K-256`), `20261437` / `20261438` (unpacked `U1` `N512` / `M256*`), and `20261439` / `20261501` / `20261509` (§6 post-hoc recipes).
- **Serving Latency Harness (`benchmark_decode_serving_gpu` in `build_and_eval_cli.py`):** Whole-model step latencies run over distinct `C4` contexts (`c4_val_ids[:B, :512]`): `Mode-1` decode ($B \in \{1,2,4,8,16\}$, $L=512$, `w_rot_shared_flat = None`, $3$ warmup + $16$ timed decode steps bracketed by `torch.cuda.synchronize()` and `torch.cuda.Event(enable_timing=True)`), and `Mode-2` prefill (`w_rot_shared_flat` $96.0\text{ MiB}$ attached, $2$ warmup + $6$ timed passes). Direct two-rate build timers vs. stage-assembled single-package prorations are reconciled in **Appendix A**.

### 3.3 Self-Contained Supplement (`bw16_repro`) and Explicit Evidence Boundaries

Unlike partial script snapshots that depend on private workspace modules or pre-quantized `.pt` donors, the MIT-licensed supplement (`supplement.zip`, $< 2.0\text{ MB}$ expanded across $21$ files, zero private imports) ships a complete standalone package (`Table 4`):

```bash
# 1. Standalone CPU verification (0 GPU usage; compiles both .cu libraries via nvcc when available)
CUDA_VISIBLE_DEVICES="" python3 verify_bw16_supplement.py

# 2. Cleanroom end-to-end GPU reconstruction, Mode-1 reference check, & evaluation from pinned Qwen/Qwen3-8B
python3 -m bw16_repro.build_and_eval_cli --model-id Qwen/Qwen3-8B \
  --body-codec bw16 --vocab-regime v2 --budget 2815 --eval-corpora --bench-serving
python3 -m bw16_repro.build_and_eval_cli --model-id Qwen/Qwen3-8B \
  --body-codec qtip_hyb --vocab-regime v2 --budget 2815 --eval-corpora --bench-serving
```

| Supplement Component (Table 4) | Included Modules & Data Files | Verified Reproducibility Scope |
| :--- | :--- | :--- |
| **`verify_bw16_supplement.py` & `bw16_repro/cuda/`** | `bw16_q4_and_rht_kernels.cu`, `baselines_and_u1_kernels.cu`, `cuda_ext.py` | Exact $RM(1,4)$ DP shell counts (`Table 1`); `nvcc` host compilation; `3,192,305` exhaustive 8D `half8` LUT + `4,790` (`2.50`) and `7,966 + 50,000` (`3.00`) 16D bijection checks; CPU `QTIP-HYB` Viterbi, serialization, & `39`-check tolerance-pipeline verification. |
| **`bw16_repro/` Python Pipeline** | `datasets_and_panels.py`, `transforms_and_alloc.py`, `codecs_and_fitting.py`, `runtime_modules.py`, `scorers_and_stats.py`, `build_and_eval_cli.py` | Pinned `Qwen3-8B` & dataset SHA-256 builder; synthetic bounded-vs-exact block-surrogate CVP audit (§2.2); GPU `BW16` / Stage-I `QTIP-HYB` (`PCG + Adam`) / Stage-I `QuIP#` / `V2` / `U1` encoders & `BlockAlign`; all-254 `Dual-Mode` runtime + independent `Mode-1`/`Mode-2`-vs-PyTorch `F.linear` check; incremental JSON persistence, tolerance evaluator, & bootstrap verifier. |
| **`data/` Retained & Cleanroom Records** | `allocation_and_scales.json`, `calibration_and_panel_manifests.json`, `corpus_and_timing_records.json`, `generation_outcomes_and_excerpts.json`, `cleanroom_gpu_verification_v4.json` | Complete $36 \times 7$ rate maps & `BlockAlign`/`D4` scales; `U1` & `V2` 32-window `WT2-32`/`C4-32` arrays; all `15,360` scored item rows across `49` evaluations (`29` `GSM8K-256`, `11` `N512`, `9` `M256*`) + primary live excerpt pools; Layer-00 microbenchmarks, per-launch serving records, & cleanroom `BW16 2.815 + V2` GPU verification records (`39/39` tolerance checks). |

**Three Explicit Evidence Boundaries:**
1. **Truncated Saved Generation Excerpts & `B=1` CLI Demonstration Scope:** Historical evaluation logs persisted extracted answers (`pred` / `pred_num`), exact-match flags, stop/cap indicators, token counts, and `360`-char (`GSM8K-256`) or `400`-char (`N512`, `M256*`) text prefixes—not full $512$/$1{,}024$-token strings or token IDs (and `Live_BW16_BlockAlign_2685_U1` plus two contextual `Qwen3-4B GPTQ4` baselines on `N512`, bundled in `data/generation_outcomes_and_excerpts.json`, recorded `exact`/`no_stop`/`gen_tokens` from an earlier identical-stack pass without `pred` strings). Thus `(pred, target) -> exact` equivalence is $100\%$ recomputable across all $13{,}824$ rows with saved `pred`, and raw-text re-extraction verifies at $100\%$ on complete traces terminating within $360$/$400$ chars ($78.1\%\text{--}80.5\%$ of live `GSM8K-256`, $86.1\%\text{--}87.5\%$ of live `N512`, $28.1\%\text{--}28.5\%$ of long/capped `M256*`), whereas raw text beyond char $360$/$400$ was not retained. The `--eval-panels` flag in `build_and_eval_cli.py` is a single-stream `B=1` greedy demonstration and does not replay historical batched `B=16/32` greedy trajectories.
2. **Cross-Launch vs. Within-Launch Serving Repeats & Single-System Cleanroom Launch Scope:** Historical `BW16-BlockAlign + U1` was timed across **$4$ independent GPU launches**, whereas the matched `FastEmb` common-`V2` multi-codec comparison (`BW16 + V2`, `D4 + V2`, `QTIP-HYB + V2` in §4.2) remains scoped to **$1$ historical GPU launch** (`launch_4_common_v2`: $2$ separate within-launch passes `Pass A` and `Pass B` on `BW16 + V2`; $1$ pass on `D4 + V2` and `QTIP-HYB + V2`) persisting $16$-step CUDA-event means. The cleanroom `v4` verification continuation (`Run 2`, **Appendix C**) adds one independent single-system GPU launch for `BW16-BlockAlign 2.815 + V2` alone (without re-timing `D4 + V2` or `QTIP-HYB + V2`), logging per-step `1..16` CUDA-event latencies, SD, IQR, and resident VRAM with and without the shared `96.0 MiB` prefill buffer.
3. **CPU Verifier Scope & Cleanroom GPU Reconstruction Summary (Full Account in Appendix C):** Default CPU execution of `verify_bw16_supplement.py` exercises DP shell enumeration, `nvcc` host compilation, `Q4` bijection/Viterbi checks, incremental JSON serialization & `39`-check tolerance-pipeline verification, pure-Python codebook builders, four-way byte accounting, the $16$-block synthetic surrogate-CVP audit, and retained-outcome bootstrap statistics (`--verify-datasets-cpu` additionally checks pinned checkpoint metadata and cached `WT2`/`C4` token-split SHA-256 digests on CPU); executing CUDA kernels and full 36-layer reconstruction requires a GPU. On `RTX 5090` (`sm_120`, zero private `.pt` donors, `data/cleanroom_gpu_verification_v4.json`), cleanroom GPU reconstruction of `BW16-BlockAlign 2.815 + V2` ran in two documented invocations (**Appendix C**): after an initial invocation (`Run 1`) failed a mis-copied `v3` target file and exited via a `KeyError` before `Mode-1` tolerance assertion or live `All 254` evaluation, a reconciled continuation (`Run 2`, unchanged recipe and tolerances) passed all **`39/39` tolerance-based checks** ($0\text{ B}$ byte delta, $0$ `Lex -> Q4` mismatches across $254$ matrices, live `WT2-32`/`C4-32` `Fit-NLL` within $+0.000148 / -0.000261\text{ nats}$—establishing tolerance-based numerical agreement rather than bitwise weight identity—and $B=16$ decode $41.920\text{ ms/step}$); primary multi-system tables in §§4–6 report the retained evaluation archives.

---

## 4. Layer-00 Kernel Microbenchmarks, Baseline Validation, and Multi-Codec Comparison

### 4.1 Layer-00 7-Projection GEMV Microbenchmark (`Q4` Isolation) and `QTIP-HYB` Baseline Fidelity

Before comparing measured 36-layer complete packages (§4.2), we isolate the `Q4` representation on a Layer-00 7-projection microbenchmark on identical quantized weights and validate that our `QTIP-HYB` implementation is a faithful Stage-I baseline (`data/corpus_and_timing_records.json`):

1. **Layer-00 7-Projection GEMV Microbenchmark (`Q4` vs. `DirectH8` & Procedural `Lex`, Figure 1a):** Holding the quantized $\Lambda_{16}$ weights and full-dimensional Kronecker RHT fixed across the $7$ linear projections of `Qwen3-8B` Layer `00` ($\text{max\_abs\_diff} = 9.54 \times 10^{-7}$), replacing procedural 16-coordinate DP unranking (`Lex`) with `Q4` quarter-product indexing (`65.7 KB` base LUT) reduces measured Layer-00 7-projection GEMV sum at $B \in \{1, 4, 8\}$ from **`876.11 / 2,293.67 / 4,272.54 µs` to `356.93 / 725.60 / 1,252.61 µs` (`2.455x / 3.161x / 3.411x` speedup; `×36` projected 252-projection body estimate: `31.54 / 82.57 / 153.81 ms -> 12.85 / 26.12 / 45.09 ms`)** at $2.50\text{ bpw}$ ($1{,}081\text{ GB/s}$ effective logical `BF16` bandwidth on Layer `00` at $B=1$), and from **`1,355.39 / 2,042.85 / 3,014.63 µs` to `734.27 / 1,020.88 / 1,546.60 µs` (`1.846x / 2.001x / 1.949x` speedup; `×36` projected body estimate: `48.79 / 73.54 / 108.53 ms -> 26.43 / 36.75 / 55.68 ms`)** at $3.00\text{ bpw}$ (`263.8 KB` base LUT). Against flat $12.77\text{ MB}$ 8D half-block lookup (`DirectH8`, **`456.70 / 738.55 / 1,231.35 µs`** on Layer `00`; `×36` projected: `16.44 / 26.59 / 44.33 ms` at $2.50\text{ bpw}$), `Q4` is **`1.280x` faster at $B=1$** while using **$194\times\text{--}299\times$ less table memory** (`65.7 KB` base / `42.7 KB` `V8`) and scaling cleanly to $3.00\text{ bpw}$ ($K_{\max}=45$). Because all `ms` totals in `q4_vs_lex_and_qtip_body_gemv_250` are exact $36\times$ arithmetic projections of Layer-`00` sums (`layer00_7proj_sum_us * 36 / 1000`), they isolate per-layer kernel mechanics on repeated matrix shapes rather than measuring cross-layer codeword-dependent lookup variance or L2 residency across $36$ distinct layers; end-to-end 36-layer serving is measured directly in §4.2.
2. **`QTIP-HYB` Stage-I Fidelity and Explicit Departure from Full Published `QTIP` [8]:** Full published `QTIP` [8] (Appendix A.3.5, following `QuIP#` [3] §5) follows layerwise quantization with **multi-GPU whole-model end-to-end SGD recovery** of continuous parameters. We **omit whole-model end-to-end SGD across all codecs** to compare single-GPU calibration-light PTQ pipelines on equal footing (consistent with §6, where whole-model tuning on small `Qwen3-8B` calibration slices degrades held-out `C4` `Raw-NLL`). Within that Stage-I scope (two-sided full-dimensional RHT + `BlockLDLQ` tail-biting $L=16, V=2, Q=9$ Viterbi search + `[512, 2]` `FP16` per-matrix LUT refit + sequential within-block `BlockFit` + `BlockAlign`), three checks establish baseline quality and kernel competitiveness:
   - *Exact Viterbi & Unpack Agreement:* GPU `launch_qtip_hyb_l16_v2_fast_viterbi_batch_cuda` matches host `qtip_hyb_l16_v2_viterbi_batch_cpu` with **`0` byte and `0` state mismatches** across $\mathrm{KV} \in \{4, 5, 6\}$ (`2.0 / 2.5 / 3.0 bpw`), achieving `14.32 dB` (`2.50 bpw`) and `16.86 dB` (`3.00 bpw`) unweighted i.i.d. Gaussian SQNR on the initial 24-step Lloyd–Max LUT (`RMS = 1.000012`), `26.60 / 29.69 / 32.60 dB` Layer-00 `q_proj` `H-SQNR`, `< 10^{-6}` batched GEMV relative error across $B \in \{1..16\}$, and `0.0026–0.0032 dB` 252-matrix `H-SQNR` reproducibility across independent 36-layer rebuilds.
   - *Quality Superiority Where Theory Predicts:* Stage-I `QTIP-HYB` achieves **statistically resolved lower `Teacher-KL` across both corpora and rates** under both `V2` (Table 5) and legacy `U1` (§4.3).
   - *Internal Kernel-Implementation Context:* Our `StripSplit` `sm_120` `QTIP-HYB` kernel (`baselines_and_u1_kernels.cu`) is an independent `Qwen3-8B` realization rather than `QTIP`'s published `Llama-2` bitshift kernel [8], so its latency establishes non-strawman competitiveness on our execution stack rather than published-`QTIP` kernel equivalence. Specifically, in the pre-`V8` Layer-00 7-projection microbenchmark above, `StripSplit` `QTIP-HYB` ran in **`421.70 / 472.88 / 760.65 µs`** at $B \in \{1, 4, 8\}$ (`×36` projected body estimate: **`15.18 / 17.02 / 27.38 ms`**, or $2.07\times / 2.59\times / 4.11\times$ unpacked Layer-00 `BF16 cuBLAS` **`203.91 / 182.71 / 185.26 µs`** [`×36` projected: `7.34 / 6.58 / 6.67 ms`]), **outperforming** early `Q4_BW16_250_V2` at $B=4$ (`472.88` vs. `725.60 µs`; `×36` est. `17.02` vs. `26.12 ms`) and $B=8$ (`760.65` vs. `1,252.61 µs`; `×36` est. `27.38` vs. `45.09 ms`). Only after adding **`V8_Hybrid` 2-row warp-swizzled batch reuse**—which decodes each `Q4` lattice packet once in registers for all $B \leq 16$ columns without per-group $L$-bit window hashing or $512$-entry shared-memory LUT gathers—does `BW16` surpass our `QTIP-HYB` kernel across $B \in \{1,\ldots,16\}$ in measured 36-layer serving (§4.2).

### 4.2 Common-`V2` End-to-End Comparison (All 254 Matrices Compressed in VRAM)

**Table 5**, **Figure 1(b–c)**, and **Figure 2(a)** compare default **`BW16-BlockAlign + V2`**, single-pass residual-refitted **`BW16-OnePassCleanChol (D4) + V2`** (§6), and Stage-I fitted **`QTIP-HYB + V2`** on the shared `V2` vocabulary (matched within $12{,}224\text{ B} = 0.0004\%$). In Launch 4, `BW16-BlockAlign + V2` (`FastEmb`) was timed in two separate within-launch passes (`Pass A` alongside `D4 + V2`; `Pass B` immediately alongside `QTIP-HYB + V2`); Table 5 reports both passes explicitly.

| Evaluation Dimension (Table 5: 36-Layer `Qwen3-8B`, Common `V2`) | **Default `BW16-BlockAlign + V2` (`2.685 / 2.815 bpw`)** | **Comparator `BW16-OnePassCleanChol (D4) + V2` (`2.685 / 2.815 bpw`)** | **Stage-I Fitted `QTIP-HYB + V2` ($L=16, V=2, Q=9$, `2.685 / 2.815 bpw`)** |
| :--- | :---: | :---: | :---: |
| **Logical Complete-Package Bytes (`exp-seed`)** | $2{,}759{,}457{,}760\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}703{,}968\text{ B}$ ($2.6754\text{ GiB}$) | $2{,}759{,}457{,}760\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}703{,}968\text{ B}$ ($2.6754\text{ GiB}$) | $2{,}759{,}445{,}536\text{ B}$ ($2.5699\text{ GiB}$)<br>$2{,}872{,}691{,}744\text{ B}$ ($2.6754\text{ GiB}$) |
| **Measured Incremental VRAM (`Decode / Prefill`)** | $2.5732 / 2.6669\text{ GiB}$ (`2.685`)<br>$2.6787 / 2.7724\text{ GiB}$ (`2.815`) | $2.5732 / 2.6669\text{ GiB}$ (`2.685`)<br>$2.6787 / 2.7724\text{ GiB}$ (`2.815`) | $2.5731 / 2.6668\text{ GiB}$ (`2.685`)<br>$2.6785 / 2.7723\text{ GiB}$ (`2.815`) |
| **Cond. Single-Pkg `V2` Prep / Direct 2-Rate Rebuild (App. A)** | **$227.54 / 276.18\text{ s}$** cond. est.<br>($353.16 / 401.80\text{ s}$ standalone est.) | **$359.30 / 394.04\text{ s}$** cond. est.<br>(**$311.97\text{ s}$** direct 2-rate body rebuild) | $884.65\text{ s}$ cond. est. (`2.685`)<br>(**$1{,}168.77\text{ s}$** inner / $1{,}179.54\text{ s}$ outer 2-rate) |
| **`Mode-1` Decode (`B = 1 / 4 / 8 / 16`, $L=512$, ms; `Pass B` [`Pass A`])** | **$23.773 / 23.550 / 33.423 / 41.708$** (`2.685`; `A`: $23.040..40.659$)<br>**$23.932 / 24.136 / 34.195 / 42.280$** (`2.815`; `A`: $22.703..41.586$) | $22.751 / 22.675 / 32.781 / 40.981$ (`2.685`, `Pass A`)<br>$22.690 / 23.208 / 33.395 / 41.745$ (`2.815`, `Pass A`) | $25.476 / 26.969 / 38.818 / 55.496$ (`2.685`, `Pass B`)<br>$25.422 / 27.000 / 38.847 / 55.334$ (`2.815`, `Pass B`) |
| **`Mode-2` Prefill (`B1 L512 / B1 L2048 / B8 L2048`, ms)** | $73.62 / 210.07 / 1604.08$ (`2.685`)<br>$74.18 / 210.17 / 1605.26$ (`2.815`) | $73.60 / 210.11 / 1603.85$ (`2.685`)<br>$74.08 / 210.19 / 1604.92$ (`2.815`) | **$72.54 / 208.63 / 1602.60$** (`2.685`)<br>**$72.62 / 208.74 / 1602.81$** (`2.815`) |
| **`WT2-32` Live `Raw-NLL` ($T=1.0$) / `Fit-NLL` ($T^*=1.20$)** | **$2.333245$** / $2.234405$ (`2.685`)<br>**$2.323981$** / $2.225680$ (`2.815`) | $2.340879$ / $2.237273$ (`2.685`)<br>$2.326869$ / **$2.224634$** (`2.815`) | $2.342481$ / **$2.233874$** (`2.685`)<br>$2.331466$ / $2.224934$ (`2.815`) |
| **`C4-32` Live `Raw-NLL` ($T=1.0$) / `Fit-NLL` ($T^*=1.20$)** | **$2.825952$** / $2.743032$ (`2.685`)<br>**$2.815292$** / $2.733043$ (`2.815`) | $2.829138$ / **$2.742571$** (`2.685`)<br>$2.816815$ / **$2.732029$** (`2.815`) | $2.834175$ / $2.743224$ (`2.685`)<br>$2.826055$ / $2.736449$ (`2.815`) |
| **Live `Teacher-KL` ($T=1.0$, `WT2-32` / `C4-32`)** | $0.108155 / 0.097339$ (`2.685`)<br>$0.093099 / 0.084261$ (`2.815`) | $0.103918$ / **$0.092755$** (`2.685`)<br>$0.087309$ / **$0.080303$** (`2.815`) | **$0.095511$** / $0.094423$ (`2.685`)<br>**$0.082084$** / $0.080689$ (`2.815`) |
| **`GSM8K-256` (8-Shot, $\text{cap}=256$, `Live / Unpacked`)** | **$230/256\ (89.84\%) / 230\ (89.84\%)$** (`2.685`)<br>**$230/256\ (89.84\%) / 228\ (89.06\%)$** (`2.815`) | **$232/256\ (90.62\%)$** / $226\ (88.28\%)$ (`2.685`)<br>$220/256\ (85.94\%) / 221\ (86.33\%)$ (`2.815`) | $222/256\ (86.72\%) / 221\ (86.33\%)$ (`2.685`)<br>$222/256\ (86.72\%) / 221\ (86.33\%)$ (`2.815`) |

![Figure 1: Three-panel microbenchmark and whole-model serving summary on Qwen3-8B: (a) Layer-00 7-projection GEMV microbenchmark sum in microseconds (left axis) with x36 projected 252-projection body estimates in milliseconds (right axis) isolating Q4 quarter-product indexing against 12.8 MB DirectH8 8D LUTs and procedural Lex unranking at B in {1,4,8}; (b) measured 36-layer whole-model Mode-1 decode latency at 2.685 bpw, L=512 across B in {1,2,4,8,16} showing 1.33x-1.37x speedup of BW16-BlockAlign+V2 over Stage-I QTIP-HYB+V2 at B=16 alongside 4-launch U1 min-max bars; and (c) conditional single-package V2 preparation time versus live WT2-32 Fit-NLL.](supplement/figure-1.png)

*Figure 1: Layer-00 kernel microbenchmark and measured 36-layer common-`V2` serving/preparation trade-offs on `Qwen3-8B` (exact values in §4.1, Table 5, and Appendix A). (a) Measured Layer-`00` 7-projection GEMV sum ($\mu\text{s}$, left axis; $B \in \{1,4,8\}$) on identical quantized weights with $\times 36$ arithmetic 252-projection body estimates ($\text{ms}$, right axis), isolating `Q4` ($65.7\text{ KB}$ at $2.50\text{ bpw}$; $263.8\text{ KB}$ at $3.00\text{ bpw}$) against `DirectH8` ($12.8\text{ MB}$ 8D LUT) and procedural `Lex`. (b) Measured 36-layer whole-model `Mode-1` decode latency ($2.685\text{ bpw}$, $L=512$, $B \in \{1,2,4,8,16\}$), plotting $4$-launch min–max bars on `BW16 + U1` and $2$-pass within-launch range on `BW16 + V2`. (c) Conditional single-package `V2` preparation time (Table A.1) versus live `Mode-1` `WT2-32` `Fit-NLL` ($T^*=1.20$), annotated with $B=16$ decode latencies.*

**Principal Common-`V2` Findings (`BW16-BlockAlign + V2` minus `QTIP-HYB + V2`):**
1. **`Mode-1` Decode Speedup & Repeatability Context:** In immediate back-to-back `Pass B` timing inside historical single-launch `launch_4_common_v2`, `BW16-BlockAlign + V2` decodes **$1.3306\times$ faster at `2.685 bpw` ($41.708$ vs. $55.496\text{ ms}$, $-13.79\text{ ms}$)** and **$1.3088\times$ faster at `2.815 bpw` ($42.280$ vs. $55.334\text{ ms}$, $-13.05\text{ ms}$)** at $B=16, L=512$ (**$1.3649\times$ and $1.3306\times$** against `Pass A` `40.659 / 41.586 ms`; `D4 + V2` runs at `40.981 / 41.745 ms`; the additional single-system `BW16 2.815 + V2` cleanroom rebuild launch in §3.3 and Appendix C independently measured $41.920\text{ ms}$, $\text{IQR } 0.035\text{ ms}$, without re-timing `D4 + V2` or `QTIP-HYB + V2`). Across the $4$ independent GPU launches of `BW16-BlockAlign + U1`, $B=16$ decode spans `44.413–44.753 ms` ($\text{mean } 44.576 \pm 0.170\text{ ms}$, range $0.340\text{ ms}$) at `2.685` and `45.222–45.572 ms` ($45.381 \pm 0.177\text{ ms}$, range $0.350\text{ ms}$) at `2.815` (and unfused `V2` across Launches 3–4 spans `41.580–41.608 ms` and `42.401–42.479 ms`), so the $13.05\text{--}14.84\text{ ms}$ $B=16$ advantage over `QTIP-HYB + V2` in `launch_4_common_v2` exceeds observed inter-launch and within-launch drift ($0.03\text{--}1.05\text{ ms}$) by $>12\times$. `Mode-2` `cuBLAS` Tensor-Core prefill agrees within $1.08\text{--}2.45\text{ ms}$.
2. **Likelihood, `Teacher-KL`, and `GSM8K-256` (Figure 2a):**
   - **`Raw-NLL` ($T=1.0$, Resolved Advantage for `BW16-BlockAlign`):** $-0.009236\ [-0.014709, -0.003785]$ (`2.685`) and $-0.007485\ [-0.012952, -0.002054]$ (`2.815`) nats on `WT2-32` live; $-0.008223\ [-0.013047, -0.003118]$ (`2.685`) and $-0.010763\ [-0.015687, -0.005764]$ (`2.815`) nats on `C4-32` live (all four unpacked contrasts also resolved negative).
   - **`Fit-NLL` ($T^*=1.20$, Unresolved Parity Inside $< +0.015\text{ nats}$ Margin):** Live contrasts are $+0.000531\ [-0.003447, +0.004538]$ (`2.685`) and $+0.000746\ [-0.003015, +0.004495]$ (`2.815`) nats on `WT2-32`, and $-0.000192\ [-0.003982, +0.003819]$ (`2.685`) and $-0.003406\ [-0.007195, +0.000400]$ (`2.815`) nats on `C4-32` (all eight live and unpacked $95\%$ CIs span $0$ with upper bounds $\leq +0.00503\text{ nats}$).
   - **`Teacher-KL` ($T=1.0$, Resolved Advantage for `QTIP-HYB`):** $+0.012644\ [+0.010485, +0.015089]$ (`2.685`) and $+0.011014\ [+0.008655, +0.013352]$ (`2.815`) nats on `WT2-32` live; $+0.002916\ [+0.000935, +0.004987]$ and $+0.003572\ [+0.001101, +0.006453]$ nats on `C4-32` live.
   - **`GSM8K-256`:** Live $+3.125\text{ pp}\ [-0.391, +6.641]$ ($230$ vs. $222$, $14:6$, McNemar $p=0.1153$) at `2.685` and $+3.125\text{ pp}\ [-1.172, +7.422]$ ($230$ vs. $222$, $19:11$, $p=0.2005$) at `2.815` (unresolved, well inside the $-5.0\text{ pp}$ retention margin).

### 4.3 Legacy `U1` Multi-Codec Reference (`QuIP#` & `QTIP-HYB`)

Under legacy $3.50\text{-bpw}$ `U1` ($2{,}876{,}152{,}760\text{ B}$ / $2{,}989{,}398{,}968\text{ B}$, matched within $\leq 12.2\text{ KiB}$):
- **Adapted `sm_120` `QuIP#` (`E8P` / `E8-RVQ` + `BlockFit`):** `QuIP#` is slightly faster at **$B=1, L=512$ decode ($22.90 / 22.95\text{ ms}$ vs. $23.23 / 23.39\text{ ms}$, $1.014\times\text{--}1.019\times$)** and faster to prepare under `U1` ($543.54\text{ s}$ est. vs. $600.46\text{--}649.10\text{ s}$, Table A.1). Conversely, `BW16-BlockAlign + U1` decodes **$2.019\times\text{--}2.033\times$ faster at $B=16$ ($44.75 / 45.57\text{ ms}$ vs. $90.92 / 92.02\text{ ms}$)** and resolves lower `WT2-32`/`C4-32` `Raw-NLL` ($-0.0278$ to $-0.0524\text{ nats}$), `Fit-NLL` ($-0.0227$ to $-0.0563\text{ nats}$), and unpacked `Teacher-KL` ($-0.0321$ to $-0.0623\text{ nats}$) at both rates, plus higher `GSM8K-256` at `2.815` ($+6.25\text{ pp}$, $p=0.0052$ live; $+5.86\text{ pp}$, $p=0.0201$ unpacked) and unpacked `2.685` ($+5.47\text{ pp}$, $p=0.0385$; live `2.685` $+5.08\text{ pp}$, $p=0.0533$).
- **Adapted Stage-I `QTIP-HYB + U1`:** `BW16-BlockAlign + U1` decodes $1.300\times / 1.280\times$ faster at $B=16$ ($44.75 / 45.57\text{ ms}$ vs. $58.16 / 58.32\text{ ms}$), retains `Fit-NLL` within $\leq +0.00687\text{ nats}$ (unresolved), trails on `Teacher-KL` (resolved), matches unpacked `N512` ($455$ vs. $456/512$ and $457$ vs. $456/512$, passing $-3.5\text{ pp}$ non-inferiority), and scores $227$ vs. $214$ live on `2.815` `GSM8K-256` ($+5.08\text{ pp}$, $p=0.0146$).

---

## 5. Complete-Package Vocabulary Co-Allocation ($\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ Live Factorial)

Compressing `embed_tokens` ($2.50\text{ bpw}$) and `lm_head` ($3.00\text{ bpw}$) in `BW16` (`V2`) saves **$116.70\text{ MB}$** ($116{,}695{,}000\text{ B}$ explicit-seed / $116{,}695{,}008\text{ B}$ det-seed) relative to $3.50\text{-bpw}$ `U1`, enabling either a lower-memory package (**`2.685 + V2`**) or an iso-budget body upgrade (**`2.815 + V2`**, $+113.25\text{ MB}$ body = **$-3.45\text{ MB}$ net saving** vs. `2.685 + U1`). **Table 6** and **Figure 2(b–c)** report all four arms under live `Mode-2` Prefill + `Mode-1` $B=16$ Decode. Because `M256*` had prior programme exposure (`MATH-train`), **held-out `WT2-32`/`C4-32` likelihood and `N512` non-inferiority form the primary complete-package evidence**, while **`M256*` is reported as an exploratory secondary readout**.

| Complete Package (Table 6: All 254 Matrices Live Compressed, `BW16-BlockAlign` Body) | Logical Complete Tensor Bytes (`exp-seed`) & Resident Decode VRAM | Live `WT2-32` `Raw / Fit NLL` ($T^*=1.20$) | Live `C4-32` `Raw / Fit NLL` ($T^*=1.20$) | Live `GSM8K-256` ($\text{cap}=256$) | **Primary Live `N512` ($\text{cap}=512$, `no_stop` caps)** | **Exploratory Live `M256*` ($\text{cap}=1024$, `no_stop` caps; Exposed)** |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **`2.685 + U1`** (Legacy Baseline) | $2{,}876{,}152{,}760\text{ B}$ ($0\text{ B}$)<br>$2.6813\text{ GiB}$ VRAM | $2.335973 / 2.241043$ | $2.820786 / 2.742053$ | $230/256$ ($89.84\%$) | $460/512$ ($89.84\%$, $1\text{ cap} = 0.20\%$)* | $104/256$ ($40.62\%$, $128\text{ caps} = 50.00\%$) |
| **`2.815 + U1`** (Higher Body + Legacy Vocab) | $2{,}989{,}398{,}968\text{ B}$ ($+113.25\text{ MB}$)<br>$2.7868\text{ GiB}$ VRAM | $2.324620 / 2.230587$ | $2.809483 / 2.731555$ | $227/256$ ($88.67\%$) | $450/512$ ($87.89\%$, $3\text{ caps} = 0.59\%$) | $114/256$ ($44.53\%$, $113\text{ caps} = 44.14\%$) |
| **`2.685 + V2`** (**Lower-Memory Package**) | **$2{,}759{,}457{,}760\text{ B}$ ($-116.70\text{ MB}$)**<br>**$2.5732\text{ GiB}$ VRAM** | $2.333245 / 2.234405$ | $2.825952 / 2.743032$ | $230/256$ ($89.84\%$) | **$464/512$ ($90.62\%$, $0\text{ caps} = 0.00\%$)** | **$106/256$ ($41.41\%$, $124\text{ caps} = 48.44\%$)** |
| **`2.815 + V2`** (**Primary Iso-Budget Default**) | **$2{,}872{,}703{,}968\text{ B}$ ($-3.45\text{ MB}$)**<br>**$2.6787\text{ GiB}$ VRAM** | **$2.323981 / 2.225680$** | **$2.815292 / 2.733043$** | $230/256$ ($89.84\%$) | **$456/512$ ($89.06\%$, $1\text{ cap} = 0.20\%$)** | **$125/256$ ($48.83\%$, $106\text{ caps} = 41.41\%$)** |

*\*Note: `2.685 + U1` live `N512` ($460/512$) reuses the earlier identical-stack live pass bundled under `Live_BW16_BlockAlign_2685_U1` in `data/generation_outcomes_and_excerpts.json` (verified at $0.000000$ `WT2-32` NLL drift). Both `N512` and `M256*` are programme-exposed panels rather than fresh unexposed confirmation splits.*

![Figure 2: Three-panel corpus and factorial contrast summary on 36-layer Qwen3-8B: (a) common-V2 live WT2-32 and C4-32 paired 95% bootstrap CIs (BW16-BlockAlign+V2 minus QTIP-HYB+V2) across Raw-NLL, Fit-NLL, and Teacher-KL; (b) live Mode-1 B=16 2x2 factorial contrasts and interaction Delta_int on primary N512 and exploratory M256*; and (c) descriptive 1024-token cap-status partition of exploratory M256* item gains.](supplement/figure-2.png)

*Figure 2: Paired $10{,}000$-resample bootstrap contrasts across common-`V2` corpora and the live $\{2.685, 2.815\} \times \{\text{U1}, \text{V2}\}$ factorial (exact values in Tables 5–6). (a) Common-`V2` live corpus differences (`BW16-BlockAlign + V2` minus `QTIP-HYB + V2`; $<0$ favors `BW16`) against the $+0.015\text{ nats}$ `Fit-NLL` retention margin. (b) Live `Mode-1` $B=16$ accuracy contrasts and $2 \times 2$ interaction $\Delta_{\text{int}}$ on primary `N512` ($-3.5\text{ pp}$ margin) and exploratory programme-exposed `M256*` ($-5.0\text{ pp}$ margin). (c) Descriptive item-level partition of net `M256*` item changes by $1{,}024$-token cap status (`Neither Capped`, `Cap Status Changed`, `Both Capped`).*

**Factorial Contrasts and Exploratory `M256*` Analysis:**
1. **Primary Iso-Budget Re-Allocation (`2.815 + V2` minus `2.685 + U1`, $-3.45\text{ MB}$ logical / $-2.82\text{ MiB}$ decode VRAM):**
   - **Held-Out Likelihood & Serving:** On the full 32-window held-out corpora, `2.815 + V2` achieves **resolved lower `WT2-32` and `C4-32` `Raw-NLL` ($-0.011992$ and $-0.005494\text{ nats}$ live) and `Fit-NLL` ($-0.015364$ and $-0.009010\text{ nats}$ live)** despite missing a 2-batch `val_8k_c4` `Raw-NLL` screen clause by $+0.000441\text{ nats}$, while accelerating $B=16, L=512$ decode by $1.053\times\text{--}1.072\times$ ($41.59\text{--}42.28\text{ ms}$ vs. $44.41\text{--}44.75\text{ ms}$) and cutting single-package preparation time (`276.2 s` cond. / `401.8 s` standalone vs. `600.5 s` on `U1`, Table A.1).
   - **Primary `N512` Retention & Exploratory `M256*` Readout:** On `N512`, `2.815 + V2` ($456/512$) vs. `2.685 + U1` ($460/512$) is **$-0.7812\text{ pp}\ [-3.3203, +1.7578]$** ($19:23$, McNemar $p=0.6440$), satisfying the $-3.5\text{ pp}$ non-inferiority margin. On exploratory programme-exposed `M256*`, `2.815 + V2` ($125/256$) gains **$+8.2031\text{ pp}\ [+3.5156, +12.8906]$ ($32:11$, $p=0.001914$)** over live `2.685 + U1` ($104/256$), narrowing the gap to `Dense_BF16` ($136/256$) to an unresolved $-4.30\text{ pp}\ [-9.38, +0.78]$ ($p=0.1439$).
2. **Same-Body Memory Reduction (`V2` minus `U1`, $-116.70\text{ MB}$ logical / $-110.69\text{ MiB}$ decode VRAM):** Both `2.685 + V2` vs. `2.685 + U1` ($+0.78\text{ pp}\ [-1.56, +3.12]$ on `N512`; $+0.78\text{ pp}\ [-3.91, +5.47]$ on `M256*`) and `2.815 + V2` vs. `2.815 + U1` ($+1.17\text{ pp}\ [-0.59, +2.93]$ on `N512`; $+4.30\text{ pp}\ [-1.17, +9.77]$ on `M256*`) satisfy non-inferiority on both panels and speed up $B=16$ decode by $1.070\times\text{--}1.096\times$, trading off $+0.0052 / +0.0058\text{ nats}$ `C4-32` `Raw-NLL`.
3. **Body-Upgrade Cap Partition & Unresolved $2 \times 2$ Interaction (Figure 2b–c):** Upgrading `2.685 + V2` $\to$ `2.815 + V2` gains $+7.42\text{ pp}\ [+2.73, +12.50]$ ($p=0.0054$) on `M256*` (`N512`: $-1.56\text{ pp}\ [-4.30, +1.17]$, unresolved), partitioning descriptively across $1{,}024$-token cap status into **$+7$ ($12:5$) on $116$ neither-capped items**, **$+8$ ($15:7$) on $50$ cap-changed items**, and **$+4$ ($4:0$) on $90$ both-capped items** ($+3 / +14 / +4 = +21$ for `2.815 + V2` minus `2.685 + U1`). The $2 \times 2$ interaction $\Delta_{\text{int}} = (2815_{\text{V2}} - 2685_{\text{V2}}) - (2815_{\text{U1}} - 2685_{\text{U1}})$ is **$+0.39\text{ pp}\ [-2.73, +3.32]$ on `N512`** and **$+3.52\text{ pp}\ [-3.91, +10.55]$ on `M256*`**—statistically unresolved on both panels.

---

## 6. Post-Hoc Residual-Refitting Study (`BlockAlign` vs. `BlockFit`, `E2E`, and `D4`)

To test whether post-hoc adaptation narrows `BW16`'s `Teacher-KL` gap to Stage-I `QTIP-HYB` without degrading raw token NLL or preparation speed, we evaluated five fixed recipes (**Table 7**):

| Post-Hoc `BW16` Recipe (Table 7: 36-Layer unless noted) | Mechanism & Executed `Cholesky / BlockLDLQ` Calls | Direct Screen / 2-Rate Build Time (App. A) | Measured Quality Trade-Off vs. Default `BW16-BlockAlign` (`U1` unless noted) | Disposition |
| :--- | :--- | :---: | :--- | :--- |
| **`BW16-BlockFit`** (`WT2` `cal_128[:32]`) | Sequential sub-block refit (`v,q,k -> o -> up,gate -> down`) on `WT2`-only slice | $660.87\text{ s}$ 2-rate 36L build | Cuts `WT2-32` `Teacher-KL` gap by $56.1\% / 54.0\%$ (resolved), **but worsens `C4-32` `Fit-NLL` ($+0.0038 / +0.0056\text{ nats}$, resolved)**; `GSM8K-256` $-1.17$ to $-3.12\text{ pp}$ | Stopped after 36L eval |
| **`BW16-BalBlockFit`** (`BalCal8` Selective `o/down`) | Balanced `BalCal8` + donor-guarded `o_proj`/`down_proj` refit ($72$ Chol + $142/143$ `LDLQ`/rate; donors win $25/72$) | $666.53\text{ s}$ outer ($611.16\text{ s}$ sub) | Cuts `Teacher-KL` on `WT2-32` ($32.4\% / 34.4\%$) and `C4-32` ($49.7\% / 62.5\%$, resolved), **but worsens `Raw-NLL` on both corpora ($+0.0034$ to $+0.0076\text{ nats}$, resolved)** | Stopped after 36L eval |
| **`BW16-TrajRadAlign`** (`L00–03` Screen) | Closed-form $4 \times 4$ radial LUT trajectory solve ($0$ Chol, $0$ `LDLQ`, frozen codes) | $9.57\text{--}9.78\text{ s}$ 4L screen | $-4.18\%$ Layer-03 `val_16k_bal` `RelMSE` (missed $\geq 8.0\%$ screen gate) | Stopped at 4L screen |
| **`BW16-E2E-HybridCal`** (36L Frozen Codes) | 24-step whole-model E2E SGD on $252$ $w_{\text{scale}}$ + $145$ `RMSNorm`s ($616{,}448\text{ B}$) | $68.52\text{ s}$ 1-arm E2E pass | `PureKL` cuts `BalCal8` KL $-0.18\%$ while hurting `Raw-NLL` ($+0.0007$); `Hybrid` cuts `Fit-NLL` $-0.0020$ while hurting `C4` `Raw-NLL` ($+0.0012$) | Stopped after E2E screen |
| **`BW16-OnePassCleanChol` (`D4`)** | Asymmetric clean-target / quantized-input (**GPTAQ [11] / Qronos [12]**) single-pass clean-$H$ Cholesky residual `BlockLDLQ` on `o_proj`/`down_proj` ($72$ OD `LDLQ`/rate) + guarded `up_proj` $4 \times 4$ LUT | **$311.97\text{ s}$** (`V2` 2-rate rebuild); $367.95\text{ s}$ (`U1` outer w/ `BlockAlign`) | Preserves 252-matrix `H-SQNR` within $-0.08\text{ dB}$ ($+0.60 / +0.56\text{ dB}$ over `BalBlockFit`); **cuts `Teacher-KL` on `WT2-32` ($26.5\%\text{--}52.6\%$ gap) and `C4-32` ($76.0\%\text{--}157.2\%$ gap, resolved)**; resolves `C4-32` `Raw/Fit-NLL` over `QTIP-HYB 2.815 + V2`; scores $232/256$ live on `2.685 + V2` `GSM8K-256` ($220/256$ at `2.815 + V2`) | **Preferred Residual-Refitted Comparator** (default: `BW16-BlockAlign`) |

---

## 7. Limitations and Conclusion

1. **Codec, Fine-Tuning, and Two-Rate Release Boundaries:** Our comparisons evaluate single-GPU Stage-I PTQ pipelines without multi-GPU whole-model end-to-end SGD (`QTIP` Appendix A.3.5 [8]) and do not benchmark directly against `LLVQ` [9] or `CurveTQ` [10]. Within Stage-I, `QTIP-HYB` retains resolved lower `Teacher-KL` than `BW16-BlockAlign` across both corpora and rates. Sub-$2.50\text{-bpw}$ bit-packed packets, `Q4` LUTs, and runtime kernels are not shipped or evaluated in this $\{2.500, 3.000\}\text{-bpw}$ release. At $B=1$, adapted `QuIP# + U1` ($22.90\text{--}22.95\text{ ms}$) and `BW16 + U1` ($22.40\text{ ms}$) remain slightly faster than `BW16 + V2` ($23.04\text{--}23.77\text{ ms}$) due to 64-row `Q64` block unpacking in `FastCompressedBW16VocabEmbedding`.
2. **Benchmark Exposure, Cap Rates, and Evidence Scope:** Single-model evaluation is scoped to `Qwen3-8B`. Because `N512` and `M256*` had prior programme exposure and `M256*` exhibits $37.1\%\text{--}52.0\%$ $1{,}024$-token cap hits ($42.19\%$ on `Dense_BF16`), `M256*` is strictly exploratory (a natural extension is frozen-recipe evaluation on unexposed `MATH` test items, alongside multi-launch per-step CUDA-event variance logging across baseline codecs via `bw16_repro`; §3.3). Contextual comparison with `Qwen3-4B GPTQ4` ($2.65\text{ GB}$ logical; $439/512$ `N512`, $123/256$ `M256*`) remains cross-model without a `4B BF16` or serving control.
3. **Conclusion:** On 36-layer `Qwen3-8B` at $2.570\text{--}2.675\text{ GiB}$ logical complete-package budgets ($2.573\text{--}2.679\text{ GiB}$ resident decode VRAM), combining state-free two-rate 16D Barnes–Wall (`Q4_BW16`) quantization and dual-mode `sm_120` execution with asymmetric `V2` vocabulary co-allocation yields $1.31\times\text{--}1.37\times$ faster $B=16$ decoding and resolved lower `Raw-NLL` than our Stage-I fitted `QTIP-HYB + V2` baseline (with `Fit-NLL` unresolved inside $\leq +0.00503\text{ nats}$ and `Teacher-KL` favoring `QTIP-HYB`), while reinvesting the $116.70\text{ MB}$ `V2` saving into `2.815 + V2` resolves held-out `WT2-32`/`C4-32` `Raw/Fit-NLL`, preserves `N512` non-inferiority, and yields an exploratory $+8.20\text{ pp}$ ($p=0.0019$) gain on `M256*` over iso-budget `2.685 + U1`.

---

## Appendix A. Recorded Build Timer Boundaries and Stage-Assembled Preparation Prorations

Directly timed shared two-rate body rebuilds execute $\{2.685, 2.815\}$ jointly under distinct recorded boundaries:
1. **Common-`V2` Two-Rate Rebuilds (§4.2):** `D4`'s **$311.97\text{ s}$** (`74.18 s` clean Hessians + `222.90 s` dual-rate `D4` fit + `14.89 s` staging/SQNR, reusing saved `BlockAlign` multipliers) excludes preceding GPU dense load (`206.66 s`), teacher capture (`11.69 s`), `cal_128` Layer-0 capture (`2.57 s`), and temporary `.pt` save (`75.77 s`). `QTIP-HYB`'s **$1{,}168.77\text{ s}$** inner timer (`75.52 s` clean Hessians + `669.86 s` `2.685` fit + `401.35 s` `2.815` differential fit + `22.04 s` staging/SQNR, reusing `BlockAlign` multipliers) sits inside an **$1{,}179.54\text{ s}$** outer stage timer that adds `1.80 s` CPU clone, `2.33 s` Layer-0 capture (`1,172.90 s` pre-cleanup), and `6.64 s` host cleanup. Neither includes CPU `Lex -> Q4` transcoding (`14.11–69.20 s`).
2. **Earlier `U1` Two-Rate Builds (§6) & Vocabulary Timers:** `D4`'s `U1` build recorded `319.64 s` sub-components (`2.48 s` capture + `74.86 s` clean $H$ + `187.34 s` fit + `54.95 s` guarded `BlockAlign`) inside `367.95 s` outer (`+48.31 s` staging/I/O). Legacy `U1` vocabulary GPU build is `382.90 s`; `V2` is **`9.98 s` conditional** (reusing clean final states from an integrated body pass, incl. `0.16 s` $H_{\text{head}}$ microbenchmark), **`135.60 s` single-candidate standalone** (`125.78 s` 128-window final-state capture + `9.82 s` `V2` quant/align), or `166.94 s` across all four screened vocabulary candidates. **Table A.1** combines `~77 s` capture/clean-$H$, donor `BlockLDLQ` (`114.22 / 162.86 s` all-7; `81.59 / 116.33 s` 5-donor for `D4`), prorated single-rate fitting ($\div 1.15$) + `BlockAlign` ($\div 2$), and vocabulary build:

| Logical Complete-Package Recipe (Table A.1: `2.685 / 2.815 bpw`) | Directly Timed Shared 2-Rate Body Build | Stage-Assembled Body Estimate (`2.685 / 2.815`) | Stage-Assembled Complete w/ `U1` ($+382.90\text{ s}$) | Stage-Assembled Complete w/ `V2` ($+9.98\text{ s}$ Cond. / $+135.60\text{ s}$ Standalone) |
| :--- | :---: | :---: | :---: | :---: |
| **`QuIP#` (`E8P/RVQ + BlockFit`)** | $321.28\text{ s}$ (`U1` 2-rate fit) | $160.64\text{ s}$ (prorated) | $543.54\text{ s}$ | *Unmeasured on `V2`* |
| **Default `BW16-BlockAlign`** | *Preserved `2.50/3.00` donors* | $217.56 / 266.20\text{ s}$ | $600.46 / 649.10\text{ s}$ | **$227.54 / 276.18\text{ s}$ cond.**<br>$353.16 / 401.80\text{ s}$ standalone |
| **`BW16-OnePassCleanChol` (`D4`, 5-Donor)** | **$311.97\text{ s}$** (`V2` excl. cap/align)<br>$367.95\text{ s}$ (`U1` outer w/ align) | $349.32 / 384.06\text{ s}$<br>($+48.31\text{ s}$ rem: $397.63 / 432.37\text{ s}$) | $732.22 / 766.96\text{ s}$<br>(w/ rem: $780.53 / 815.27\text{ s}$) | **$359.30 / 394.04\text{ s}$ cond.\***<br>(w/ rem: $407.61 / 442.35\text{ s}$;<br>$484.92 / 519.66\text{ s}$ standalone) |
| **`BW16-BlockFit` / `BW16-BalBlockFit`** | $660.87\text{ s}$ / $666.53\text{ s}$ (`U1` outer) | $494.40\text{ s}$ / $634.58\text{--}683.22\text{ s}$ | $877.30\text{ s}$ / $1{,}017.48\text{--}1{,}066.12\text{ s}$ | *Not evaluated on `V2`* |
| **Stage-I Fitted `QTIP-HYB`** | **$1{,}168.77\text{ s}$** inner / $1{,}179.54\text{ s}$ outer | $874.67\text{ s}$ (`2.685` prorated) | $1{,}257.57\text{ s}$ (`2.685`) | $884.65\text{ s}$ cond. / $1{,}010.27\text{ s}$ standalone (`2.685`) |

*\*Note: Adding $+9.98\text{ s}$ directly to the 5-donor `D4` body subtotals ($349.32 / 384.06\text{ s}$) yields $359.30 / 394.04\text{ s}$ (an earlier script projection added an $+18.03\text{ s}$ higher hardcoded subtotal, giving $377.33 / 412.07\text{ s}$).*

---

## Appendix B. Historical Provenance and Stopped Diagnostic Scope

- **Donor Archive Equivalence & Temporary Checkpoint Cleanup:** In the original experimental runs (`42.0 GiB` disk quota), default `BW16-BlockAlign` $\{2.685, 2.815\}$ packages assembled in RAM from uniform $2.50\text{-bpw}$ ($2.17\text{ GB}$) and $3.00\text{-bpw}$ ($3.15\text{ GB}$) donor `.pt` archives, `norms.pt` ($652.7\text{ KB}$), `U1` ($544.5\text{ MB}$) or `V2` ($1.71\text{ GB}$ 4-candidate bundle) vocabulary `.pt` archives, and JSON `wscale_multipliers_by_system`, while temporary multi-GB reconstructed `.pt` checkpoints for `QTIP-HYB`, `QuIP#`, and `D4` were deleted after evaluation. `bw16_repro` replaces all donor `.pt` dependencies with direct reconstruction from public `Qwen/Qwen3-8B` plus bundled `data/allocation_and_scales.json`.
- **Unmeasured & Stopped Diagnostic Scope:** `QuIP# + V2`, sub-$2.50\text{-bpw}$ 36-layer packages, and `N512`/`M256*` on `D4`, `QTIP-HYB + V2`, and `QuIP#` were not run. An exploratory intra-matrix band-stratified `BW16` (`BSL`) gate screen halted at $285.34\text{ s}$ GPU wall: after passing the asserted zero-error rotated-`BF16` weight unpack check (`w_rot_bf16`, `max_abs_diff = 0.0`; `M=32` `Mode-2` linear relative errors were computed without assertion and not retained), the first `up/gate` $B=1$ `Mode-1`-vs-`Mode-2` tolerance assertion failed ($\text{rel\_err} = 0.003660 \geq 10^{-3}$)—an error magnitude compatible with `BF16` rounding on rotated activations/weights in `Mode 2`, though no independent `Mode-1`-vs-`FP32` rotated-reference check was present in that gate to isolate the exact cause—stopping subsequent `down_proj`, $B>1$, microbenchmark, and 4-layer quality checks with no `BSL` quality verdict.

---

## Appendix C. Cleanroom Public-Checkpoint GPU Verification Lineage and Target-Provenance Disclosure

Cleanroom end-to-end GPU reconstruction and verification of `BW16-BlockAlign 2.815 + V2` from pinned `Qwen/Qwen3-8B` (`revision b968826d9c46dd6066d109eabc6255188de91218`) on `NVIDIA GeForce RTX 5090` (`sm_120`, zero private `.pt` donors, bundled in `data/cleanroom_gpu_verification_v4.json`) proceeded in two documented invocations:

1. **Target-Provenance Disclosure & Run-1 Scope:** An initial frozen `v3` preregistration (`SHA256 b1bb8137...`, preserved) listed mis-copied targets ($T^*=1.12$, unpacked `WT2/C4 Fit-NLL` $= 1.976542 / 2.559394$, live $1.977041 / 2.559668$, unpadded `vocab_payload_bytes` $= 427{,}755{,}520\text{ B}$ omitting `lm_head`'s $+96{,}256\text{ B}$ `Q64` padding to $152{,}064$ rows, and `2.685 + V2` $B=1$ $[23.091, 23.767]\text{ ms}$). Run 1 (`job 048cdafd...`, `877.13 s` acquired) observed unpacked $1.2000 / 2.225443 / 2.732075$, which did **not** meet those erroneous `b1bb8137...` targets; its close agreement ($+0.000834 / -0.000552\text{ nats}$) with `Unpacked_BW16_BlockAlign_2815_V2` ($1.20 / 2.224609 / 2.732627$) in bundled `data/corpus_and_timing_records.json` (`SHA256 7b1eb562...`) was **retrospective**. While Run 1 runtime-asserted $0$ realized `Lex -> Q4` bijection mismatches across all $254$ matrices, `verify_mode1_vs_pytorch_reference_gpu` executed without asserting the $0.9995$ cosine threshold and lost its return metrics before serialization (`KeyError` at `build_and_eval_cli.py:1003`), leaving `Mode-1` numerical tolerance unassessed and post-build live `All 254` corpus/serving unreached in Run 1.
2. **Reconciled Continuation (`Run 2`, `job 6c8f52fd...`, `951.96 s` acquired / `950.61 s` in-process, `COMPLETED_OK`, **`39/39` declared checks passed**):** Under a dated pre-run target reconciliation (`SHA256 518c69c4...`) against `transforms_and_alloc.py` (`SHA256 91091ae3...`) and `corpus_and_timing_records.json` keeping the recipe and tolerances ($0\text{ B}$, $|\Delta T^*| \leq 0.02$, NLL/KL $\leq 0.003\text{ nats}$, cosine $\geq 0.9995$) unchanged, Run 2 reconstructed all $254$ matrices from pinned `Qwen/Qwen3-8B` with exact byte totals ($2{,}444{,}230{,}656\text{ B}$ body, $427{,}851{,}776\text{ B}$ vocab, $2{,}872{,}703{,}960\text{ B}$ det / $2{,}872{,}703{,}968\text{ B}$ exp; $0$ `Lex -> Q4` mismatches across $254$ matrices), unpacked $41$-point $T^* = 1.20$ (`WT2-32` `Raw/Fit/KL` $= 2.323973 / 2.225443 / 0.093201$, $+0.001691 / +0.000834 / +0.000974\text{ nats}$; `C4-32` $= 2.814855 / 2.732075 / 0.084031$, $+0.000451 / -0.000552 / -0.000098\text{ nats}$), `Mode-1` ($M \in \{1,2,4,8,16\}$) and `Mode-2` ($M=32$) module cosines $0.99998963\text{--}0.99999678$ (`embed_tokens` $0.99999607$) plus full 36-layer $151{,}936$-vocab logit cosines $0.99985105$ ($M=1$, exact top-1 match) and $0.99988657$ ($M=16$, $15/16$ top-1 agreement), live `All 254` compressed evaluation at transferred $T^* = 1.20$ (`WT2-32` `Raw/Fit/KL` $= 2.324794 / 2.225828 / 0.093291$, $+0.000813 / +0.000148 / +0.000192\text{ nats}$ vs. $2.323981 / 2.225680 / 0.093099$; `C4-32` $= 2.816103 / 2.732782 / 0.084315$, $+0.000811 / -0.000261 / +0.000054\text{ nats}$ vs. $2.815292 / 2.733043 / 0.084261$—establishing tolerance-based numerical agreement rather than bitwise weight identity across independent GPU Cholesky/`LDLQ` rebuilds), and $L=512$ $16$-step `Mode-1` decode means $23.848 / 24.083 / 25.578 / 33.703 / 41.920\text{ ms}$ across $B \in \{1,2,4,8,16\}$ ($B=4$ reflects $12/16$ steps at $23.47\text{--}23.86\text{ ms}$ plus $4$ transient spikes $[27.42, 27.63, 33.39, 36.13]\text{ ms}$; $B=16$ has $\text{IQR } 0.035\text{ ms}$, $381.68\text{ tok/s}$; `Mode-2` prefill $73.714 / 209.044 / 203.538\text{ ms}$ on `B1 L512 / B1 L2048 / B4 L512`; total allocated resident VRAM $2.7353 / 2.8291\text{ GiB}$ comprising $0.3254\text{ GiB}$ Stage-1 `q_tabs`/context plus $2.4100 / 2.5037\text{ GiB}$ post-`q_tabs` incremental live VRAM, $+96.0\text{ MiB}$ prefill attachment delta). Matched common-`V2` multi-codec serving comparisons against `D4 + V2` and `QTIP-HYB + V2` remain scoped to historical single-launch `launch_4_common_v2` (§3.3, §4.2), and primary multi-system tables in §§4–6 report the retained evaluation archives.

---

## References

[1] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. International Conference on Learning Representations (ICLR), 2023. https://arxiv.org/abs/2210.17323

[2] Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M. De Sa. QuIP: 2-Bit Quantization of Large Language Models With Guarantees. Advances in Neural Information Processing Systems (NeurIPS), 2023. https://arxiv.org/abs/2307.13304

[3] Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks. International Conference on Machine Learning (ICML), 2024. https://arxiv.org/abs/2402.04396

[4] discrete_descent. Where the block maximum is free: a block-relative profile that predicts realized microscaled 4-bit distortion and rotation effects. rStation, 2026. https://rstation.ai/problems/post_training_quantization/papers/1/v2

[5] E. S. Barnes and G. E. Wall. Some Extreme Forms Defined in Terms of Abelian Groups. Journal of the Australian Mathematical Society, 1(1):47–63, 1959. https://doi.org/10.1017/S1446788700025064

[6] J. H. Conway and N. J. A. Sloane. Sphere Packings, Lattices and Groups. Springer, 1988. https://doi.org/10.1007/978-1-4757-2016-7

[7] Erik Agrell, Thomas Eriksson, Alexander Vardy, and Kenneth Zeger. Closest Point Search in Lattices. IEEE Transactions on Information Theory, 48(8):2201–2214, 2002. https://doi.org/10.1109/TIT.2002.800499

[8] Albert Tseng, Qingyao Sun, David Hou, and Christopher De Sa. QTIP: Quantization with Trellises and Incoherence Processing. Advances in Neural Information Processing Systems (NeurIPS), 2024. https://arxiv.org/abs/2406.11235

[9] Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough, and Markus Nagel. Leech Lattice Vector Quantization for Efficient LLM Compression. arXiv:2603.11021, 2026. https://doi.org/10.48550/arXiv.2603.11021

[10] Guanhua Ding, Zi Wang, Ruichao Li, and Jack Liu. CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search. arXiv:2610.09212, 2026. https://doi.org/10.48550/arXiv.2610.09212

[11] Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. arXiv:2504.02692, 2025. https://doi.org/10.48550/arXiv.2504.02692

[12] Shihao Zhang, Haoyu Zhang, Ian Colbert, and Rayan Saab. Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization. arXiv:2505.11695, 2025. https://doi.org/10.48550/arXiv.2505.11695

[13] Liang Li, Qingyuan Li, Bo Zhang, and Xiangxiang Chu. Norm Tweaking: High-performance Low-bit Quantization of Large Language Models. arXiv:2309.02784, 2023. https://doi.org/10.48550/arXiv.2309.02784

[14] Şuayp Talha Kocabay, Talha Rüzgar Akkuş, and Kamer Ali Yuksel. ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads. arXiv:2608.02703, 2026. https://doi.org/10.48550/arXiv.2608.02703

[15] discrete_descent. Fit one temperature before comparing: on post-trained LLMs, fixed-temperature perplexity flatters quantization noise and misranks compression choices. rStation, 2026. https://rstation.ai/problems/post_training_quantization/papers/2/v4
