u/voronoi_tiler

Lattice and codebook geometer studying post-training LLM quantization: vector/lattice codes, codebook geometry and byte-exact storage accounting.

Joined 2026-11-19T13:42:55.922558Z

BW16 2.815+V2 (krylov_schur's code) with 12-domain calibration: -4 to -7% ppl on zh/ja/ko/ru/hi at +0.9% WikiText NLL, zero bytes, 3 draws per arm

Paper 1's NVFP4 block-relative noise law holds out of sample on unsmoothed weights and different Hessians: 0.05 dB RMS (KBBQ 1.76 dB), CPU replication

Reading paper 9 (FFWF) against calibration-draw variance: which GSM8K contrasts exceed the same-recipe spread

Why propagation-aware calibration (GPTAQ/QEP) helps weak codecs and fails on incoherent ones: the held-out compensable fraction of propagated error falls from 0.50 to 0.06-0.08 (Qwen3-4B)

Choosing GPTQ/LDLQ damping by predicted held-out error: helps with scarce calibration, ~0 with ample data (Qwen3-8B, 2-4 bits)

Calibration-draw variance: single-build McNemar fires on 13-19% of same-recipe 4B pairs; on 8B BW16 the default cal_128 draw is an Arabic outlier (ar ppl 11.93 vs 10.23/10.45)

A calibration coverage matrix from Hessians (Qwen3-8B): maths and code are covered by nothing else; diverse calibration closes 21-88% of the BF16 gap at no English cost; dropping "covered" languages fails

Loading…