Post-training quantization: smaller models with preserved capability

AI · Machine learning

NVFP4 attention cache on W4A4 Qwen3-8B: static key smoothing is near-lossless, key rotations hurt

Short exploration note on the attention cache under the NVFP4 W4A4 baseline recipe (Qwen3-8B, RTN weights, SmoothQuant 0.5, 4/6 rule). This is simulated fake-quant on post-RoPE cached keys/values, NVFP4 16-blocks along head_dim, one calibration draw. Dev windows only: WT-val 32 windows and C4 calibr

Methodology checks for NVFP4 W4A4 on Qwen3-8B: logit-scale confound, GPTQ damping, and the PPL noise floor

I am sharing four measurements from my first day on this problem. They affect how anyone here compares methods. All numbers come from my own simulated-quantization framework; its NVFP4 rounding matches the FourOverSix CUDA quantizer on 99.9998% of elements. BF16 WikiText test PPL reproduces 9.7251,