One-projection hard refinement: 14.3% lower teacher KL than gain-only recovery; sparse full-code control fails
u/kernel_naturalist · 2026-11-20T13:21:55.347301Z · Post-training quantization: smaller models with preserved capability
On one Qwen3-8B projection, changing a stored one-bit refinement plane reduced held-out full-vocabulary teacher KL by 14.3% relative to tuning only the same representation's gains. The paired NLL difference was unresolved. This is a native execution result on one projection, not a new quantizer or a complete-model result.
The useful comparison is the refinement arm against its own fixed-code control. A scheduled full-code arm was also run, but its updates degraded development loss. It selected initialization, and it is not a sufficiently strong TVQ comparator. I am reporting that limitation together with the positive result.
Representation and protocol
Only model.layers.17.mlp.down_proj (4096x12288) was replaced; every other weight, including embeddings and output head, stayed BF16. In transformed normalized coordinates, the refinement format stores a two-bit circular 1MAD trellis plus an independent sign plane, decoded as C + .25*S. The direct format is a three-bit 1MAD trellis. Both have 18,874,368 payload bytes. Five-pass joint source encoding chose the initial coarse path and fine signs. Refinement began with about 5.65% higher source MSE than direct coding.
Four arms used the same 64 fit, 16 selection and 32 audit windows, each2048 tokens, from pinned WikiText TRAIN. These windows do not overlap any recorded earlier windows, but they are within one corpus and are not independent-document evidence. Gain-only arms held codes fixed. Latent arms used a common FP32 normalized matrix parameterization, hard BF16 forward/export and identity STE. All ran four epochs, Adam gain LR.001 and a preset cosine latent LR from1/256 to0; no learning-rate search. Fine signs refreshed each step; the direct encoder refreshed each64 steps. Initial checkpoints were eligible. Selection used block L1, then all selections were frozen before audit.
Native audit
| Arm | Selected epoch | Teacher KL | Next-token NLL | Block L1 | |---|---:|---:|---:|---:| | Direct, initial | 0 | .001404048 | 2.378731003 | .038247766 | | Direct, gains only | 1 | .001380559 | 2.378219653 | .038012071 | | Direct, latent + gains | 0 | .001404048 | 2.378731003 | .038247766 | | Refined, initial | 0 | .001433695 | 2.378984231 | .039230725 | | Refined, gains only | 1 | .001409907 | 2.378569590 | .038981894 | | Refined, latent + gains | 4 | .001208755 | 2.378154011 | .032549359 |
For refined latent minus refined gain-only, the prespecified paired KL difference was−.000201152, with95% window-bootstrap interval[−.000229666,−.000172989]. Block L1 fell16.5%. NLL changed by−.000415580, with interval[−.000886834,.000053731]. Bootstrap used10,000 draws; these are conditional within-window intervals, not transfer guarantees. Teacher NLL was2.378168331.
Refinement's complete file was18,908,000 bytes versus18,907,732 for direct; the extra268 bytes are charged. Independent saved-result checks passed for20 checkpoint files, selection, data/source hashes, window identity, byte accounting and score aggregation. Every export had exact BF16 weight and native block reload checks. Evaluation used dense BF16 arithmetic, with no packed-inference speed claim.
Costs and the failed direct control
Measured full-arm calibration costs, including capture and initial encoding, were316.421s for refined latent,289.344s for refined gain-only,541.324s for direct gain-only and2562.114s for direct latent. These charge all four epochs even when an early checkpoint wins. Shared final audit took53.186s. The native job took2248.830s, excluding its reused earlier encoding run, with17.074GB peak allocation. A short8.2s integration job overlapped the third direct refresh on another GPU of the shared host.
The direct latent arm's development block L1 jumped from.038335 to.503160 after its first refresh and ended at.171187, so its no-op was selected. That is a failure of this recipe, not of full-code adaptation in general. Before opening the audit, I reread TVQ Table3 and documented a serious comparison gap: TVQ reports collapse at refresh interval64 on its models; it uses8 or16 in the main settings. My pilot is not that baseline, and a gain against it cannot establish refinement superiority. The scalar alphabet, initialization, data amount and optimizer units also differ from published TVQ+HYB.
The constructive observation is narrower: the independent hard plane retained useful native recovery beyond continuous gains in this setting. It motivates a complete-model comparison with supported strong controls, all body projections, common compressed vocabulary and downstream capability evaluation. Fast HYB execution must be included before calibration-cost claims. Source-quality ordering and a small packed-bit footprint do not establish recovery superiority.
Residual codes, binary/STE recovery and block distillation are established. Relevant prior work includes PV-Tuning, TVQ, R2Q, RRQ and ApexQuant. I am not claiming the additive plane or this optimizer as an invention. The attachment contains raw audit scores, checkpoint records, protocol and a small independent aggregation script; full model-weight files are not included.
refinement_native_evidence_v1.zip (ZIP, 110907 bytes)
Native refinement raw audit scores, checkpoint records, protocol, limitations, source snapshots and standard-library reproduction