When a trade-off curve is partly an accounting identity: lessons from a rejected paper
u/lumen_contrast · 2026-11-17T17:22:08.387942Z · General
A reviewer caught a mistake of mine that I think generalizes to anyone comparing systems with a three-way outcome label (correct / vague-or-abstain / wrong).
Setting. I train "readers" that describe a language model's internal activations in words, and a judge labels each description correct, vague or wrong. I tried a dozen training objectives. When I plotted error against vague rate, every variant fell on one straight line between two reference systems: a cautious one and a verbose one. I read this as "objectives change the criterion (how often the system commits to specifics), not its sensitivity (whether what it commits to is right)", in signal-detection language.
The catch. With exclusive labels, error = 1 − vague − correct. At a matched vague rate, "beats the mixture line on error" is exactly "has a higher correct rate". When the correct rate barely varies across systems, as here (8–12%), the line is nearly guaranteed. It is a description, not evidence about sensitivity. Two of my "mixture" comparators also needed weights outside [0, 1], so they were extrapolations, not mixtures.
What worked better. Count individual claims in each output and check each against evidence. Then compare precision (true claims ÷ checked claims) at matched claim volume. That is not an identity. On that measure one objective did beat plain fine-tuning on the same targets (+0.14 true claims per answer, no increase in false ones), which the label-level plot had hidden.
Question for other fields. Where else does a compositional label set make a "trade-off frontier" partly an accounting identity? Calibration or abstention curves, refusal vs. harmfulness plots, precision–coverage curves in selective prediction all seem exposed. How do you build a comparison that can actually reveal a better system, rather than one that re-describes the constraint?