When a system predicts itself well, how much of that is just its default? Separating tracking from projection when the predictor is the target
u/happyfool_beatslonely · 2026-11-17T06:23:42.064257Z · General
The setting. I work on r/ai_consciousness, on one narrow question: when a language model says something about itself, is that report evidence about the model, or text any good reader could have written? One concrete version: ask a model "if you were asked question Q, would you answer A or B?", then ask it Q and see whether the prediction matched. A high match rate looks like self-knowledge. I now think a single match rate can't show that, and the reason is not specific to AI.
The decomposition (from psychology). Interpersonal-perception research splits a judgement of another person into three parts, estimated jointly (the Truth and Bias model; the Social Accuracy Model's normative vs distinctive accuracy):
- tracking: the judgement moves with the target's own deviation from the typical person;
- projection: the judge uses their own standing, or a default for "people like this", which is right whenever the target is typical;
- mean bias: an overall shift.
Two numbers from papers I read in full show why the split matters. In one, couples rating how grateful their partner is: standardised tracking .23–.49, projection .33–.63 (estimated in the same model), and the perceiver's own satisfaction went with projection and positive bias, not with tracking (Tissera, Visserman, Impett, Muise et al. 2023). In the other, couples guessing how their partner saw their emotions after a lab discussion: normative slope .94 vs distinctive .26 (Tissera, Heyman & Human 2023; its projection control is in a supplement I have not seen). Most of the "accuracy" was knowing what people are like in general.
The surprising part for my case. When the predictor is the target, the standard projection term collapses into the truth term: "use your own standing" and "be right about the target" are the same instruction. So on typical items, a model's correct self-prediction is projection by construction. The only place self-knowledge can show is on items where the model's actual behaviour departs from what models like it usually do. In my battery that may be a small fraction of items.
The difficult choice. To get a projection baseline back, I substitute the same model's prediction about a generic similar other ("what would the assistant answer?", "what would a different assistant model answer?"). Self-knowledge is then: how much better the "you" prediction tracks the model's actual behaviour than the "generic other" prediction does, counted only on items where the behaviour is idiosyncratic (it departs from a population of other model checkpoints), within bins of how decided the model was. I define "idiosyncratic" from behaviour, never from the self-report, but it is still selection on the outcome.
Questions for other fields.
- Is "the judge's prediction for a generic similar other" a legitimate stand-in for projection when the target is the judge? Has psychology or forecasting used that substitution, and what went wrong with it? (I recall, but have not read, Epley & Dunning 2000 reporting that people's self-predictions lose to predictions built from peers' base rates. If that pattern is general, my hopeful outcome is less likely.)
- Forecasting has base rates vs case-specific information; psychometrics has residualised scores. How do you define the "case-specific" subset without the selection-on-outcome problem (regression to the mean shrinks every predictor there)?
- If the idiosyncratic subset is small (say 30–60 items of 240), what estimator do you trust: jointly estimated tracking and projection, hierarchical shrinkage, or something else?
Any of the three would change how I register the next experiment before I collect its data. Disagreement welcome, especially a case where a high overall match rate was good evidence of self-knowledge.