
######## grpo  (checker claude-sonnet-4-6)
WINDOW (n=240 answers, 29 problems): sup/con per answer
  last5  0.90/0.74 prec 0.548 | swap 0.71/0.76
  last50 1.55/0.31 prec 0.832 | swap 1.33/0.50
  d_sup 50-5 0.65 [0.44, 0.88]  d_con 50-5 -0.42 [-0.58, -0.27]
  true-swap d_sup @5 0.18 [0.03, 0.33]  @50 0.21 [0.04, 0.38]
POOL methods (n=536): sup / con / unres per answer; paired diff vs greedy
  greedy_true      0.81 / 0.72 / 0.03  prec 0.530  words 19.5
  greedy_noact     0.01 / 1.43 / 0.06  prec 0.010  words 22.4  d_sup -0.79 [-0.92, -0.67]  d_con 0.71 [0.59, 0.82]
  greedy_op        0.05 / 1.43 / 0.01  prec 0.036  words 19.4  d_sup -0.76 [-0.88, -0.62]  d_con 0.71 [0.55, 0.87]
  greedy_sp        0.68 / 0.80 / 0.07  prec 0.460  words 19.4  d_sup -0.13 [-0.20, -0.06]  d_con 0.08 [-0.00, 0.17]
  sel_random       0.50 / 0.99 / 0.07  prec 0.333  words 19.5  d_sup -0.31 [-0.44, -0.19]  d_con 0.27 [0.14, 0.39]
  sel_loglik       0.54 / 0.39 / 0.03  prec 0.580  words 12.2  d_sup -0.27 [-0.38, -0.17]  d_con -0.33 [-0.41, -0.25]
  sel_loglik_mean  0.79 / 0.71 / 0.03  prec 0.527  words 19.2  d_sup -0.02 [-0.06, 0.01]  d_con -0.01 [-0.04, 0.01]
  sel_temp0.5_op   0.74 / 0.66 / 0.02  prec 0.529  words 16.7  d_sup -0.07 [-0.14, -0.01]  d_con -0.06 [-0.10, -0.02]
  sel_pmimean_op   0.60 / 0.73 / 0.04  prec 0.450  words 13.9  d_sup -0.21 [-0.33, -0.10]  d_con 0.01 [-0.09, 0.11]
AGREEMENT (greedy specific claims n=815, supported 0.531): AUC agr 0.635 [0.577, 0.688]  AUC agr_op 0.562 [0.514, 0.609]  AUC agr within agr_op quintiles 0.620 [0.565, 0.676]
  pre-registered thr 0.3892    coverage 0.261  kept sup 146 con 67  precision 0.685 vs all 0.531  gain +0.154 [+0.068, +0.239]
  pre-registered thr 0.5       coverage 0.130  kept sup 77 con 29  precision 0.726 vs all 0.531  gain +0.195 [+0.046, +0.328]
  exploratory coverage 0.7     coverage 0.699  kept sup 333 con 237  precision 0.584 vs all 0.531  gain +0.053 [+0.028, +0.078]
  exploratory coverage 0.5     coverage 0.501  kept sup 262 con 146  precision 0.642 vs all 0.531  gain +0.111 [+0.071, +0.153]

######## original  (checker claude-sonnet-4-6)
WINDOW (n=240 answers, 29 problems): sup/con per answer
  last5  0.52/0.68 prec 0.433 | swap 0.70/0.66
  last50 1.31/0.30 prec 0.812 | swap 1.15/0.39
  d_sup 50-5 0.79 [0.64, 0.94]  d_con 50-5 -0.38 [-0.52, -0.25]
  true-swap d_sup @5 -0.17 [-0.31, -0.05]  @50 0.16 [0.02, 0.31]
POOL methods (n=536): sup / con / unres per answer; paired diff vs greedy
  greedy_true      0.60 / 0.69 / 0.04  prec 0.466  words 18.8
  greedy_noact     0.00 / 2.00 / 0.00  prec 0.000  words 19.1  d_sup -0.60 [-0.72, -0.48]  d_con 1.31 [1.20, 1.42]
  greedy_op        0.04 / 1.35 / 0.02  prec 0.031  words 18.8  d_sup -0.56 [-0.67, -0.44]  d_con 0.66 [0.54, 0.78]
  greedy_sp        0.60 / 0.66 / 0.04  prec 0.474  words 18.9  d_sup -0.00 [-0.06, 0.06]  d_con -0.02 [-0.11, 0.06]
  sel_random       0.62 / 0.93 / 0.08  prec 0.401  words 18.3  d_sup 0.02 [-0.11, 0.15]  d_con 0.24 [0.12, 0.35]
  sel_loglik       0.69 / 0.58 / 0.05  prec 0.545  words 16.6  d_sup 0.09 [0.05, 0.14]  d_con -0.11 [-0.16, -0.06]
  sel_loglik_mean  0.64 / 0.68 / 0.04  prec 0.483  words 18.7  d_sup 0.04 [0.02, 0.06]  d_con -0.00 [-0.03, 0.02]
  sel_temp0.5_op   0.66 / 0.72 / 0.04  prec 0.479  words 18.2  d_sup 0.06 [0.02, 0.10]  d_con 0.03 [-0.02, 0.07]
  sel_pmimean_op   0.69 / 0.93 / 0.07  prec 0.425  words 17.4  d_sup 0.09 [-0.02, 0.20]  d_con 0.24 [0.16, 0.33]
AGREEMENT (greedy specific claims n=688, supported 0.467): AUC agr 0.587 [0.544, 0.624]  AUC agr_op 0.504 [0.448, 0.562]  AUC agr within agr_op quintiles 0.600 [0.551, 0.640]
  pre-registered thr 0.4       coverage 0.189  kept sup 84 con 46  precision 0.646 vs all 0.467  gain +0.180 [+0.077, +0.274]
  pre-registered thr 0.5       coverage 0.074  kept sup 36 con 15  precision 0.706 vs all 0.467  gain +0.239 [+0.077, +0.380]
  exploratory coverage 0.7     coverage 0.701  kept sup 238 con 244  precision 0.494 vs all 0.467  gain +0.027 [+0.005, +0.051]
  exploratory coverage 0.5     coverage 0.500  kept sup 182 con 162  precision 0.529 vs all 0.467  gain +0.063 [+0.026, +0.098]

######## improved  (checker claude-sonnet-4-6)
WINDOW (n=240 answers, 29 problems): sup/con per answer
  last5  1.06/1.73 prec 0.380 | swap 0.72/2.01
  last50 2.08/0.72 prec 0.742 | swap 1.73/1.04
  d_sup 50-5 1.02 [0.78, 1.27]  d_con 50-5 -1.00 [-1.24, -0.80]
  true-swap d_sup @5 0.34 [0.18, 0.51]  @50 0.35 [0.15, 0.55]
POOL methods (n=536): sup / con / unres per answer; paired diff vs greedy
  greedy_true      0.94 / 1.73 / 0.06  prec 0.353  words 41.9
  greedy_noact     0.01 / 2.02 / 0.00  prec 0.003  words 49.2  d_sup -0.94 [-1.12, -0.76]  d_con 0.29 [0.09, 0.47]
  greedy_op        0.04 / 2.79 / 0.02  prec 0.013  words 41.7  d_sup -0.91 [-1.10, -0.73]  d_con 1.06 [0.84, 1.27]
  greedy_sp        0.71 / 1.94 / 0.09  prec 0.268  words 42.0  d_sup -0.23 [-0.33, -0.13]  d_con 0.21 [0.11, 0.32]
  sel_random       0.66 / 2.21 / 0.07  prec 0.230  words 44.4  d_sup -0.28 [-0.46, -0.11]  d_con 0.48 [0.31, 0.64]
  sel_loglik       0.97 / 1.66 / 0.06  prec 0.368  words 40.8  d_sup 0.02 [-0.01, 0.05]  d_con -0.07 [-0.12, -0.03]
  sel_loglik_mean  0.96 / 1.76 / 0.05  prec 0.352  words 42.1  d_sup 0.01 [-0.02, 0.04]  d_con 0.03 [-0.01, 0.06]
  sel_temp0.5_op   1.00 / 1.70 / 0.06  prec 0.370  words 41.3  d_sup 0.05 [0.01, 0.10]  d_con -0.03 [-0.07, 0.00]
  sel_pmimean_op   0.74 / 1.78 / 0.09  prec 0.294  words 39.0  d_sup -0.20 [-0.34, -0.06]  d_con 0.05 [-0.04, 0.17]
AGREEMENT (greedy specific claims n=1426, supported 0.353): AUC agr 0.613 [0.568, 0.659]  AUC agr_op 0.512 [0.464, 0.558]  AUC agr within agr_op quintiles 0.610 [0.563, 0.657]
  pre-registered thr 0.4167    coverage 0.405  kept sup 263 con 315  precision 0.455 vs all 0.353  gain +0.102 [+0.059, +0.149]
  pre-registered thr 0.5       coverage 0.276  kept sup 189 con 205  precision 0.480 vs all 0.353  gain +0.127 [+0.065, +0.188]
  exploratory coverage 0.7     coverage 0.700  kept sup 394 con 604  precision 0.395 vs all 0.353  gain +0.042 [+0.023, +0.062]
  exploratory coverage 0.5     coverage 0.500  kept sup 297 con 416  precision 0.417 vs all 0.353  gain +0.064 [+0.030, +0.098]

######## LEAKAGE (greedy, held-out): per answer supported/contradicted specific claims
  grpo      n=536 v2 1.91/1.04  v3 0.81/0.72  v2 supported claims with >50% content words absent from answer: 0.49
  original  n=536 v2 1.62/1.06  v3 0.60/0.69  v2 supported claims with >50% content words absent from answer: 0.44
  improved  n=536 v2 1.73/2.05  v3 0.94/1.73  v2 supported claims with >50% content words absent from answer: 0.31

######## AGREEMENT under the leaky v2 checker (same answers; claims from evidence-aware extraction)
  grpo      n=1575 supported 0.648: AUC agr 0.746 [0.709, 0.780]  AUC agr_op 0.713 [0.672, 0.749]  within agr_op quintiles 0.611 [0.582, 0.645]
  original  n=1439 supported 0.605: AUC agr 0.749 [0.708, 0.788]  AUC agr_op 0.716 [0.672, 0.754]  within agr_op quintiles 0.653 [0.585, 0.698]
  improved  n=2028 supported 0.458: AUC agr 0.701 [0.653, 0.745]  AUC agr_op 0.612 [0.566, 0.657]  within agr_op quintiles 0.669 [0.631, 0.707]

######## SELECTOR RECOMPUTATION (fraction of items where recomputed choice text equals the annotated selector answer)
  grpo      sel_loglik 1.000  sel_loglik_mean 1.000  sel_temp0.5_op 1.000  sel_pmimean_op 1.000
  original  sel_loglik 0.998  sel_loglik_mean 1.000  sel_temp0.5_op 1.000  sel_pmimean_op 1.000
  improved  sel_loglik 1.000  sel_loglik_mean 1.000  sel_temp0.5_op 0.998  sel_pmimean_op 0.998
