Friction-Guided Inference: Calibrating Correction Strategies and Abstention from Logprob Signals
Pødenphant Lund, T. (2026) · Preprint · Live on Zenodo
Large language models frequently possess the knowledge needed to answer correctly yet commit to the wrong response: the correct answer sits in the top-k distribution, narrowly outranked. The paper uses the model’s own logprob distribution for two jobs, each with its own quantity. The one it proposes is calibration: a competing-routes profile assigns correction strategies per region of a model–benchmark cell. The one it deploys rather than proposes is abstention on a Shannon-entropy threshold, which is the reject option applied to a language model. CR guides where to intervene; entropy, not CR, decides when to decline; neither decides which answer to keep.
| DOI (concept) | 10.5281/zenodo.20014121 |
| Status | v4 live on Zenodo (2026-07-23); the master has since been revised and declared publish-ready |
| Author | Tomas Pødenphant Lund [ORCID] |
Three limits, stated up front
- The deployed abstention component is thresholded on Shannon entropy, not on CR. Both are measured and entropy wins.
- Whether a correction strategy helps is decided by counting rescues, not by reading CR.
- CR does not reliably determine which answer is correct at the individual-question level (§5.1).
Results: four held-out cells
All four cells are statistically significant, with bootstrap 95% confidence intervals excluding zero; the mean is +11.8 pp. Results are on held-out questions, calibration questions excluded.
| Model | Architecture | Benchmark | n | Vanilla | Adaptive | Lift | 95% CI |
|---|---|---|---|---|---|---|---|
| Qwen2.5-7B | Dense transformer | MATH-500 L4–5 | 212 | 45.8% | 66.5% | +20.8 pp | [+15.1, +26.4] |
| Qwen3-235B | Mixture-of-experts | SimpleQA | 3,810 | 41.1% | 51.8% | +10.6 pp | [+9.1, +12.1] |
| Qwen3-235B | Mixture-of-experts | MMLU-Pro STEM | 4,498 | 55.2% | 62.9% | +7.7 pp | [+6.4, +9.0] |
| LiquidAI LFM2 | Liquid Neural Network | MMLU-Pro STEM | 4,701 | 33.8% | 41.9% | +8.1 pp | [+7.1, +9.2] |
On SimpleQA the strategy pipeline lifts Qwen3-235B to 51.8%, above GPT-4o (38.0%) and GPT-4.1 (40.0%) on the same metric.
Ablation: what CR actually contributes
The central question is whether CR adds value beyond re-running with a fixed strategy. It does not, at the answer-selection level:
| Cell | Vanilla | Blind strategy | CR-weighted | CR advantage |
|---|---|---|---|---|
| SimpleQA / Qwen3 | 41.6% | 52.5% | 50.7% | −1.7 pp |
| MMLU-Pro / Qwen3 | 55.2% | 64.8% | 63.2% | −1.6 pp |
| MMLU-Pro / LiquidAI | 33.8% | 48.1% | 41.9% | −6.1 pp |
CR-weighted commit does not outperform blind strategy commit on any of the three cells. The lift comes from running the right strategy, not from CR-weighted answer selection. CR’s value is diagnostic and sits at calibration time: its profile shows where a cell’s friction lies and which regions no strategy reaches. It is not a per-question answer selector.
Abstention: entropy, not CR
Abstention ranks questions by a per-question uncertainty signal, declines the highest-uncertainty fraction, and scores accuracy on the answered set. Three signals are free from the same distribution. Error-detection AUROC for predicting that the vanilla answer is wrong:
| Cell | n | CR | Entropy | Max-prob |
|---|---|---|---|---|
| MATH-500 / Qwen2.5-7B | 262 | 0.746 | 0.754 | 0.614 |
| SimpleQA / Qwen3-235B | 4,326 | 0.771 | 0.818 | 0.819 |
| MMLU-Pro / Qwen3-235B | 4,751 | 0.651 | 0.690 | 0.686 |
| MMLU-Pro / LiquidAI | 4,751 | 0.636 | 0.692 | 0.688 |
| GPQA Diamond / GPT-oss-20B | 198 | 0.412 | 0.467 | 0.467 |
Entropy equals or exceeds CR on all five cells, and not as an artefact of the threshold: it matches or beats CR at every cut from 0.05 to 0.25, and CR’s AUROC falls monotonically as the threshold rises, so the 0.10 operating point is close to the best available to CR. Max-probability collapses on MATH, where answers are multi-token expressions on which the model is locally confident token by token. Entropy is the only signal robust across all five cells and is the deployed abstention signal. Thresholding a confidence signal to decline is the reject option (Chow, 1970) and selective classification (Geifman & El-Yaniv, 2017); the paper deploys it rather than proposing it.
At 20% abstention the entropy threshold yields +9.0 to +15.7 pp success-rate improvement on the four cells, reported as a deployment note and computed on full-set fresh vanilla generations rather than a held-out split. Deploying strategy and abstention together gave +12 to +21 pp end-to-end, but those combined runs used CR-ranked abstention, which the ablation shows is slightly weaker than entropy-ranked; that figure is therefore a deployment note rather than a primary claim.
The friction ceiling
Confident-wrong is structurally invisible to friction-based methods. Signals computed from logprobs measure the cost of choosing between routes, not which route is correct, and confident-wrong is the regime in which the model is not uncertain. GPT-oss-20B on GPQA Diamond (198 PhD-level science questions) is included as the negative case: none of the three signals detects errors there. Its stored CR is a different estimator from the other rows; re-scored with the canonical count on the same rows it gives 0.431, the same side of 0.5, so the conclusion stands and what changes is the reason: the signal points consistently the wrong way rather than being noise.
Under the companion framework this pattern is interpreted as race-resolution under bounded resources. That interpretation is offered as context, not as a premise the empirical results rely on.
Strategy benefit is regime-dependent
The optimal strategy is model-specific: step-by-step for Qwen2.5 on MATH, verification for LiquidAI, narrowing for Qwen3 on MMLU-Pro. Challenge prompting is destructive on SimpleQA and on MMLU-Pro/Qwen3, neutral on LiquidAI, and constructive only on Cogito. Pre-mortem’s gain is concentrated where a rushed prompt has suppressed the model’s deliberation into excess explicit abstention over questions it can in fact answer, and is small where the model already deliberates. No simple logprob pre-selector tested predicts which questions a strategy will rescue, so the extra call is best gated post hoc on answer disagreement.
Calibration
- Breadth. Run every available strategy on 50–200 questions and classify each as constructive, destructive or neutral by net rescues.
- Depth. Repeat the best constructive strategy up to 20 times on 50 questions to find the oracle plateau, where further rounds yield no new correct answers for three or more consecutive rounds.
- Audit. An automated check of data quality, judge bias, grading consistency and plateau produces a JSON report and blocks the adaptive pipeline if any check fails.
Calibration is per model and per benchmark, and transferring it between cells is prohibited. The released thresholds were fitted on CR values carrying a double-count; they are self-consistent within the work but not portable to a corrected implementation, so a re-implementation should re-run calibration rather than reuse the numbers. Calibration requires no weight update and a labelled set of 50–200 questions per cell, at a cost that is provider- and checkpoint-dependent.
Scope
The extraction code transfers without modification across the architecture families in the panel: a dense transformer, a mixture-of-experts model and a Liquid Neural Network. This is evidence that the signal generalises across the autoregressive architectures tested, not a claim of full architectural universality. The MoE model is still transformer-based, and the Liquid Neural Network result is one model on one benchmark with a low vanilla baseline. Establishing universality would require a wider panel including stronger non-attention models.
Companion papers
- Paper 0 (BFT) — the behavioural framework behind the notion of friction.
- Paper 21 — the same readout used as a measurement model rather than to improve answers.