Research Round 12 · 4 October 2026
After the last revision: calibrating release risk
Reasoning & VerificationEvaluation SciencePublished · experiment proposedArchitecture impact: High
Research question
Can the final revised answer be released under a defined selective-risk target?
Summary
Proposes calibrating the terminal answer object rather than the first draft, and tests whether revision and provenance features add useful release coverage. Includes exact synthetic diagnostics; no new model benchmark.
Major findings
- Exact synthetic diagnostics show how candidate thresholds trade coverage for an upper risk bound at fixed δ = 5% and family size 3. These are worked calculations, not model performance.
Experiments
- E12-A Exact synthetic calibration diagnostics Executed (diagnostic)
- E12-B Terminal release calibration study Proposed
Hypotheses
- H12-1 Terminal evidence-and-revision features improve held-out release coverage by at least five points at a 2% selective-risk target.
- H12-2 Revision state adds information beyond task difficulty.
- H12-3 Answer binding catches content changed after scoring.
- H12-4 Pooled performance conceals predictable regressions in revised and conflicted episodes.
Architecture implications
- Revision-drift guard. Flag synthesis that omits supported risks or adds untraced claims, rather than presenting it as consensus.
- Uncalibrated release-risk contract. Return a null risk score until version-bound, externally labelled calibration exists.
Open questions
- Do terminal evidence and revision features improve useful coverage at identical generation compute?
- Do revised and conflicted episodes need separate risk calibration?
Connected research
Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra Research.
