Research Round 28 · 19 October 2026
Round 28 - Confidence About What? Separating Target Probability, Structural Ambiguity, and Model Adequacy
Research question
Under a fixed evidence and compute budget, does separating target probability, structural ambiguity, and model adequacy support safer abstention than a single confidence scalar?
Summary
Model-set intervals were conditionally valid in-model, but passive adequacy signals failed on the hidden-common-origin alias. TSAR did not universally dominate a scalar. E28-A is an exact constructed trace replay of 9,744 frozen histories and 29,232 weighted rows, not a production, language-model, or Encevra answer benchmark. Round 29 remains a proposed study of active dependency interventions that can reveal omitted dependencies.
Major findings
- Declared-variant model intervals covered exact conditional probabilities in 100.00% of weighted histories under all three authored priors. Held-out hidden-common-origin coverage was 72.08% (uniform), 60.57% (benign), and 65.43% (issuer shift). These are exact constructed results, not production estimates.
- All hidden-common-origin histories remained compatible with the declared set. Passive adequacy AUROC ranged from 0.488 to 0.577. TSAR did not systematically dominate scalar routing at matched coverage; most cells tied.
Experiments
- E28-A Exact fixed-compute constructed trace replay Executed (diagnostic)
Hypotheses
- H28.1 For every evaluated history generated by a declared model with matched prior and likelihood, the exact conditional target probability lies inside the reported model interval.
- H28.2 At matched coverage, TSAR's conservative set risk yields lower delayed-correction risk than pooled scalar risk across both declared variants and all preregistered priors.
- H28.3 Surprise, interval width, or compatibility detects the held-out hidden-common-origin variant with AUROC at least 0.75 and useful precision above prevalence.
- H28.4 Replacing one passive repeat check with a provenance-changing intervention will raise held-out model detection above AUROC 0.75 and reduce false-clean rate at equal total cost.
- H28.5 On a preregistered multi-agent benchmark with delayed adjudication, an interoperable TSAR schema will improve correction-risk Brier score and 90%-coverage risk over a one-number confidence message, without reducing task accuracy or increasing total tokens/tool cost.
Architecture implications
Research-only uncertainty record. No production or language-model benchmark was run. No deployment change.
Open questions
- Can a fixed-cost active intervention discover a missing evidence dependency that passive confidence, disagreement, and surprise cannot detect?
Connected research
Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra Research.
