← Notebook library

Research Round 27 · 18 October 2026

Round 27 - Which Check Should Run Next? Model-Set-Safe Verification Under a Fixed Budget

Reasoning & VerificationEvaluation ScienceArtificial Cognitive ArchitectureOne exact finite constructed diagnostic executed · one shadow experiment proposedArchitecture impact: None

Research question

Under a fixed observation budget, can a sequential verification policy improve correct resolution while avoiding harmful false confidence when the cost, efficacy, and independence of checks are uncertain?

Summary

E27-A is exact only for the declared scenario set, deterministic observation rules, authored cost tables, priors, and policies. It enumerates 58 worlds per dependency variant, a 116-scenario planning set, 9,744 policy-world runs, and 504 aggregate evaluation cells. Four model-set gate policies produced no wrong categorical verdict in 3,712 in-model runs; in the out-of-model hidden_common_origin stress the exact policy's false-clean rate reached 33.68 percent at budget 9. A myopic adaptive heuristic sometimes underperformed a fixed target-first schedule. E27-A is not a production trial, not a language-model reasoning benchmark, and no live Encevra benchmark was run. E27-B and H27.1 to H27.7 are proposed. Architecture impact: None. Production impact: None. The archive ended at Round 26, making Round 27 sequential.

Major findings

  • A model-set unanimity gate separates verdict safety from scheduling efficiency under fixed verification budgets: four policies using the 116-scenario gate produced no wrong categorical verdict in any of the 3,712 in-model policy-world runs, while a nominal target-entropy policy resolved 100 percent of cases but falsely emitted NO_CHANGE in 11.23 percent of changed cases in one regime.
  • An out-of-model common-origin stress shows that extra evidence can increase false confidence: with the hidden_common_origin variant excluded from the planning set, the exact MSAV policy's false-clean rate reached 33.68 percent among changed worlds at budget 9. Myopic adaptivity can underperform a fixed target-first schedule.

Experiments

  • E27-A Exact finite constructed diagnostic of budgeted verification policies Executed (diagnostic)
  • E27-B Fixed-compute shadow comparison of model-set adaptive verification Proposed

Hypotheses

  • H27.1 For every scenario inside a correctly specified finite model set, a target-unanimity gate will never emit the wrong categorical target.
  • H27.2 At matched compute in strata with correlated evidence roots, a model-set gate will have a lower harmful false-clean rate than a nominal independence gate, while retaining nonzero correct resolution.
  • H27.3 Given the same declared model, costs, and planning distribution, finite-horizon optimization will weakly dominate any included fixed schedule on expected categorical resolution.
  • H27.4 A greedy next-check policy can underperform a target-first fixed schedule because an attractive immediate reduction can block an affordable completing path.
  • H27.5 When apparently independent checks share an omitted root, increasing budget can increase the conditional false-clean rate.
  • H27.6 A scheduler that models both target value and check efficacy will improve cost-to-correct-resolution or selective risk under shifts in check reliability.
  • H27.7 Among abstained cases, the recorded number and structure of cross-target scenario pairs will predict which additional checks later change the answer.

Architecture implications

Research-only proposal. MSAV was not installed. No release gate, claim state, retention policy, or production behavior changed. Production impact: None.

Open questions

  • Does model-set adaptive verification reduce harmful false-clean rates at fixed compute with hidden independent adjudication in E27-B?

Connected research

Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra Research.