All research
Contents

Encevra Research Round 11

When Should Collaborators Spend Their Next Unit of Compute? Marginal-Value Allocation for Verification Under a Shared Budget

Expanded revision 2 | 3 October 2026
Encevra Research | Research synthesis, protocol proposal, and exact illustrative diagnostics

Evidence status. No new Encevra or language-model benchmark was run. The numerical results in Section 9 are exact calculations under explicitly stipulated toy assumptions. They test logical weaknesses in proposed decision rules; they do not establish that the proposed controller improves an AI system. Literature results remain author-reported and were not reproduced here.

Abstract

Reliable collaboration requires deciding which additional check is worth its cost. This expanded Round 11 examines that decision after Encevra's work on verified experience, evidence exchange, and selective revision. An audit of the archive and primary literature substantially narrows the proposed contribution: adaptive compute allocation, learned verification routing, and value-of-computation control already exist. The useful next question is whether a controller can learn, from information available before a check, which claim-specific action or short action bundle will repair an answer without damaging correct material, at a fixed end-to-end budget.

The proposed protocol predicts repair, damage, and cost; maintains a frozen initial answer and evidence dependencies; compares checking with stopping; and evaluates short bundles where individual checks are uninformative. It treats verifier acceptance, factual correctness, and confidence as separate quantities. We specify counterfactual data collection, leakage controls, matched-cost baselines, accounting rules, regression metrics, and rejection criteria. Exact examples show why uncertainty alone is insufficient, why a greedy stopping rule can fail, and why repeated agreement can remain wrong. The proposed experiment can disconfirm the controller's value. Its outcome should determine whether Encevra keeps a learned policy or the best simple fixed workflow.

1. What this revision changes

The first Round 11 draft correctly identified resource allocation as an important operational question, but left the value estimator and experimental identification underspecified. This revision makes five substantive corrections.

  1. Prior work is closer than the first draft acknowledged. BudgetVerify, posted 27 September 2026, already proposes routing among verification levels using pre-verification information [S3]. Encevra cannot claim that general idea as new.
  2. Repair and damage must be measured separately. A check can fix some wrong answers while reducing overall accuracy. Net accuracy already includes both directions; an additional damage penalty must represent an explicit preference, not accidental double counting.
  3. A one-step stopping rule is incomplete. Two checks can jointly reveal an answer even when neither improves the answer alone. Short bundles are a candidate remedy with their own overhead.
  4. Cost equality needs an operational definition. Equal requested reasoning limits, equal model-call counts, equal actual spending, and equal hardware work are different comparisons. Reported reasoning usage can also be a subset of output usage.
  5. The archive and equations are corrected. This is an expansion of the existing Round 11, not a new Round 12. Equations use readable code notation to avoid broken escape sequences in the current Markdown renderer. Source versions, access limitations, and the distinction between verification scores and truth are explicit.

2. Archive continuity and scope

The current Lovable archive contains Round 11 with this title. The research overview and rounds navigation link it, and the publications data include it. The review inspected the existing Round 11, full Round 9 and Round 10 page texts, and relevant passages from earlier work. Round 4 now redirects to the combined Rounds 4 to 6 page; its standalone redirect is not its research text. Archive references [A1–A6] identify the inspected pages.

Earlier workCarry-forward into Round 11
Round 3: memory, replay, compositional reasoningA controller with explicit call, time, and cost limits was already an engineering proposal.
Round 4, in the combined reviewCounterevidence and uncertainty were already studied; seeing evidence and using it are distinct problems.
Round 8: change while preserving dependable functionWorkload regulation and retaining valid behavior were already proposed experiments.
Round 9: reusable procedures from verified experienceA small check-selection controller was already contemplated. This round specifies how to collect its training evidence and evaluate it.
Round 10: evidence exchange and selective revisionClaim dependencies, correction-versus-damage measurements, and fair budgets are existing commitments. This round makes the scheduling decision concrete.

Round 10's roadmap places expected check value and learned control in a later block, while placing fair budgets and initial correction-versus-damage measurements in Rounds 11–15. This revision develops a bounded control experiment early because the new literature directly challenges the original Round 11 framing. It does not claim an untouched topic or that prior Encevra proposals have already been validated.

Research question. Can a controller trained on pre-check information and independently measured outcomes select claim-specific checks and short bundles that improve task success over a strong whole-answer verification router at matched cost, while preserving initially correct material claims?

Candidate contribution. A test of the combination of localized action choice, explicit repair/damage prediction, and limited lookahead under audited costs. This is a research hypothesis with uncertain novelty at the implementation level, not a first-in-field claim.

Scope. One inference episode with a frozen backbone, a bounded action menu, and a controller trained before evaluation. The main experiment uses shared initial answers to isolate verification policy. A secondary experiment includes initial generation to assess end-to-end utility.

Exclusions. Learning reusable procedures during test episodes; persistent memory architecture; general confidence-estimation theory; unrestricted agent topology search; claims of AGI; biological equivalence; and certifying truth from a model's verbal confidence. This round consumes the evidence interface proposed earlier rather than redesigning it.

3. Literature map and competing positions

The search covered metareasoning, adaptive inference, generative and multi-agent verification, budget-matched reasoning, sequential inference, off-policy evaluation, and the expected-value-of-control theory. It used primary papers, proceedings, author material, and official technical documentation. This is a targeted research review, not an exhaustive systematic review. Retrieval and inspection occurred on 3 October 2026. The source ledger records dates, inspected sections, and access limits.

Research lineFinding and boundaryConsequence for the proposal
Metareasoning [S1]Computation selection is a sequential decision problem. Myopic policies can stop before a useful sequence is completed.Marginal-value scheduling is established prior art; compare limited lookahead with greedy control.
Learned compute allocation [S2]Learned marginal rewards support adaptive allocation. Some evaluations use oracle verification/pass-at-k; estimation errors can undermine online allocation.Learn check value, but test an attainable selector and a simpler regularized policy.
Learned verification tiers [S3]The financial-QA abstract describes no-check, light-check, and solve-first/compare routing from pre-check features and offline outcomes. Full text was inaccessible.A whole-answer tier router is a required close baseline; exact reproduction awaits full methods.
Budgeted multi-attribute verification [S4]BMA-GAI allocates checks across fixed candidates and attributes under a global budget. Its certified property concerns mean verifier scores.Distinguish batch certification from revising one answer and distinguish score validity from truth.
Generate versus verify [S5]Self-consistency is often more efficient at lower budgets; generative verification becomes competitive in some higher-budget regimes.A verification-heavy baseline is not automatically strong at every budget.
Multiple verifiers [S6]Aspect-specialized verifiers show gains in the studied tasks, with low-budget overhead. Scaling is largely expressed in combined model-call counts.Specialization is plausible, but token, price, and latency comparisons still need measurement.
Budget-aware reasoning [S7]Strong simple sampling competes well with elaborate methods; outcomes vary by model and task.Tune simple baselines and count the whole pipeline.
Matched thinking tokens [S8]Single-agent parity or advantage is reported on multi-hop QA. Prompts and final answers are excluded from the controlled thinking-token quantity.This is a scoped finding, not a full-cost comparison or a universal single-agent theorem.
Estimated hardware runtime [S9]Debate and mixture-of-agents improve some Pareto frontiers on MMLU-Pro and BBH with Llama 3.1. Cost is modeled from arithmetic and memory transfer.Apparently conflicting findings can reflect different tasks, backbones, and resource measures.
Intrinsic self-correction [S10]Older tested models sometimes damage correct answers when asked to reconsider without external feedback; prompt quality affects comparisons.Preserve a strong initial-answer baseline and measure both transition directions.

These positions need not contradict one another. An extra verifier can help relative to a weak chooser, lose to more candidate generation at a small budget, and help again with sufficient spending. Specialization can be useful when it exposes evidence a unified context fails to use. None of these observations establishes that an arbitrary team of agents should be used for every question.

Two distinctions are particularly consequential. Damani et al.'s oracle-based coverage evaluations do not by themselves solve selection with an imperfect verifier [S2]. Tran and Kiela's information argument concerns optimal predictors of a full context and messages derived from it; it does not establish dominance for finite models with unequal tool access. Their comparison of visible thought length with reported thought tokens also cannot establish hidden-compute inflation: the official API describes returned thought content as summaries [S8, S14].

4. Claim and evidence assessment

ClaimAssessmentEvidence and remaining uncertainty
Choosing computation adaptively is already an established research problem.Established theoretical framingMetareasoning [S1] and learned allocation [S2]; effectiveness depends on the environment and estimator.
More checking always improves a reliable answer.UnsupportedSelf-correction regressions [S10]; the repair/damage identity below gives a general counterexample.
Multi-agent verification has a universal cost advantage.Mixed evidenceStudies [S5–S9] use different tasks and resource definitions. No universal ordering follows.
A statistically valid verifier certificate establishes factual truth.False without an additional validity linkThe certified estimand in [S4] is a score mean. The score-to-truth relation requires external validation.
A calibrated current-correctness probability identifies the best check.InsufficientAction-dependent repair, damage, and cost remain unknown; Section 6 derives the distinction.
Localized checks plus short bundles beat a whole-answer router.Open Encevra hypothesisNo new model experiment in this round. Section 11 specifies the comparison that could reject it.
Controller risk estimates guarantee safe individual revisions.Not establishedHeld-out averages and calibration do not provide a per-instance truth guarantee.
A brain-inspired controller copies a brain mechanism.Not claimedThe analogy supplies an experimental manipulation, not a biological implementation claim.

The BMA-GAI model uses fixed candidate/attribute means, known costs, and conditional concentration assumptions. Its language-model illustration resamples verifier outcomes for 100 fixed GSM8K answers [S4]. For Encevra, changing the answer changes the estimation target. A consistently wrong verifier can have a correctly estimated acceptance mean. This limits what a certificate means without invalidating its statistical theorem.

5. The proposed controller

5.1 State and action contract

Each episode stores the question, the initial answer, a claim ledger, source identifiers and versions, a dependency graph, action history, residual resources, and the current candidate answer. Initial claims receive immutable identifiers. Every revision records which claims changed and which dependencies were rechecked. The initial answer is preserved for evaluation and rollback; preserving it does not imply it was correct.

The controller receives only information available at that moment. It may use answer length, missing citations, contradictions between named sources, tool availability, a model's reported confidence as an untrusted feature, and prior action results. It cannot see evaluation labels, future tool returns, or a hidden oracle warning that the answer is wrong.

ActionRequired returnCost and failure to record
StopCurrent answer with scoped unresolved pointsFinal formatting and evidence references still incur cost.
Deterministic checkCalculation, schema result, or executable assertion with inputsWrong extraction or a misapplied rule can defeat a correct calculator.
Targeted source lookupSource passage, version/date, and affected claimRetrieval may find a duplicate, obsolete, or inapplicable source.
Independent solveAn answer produced before the checker sees the candidate answerIndependence of prompts does not imply independent model errors.
Conflict resolutionComparison of named sources and their scopeNewer does not automatically mean authoritative or applicable.
Targeted revisionA bounded patch and dependency revalidationA local change may invalidate conclusions elsewhere.
Short bundleA predefined two-check plan with an integration stepInclude both checks, routing, integration, and finalization in its cost.

Cap the initial menu at six non-stop action templates, with at most two eligible bundles per state. This is an experimental restriction, not an optimal number. Use the same tool permissions and action implementations across policies. Human escalation is excluded from the primary benchmark because it introduces a different resource class; an explicit unresolved answer remains available.

5.2 Decision loop

  1. Freeze the initial answer and construct the ledger. Charge ledger construction to the policy that needs it.
  2. Enumerate eligible actions and bundles using observable evidence dependencies.
  3. Estimate each action's terminal utility change, damage risk, and resource use.
  4. Remove actions that cannot fit the remaining budget with a finalization reserve. Apply a frozen empirical risk gate where configured.
  5. Compare stopping, feasible individual actions, and short bundles. Select by predicted terminal value, with cost and risk preferences declared explicitly.
  6. Execute the chosen action, store its evidence, and update only claims justified by that evidence. Revalidate affected dependencies.
  7. Recompute the state and repeat until stopping, a resource limit, or the predefined horizon. Preserve unresolved qualifications.

Do not turn “the controller stopped” into “the answer was verified.” Stopping says that this policy did not select further work under its estimates and constraints. That is a decision about resources, not a certificate of correctness.

6. Predicting value before paying for a check

For a single binary claim, define the following conditional probabilities at state s for action a:

q = P(pre-action claim is correct | s)
r = P(post-action claim is correct | pre-action claim wrong, a, s)
d = P(post-action claim is wrong | pre-action claim correct, a, s)
expected_accuracy_gain(a | s) = (1 - q) * r - q * d

This follows by partitioning the pre-action state into correct and incorrect cases. It is an identity for the binary setting, not a learned result. Later actions use the current claim state; episode-level regression metrics separately retain the frozen initial answer. Knowing q alone leaves both r and d unknown. A very uncertain claim can be uncheckable with the available tools; a fairly certain claim can be cheap to verify deterministically.

For illustration, set r = 0.60 and d = 0.05. At q = 0.95, the expected gain is -0.0175, a 1.75 percentage-point loss. At q = 0.60, it is +0.21. All three probabilities are stipulated for the example. They are not estimates for any current model.

For a multi-claim answer, define a fixed task rubric before evaluating a policy. Required claims, importance weights, abstention treatment, and material errors come from the task specification, not the controller's choice of what to mention. Let U(y) score required-claim correctness and coverage against this rubric. A controller must not improve its score by deleting difficult claims or splitting one easy claim into many.

Track a separate harm outcome H: an initially correct material claim becomes incorrect, or a required supported claim is silently removed. Net utility already incorporates ordinary mistakes. If the objective also penalizes H, state that this is extra aversion to regressions. The main experiment instead uses net task success plus a separate harm constraint, making that preference visible.

6.1 Learning data and counterfactuals

Create development states by generating an initial answer and saving all observable inputs. For a small fixed menu, independently execute every eligible action from the same saved state, including stop and selected bundles. Label the resulting answers with executable ground truth where possible and blinded source-based adjudication elsewhere. Record the whole answer after the action, not just the checker's verdict.

This produces paired comparisons of potential actions at development states. It estimates average action value across repeated calls; it does not reveal every outcome an action could ever produce. At least two independent draws for stochastic actions help expose variance in the instrumentation pilot. More draws should be justified by measured uncertainty and cost.

Split by document family, task template, and source lineage before fitting any predictors. Near-duplicate questions and revised editions of the same document belong in the same split unless temporal transfer is the explicitly held-out test. Use separate training, tuning/calibration, and final test sets. Freeze extraction prompts, action templates, models, routing thresholds, and feature transforms before test execution.

Start with an interpretable regularized model or a small tree model predicting utility change, cost, and harm. Compare direct prediction of net change with factorized q/r/d prediction. The factorization may improve diagnosis but creates sparse conditional subsets and compounds estimation error. It is an ablation, not a requirement that three probabilities are always easier to learn.

A free-form model explanation is not a calibrated value estimate. Evaluate predictions out of sample using calibration plots and proper scoring rules where probabilities are predicted, and using decision regret and outcome ranking for action choice. Report performance on both the available action set and the action actually selected; the latter alone hides selection bias.

6.2 What logged traces can and cannot identify

Routine production logs often contain only the chosen action's result. A policy cannot safely infer an unchosen check's outcome by treating missing data as failure. Randomized exploration with recorded nonzero action probabilities supports appropriate one-step off-policy evaluation. Doubly robust estimation combines outcome modeling and propensity correction, but depends on its assumptions and action support [S11]. It does not automatically validate a multi-step revision policy.

For the first experiment, prefer full-menu development data and prospective held-out policy execution. Train bundle values using actual bundle outcomes. A one-step action table cannot identify arbitrary later trajectories because actions change the context seen by later calls. Replay is valid only for matching state/action contexts with a declared sampling scheme; do not manufacture unseen branches by reusing an answer generated under a different history.

6.3 Selection makes prediction error consequential

The controller selects the largest predicted benefit among several candidates. Even an unbiased predictor can overestimate the chosen maximum when its errors vary. To expose this effect, report selected-action prediction error separately, compare a small action menu with an expanded menu, and evaluate a conservative policy that shrinks uncertain estimates toward a fixed baseline. These are Encevra design checks. No guarantee is claimed that a bootstrap interval remains valid after adaptive selection or distribution shift.

7. Sequential value, stopping, and risk

The ideal finite-budget objective is to maximize expected final task utility over a policy. In a state sufficient to represent relevant history, a conceptual recursion is:

V(s, b) = max(
    expected utility of stopping in s,
    max over feasible a of E[V(next_state, b - actual_cost(a))]
)

Feasibility includes a finalization reserve and the configured resource cap. Unknown costs require a conservative reservation or a tool-enforced maximum. In a pure budget-constrained objective, spending is constrained through b. A separate deployment objective can subtract an explicit price for resource use, but the two interpretations should not be silently mixed.

This recursion is a specification, not an implemented optimal solver. A learned one-step gain/cost ratio is only a heuristic. Discrete costs can leave unusable residual budget; useful computations can be complementary; action outcomes can change the best next step. The metareasoning literature already establishes the limitations of myopic stopping [S1].

The practical proposal is a horizon-two controller over a bounded menu, with empirically learned bundle values. It compares predicted final outcomes of completing a bundle, including integration, with stopping and individual checks. A bundle is re-evaluated after its first result; its second step is skipped if it is no longer useful or feasible. Development data must contain the resulting contingent policy or clearly evaluate the fixed bundle instead. Do not claim that fixed-bundle data validate arbitrary contingent stopping.

Risk gate. On a held-out calibration set, estimate harmful-revision rates for action families and declared task strata. Freeze a conservative gate or shrinkage rule before testing. Sparse strata should fall back to a simpler policy. These are empirical population estimates. They do not certify an individual edit or guarantee behavior after a model, source, or task distribution changes.

If evaluation is monitored repeatedly, fixed-sample intervals cannot simply be inspected until a favorable result appears. A prespecified fixed test avoids that problem; sequential monitoring requires an appropriate time-uniform method and its conditional assumptions [S12]. A valid uncertainty bound on an observed score still does not establish the validity of that score as a measure of truth.

8. Worked example: a local factual check with a dependent conclusion

Consider an explicitly fictional technical review task. A report must state a component's measured load, whether its test configuration matches a certification table, and whether it passes a 20 kN requirement. An initial answer says: “The load is 22 kN; the configuration is certified; the component passes.”

The source packet contains a table with “22” under a column labeled lbf and a separate configuration appendix. The answer's three claims are linked: the unit conversion affects the load and the pass/fail conclusion; the configuration claim depends on a different source. The controller does not initially know which claim is wrong.

Candidate actionIllustrative predicted effectController implication
Reread the whole answer with another general criticLow predicted net benefit; nonzero chance of changing the correct configuration claimDo not prioritize merely because a critic is available.
Fetch the table header, then convert the valueA short bundle can distinguish 22 lbf from 22 kNReserve both retrieval and calculation/integration costs.
Recheck the configuration appendixUseful only if its scope or provenance is disputedKeep independent evidence intact unless the new finding affects it.

Suppose the table lookup confirms lbf. Using the task's supplied conversion, 22 * 4.44822 = 97.86084 N, or about 0.09786 kN. The load statement and pass/fail conclusion must change. The configuration statement should remain if its independent source is still applicable. This example supplies the conversion as task data; it is not an empirical extraction test.

The calculator alone cannot resolve a missing or misunderstood unit. A second general critic might repeat the original assumption. The bundle matters because retrieval and computation contribute different necessary pieces. After revision, the system checks that the final prose no longer implies a 20 kN pass and that citations refer to the correct table version. A cost ledger charges the lookup, conversion, revision, dependency check, and final output.

9. Exact diagnostic calculations run in this revision

The companion diagnostics.py uses Python's standard library and exact rational arithmetic for the probability calculations. It exhaustively enumerates the two-bit example and feasible action subsets. It makes no model or network calls. Assertions check the intended counterexamples. The generated JSON records all assumptions and outputs.

DiagnosticExact or rounded outputWhat it establishes
Repair versus damageWith repair 0.60 and damage 0.05, initial correctness 0.95 yields final correctness 0.9325; initial correctness 0.60 yields 0.81.The same check can help or hurt depending on starting correctness.
Complementary evidenceFor two independent fair bits whose XOR is the answer, observing neither or either one yields accuracy 0.50; observing both yields 1.00.A zero one-step accuracy gain does not imply a useless sequence.
Cost-sensitive stoppingIf each bit lookup costs 0.05 utility units, stop gives 0.50; one lookup gives 0.45; both give 0.90.A strictly myopic net-value rule stops, although a two-step plan is better.
Discrete gain/cost rankingAt budget 3, action A costs 2 for benefit 0.30, B costs 3 for 0.40. Greedy ratio chooses A; exhaustive selection chooses B.Gain per unit cost is not an optimal finite-budget rule in general.
Correlated verifier errorsWith a 20% common failure event and otherwise independent 10% errors, majority error is 28.00%, 22.24%, 20.6848%, and 20.0713% for 1, 3, 5, and 9 verifiers.Adding voters cannot remove the stipulated common failure floor.
Biased acceptance scoreA false claim with an always-accept verifier has truth value 0 and acceptance mean 1.Perfect score estimation can coexist with completely wrong factual judgment.

In the correlated-error example, the common failure event makes every verifier wrong; otherwise each independently errs with probability 0.10. The mixture is deliberately specified rather than estimated. In the XOR example, there is no clue about either bit before checking and no single bit predicts their parity. Real tasks need not exhibit this degree of complementarity.

These calculations establish counterexamples to universal rules. They do not estimate how frequently those cases occur in Encevra work, demonstrate that the proposed features recognize them, or show that the overhead of bundle selection pays for itself. Those are questions for the next experiment.

10. A brain-inspired idea with a separable test

The expected-value-of-control account proposes that allocation of cognitive control depends on expected payoff, required control, and effort cost [S13]. It is a theoretical synthesis, not evidence that an LLM controller should be mapped onto a brain region.

The engineering hypothesis is narrower: predict the benefit of a particular available intervention, rather than allocate effort solely according to conflict or uncertainty. Construct paired tasks with similar initial uncertainty but different checkability. In one condition, an inexpensive tool resolves the issue; in another, the missing evidence is unavailable. Independently vary check cost and error importance.

A controller consistent with the hypothesis should shift spending toward resolvable, consequential uncertainty and away from fruitless repeated checks. Compare it with uncertainty-first and conflict-first policies using identical features and budgets. If the value model does not outperform these simpler policies, retain the simpler rule. Success would support this engineering decision principle; it would not validate the neural theory or establish literal brain copying.

11. Proposed experiment and falsifiable hypotheses

11.1 Task collection and staged execution

Use document-grounded tasks with externally inspectable answers. Build four strata: numeric extraction and units; conflicting document versions; multi-hop conclusions requiring complementary evidence; and genuinely underspecified or unavailable-evidence cases. Include correct initial answers and answers sharing a plausible source error. Use both natural initial mistakes and a separately reported set of controlled perturbations. Artificially injected errors must not determine the headline result.

An instrumentation pilot can begin with 120 development tasks, balanced across strata, to measure extraction quality, label agreement, action eligibility, costs, and outcome variance. Generate full-menu outcomes here. This pilot is for fixing the protocol and estimating feasibility. Do not reuse it as confirmatory evidence.

A separate exploratory evaluation of approximately 320 held-out tasks can identify large effects and failure patterns. It is unlikely to establish a small improvement or a tight harm bound. After locking the protocol, size a new confirmatory set using observed paired discordance, source clustering, and the harm endpoint. Preserve a genuinely untouched temporal/source-family transfer set.

For scale, a normal-approximation calculation with 16% paired discordance, a two-percentage-point accuracy difference, two-sided 5% significance, and 80% power suggests about 3,140 independent tasks. This is an illustrative planning calculation, not a sample-size guarantee. Clustering, model randomness, multiple endpoints, and the regression gate may require more data. Repeated calls on one task do not create independent task families.

11.2 Baselines and primary comparison

PolicyWhy it is necessary
Strong single answer with toolsMeasures the incremental value and harm of any extra checking.
Tuned fixed check scheduleTests whether adaptation improves on a well-chosen simple workflow.
Self-consistency within budgetTests whether the same resources are better spent generating alternatives.
Whole-answer verification-tier routerClosest primary comparison: no check, light check, or solve-first/compare, trained on the same allowed evidence and costs. Inspired by [S3]; not a claimed reproduction while methods remain inaccessible.
Uncertainty-first local checkerTests whether predicted action usefulness adds value beyond uncertainty ranking.
Proposed claim/bundle controllerTests localized action choice, damage-aware decisions, and limited lookahead.

For the main comparison, both routers receive the same initial answers and permitted pre-check signals, use the same backbone and tools, and have the same development-label and tuning budget. The whole-answer router can use the same derived claim features if their construction costs are charged; the treatment then concerns available interventions, not privileged evidence. Report a simpler raw-feature version separately if appropriate.

The fixed schedule and self-consistency baselines get equally serious development tuning. Add any useful corrective instructions to the initial-answer prompt for all methods. If a baseline cannot meet the budget because its minimum action is too expensive, record that operating point as infeasible rather than quietly giving it more resources.

11.3 Ablations with a defined purpose

Run four primary ablations at the prespecified middle budget: remove explicit damage prediction; disallow bundles; remove source-lineage and dependency features; and replace learned action values with a fixed action ranking. Keep the action executor, initial answers, and cost accounting unchanged. The net-value predictor versus factorized q/r/d predictor is a focused estimator comparison.

A secondary topology comparison can execute the same checking policy with one model session versus multiple isolated checker sessions. Its purpose is to separate the scheduler's value from an assumed benefit of multiple agents. Do not conflate the number of model calls with the number of informationally independent checks.

An overhead-free rerun is a diagnostic only: it estimates how much controller cost explains losses. It cannot support a deployment superiority claim. Likewise, a chooser that sees realized gold outcomes is a hindsight ceiling on the sampled candidate pool, not an achievable policy or an unbiased estimate of the true best action. Repeated independent outcome draws are needed to estimate an action's expected value.

11.4 Hypotheses and rejection criteria

H1: useful allocation. At the preregistered middle cost ceiling, the proposed controller improves task success relative to the whole-answer router. A suggested practical target is at least two percentage points, with the paired 95% interval excluding zero. Failure to meet both conditions is not positive evidence for the proposal.

H2: preserved correctness. The upper one-sided 97.5% confidence limit for the increase in the harmful-revision rate is at most 0.5 percentage points. This is a proposed noninferiority margin to freeze before evaluation, not a known safe tolerance. The denominator is fixed initially correct material claims; inference clusters by task and source family. If the data are too sparse to establish the bound, the gate is inconclusive.

H3: complementarity matters. Bundle access improves task success on the prespecified complementary-evidence stratum after including its overhead. If the effect disappears against a fixed two-step bundle or under end-to-end accounting, the learned lookahead contribution is unsupported.

H4: action usefulness adds information. On pairs matched for initial uncertainty, predicted repair/damage and checkability outperform uncertainty-first routing. If gains are explained by easier instances, special tool access, or source leakage, reject the intended causal account.

The main advancement gate requires H1 and H2 together. H3 and H4 are mechanism tests with separately labeled, multiplicity-aware analyses. Do not select the most favorable budget or subgroup after looking at test outcomes. A null or harmful result is an actionable outcome: keep the best simple policy and document the regimes where the controller failed.

12. Resource accounting and controlled total cost

“Equal compute” is too broad without a measurement contract. The primary API experiment should use a frozen operational cost schedule and a hard episode ceiling. Record actual usage by every model and tool. A same-backbone experiment also reports total logical tokens. A local-model replication can measure hardware work and elapsed time under fixed hardware and batching. These are separate analyses.

Set low, middle, and high ceilings using the frozen development distribution, for example 1.5, 3, and 6 times the median one-pass episode cost. Lock the actual numerical values before test execution. The relative values are proposed starting points, not chosen after examining test accuracy. Primary inference uses the middle ceiling; the other points describe the frontier.

QuantityAccounting rule
Input tokensCount every request's full input and record cached portions separately. Record both logical work and the provider's billed cache treatment.
Output and reasoning usageInspect each API schema. If reasoning tokens are included within completion/output totals, report the subset without adding it again. Retain raw usage records.
Requested limitsRecord them separately from actual usage; a requested reasoning setting is not an observed work count.
Controller overheadInclude claim extraction, routing, candidate descriptions, rejected proposals, formatting, retries, and integration.
Tool and verifier callsInclude successful, empty, failed, duplicate, and retried calls, with frozen per-tool prices or a separately declared resource unit.
LatencyReport episode wall time and tail latency. Parallel call durations sum to work, not to critical-path latency. Record concurrency.
Training and tuningReport total offline data-generation, fitting, and search cost. Exclude it from per-episode inference only when explicitly reporting amortized deployment economics.
EvaluationKeep hidden gold labels and research adjudication outside the live controller. Their study cost is reported separately; any judge used by the policy counts as inference cost.

For a tool with variable cost, reserve a configured maximum and refund unused allocation. Do not begin a call that cannot fit with the finalization reserve. If the provider cannot enforce the relevant limit, label the ceiling as nominal and report violations; do not call it hard-controlled. Count overruns, timeouts, and incomplete tasks among all started episodes.

Two comparisons answer different questions: performance under the same maximum allowance, and performance at matched average actual spending. Report both when feasible. A policy that stops early may deserve credit for saving cost, but that alone does not show better accuracy at equal consumed resources. Match average spending through parameters selected on development data or prespecified mixtures of policies, not by discarding expensive failures after evaluation.

A shared budget within one answer is the primary setting here. Allocating across a batch of different questions changes the objective toward coverage or triage; report that as a separate experiment. A controller must not silently abandon hard users so that the average score of completed answers looks better.

13. Outcomes that expose harmful changes

The headline outcome is task success against a frozen external rubric. Report a companion weighted required-claim score and answer coverage so partial answers and abstentions are visible. Use task-level paired differences for the primary comparison; cluster uncertainty estimates by source family where tasks share evidence. Report every rate's numerator and denominator; a zero eligible denominator is not applicable, not a measured zero error rate.

MetricRequired definition
Repair rateInitially wrong material claims becoming correct, divided by initially wrong material claims.
Harmful-revision rateInitially correct material claims becoming wrong or silently removed, divided by initially correct material claims.
Wrong-to-wrong changesChanged incorrect claims that remain incorrect; useful to detect activity without progress.
Collateral damageErrors introduced in claims outside the intended patch, including downstream inconsistencies.
Coverage and abstentionRequired claims addressed, unresolved, or omitted; error among answered claims is reported with coverage.
Evidence validityCorrect source/version, passage support, and applicability; a citation's presence alone is insufficient.
Value prediction qualityHeld-out predicted versus realized utility changes, selected-action error, and decision regret on fully observed development menus.
CalibrationBrier score and reliability plots for explicit probabilities, separated from factual accuracy and stratified by action/task.
Resource complianceActual cost distribution, controller share, completion rate, overruns, and median/tail latency over all started tasks.
RobustnessHeld-out source families, source updates, correlated errors, missing tools, and backend/version changes.

Use blinded adjudication for disputed semantic claims, record disagreement, and provide resolution rules. An automatic judge sharing the solver's model or sources can reproduce its error. Report executable, human-adjudicated, and model-judged subsets separately rather than averaging away their different reliability.

The evaluation ledger must retain initial and final answer versions, semantic claim mappings, tool inputs/outputs, and resource records. If a patch changes the meaning enough to break claim identity, an adjudicator maps it to the fixed task rubric. The controller cannot rename a damaged claim to erase its regression.

14. Alternative explanations and expected failure modes

Apparent finding or failureAlternative explanation or mechanismDiscriminating check
Controller beats the initial answerIt merely receives better instructions or more evidenceGive the same instructions and tool access to the strong baseline.
Local checking beats tier routingIt receives privileged labels, features, or tuningEqualize allowed features, development data, and tuning budgets.
Multiple critics agreeShared training/source error or answer anchoringBlind independent solves; vary source lineage; include common-error tasks.
Risk gate looks safeIt avoids most useful work or drops hard claimsReport coverage, repair, completion, and omission penalties together.
High predicted gains disappointWinner selection amplifies estimator noiseMeasure selected-action calibration; shrink estimates; reduce the menu.
Bundles help only curated examplesConstruction artifacts reveal the right planHold out task templates and test natural multi-hop documents.
Cheap checks dominateCost estimates omit extraction, integration, or retriesAudit the full request ledger and rerun cost attribution.
Local patch creates contradictionsClaim graph misses a dependencyRevalidate downstream conclusions and measure collateral damage.
Learned controller degrades after updatesModel or source drift changes action effectivenessFreeze versions for tests; treat updates as new calibration conditions.
Verification spiralsEvery critique generates new speculative issuesBound menu and horizon; compare a fixed stopping rule; log low-value loops.
Source lookup creates false confidenceRetrieval finds a plausible but inapplicable passageScore source scope and version against external adjudication.

Model diversity and source diversity should be measured separately. Changing a persona or random seed is not proof of independence. Conversely, shared dependence does not invalidate every statistical method: validity depends on the estimand and conditional assumptions. The experiment should measure the relevant failure structure rather than use “correlation” as an all-purpose explanation.

15. Limitations and decisions justified now

This review supports a more careful experiment, not a deployment performance claim. The newest routing source was available only through its indexed primary abstract. Several other recent findings are preprints. Full texts were inspected for selected methods and limitations; no external codebase or complete theorem proof was reproduced. The target task distribution remains to be collected and may differ substantially from published math, coding, and multiple-choice benchmarks.

Repair and damage estimates may be too noisy to improve on a fixed policy. Claim extraction may cost more than it saves. Dependence graphs can omit important links. Tool availability may dominate the scheduler. Conservative stopping can lose useful coverage; aggressive checking can damage correct answers. A horizon of two misses longer dependencies and may already be too expensive. These are reasons to include simpler alternatives, not to assume the controller will work.

The exact diagnostics justify rejecting universal claims about greedy checking and agreement. They do not justify an expected improvement for Encevra. The strongest practical decision now is to instrument the existing workflow, collect independent action outcomes on a bounded menu, and use the advancement gate before adding controller complexity.

16. Next-round handoff

The next numbered round should report the instrumentation pilot and action-value identifiability, rather than another broad literature essay on why verification matters. Its deliverables should be a versioned task manifest, fixed rubric, claim/source schema, action contracts, raw cost adapter, blinded labeling instructions, and the observed distribution of repair, damage, and check costs.

Before that round can claim a benchmark result, it must run the specified policies with recorded model versions and usage. Resolve access to BudgetVerify's full methods and code, if available, and revise the baseline accordingly. Report whether there is enough repeated outcome data to learn action value; if not, keep the fixed schedule and identify the missing evidence.

A useful next decision is concrete: does the small controller earn its overhead on held-out task families? If the answer is no, that narrows Encevra's architecture and avoids accumulating redundant orchestration. If the answer is yes under the accuracy and harm gates, the following work can examine transfer and longer-horizon control with a justified baseline.

17. Sources and inspection ledger

All links below were retrieved or attempted on 3 October 2026. “Full-text sections” means the specified methods/results were inspected directly; it does not imply independent replication or a proof audit. Dates distinguish initial preprints, inspected revisions, and conference publication where available. No secondary summary is used as substantive research evidence.

[S1] Hay, N.; Russell, S.; Tolpin, D.; Shimony, S. E. Selecting Computations: Theory and Applications. UAI 2012; arXiv posted 25 July 2012. Primary text. Inspected metalevel formulation, myopic policy/Theorem 7, and approximation discussion. Supports sequential computation selection and limits of myopic stopping; does not establish this controller's performance.

[S2] Damani, M.; Shenfeld, I.; Peng, A.; Bobu, A.; Andreas, J. Learning How Hard to Think: Input-Adaptive Allocation of LM Computation. Preprint 7 October 2024; ICLR 2025. Inspected v1 text; conference record. Inspected Sections 3–4 and evaluation assumptions. Supports learned marginal allocation; oracle selection and estimator errors limit transfer to realistic checking.

[S3] Jenq, J.; Shen, H. BudgetVerify: Budget-Tiered Verification for Financial QA. Preprint, 27 September 2026. Primary record. Indexed primary abstract only; direct full-text access failed. Financial QA is its reported scope. Detailed methods and controls remain uninspected.

[S4] Xue, B.; Cheng, J.; Lyu, S.-H.; Wan, Y.; Qiu, S. Test-Time Scaling via Budgeted Multi-Attribute Verification. Preprint v1, 28 September 2026. Primary text. Inspected Sections 3–4 assumptions/algorithm and Section 6 experiments. Supports adaptive allocation and certification of verifier-score means under stated assumptions; not factual correctness certification or a revision-policy result.

[S5] Singhi, N.; Bansal, H.; Hosseini, A.; Grover, A.; Chang, K.-W.; Rohrbach, M.; Rohrbach, A. When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning. COLM 2025; preprint 1 April 2025, inspected v2 dated 19 October 2025. Primary text. Inspected cost formulation, setup, and scaling results. Supports budget-dependent generation/verification trade-offs; its model/token compute proxy is not complete API spending.

[S6] Lifshitz, S.; McIlraith, S. A.; Du, Y. Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers. Preprint v1, 27 February 2025. Primary text. Inspected BoN-MAV method, verifier selection, task results, and call-count scaling. Supports potential benefits of specialized verification with a cost-metric caveat.

[S7] Wang, J.; Jain, S.; Zhang, D.; Ray, B.; Kumar, V.; Athiwaratkun, B. Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies. EMNLP, November 2024. Published paper. Inspected budget definitions and task/model comparisons. Supports strong simple baselines and sensitivity to accounting; its parallel-prompt accounting convention requires care when transferred to another serving setup.

[S8] Tran, D.; Kiela, D. Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets. Preprint 2 April 2026; inspected v2, 11 April 2026. Primary text. Inspected theory, budget definition, results, and Appendix G. Supports scoped multi-hop comparisons. Visible-summary length does not independently measure hidden reasoning work.

[S9] Wunderlich, F. V.; Kaesberg, L. B.; Wahle, J. P.; Ruas, T.; Gipp, B. Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling. ACL Student Research Workshop, July 2026. Published paper; record. Inspected Sections 3–5, limitations, and Appendix B. Supports gains in its tested setting using estimated runtime including memory transfer, not measured universal deployment savings.

[S10] Huang, J.; Chen, X.; Mishra, S.; Zheng, H. S.; Yu, A. W.; Song, X.; Zhou, D. Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024; preprint 3 October 2023, inspected v2 dated 14 March 2024. Primary text. Inspected Sections 3–5, transition analysis, prompts, and comparison controls. Supports harmful intrinsic revisions in tested older models; not an impossibility result for external feedback or future systems.

[S11] Dudík, M.; Langford, J.; Li, L. Doubly Robust Policy Evaluation and Learning. ICML, 2011. Published paper. Inspected contextual-bandit setup and estimator assumptions in Sections 2–3. Supports careful use of logged outcomes and propensity information; does not remove missing action support or automatically solve sequential off-policy evaluation.

[S12] Howard, S. R.; Ramdas, A.; McAuliffe, J.; Sekhon, J. Time-uniform, nonparametric, nonasymptotic confidence sequences. Annals of Statistics 49(2), 1055–1080, 2021; inspected arXiv v9 dated 6 August 2022. Primary author manuscript; DOI. Inspected uniform-coverage definition and conditional-assumption discussion. Supports valid sequential monitoring under appropriate assumptions, not truth guarantees for biased verifiers.

[S13] Shenhav, A.; Botvinick, M. M.; Cohen, J. D. The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function. Neuron 79(2), 217–240, 24 July 2013. Primary article record and figures; DOI. Inspected abstract, normative framing, and cost/payoff figure explanations; an author-PDF retrieval later timed out. A theoretical analogy with an engineering test, not direct evidence about language-model architecture.

[S14] Google. Gemini thinking, official API documentation, current page inspected 3 October 2026; publication date not stated in the inspected section. Documentation. Inspected “Thought summaries” and output examples. Supports the distinction between returned summaries and the internal process; API accounting must be checked against the specific version actually used.

Encevra archive sources

[A1] Research overview. Preview. Inspected the current project source/navigation; the existing Round 11 remains the item being revised.

[A2] Round 9. A research agenda: procedures, conditions, and change, dated 30 September 2026 in page metadata. Preview. Full page text inspected for the verified-procedure agenda and controller proposal.

[A3] Round 10. Evidence exchange, selective revision, and a testable path toward brain-inspired intelligence, metadata October 2026. Preview. Full page text inspected, including metrics and the staged roadmap.

[A4] Round 3. Review of memory, replay, and compositional reasoning, metadata August 2026. Preview. Relevant controller and scope passages inspected.

[A5] Combined Rounds 4 to 6. Metadata September 2026. Preview. Round 4 counterevidence/uncertainty passages and current navigation inspected; the old Round 4 route redirects here.

[A6] Round 8. Review of changing neural systems while preserving dependable function, metadata September 2026. Preview. Relevant workload-control, preservation, and analogy limitations inspected.

Revision integrity. This document expands Round 11 in place. All proposed thresholds, budgets, models of check outcomes, and evaluation designs remain proposals unless explicitly labeled as exact illustrative calculations. No benchmark success, model-setting change, or site deployment is claimed.