by EncevraHow we build and tune Noetis
Noetis is not a single model with a prompt attached. The language models behind it come from external providers. Encevra builds everything around them: the roles, the evidence checks, the review thresholds, the panel shape, and the gate that decides whether an answer is released.
This page describes our method and our release practice. It is not a benchmark result. Measured comparisons live on the benchmark page, with the cases where a single strong model did better.
Three layers
A claim about how an engine improves is only useful if you know which layer changed and who owns it. Here is the split.
Licensed from model providers
Each role in the panel calls a language model from an external provider through our model gateway. Encevra does not train or own those model weights. What we own is the choice of which model sits behind each role, its effort budget, and the record of which model version produced a given run.
Per role model choice and effort budget.
Ours, and retuned every release
Role instructions, evidence location rules, claim review prompts, panel composition, synthesis behaviour, and the gate that decides whether a run is released or returned incomplete. These are tuned on our own cases and tested before release.
Instructions, rules, thresholds, panel shape.
Yours, used only by agreement
Where a customer wants panel tuning on their own documents, we can build a small agreed evaluation set with them and hold it separate from every other account. Tuning at instruction and panel level is the default; we do not train models on your documents.
Panel variants against an agreed evaluation set.
The loop
Every step exists to keep one decision attributable. A change that cannot be pointed at cannot be defended, and a change that cannot be rolled back will eventually need to be.
Runs that end badly are kept as cases: the question, the material, the finding that was wrong, the passage it was drawn from, the verdict that should have caught it, and the cost of the run. Cases come from internal evaluation and from customer feedback where an account allows it.
Each case is sorted before anything changes. A missed term, a quotation that does not match the source, a finding that claims more than its passage says, a dropped caveat, a brief that drifted past the findings, a revision that damaged a correct answer, or a run that simply cost too much. Different kinds need different fixes, and a fix aimed at the wrong kind makes the engine worse somewhere else.
A change is usually a role instruction edit, an evidence location rule, a review threshold or a roster adjustment, or a change of which third party model serves one role. We keep changes small and separate so a result can be traced back to one decision rather than a batch of edits whose effects cancel out.
The internal evaluation set does not move while a change is being judged. The same cases, the same scoring, the same comparators, so a candidate is compared with the version directly before it and not with a moving target. Where a case needs a human judgement, the reviewer does not see which version produced the answer.
A change that improves findings while raising calls, tokens and time is not automatically accepted. Every candidate is compared with the current version at comparable cost, and the record keeps the token, call and duration figures alongside the quality result so the trade is visible rather than implied.
Where the automatic check and a human reviewer disagree, that gap is the interesting part. The reviewer's note becomes the next case, and the scorer itself gets revised. A change in the scorer means every prior result is re-read before it is quoted anywhere.
Accepted changes ship as a version bump. They run on a small share of traffic first, with the evidence location and claim check diagnostics watched on live material. If a diagnostic moves the wrong way the version is rolled back rather than patched forward, and the case that triggered it stays in the set.
What gets tuned
Model choice, instructions, rules and thresholds all sit on this list. Each is a decision inside the engine that can be written down, tested, and reversed.
| Part | What changes | How it is judged |
|---|---|---|
| Role model assignment | Which provider model is assigned to which role, and at what effort budget. | Held-out cases of that role's own failure kinds, compared at matched cost against the version directly before it. |
| Role instructions | What each specialist is told to look for, what it must not assume, and what shape its answer must take. | Whether the role finds the cases it exists to find, without pulling its verdicts toward another role's job. |
| Evidence location | The rules that match a quoted passage back to the submitted text and record its position. | Located quote accuracy and offset precision, checked mechanically rather than by judgement. |
| Claim review | What separates a supported finding from a narrowed one and an unsupported one, including scope, conditions and exceptions. | Agreement with reviewer labels on sampled findings, plus how often an unsupported finding reaches a brief. |
| Panel composition | Which roles run, how many there are, and which model sits behind each one. | Matched cost comparison against the previous roster on the fixed set, not a headline average. |
| Synthesis | How reviewed findings become a brief, and what the brief is forbidden to add. | Unsupported claim rate in the finished brief, and whether open questions survive into it. |
| Release gate | The threshold at which a run is returned incomplete with its reasons instead of released as finished. | Open questions surfaced against false completions. A gate that never stops anything is not doing its job. |
Boundaries
These are the sentences that make a method page worth reading. A method with no limits is a marketing claim.
Keeping the record
If a result cannot be traced to the version that produced it, it cannot be defended when a customer, an auditor, or next quarter's team asks how it was reached.
Each run stores the exact version of each model behind its roles, the stage it reached, its token counts, its cost and its duration, so a result can be re-read later rather than reconstructed from memory.
Engine changes carry the hypothesis behind them, the comparison that judged them, and the diagnostics that led to acceptance or rejection. Rejected changes stay in the record so the same idea is not re-tested every quarter.
The internal set, the scoring rules and the reviewer protocol are versioned with the engine. A change to the scorer is treated as an engine change and re-reads the results it touches.
Engine changes follow the same gates as our research rounds: a hypothesis, a matched comparison, and a written record of what did not improve. That is why releases arrive more slowly than a prompt edit, and why the engine behaves predictably once it ships.
Evaluation on your own questions is where a panel either earns its complexity or does not. Start with the use case and the documents, and we will tell you which parts would need tuning.
