Benchmark
Noetis benchmark: a historical internal evaluation.
We create and gather various types of business contracts, run through Noetis and through a single frontier model given the same question and the same material. Wins and losses included.
Run by Encevra, not independently verified. These are historical results for an earlier engine configuration, not a guarantee of current performance. Encevra's newer research rounds propose experiments and report no new benchmark results.
How to read evidence on this site
| Evidence state | What it means | Engine catalog today |
|---|---|---|
| Planned experiment | Designed, not run | 180 generated cases, unrun |
| Deterministic development check | Code level test of a component | Automated tests in the codebase |
| Unscored smoke run | One bounded live run to confirm the workflow executes | One per engine, all 36 recorded, all ended incomplete |
| Scored model evaluation | Fixed task set, answer key, scorer and baseline | No published result for any engine yet |
| Production observation | Measured on real customer traffic | None published |
| Authorized customer outcome | A result a customer agreed to publish | None |
A scored result will show task definition, date, engine version, model configuration, sample count, scorer, baseline, budget, latency, cost, failures and losses. See the evaluation design guide.
Average analysis quality
Scored blind per case out of 100. +4.5% relative to a single frontier model answering the same question on the same document.
Findings grounded in exact quotes
Every Noetis finding must point to a verbatim quote in the source and is verified before it can appear in the brief. A single prompt offers no citation layer at all.
Time per document
Nine coordinated specialists cost more time and tokens than one prompt. You are paying for verification, not speed.
The headline result
Higher average analysis quality. On every kind of document that matters.
How to read this: the score is an automated keyword and quote overlap diagnostic, not an independently adjudicated measure of analytical accuracy. These figures were produced before we added negative controls for copied source text, mentions without a conclusion, and rejected false claims. We are rebuilding the evaluation around claim-level human labels and will republish. Grounding, fabrication and timing figures are measured directly and are unaffected.
The gap is widest exactly where the work is hardest: operations incidents (+13%), engineering investigations (+11%), and safety reviews (+17%). Where Noetis lost — compliance triage — we say so, on this page, with the numbers.
90.3
Median Noetis score vs 87.5
5 of 6
Document categories won
Breakdown by document category
Average quality score and planted faults caught, per category.
| Category | Cases | Noetis | Single model | Difference | Faults caught |
|---|---|---|---|---|---|
| Contract | 9 | 90.2 | 86.2 | +4.6% | 45/45 vs 45/45 |
| Operations | 6 | 91.2 | 80.5 | +13.3% | 29/30 vs 28/30 |
| Finance | 6 | 87.9 | 85 | +3.4% | 27/29 vs 28/29 |
| Engineering | 3 | 97.5 | 87.5 | +11.4% | 15/15 vs 15/15 |
| Safety | 2 | 88 | 75.5 | +16.6% | 10/10 vs 10/10 |
| Compliance | 4 | 73 | 87.5 | -16.6% | 17/20 vs 20/20 |
The claim we stand behind
A single strong model gives you an answer you have to trust. Noetis gives you an answer you can check. 80% of Noetis findings carry an exact, character-offset quote from your own material, verified before the brief is written — findings that fail verification are narrowed or excluded, not asserted. The single model's grounding rate is 0% because a prompt has no citation layer to verify.
Noetis also committed 7 fabrications across the suite versus 4 for the single model — each one caught and flagged by its own verification pass before reaching the brief. The point of the architecture is not that the engine never errs; it is that errors are caught, labeled, and traceable.
Where Noetis is not the right choice
- Speed. At roughly 75 seconds per document versus 12, Noetis is ~6× slower. If you need instant answers, use a single model.
- Compliance triage. The single model outscored Noetis 87.5 to 73.0 on compliance documents and caught all 20 planted faults. We are re-tuning that panel before claiming parity.
Method
Cases. 30 original business documents across contracts, finance, operations, engineering, compliance, and safety, each with planted faults (wrong totals, conflicting clauses, missing evidence) and lures — plausible figures absent from the material. Asserting a lure counts as a fabrication.
Scoring. Each answer is scored out of 100 on fault coverage, correctness, and usefulness. Fault detection counts alternative phrasings, so exact wording is not required. Fabrications are counted separately and reported above.
Baseline. A single frontier model answering the identical question on the identical material, with no citation requirement — the way most teams use AI on documents today. Full per-case outputs are retained and available under NDA with a quote request.
Run your own documents through the same engine.
The benchmark panel is the production panel. License Noetis and test it on the documents your business actually runs on.