← Notebook library

Research Round 13 · 4 October 2026

From final scores to measurable corrections

Evaluation ScienceReasoning & VerificationPublished · pilot proposedArchitecture impact: Medium

Research question

Can saved Noetis records show whether a correction actually helped?

Summary

Audited existing Noetis evaluation exports and built an offline trace validator. The saved records reproduce, but the scoring endpoint cannot carry claims about architectural benefit.

Major findings

  • Replaying the current scorer reproduced all 119 nonempty stored scores.
  • Appending the full source as a labeled appendix removed detected-issue credit in all 114 outputs that had it.
  • Removing appended approved findings changed detection counts in 44 of 59 nonempty panel outputs.
  • Saved results keep the final brief and latest check, but not before/after pairs, so a repair flag cannot distinguish a fix from a new error.
  • Usage data existed for 19 panel cases and no baseline cases.

Experiments

  • E13-A Audit of saved Noetis evaluation records Executed (audit)
  • E13-B Local revision pilot with evidence receipts Proposed

Hypotheses

  • H13-1 Evidence receipts at the revision boundary improve task success over ordinary reconsideration.

Architecture implications

  • Revision-drift guard. Flag synthesis that omits supported risks or adds untraced claims, rather than presenting it as consensus.

Open questions

  • Does evidence packaging change outcomes at a fixed state?
  • Does Noetis outperform a strong single agent at matched evidence access and measured cost?

Connected research

Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra Research.