Research Round 13 · 4 October 2026
From final scores to measurable corrections
Evaluation ScienceReasoning & VerificationPublished · pilot proposedArchitecture impact: Medium
Research question
Can saved Noetis records show whether a correction actually helped?
Summary
Audited existing Noetis evaluation exports and built an offline trace validator. The saved records reproduce, but the scoring endpoint cannot carry claims about architectural benefit.
Major findings
- Replaying the current scorer reproduced all 119 nonempty stored scores.
- Appending the full source as a labeled appendix removed detected-issue credit in all 114 outputs that had it.
- Removing appended approved findings changed detection counts in 44 of 59 nonempty panel outputs.
- Saved results keep the final brief and latest check, but not before/after pairs, so a repair flag cannot distinguish a fix from a new error.
- Usage data existed for 19 panel cases and no baseline cases.
Experiments
- E13-A Audit of saved Noetis evaluation records Executed (audit)
- E13-B Local revision pilot with evidence receipts Proposed
Hypotheses
- H13-1 Evidence receipts at the revision boundary improve task success over ordinary reconsideration.
Architecture implications
- Revision-drift guard. Flag synthesis that omits supported risks or adds untraced claims, rather than presenting it as consensus.
Open questions
- Does evidence packaging change outcomes at a fixed state?
- Does Noetis outperform a strong single agent at matched evidence access and measured cost?
Connected research
Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra Research.
