ENCEVRA RESEARCH ROUND 21
Where Did the Trace Break? An Attempt-Bound, Fail-Closed Measurement Gate for Noetis
Version 1.0 | 12 October 2026 | Research paper
Evidence status. This round combines primary-source review, static inspection of the current Encevra project, and an executed deterministic development diagnostic. It made zero language-model calls, zero provider network calls, and zero production-database calls. It is not a model benchmark, production integration test, human study, security audit, or claim of improved reasoning. No production Noetis code was changed and no site was deployed.
Abstract
Encevra's recent program has moved from asking whether agents can exchange evidence to asking whether the system can prove what crossed the provider boundary. Round 20 specified minimal-disclosure receipts for that purpose, but left an operational question unresolved: where must measurement be captured, and when may a run be described as audit-qualified? This round inspects the current Noetis collector at source commit 5aba83a193c988eefc8622292155afa085f3fc9e and asks whether a boundary-tapped, attempt-bound, atomic measurement gate can reconstruct every provider attempt and refuse audit-qualified release when its trace is incomplete, without changing model compute.
The source audit finds meaningful progress since Round 19. Ten of 31 requirements are present and nine are partial. The implementation records model choices, fallback information, latency, usage provenance, input and system digests, outcome, code version, artifact stages, and submitted-source order. Twelve requirements remain absent. The most consequential gaps are provider request and response identifiers, a canonical commitment to the full dispatched envelope, immutable source-version and per-attempt slice records, provider-specific stream-finality evidence, stable attempt lineage, an attempt-bound response commitment, atomic persistence, an explicit completeness state, a release gate, and append-only or independently checkpointed history.
Round 21 proposes the Attempt-Bound Measurement Gate (ABMG). ABMG emits a dispatch event at the provider boundary, closes each attempt only on a provider-specific terminal event or explicit error, commits the run terminal state and required measurement events atomically, and labels output as audit-qualified only when a completeness predicate passes. Raw content remains off by default; canonical request, source-manifest, response, and terminal-artifact commitments provide binding without routine content capture. Independent checkpointing remains a higher assurance tier, not a hidden implication of the base mechanism.
An executable finite-world diagnostic evaluates the logic of the inspected implementation and the proposed contract. All 34 named checks passed. Under the current three-write best-effort persistence model, seven of eight possible write states are incomplete yet do not block release; ABMG audit-qualifies none of those seven. Ten controlled provider-envelope mutations all collide under the current content-only fingerprint but are detected by a full-envelope commitment. Twenty-four equal-byte, different-version source pairs collapse under the current source projection but remain distinct under the ABMG manifest. Twenty-four distinct provider-response pairs share the same current attempt record but are separated by ABMG response receipts. Five malformed, truncated, or error stream fixtures are falsely accepted by a development model of the current parser and rejected by the ABMG state machine. These are constructed conformance findings, not estimates of production incident rates.
The result is a narrow engineering conclusion: the current collector is useful operational telemetry, but it cannot yet support a claim that every released run is reconstructable at the provider boundary. ABMG is a falsifiable bridge from that telemetry to an audit-qualified trace. The next step is a shadow implementation on real provider traffic with fixed model calls, failure injection, privacy review, and preregistered adoption thresholds.
1. Archive continuity and round numbering
The Lovable archive was inspected before topic selection. The Research overview, rounds index, publications registry, full texts of Rounds 20, 19, 10, and 9, the trial-readiness plan, and the current measurement and provider paths were reviewed at source commit 5aba83a193c988eefc8622292155afa085f3fc9e [S01-S05]. The archive ends at Round 20, so Round 21 is the correct next sequential number. No later completed round was found.
The closest antecedents are:
- Round 9: reusable procedures require versioned rules, feedback, graders, costs, negative results, and revalidation after relevant changes. That principle applies here to the measurement procedure itself [S05].
- Round 10: instrumentation must precede stronger claims about collaborative evidence exchange and answer revision. Candidate claims, source references, and final changes should be captured precisely [S04].
- Round 19: the then-current collector could not reconstruct exact provider-bound evidence delivery. Its handoff was implementation, not another conceptual protocol [S03].
- Round 20: a minimal-disclosure receipt can separate content binding from routine content disclosure, but its live-boundary and independent-witness tests had not been run. Its explicit handoff was the smallest real provider-bound collector slice [S02].
Older indexed work was checked through the overview and index for overlap. Rounds 3, 4, and 8 concern earlier collaboration, critique, and evidence mechanisms, but none answers the present systems question: whether the existing collector can fail closed at the exact provider-attempt boundary. The selected topic therefore advances the archive instead of repeating evidence exchange, selective revision, or procedural learning.
2. Research question, decision, and novelty
Research question. Can a boundary-tapped, attempt-bound, atomic measurement gate reconstruct every Noetis provider attempt and refuse audit-qualified release when the trace is incomplete, at fixed model compute?
Decision sought. Decide whether Noetis should replace best-effort, post-hoc measurement with an explicit gate whose success is necessary for the label audit_qualified_released, while retaining an availability-preserving unverified_output_retained path for ordinary product use.
The question is distinct from the nearest alternatives:
| Candidate | Why consequential | Why not selected as the sole Round 21 question |
|---|---|---|
| Confidence calibration across collaborating agents | Could improve how disagreement and uncertainty are routed | It depends on trustworthy attempt and evidence records. Running it first would place calibration claims on an incomplete measurement substrate. |
| Verification-budget allocation | Could improve accuracy per token or dollar | It is downstream of exact attempt accounting, usage provenance, and evidence-delivery reconstruction. |
| Long-term memory provenance and conflict resolution | Could reduce stale or contradictory memories | It is broader than the immediate Round 20 handoff and risks conflating storage policy with provider-bound measurement. |
| Attempt-bound measurement completeness | Determines whether later experiments can be trusted at all | Selected because current source contains a real collector, making the question immediately falsifiable against implementation and live traffic. |
The novelty is not that traces, hashes, transactions, or terminal stream events exist. The contribution is the combined release contract: capture at emission; assign stable attempt lineage; bind the exact provider request, evidence manifest, and provider response; require provider-specific stream finality; atomically persist a completeness bitmap with the terminal run; and reserve audit-qualified release for complete records. This combination targets the precise failure modes exposed by the current Noetis path.
3. Scope and exclusions
3.1 In scope
- Static inspection of the provider, panel-run, measurement-builder, measurement-recorder, queue, API, and migration paths frozen at the named commit.
- A field-level contract audit of the current collector.
- An executable, deterministic diagnostic over controlled write states, envelope mutations, source-version pairs, provider-response pairs, and stream fixtures.
- A proposed event schema, state machine, privacy boundary, assurance ladder, and production experiment.
- Regressions and harm metrics, including availability, latency, privacy exposure, observability reliability, and answer-quality non-degradation.
3.2 Out of scope
- Measuring whether Noetis answers are more accurate, calibrated, useful, or safe.
- Claiming that a submitted source was true, complete, attended to, or causally used by a model.
- Inspecting provider-internal computation.
- Proving cryptographic security, key custody, non-equivocation, or resistance to a dishonest operator.
- Running live provider calls, querying production data, changing production code, or deploying the site.
- Repeating Round 20's full minimal-disclosure receipt design or Round 9's procedure-learning question.
The distinction matters: reconstructability is a prerequisite for credible evaluation, not evidence that the evaluated system is good.
4. Current implementation: what exists and where it can break
The code review used the current Lovable source, not a recollection of the prior round. The following map summarizes the inspected path [S01].
| Layer | Current behavior | Measurement consequence |
|---|---|---|
| Provider dispatch | Direct streaming fetch paths for OpenAI and Anthropic | The exact serialized request is created here, but the measurement record is assembled later from a narrower logical projection. |
| Stream reader | Text deltas are accumulated; parse failures are tolerated; OpenAI [DONE] and Anthropic terminal semantics are not recorded as receipt fields | EOF or partial text can look like a successful completion in the collector model. An Anthropic error event after HTTP 200 has no explicit terminal classification. |
| Run trace | Tracks requested and actual model, fallback, latency, usage, outcome, error, and prompt fingerprints | Useful operational telemetry, but no stable provider request ID, response ID, finish or stop reason, terminal marker, or response commitment. |
| Source projection | Stores label, position, SHA-256, character count, delivered characters, and a truncation note | Records the submitted source set, but not immutable source version, exact per-attempt slice and order, or range identity. |
| Measurement persistence | Three independent upserts run concurrently and errors are caught | Any subset can commit. The caller cannot distinguish complete from partial measurement. Upserts can replace prior history. |
| Run finalization | Terminal run state is recorded before the best-effort measurement recorder | User-visible completion and downstream actions do not depend on measurement completeness. |
The current collector is therefore best described as A0 operational telemetry. It helps diagnose a run, estimate usage, and inspect the broad execution path. It does not yet establish A1 provider-bound completeness.
4.1 Contract audit
The audit assessed 31 requirements derived from the prior-round handoff, present provider contracts, current observability guidance, database transaction semantics, and the minimum fields needed for attempt reconstruction. Ten are present, nine partial, and twelve absent.
| Status | Count | Representative fields |
|---|---|---|
| Present | 10 | Requested and actual model; fallback flag and reason; latency; usage and provenance; prompt digests; outcome/error; code version; artifact stages; submitted-source order/hash/length. |
| Partial | 9 | Every physical attempt; provider identity; exact provider request; response completion; per-call evidence mapping; ordering; terminal-answer linkage; content-privacy separation; run referential integrity. |
| Absent | 12 | Provider request/response IDs; response commitment; canonical full-envelope commitment and profile; immutable source version; per-attempt slices; stable attempt/parent IDs; terminal stream marker; atomic persistence; completeness state; release gate; append-only/checkpointed history; protected salted commitment/reveal flow. |
The count is not a maturity score. A single absent field can be decisive. For example, stable model and token fields do not compensate for an absent terminal-stream marker when a truncated stream is accepted as complete.
4.2 Four failure surfaces
Boundary mismatch. The current fingerprints bind user input and system text, not the exact HTTP envelope. Endpoint, stream flag, maximum tokens, temperature, reasoning configuration, response format, tools, message roles, or even provider can change without changing those content-only fingerprints.
Attempt ambiguity. A run-local sequence assembled after execution does not identify the event emitted at dispatch. Provider request and response identifiers, parent attempt lineage, actual terminal reason, and an attempt-bound response commitment are absent.
Stream-finality ambiguity. A streaming HTTP 200 is not the same as a complete model response. OpenAI notes that interrupted streams may omit final usage, and Anthropic documents that an error may occur in a stream after the initial HTTP 200 [S10-S12]. A terminal event must therefore be evidence, not an assumption based on EOF or accumulated text.
Persistence and release ambiguity. The artifact, attempt, and source sets are written independently. With three writes there are eight possible success/failure states. Only one is complete. Because measurement errors are swallowed and terminal state is already recorded, all seven incomplete states can coexist with ordinary release. This does not mean all seven occur in production; it means the current control flow cannot exclude them.
5. Literature and standards map
This round inspected primary or authoritative sources directly. The literature supports components of ABMG but does not establish that the full mechanism will work in Noetis.
5.1 Position A: conventional operational tracing is sufficient
OpenTelemetry's generative-AI semantic conventions recommend spans that begin with request initiation and end after the full response or error. They include provider, requested and response model, response ID, usage, and errors, with separate spans for retries [S06]. W3C Trace Context supplies stable trace and parent identifiers for cross-component correlation [S07]. Together, these conventions are an excellent A0 baseline.
The limitation is explicit in what the standards do not promise. Trace Context correlates operations but does not bind payloads or prove trace completeness. The OpenTelemetry convention is in development status, advises against capturing sensitive content by default, and leaves its streaming-chunks section unfinished [S06]. A conforming operational trace can still lack the exact provider envelope, terminal-event proof, atomic measurement state, or non-equivocation.
5.2 Position B: attempt-bound commitments are needed
Provider APIs expose identifiers and terminal fields that can close a receipt. OpenAI responses and chunks include completion identifiers, models, finish reasons, and usage; Anthropic exposes request IDs, response IDs, models, stop reasons, usage, and typed streaming errors [S10-S12]. Capturing these at the boundary reduces ambiguity between the application's intended call and the provider's acknowledged attempt.
Hashing structured requests requires deterministic serialization. RFC 8785's JSON Canonicalization Scheme defines invariant property ordering, primitive serialization, and UTF-8 generation for repeatable hashes [S08]. This supports a named canonicalization profile, not the stronger claim that the committed content is true or confidential. Low-entropy data remains guessable under plain hashing, and incomplete capture remains incomplete.
5.3 Position C: only independently witnessed history is credible
RFC 9942 defines COSE receipt structures for inclusion and consistency proofs, while RFC 9943 describes append-only transparent statements, registration policy, and receipts [S13-S14]. These mechanisms can make later rewriting detectable under their trust assumptions. They also clarify an important limit: transparency does not prevent a dishonest issuer from making a false statement in the first place.
ABMG therefore separates two assurance levels. A1 binds the application-observed provider boundary and fails closed on local incompleteness. A2 adds an independent checkpoint or transparency service for later non-equivocation. This round does not collapse A1 and A2.
5.4 Position D: strict failure is too costly for a product system
A hard gate can reduce availability if telemetry fails while model inference succeeds. PostgreSQL transactions can make a group of writes all-or-nothing [S09], but that does not settle product policy. The proposed compromise is fail closed for the audit-qualified label, not necessarily for retaining the user-visible answer. A run with incomplete measurement may become unverified_output_retained; it must not silently become audit_qualified_released.
This is an Encevra hypothesis, not a general standard. The correct retention policy depends on product risk, latency budget, incident-response needs, and user expectations. The production experiment must measure these costs.
5.5 Established findings, mixed evidence, and open questions
| Epistemic status | Finding |
|---|---|
| Established by inspected specifications | Cross-component correlation needs stable trace lineage [S07]. Provider APIs expose request/response and terminal fields that are useful for attempt closure [S10-S12]. Database transactions can prevent partial multi-write commits [S09]. Canonicalization is needed for repeatable structured-data commitments [S08]. |
| Established by inspected source | Current Noetis measurement uses three independent best-effort writes, omits provider IDs and stream terminal markers, and finalizes the run before the recorder [S01]. |
| Mixed or incomplete | GenAI tracing conventions cover many operational fields but remain in development and do not finish streaming-chunk guidance [S06]. Independent transparency improves later rewrite detection but adds trust, privacy, and operational complexity [S13-S14]. |
| Open | Whether ABMG captures at least 99.5 percent of live attempts exactly; whether synchronous gate overhead meets budget; whether failure labeling improves reviewer reconstruction; whether the privacy design withstands review. |
| Encevra hypothesis | A boundary tap plus atomic completeness gate will improve reconstructability more than adding further post-hoc fields, at unchanged model compute. |
NIST's generative-AI profile supports lifecycle-wide risk management rather than treating one output score as trustworthiness [S15]. It does not prescribe ABMG, its fields, or the thresholds below.
6. Claim and evidence ledger
| Claim | Evidence in this round | Strength and boundary |
|---|---|---|
| The archive's next number is 21 | Lovable overview, index, registry, and routes end at Round 20 [S01-S05] | Direct archive inspection at the frozen commit. |
| The collector has advanced beyond the Round 19 baseline | Current schema and builders record ten fully present and nine partial requirements | Direct static-source comparison; no production execution. |
| The current trace cannot uniquely bind all provider envelopes | Ten controlled envelope-only mutations preserve current content fingerprints | Exact for the constructed fixtures and inspected fingerprint projection; prevalence unknown. |
| The current source record cannot distinguish immutable versions with equal bytes | Twenty-four equal-byte/different-version pairs collapse | Exact finite-world result; does not claim such pairs are common. |
| The current attempt record cannot distinguish provider responses that differ only in omitted receipt fields | Twenty-four response pairs collapse | Exact finite-world result; provider fields were constructed from documented response shapes. |
| Best-effort three-write persistence permits incomplete records without blocking release | Seven of eight Boolean write states are incomplete and current modeled control flow still releases | Logical state-space result from inspected control flow, not a measured production rate. |
| The current stream parser model accepts malformed or nonterminal fixtures | Five of five incomplete/error fixtures are false accepts | Development model based on inspected code; must be confirmed in integration tests. |
| ABMG would reject the constructed incomplete cases | Proposed reference state machine rejects all seven incomplete write states and all five bad streams | Conformance behavior of the proposal, not proof of production implementation. |
| ABMG improves model reasoning | No evidence collected | Not claimed. No model benchmark was run. |
| ABMG prevents operator equivocation | No independent witness implemented | Not claimed at A1. A2 requires an independent checkpoint. |
7. Proposed mechanism: Attempt-Bound Measurement Gate
ABMG is a three-event contract plus a release predicate. It does not require raw prompts in ordinary telemetry.
7.1 Event 1: attempt_dispatched
Emit this record at the last application-controlled boundary before bytes are sent to the provider. Required fields are:
schema_versionandcanonicalization_profile;run_id, stableattempt_id, optionalparent_attempt_id, and monotonicsequence;- orchestration
role, explicitprovider,endpoint, andrequested_model; request_envelope_commitmentover the canonical exact dispatched object, including provider options;source_manifest_commitmentover the ordered per-attempt source manifest;code_versionandstarted_at.
Each manifest item contains source_id, immutable version_id, position, byte or token range, truncation flag, and content commitment. If a call has no source evidence, the manifest is explicitly empty, not missing.
7.2 Event 2: attempt_completed
Close the same attempt after a provider-specific terminal event or an explicit failure. Required fields are:
attempt_idandoutcome;- provider request ID and provider response ID, or an explicit
unavailablereason for each; - actual model, or explicit unavailability;
- finish or stop reason, or explicit unavailability;
terminal_event_seen;response_commitmentfor completed output, or normalized error class for failure;- input/output usage, usage provenance, latency, and
completed_at.
Retries and fallbacks are new physical attempts with new IDs. The retry points to its parent and records the policy reason. It never overwrites the prior attempt.
7.3 Event 3: run_terminal
The terminal record binds the run output to the attempts that produced it:
terminal_artifact_commitment;- ordered
producing_attempt_ids; - a required-field
completeness_bitmap; measurement_state,release_state, andcommitted_at.
7.4 Completeness and release predicate
Let (R) be the set of required event fields for the applicable provider path, (P) the persisted fields, (T(a)) the provider-specific terminal predicate for attempt (a), and (L) the lineage predicate that every producing attempt is reachable from the run. Define:
complete(run) = (R subset P) AND all T(a) AND L AND atomic_commit_succeeded.
Then:
audit_qualified_released(run) iff complete(run).
An inference result may still be retained under unverified_output_retained when product policy permits. The important property is non-conflation: a measurement failure cannot silently receive the stronger label.
7.5 Stream state machine
| State | Accepted input | Next state | Audit consequence |
|---|---|---|---|
| Dispatched | Valid data event | Streaming | None yet |
| Streaming | More valid data | Streaming | None yet |
| Dispatched or Streaming | Provider-specific terminal event | Complete | Attempt may close if required receipt fields exist |
| Dispatched or Streaming | Typed provider error | Failed | Attempt closes as failure; retry is a new attempt |
| Dispatched or Streaming | EOF without terminal event | Incomplete | Cannot be audit-qualified |
| Any | Unparseable event that cannot be safely classified | Incomplete | Cannot be audit-qualified |
For OpenAI, [DONE] or the documented equivalent and applicable finish metadata are explicit evidence. For Anthropic, message_stop closes the stream; a typed error event fails it. The provider adapters own these predicates because one generic EOF rule is insufficient [S10-S12].
7.6 Persistence rule
Persist required attempt events, source manifests, artifact commitments, and the terminal run state in one database transaction, or set the measurement to measurement_failed. The implementation may stage records asynchronously before the transaction. The label transition must occur in the same atomic unit as the completeness record. PostgreSQL transactions provide the necessary all-or-nothing primitive [S09].
ABMG is append-only at the event layer. A correction is a linked amendment event with actor, reason, prior-event commitment, and new commitment. It is not an upsert that erases the prior value.
7.7 Privacy and assurance tiers
Routine records are content-free by default, consistent with OpenTelemetry's warning that prompts and responses may contain sensitive information [S06]. Raw content and fresh commitment salts, when retained for an authorized reveal, live behind separate authorization and retention controls. Plain unsalted hashes are identifiers, not confidentiality mechanisms.
| Tier | Name | What it supports | What it does not support |
|---|---|---|---|
| A0 | Operational trace | Correlated metadata for debugging | Payload binding, completeness, or non-equivocation |
| A1 | ABMG complete | Provider-bound request/response commitments, source manifest, stream finality, and atomic completeness | Provider-internal processing, truth, semantic use, or independent non-equivocation |
| A2 | ABMG plus independent checkpoint | A1 plus evidence against later rewrite under the checkpoint assumptions | Truth, completeness before capture, provider-internal processing, or causal use |
RFC 9942 and RFC 9943 offer patterns for A2 receipts and append-only transparency [S13-S14]. This round does not implement them.
8. Worked example: fallback after a mid-stream error
Consider one Noetis role that first calls Anthropic and falls back to OpenAI.
- Noetis canonicalizes the exact Anthropic request at dispatch, including system prompt, ordered messages, model, maximum tokens, stream flag, and the ordered source manifest. It emits
attempt_dispatchedwithattempt_id=A1. - Anthropic returns HTTP 200 and several text events, then a typed stream error. The adapter records the provider request ID,
terminal_event_seen=false, normalized error class, partial-response commitment if policy allows, andoutcome=failed. A1 is not complete. - The fallback policy creates
attempt_id=A2withparent_attempt_id=A1. It commits the exact OpenAI request envelope. The fact that evidence slices were rearranged or truncated would change the A2 manifest commitment even if the high-level source labels were identical. - OpenAI returns a response ID, model, usage, finish reason, and terminal marker. The adapter emits
attempt_completedfor A2 with a response commitment. - The final answer artifact commits to the bytes produced by A2. The terminal event lists A1 and A2 in lineage and A2 as the producing attempt.
- The database transaction either persists the complete event set and moves to
audit_qualified_released, or rolls back the label transition and recordsunverified_output_retainedormeasurement_failedunder product policy.
The trace supports the statement that Noetis dispatched particular committed envelopes, observed an Anthropic stream error, retried on OpenAI, and bound the terminal artifact to A2. It does not support the statements that the sources were true, that either model used them internally, or that the resulting answer was correct.
9. Brain-inspired design hypothesis, kept narrow
The only biological analogy used in this round is the motor-control concept of efference copy: a copy of an outgoing command can help distinguish consequences of self-generated action from external input. Kim et al. empirically study this distinction in a hopping task [S16].
The engineering translation is deliberately modest: copy the exact outbound provider command at emission, not a later reconstruction of what the application believes it sent. This yields a testable design choice: compare a boundary tap against an otherwise identical post-hoc collector. If both reconstruct provider-bound attempts equally well under controlled mutations and failures, the analogy contributed no engineering value and should be dropped. No claim is made that Noetis copies a brain or that the cited motor system explains agent collaboration.
10. Executed diagnostic D21-A
10.1 Purpose
D21-A tests whether the identified failure classes are logically observable under the current projection and rejectable under the proposed ABMG contract. It is a conformance diagnostic, not an outcome benchmark.
10.2 Inputs and execution
The versioned harness contains a machine-readable contract audit, constructed fixtures, the proposed protocol, and assertions. It executed locally against its fixtures. It made zero model calls, zero provider calls, and zero production-database calls.
The diagnostic families were:
- all eight success/failure combinations of three independent measurement writes;
- ten changes to provider-envelope fields not covered by the current content-only fingerprint;
- twenty-four equal-byte source pairs with distinct immutable versions;
- twenty-four provider-response pairs that differ only in receipt fields absent from the current attempt record;
- seven stream scenarios, two complete and five truncated, malformed, or explicitly erroneous;
- overwrite versus append-only amendment behavior;
- field-status totals for the 31-requirement contract audit.
10.3 Oracles
The current oracle is a development model derived from the inspected source paths. The ABMG oracle is the proposed state machine and canonical commitment projection. Each assertion is named and fails the harness if its expected classification changes. The source commit is embedded in the result artifact.
11. D21-A results
All 34 of 34 named checks passed. Passing means the harness behaved as specified; it does not mean the production system passed ABMG.
| Diagnostic | Current projection | ABMG projection | Interpretation |
|---|---|---|---|
| Contract requirements | 10 present, 9 partial, 12 absent | Proposed fields cover all 31 in the reference contract | Proposal coverage is not implementation. |
| Incomplete three-write states still compatible with release | 7 of 7 incomplete states | 0 of 7 audit-qualified | The release predicate removes silent partial qualification in the finite state model. |
| Envelope-only mutations missed | 10 of 10 | 0 of 10 | Full canonical-envelope commitment detects all constructed changes. |
| Equal-byte, different-version source pairs collapsed | 24 of 24 | 0 of 24 | Immutable version identity prevents these constructed provenance collisions. |
| Distinct response pairs sharing one attempt projection | 24 of 24 | 0 of 24 | Provider IDs, terminal reason, and response commitment disambiguate the pairs. |
| Bad stream scenarios accepted as complete | 5 of 5 | 0 of 5 | Provider-specific terminal state rejects the constructed EOF/error cases. |
| Complete stream scenarios accepted | 2 of 2 | 2 of 2 | The stricter state machine did not reject the two valid fixtures. |
The diagnostic is deliberately finite and adversarial. It establishes that each proposed field has discriminating power in at least one controlled case. It does not estimate sensitivity over unknown future failures, prove collision resistance of an implementation, or validate concurrency and transaction behavior in the deployed stack.
12. Interpretation and alternative explanations
The simplest interpretation is that the current collector was optimized for operational observability and incremental deployment, not for audit-qualified reconstruction. That is not a defect in the recorded fields; it is a mismatch between the current implementation contract and a stronger claim.
Several alternatives could explain the diagnostic gap:
A hidden lower-level trace already captures the boundary. If infrastructure outside the inspected Lovable source records exact envelopes, provider IDs, terminal events, and atomic release linkage, the present audit understates actual capability. E21-B must inventory all telemetry layers and reconcile them by attempt ID.
Provider SDK or gateway metadata can fill the gaps. A gateway may expose request IDs and stream completion without application changes. That could reduce implementation work, but it still needs a stable lineage, canonical envelope commitment, source manifest, completeness state, and release semantics.
A database transaction alone is enough. Atomic writes solve partial durability, not boundary mismatch or stream-finality ambiguity. Conversely, a perfect boundary tap without an atomic gate can still lose fields and overstate completeness.
The strict label gate harms availability more than it helps auditability. This is plausible. The proposed two-path release policy exists so a measurement incident can preserve an answer while clearly withholding the audit-qualified label. E21-B must quantify both paths.
Raw-content logging is simpler. It may ease reconstruction, but it expands privacy, retention, breach, and access risks. ABMG treats content capture as separately authorized and tests whether commitments plus controlled reveal are sufficient.
The constructed fixtures merely encode the proposal. Partly true. D21-A is a contract sanity check, not independent empirical confirmation. Its value is exposing falsifiable cases and preventing vague claims. The live shadow experiment is the decisive next test.
13. Falsifiable hypotheses
All thresholds below are preregistration candidates for E21-B. They are targets, not results.
| ID | Hypothesis | Falsifier |
|---|---|---|
| H21-1 | The boundary-tapped ABMG collector reconstructs at least 99.5 percent of provider attempts exactly and improves reconstructability by at least 20 percentage points over the current collector on the same traffic. | Either absolute coverage is below 99.5 percent or improvement is below 20 points. |
| H21-2 | Under injected single-write, process-crash, and retry-race failures, false audit_qualified_released is at most 0.1 percent with a two-sided 95 percent upper confidence bound at or below 0.5 percent. | Observed rate or confidence bound exceeds the threshold. |
| H21-3 | Under truncated streams, malformed events, and HTTP-200-then-error cases, false-complete classification is at most 0.1 percent and recall for valid complete streams is at least 99.5 percent. | Either safety or valid-completion threshold fails. |
| H21-4 | With model calls held identical, A1 instrumentation adds at most 5 ms median and 20 ms p95 synchronous latency, at most 15 KB median content-free storage per provider call, and no statistically or practically meaningful answer-quality regression. | Any overhead ceiling is exceeded or quality shifts beyond the preregistered equivalence margin. |
| H21-5 | Reviewers given ABMG traces reconstruct attempt lineage and terminal status at least 30 percent faster, with at least 95 percent correctness, than reviewers given current traces. | Time improvement is below 30 percent or correctness below 95 percent. |
| H21-6 | The boundary tap outperforms an otherwise equivalent post-hoc collector under envelope mutation and fallback. | The lower 95 percent bound on the reconstruction difference is not above zero. |
14. Proposed production experiment E21-B
14.1 Design
Implement ABMG in shadow mode at the provider adapters, using the existing production call as the sole inference. Every arm consumes the same recorded provider traffic and the same terminal answer. No arm may make an additional model call. This keeps total model tokens, model latency, and model cost fixed by construction.
Use a preregistered traffic window large enough to include both providers, streaming and non-streaming paths if present, retries, fallbacks, empty-source calls, truncated-source calls, and terminal failures. Exclude raw content from ordinary analysis. Generate controlled faults in a staging replay or transactional fault-injection environment, never by corrupting live user runs.
14.2 Baselines and ablations
| Arm | Description | Question isolated |
|---|---|---|
| B0 | Current collector | Existing reconstructability and overhead |
| B1 | Generic OpenTelemetry-style operational trace | Whether standard correlation and provider fields are sufficient |
| B2 | Full A1 ABMG | Combined mechanism |
| A1 | ABMG without boundary tap; request reconstructed post hoc | Value of emission-time capture |
| A2 | ABMG without atomic release gate | Value of fail-closed completeness |
| A3 | ABMG without immutable source version and slice IDs | Value of exact evidence provenance |
| A4 | ABMG without provider-specific terminal-state logic | Value of stream finality |
| A5 | ABMG with plain unsalted commitments only | Privacy and authorized-reveal risk; never treat as confidential |
| A6 | Full A1 plus independent checkpoint | Marginal benefit and cost of A2 non-equivocation |
14.3 Ground truth and adjudication
For staging replays, a controlled proxy records the exact outbound bytes, inbound event sequence, connection termination, and injected fault. The proxy record is the test oracle, not the ABMG collector itself. For live shadow traffic, compare ABMG to provider-returned IDs and gateway logs where available, then manually adjudicate a stratified sample. Disagreements remain failures until explained; they are not silently discarded.
14.4 Primary metrics
- Attempt reconstructability: fraction of physical provider attempts for which provider, endpoint, exact canonical envelope commitment, lineage, terminal state, response/error receipt, usage provenance, and producing-artifact link agree with the oracle.
- False audit-qualified release: incomplete runs carrying the strong release label divided by all incomplete runs.
- False complete stream rate and valid-stream recall.
- Source-manifest fidelity: exact agreement on source version, order, range, and truncation for every call.
- Atomicity: number of visible partial event sets after injected failures.
- Reviewer reconstruction correctness and time.
14.5 Regression and harm metrics
- Provider call count, input/output tokens, and total model cost, which must be identical across replayed arms.
- End-to-end latency, gate latency, timeout rate, retry rate, terminal-run failure rate, and output-retention rate.
- Answer bytes and existing answer-quality scores, tested for equivalence because instrumentation should not alter inference.
- Storage bytes per call, event-ingest throughput, queue lag, and measurement availability.
- Sensitive-field incidence in routine telemetry, unauthorized reveal attempts, salt/key access, retention-policy conformance, and deletion behavior.
- Disagreement between audit-qualified labels and actual completeness.
- User-visible availability lost solely to measurement failures.
- Operational alert burden and false-positive incident rate.
14.6 Analysis and decision rule
Report denominators, confidence intervals, missingness, provider/path strata, and all injected-fault classes. Do not aggregate away a provider-specific failure. Treat missing oracle data as unscorable, not success.
Adopt A1 for audit-qualified research runs only if H21-1 through H21-4 pass, no critical privacy finding remains open, all model-compute counters match, and every false audit-qualified event is root-caused. Adopt A2 only if its independent checkpoint produces material non-equivocation benefit relative to cost. If thresholds fail, retain A0 operational telemetry, label research runs unverified, and publish the negative result.
15. Expected failure modes and mitigations
| Failure mode | Why it matters | Detection or mitigation |
|---|---|---|
| Boundary hook executes before final request mutation | The commitment does not represent transmitted bytes | Tap the final serialized body or gateway request; compare against proxy oracle. |
| Provider adds or changes terminal semantics | Valid streams are rejected or incomplete streams accepted | Version provider adapters and fixtures; default unknown terminal event to incomplete. |
| Request ID unavailable | Attempt cannot be reconciled with provider logs | Record explicit unavailability reason; retain application attempt ID; do not fabricate. |
| Transaction succeeds but event content is wrong | Atomic falsehood remains possible | Independent oracle samples, schema validation, canonicalization test vectors, and A2 checkpoint only for later rewrite detection. |
| Gate outage blocks useful output | Product availability degrades | Preserve unverified_output_retained where policy allows; alert and withhold strong label. |
| Hash dictionary attack | Sensitive low-entropy text can be guessed | Separate fresh salts and reveal authority; do not expose raw or unsalted commitments as confidential proof. |
| Clock skew | Event order becomes ambiguous | Use monotonic per-run sequence plus IDs; timestamps are supplemental. |
| Retry overwrite | Failure evidence disappears | Append-only events and explicit parent lineage; amendments never erase originals. |
| Source bytes identical across versions | Content digest alone hides version identity | Bind immutable source version and selected ranges in the manifest. |
| Evidence added after prompt assembly | Source registry overstates delivery | Build manifest from exact per-attempt slices at serialization time. |
| Telemetry and answer paths diverge | The trace describes a different request | Compute commitment in the same boundary adapter that transmits the object. |
| Checkpoint service colludes or fails | A2 assurance is overstated | State trust assumptions, use independently operated checkpoints, and retain A1/A2 distinction. |
16. Limitations and non-claims
- The source audit is static. It did not execute the deployed service, inspect runtime configuration, observe production concurrency, or query the production database.
- D21-A models current behavior from inspected code. It is not a byte-for-byte test of the live TypeScript parser or persistence layer.
- The fixture families are constructed to isolate known gaps. They do not estimate how often those gaps cause real incidents.
- ABMG is a proposal and reference projection. No production schema, transaction, adapter, privacy-control, key-management, or checkpoint code was implemented in this round.
- Cryptographic commitments bind captured bytes only under their algorithm, canonicalization, salt, and custody assumptions. They do not establish truth, semantic use, causality, or completeness before capture.
- Provider-documented IDs and terminal fields may change. Versioned adapters and contract tests are required.
- The proposed thresholds have not been statistically powered against real traffic. E21-B must determine sample size before data collection.
- No new Encevra model benchmark, human evaluation, or product-quality experiment was run. No result in this paper supports a claim that Noetis reasoning improved.
- The brain-inspired analogy is an engineering prompt with an ablation, not a biological equivalence claim.
- No site publication or deployment is authorized by this paper.
17. Conclusion
Noetis now records materially more than the Round 19 baseline, but the current collector still combines narrow content fingerprints, post-hoc attempt assembly, permissive stream handling, independent best-effort writes, and release before measurement completion. That combination is adequate for many debugging tasks and inadequate for the stronger statement that every released research run is reconstructable at the provider boundary.
ABMG makes that statement testable. It captures the exact dispatch, binds every physical attempt and response, requires provider-specific finality, persists the completeness decision atomically, and separates ordinary output retention from audit-qualified release. Its privacy posture keeps raw content off by default, and its assurance ladder avoids treating a local trace as independently witnessed history.
D21-A shows that the proposed fields and state machine distinguish all constructed failure pairs while making no model or product claim. The decisive evidence must now come from E21-B on real provider traffic at fixed model compute.
18. Next-round handoff
Round 22 should proceed only if the team can implement the smallest real ABMG shadow slice at the provider boundary. The minimum deliverable is:
- final serialized-request capture with a named canonicalization profile;
- stable attempt and parent IDs across retry and fallback;
- provider request/response IDs and provider-specific terminal status;
- exact per-attempt source version, order, range, and truncation manifest;
- response and terminal-artifact commitments;
- one atomic completeness transaction and explicit release label;
- fault-injection tests for stream interruption, typed mid-stream error, write failure, retry race, and process crash;
- privacy review of salts, reveals, access, retention, and deletion;
- fixed-call shadow evaluation against an independent proxy oracle.
Do not resume the Round 18 versioned-repair pilot or make stronger collaborative-reasoning claims until the live-boundary and completeness thresholds pass. If implementation access is unavailable, the next round should publish that constraint rather than invent results.
19. Reproducibility and inspected artifacts
The accompanying reproducibility package contains:
- this manuscript and rendered PDF;
diagnostics.py, the executable finite-world harness;fixtures.json, the exact constructed cases;collector_contract_audit.json, the 31 field assessments and code evidence;diagnostic_results.json, all 34 named checks and limitations;proposed_protocol.json, the machine-readable ABMG contract;source_ledger.json, inspection dates, inspected portions, supported claims, and limitations;delivery_status.jsonandMANIFEST.json.
To reproduce D21-A from the package root, run:
python3 diagnostics.py
Expected summary for version 1.0: 34/34 named checks pass; 10 present, 9 partial, and 12 absent requirements; seven incomplete write states compatible with current release and zero audit-qualified under ABMG; 10/10 current envelope misses and 10/10 ABMG detections; 24/24 current source-version collisions and 0/24 under ABMG; 24/24 current response ambiguities and 0/24 under ABMG; five current stream false accepts and zero under ABMG. Model, provider-network, and production-database call counts are all zero.
20. Sources
[S01] Encevra Lovable project. Frozen source commit 5aba83a193c988eefc8622292155afa085f3fc9e, dated 11 October 2026. Inspected 12 October 2026. Reviewed the Research overview, rounds index, publications registry, full Rounds 20, 19, 10, and 9, trial-readiness plan, run-measurement.ts, run-measurement.server.ts, ai-providers.server.ts, panel-shared.ts, panel-runs.server.ts, migrations 0001 and 0003, public run API, and queue paths. Supports numbering, prior conclusions, and current collector behavior. Static connector inspection only; no deployed execution or production data. https://lovable.dev/projects/8483bc59-da36-492e-b8b8-6bf684070241
[S02] Encevra Research Round 20. What Must the Log Reveal? Minimal-Disclosure Receipts for Verifiable Evidence Delivery. 11 October 2026. Inspected full manuscript, MDIR mechanism, E20-A, decision gates, limitations, and handoff. Supports the provider-bound collector handoff and separation between local receipts and independent checkpoints. Internal research artifact; live E20-B was not run. https://id-preview--8483bc59-da36-492e-b8b8-6bf684070241.lovable.app/research-round-20
[S03] Encevra Research Round 19. Collector and reconstructability analysis. 10 October 2026. Inspected full manuscript, collector contract audit, result tables, and handoff. Supports the pre-Stage-4 baseline. It does not describe the current post-Stage-4 source. https://id-preview--8483bc59-da36-492e-b8b8-6bf684070241.lovable.app/research-round-19
[S04] Encevra Research Round 10. Evidence exchange, selective revision, and a testable path toward brain-inspired intelligence. 1 October 2026. Inspected full route, instrumentation sequence, claims, exclusions, and experiment recommendations. Supports the rule that precise instrumentation should precede stronger reasoning claims. https://id-preview--8483bc59-da36-492e-b8b8-6bf684070241.lovable.app/research-round-10
[S05] Encevra Research Round 9. Learning reusable procedures from verified experience. 30 September 2026. Inspected full route, validation conditions, and publication requirements. Supports versioning, costs, negative-result retention, and revalidation. It does not address provider-bound telemetry. https://id-preview--8483bc59-da36-492e-b8b8-6bf684070241.lovable.app/research-round-9
[S06] OpenTelemetry. Semantic conventions for generative client AI spans. Development status; continuously updated. Inspected 12 October 2026: span lifetime, retries, inference and retrieval attributes, response IDs, errors, content privacy, external storage, and streaming-chunks section. Supports operational trace fields and content-off-by-default guidance. Does not guarantee integrity, completeness, or non-equivocation. https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md
[S07] W3C. Trace Context. W3C Recommendation, 23 November 2021. Inspected overview, traceparent, parent-ID semantics, processing model, privacy, and security. Supports cross-component correlation and sensitive-header caution. It does not bind payloads or prove completeness. https://www.w3.org/TR/trace-context/
[S08] Rundgren, D.; Jordan, B.; Erdtman, A. JSON Canonicalization Scheme (JCS). RFC 8785, June 2020. Inspected abstract, introduction, I-JSON constraint, deterministic sorting, primitive serialization, and UTF-8 output. Supports invariant JSON representation for repeatable hashing. Canonicalization does not prove truth, capture completeness, authorization, or confidentiality. https://www.rfc-editor.org/rfc/rfc8785.html
[S09] PostgreSQL Global Development Group. PostgreSQL 18 Documentation: Transactions. Current documentation, inspected 12 October 2026. Inspected all-or-nothing semantics, durability, visibility, BEGIN, COMMIT, ROLLBACK, and savepoints. Supports atomic persistence. It does not guarantee correct boundary capture or independent witnessing. https://www.postgresql.org/docs/current/tutorial-transactions.html
[S10] OpenAI. API Reference: Chat Completions response and streaming fields. Current reference, inspected 12 October 2026. Inspected completion IDs, finish reasons, models, usage, streamed chunk IDs, service tier, and interruption warning. Supports receipt fields and the possibility that interrupted streams omit final usage. Provider identifiers do not prove internal semantic use. https://developers.openai.com/api/reference/resources/chat
[S11] Anthropic. Claude API errors. Current documentation, inspected 12 October 2026. Inspected HTTP and mid-stream errors, typed error structure, retries, and request-ID exposure. Supports explicit recording of retry and stream-error attempts. The inspected Noetis path uses direct fetch rather than the SDK. https://platform.claude.com/docs/en/api/errors
[S12] Anthropic. Create a Message and stop-reason documentation. Current reference, inspected 12 October 2026. Inspected response ID, model, stop reason, usage, stream behavior, and request shape. Supports terminal receipt fields. These fields do not prove that supplied evidence affected inference. https://platform.claude.com/docs/en/api/messages/create
[S13] Steele, O.; Birkholz, H.; Delignat-Lavaud, A.; et al. COSE Receipts. RFC 9942, June 2026. Inspected abstract, receipt types, inclusion and consistency structures, verification order, and privacy. Supports an optional higher-assurance receipt layer. ABMG does not implement COSE or a verifiable data structure in this round. https://www.rfc-editor.org/rfc/rfc9942.html
[S14] Birkholz, H.; et al. An Architecture for Trustworthy and Transparent Digital Supply Chains. RFC 9943, June 2026. Inspected transparency definition, append-only records, registration policies, receipts, signed statements, and accountability limits. Supports independent checkpoint concepts and the limit that transparency cannot prevent false issuer statements. Applying the pattern to AI runs is an Encevra hypothesis. https://www.rfc-editor.org/rfc/rfc9943.html
[S15] Autio, C.; Schwartz, R.; Dunietz, J.; et al. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1, 26 July 2024; page updated 8 April 2026. Inspected publication identity, abstract, scope, and lifecycle framing. Supports lifecycle-wide trustworthiness management. It does not prescribe ABMG or its thresholds. https://doi.org/10.6028/NIST.AI.600-1
[S16] Kim, H.; Veloso, A.; Joao, F.; Kohles, S. Efferent Copy and Corollary Discharge Motor Control Behavior Associated with a Hopping Activity. 23 July 2013. Inspected metadata, abstract, stated experiment, and distinction between external sensory input and self-generated reafference. Supports only the emission-time-copy analogy. It is a narrow motor-control study, not evidence that AI agents literally implement a brain mechanism. https://pmc.ncbi.nlm.nih.gov/articles/PMC8849611/
21. Source-inspection statement
Every external source in the reference list was opened and inspected directly during this round. The machine-readable ledger records the publication date available, the exact portions inspected, the claim each source supports, and its limitations. Search-result snippets were not treated as sufficient evidence. No source was cited as proof of a claim it does not make.
Round 21 result in one sentence: current Noetis telemetry is materially improved but cannot yet justify audit-qualified provider-bound reconstruction; ABMG is a concrete, privacy-aware, fail-closed contract whose decisive live shadow test remains to be run.
