All research

Round nine · 30 September 2026

A research agenda: procedures, conditions, and change

This round turns the earlier mechanism reviews into a testable question for the lab: can an AI system acquire useful procedures from verified experience, reuse them on unfamiliar problems, and revise them when the world changes? Noetis is the experimental system for answering it, with Desk and Lexplio supplying consequential failure cases from real work. The evidence does not establish a route to superintelligence; it establishes a credible direction a serious lab can hold while making precise claims about what its experiments demonstrate.

How to read the evidence

Findings from published studies and preprints appear with numbered references. Proposed designs, thresholds, and budgets are recommendations for Encevra, not results. No Encevra implementation, benchmark run, or new performance result is claimed in this round. Recent preprints receive less weight than replicated findings, and author-reported product results remain author-reported. The strongest opportunity identified here is learning when a previously useful procedure still applies; its novelty would depend on the implementation and the comparisons.

FEEDBACK: UNRESOLVED CLAIMS RE-ENTER THE LOOPLearn01Condition02Detect03Revise04

The working thesis

Encevra studies how AI systems turn verified experience into reusable reasoning procedures under changing conditions.

Each word is doing work. "Verified" needs an operational meaning: a deterministic tool result, a hidden software test, and an expert judgment carry different strengths of evidence, while an agent agreeing with its own earlier explanation carries much less. The system must preserve those distinctions when it updates memory. "Reusable" means success on new instances and new combinations of familiar operations; replaying a previous answer is retrieval, not a new procedure. "Changing conditions" means encountering something that can invalidate a past success, such as an appended exception, a changed tool contract, or a different objective.

The motivating failure is concrete. A system correctly applies a supplier approval policy. Later, an exception is appended to the document while the original supporting sentence remains verbatim. A quote checker can still pass even though the original decision is now wrong. The research problem is to connect a decision to the assumptions that made it valid, detect a consequential change, and update the relevant procedure without degrading unrelated ones. This is a proposed experimental case, not a claim about current Noetis behavior.

What would make this a research lab rather than a product team with ambitions: a versioned hypothesis, a comparison against the best feasible alternatives, a record of results that could disprove the hypothesis, and enough published detail for another team to reproduce the central finding, including a useful benchmark or failure analysis when a proposed mechanism loses. "Superintelligence" stays a long-term aim involving broad performance beyond human capabilities, never a label inferred from one benchmark, and no reviewed result licenses a reliable date for reaching it.

What the current systems research establishes

Learned coordination is already a serious field. Conductor trains a 7B model with reinforcement learning to specify tasks, worker models, and visibility between workers, reporting gains over individual workers and several multi-agent baselines under its own tested conditions. Sakana Fugu packages related orchestration research as a commercial API, with company-reported comparisons. Two scaling studies qualify the "more agents" instinct: across 260 configurations, results depend strongly on task structure, and in teams of up to 30 models, having at least one correct answer becomes more likely while aggregation often fails to select it. Recursive Language Models treat a long prompt as an external environment the model can inspect programmatically, with benefits that depend on the model and task and some trajectories incurring large call costs.

The implication for Noetis: measure the candidate pool and the selector separately. If one worker already produced the right solution and Noetis chose another, adding workers addresses the wrong failure. If every worker shares a missing assumption, the fix is an independent observation or executable check, not another debate round that repeats the same error more confidently. A scientific contribution needs to identify what the controller learns that existing coordination methods do not reliably handle; changing procedure validity and accountable learning from corrections are a more specific target than worker-pool adaptation alone.

Research directionEstablished nearby workEncevra must demonstrate
Adaptive orchestrationConductor and FuguA benefit beyond worker selection and workflow generation.
Long-context computationRecursive Language ModelsBetter task outcomes at measured cost, including difficult tails.
Prompt improvementGEPAGains beyond a strong optimized prompt.
Experience playbooksACE and CLINTransfer beyond closely related task repetitions.
Conditions on memory useBASMReliable revision of conditions after meaningful changes.
Local correction of workflowsNLPGBenefits beyond route-specific policy repair.
Automated improvementDarwin Gödel Machine and AlphaEvolveGeneralization beyond the evaluator used during search.

These methods differ in training access, feedback, and computational budget. A fair comparison implements the relevant mechanisms under matched conditions or states the mismatch explicitly. A weak home-built approximation cannot support a claim of defeating the published method.

Continual learning is the central opportunity

CLIN demonstrated persistent textual memory with causal abstractions in 2023, ACE develops evolving playbooks through generation, reflection, and curation, and GEPA searches for better prompts using execution feedback. Substantial prior art exists for improving an agent without updating its weights; none of it establishes unrestricted learning across arbitrary domains. The distinction that matters for Noetis is between storing "this attempt worked" and learning "this operation tends to work under these observable conditions." The latter still requires evidence about exceptions, because a fluent explanation produced after success may identify an incidental feature instead of the reason the attempt succeeded.

BASM, which stores applicability conditions, risk boundaries, avoidance strategies, and recovery information with experience, is the closest prior art to the proposed memory design, and NLPG's local policy revision shows the direction is active. Claiming ownership of "memory with scope" would be inaccurate. The candidate contribution is narrower: test whether a procedure's applicability conditions can themselves be learned, challenged, and revised after a change, with independent evidence for promotion. The contribution might be a better algorithm, a benchmark that exposes an overlooked failure, or evidence that an existing method already solves the problem.

Accumulated state can also make a system worse. AgentCL found memory can help when earlier experience is useful while degrading performance on unrelated work. Continual Learning Bench found simple in-context baselines strong and dedicated memory systems not consistently justifying their cost. GovMem provides the sharpest caution: a method can reduce bad memory writes partly by refusing useful writes, so Encevra must measure both learning retained and harmful information excluded. Model training enters only when the experiments locate a bottleneck that better prompts and external state do not resolve: a retrieval failure calls for better selection, a controller that repeatedly chooses poor checks calls for training that controller, and a backbone that cannot execute a required reasoning operation despite good context calls for targeted adaptation. Those are different interventions and should be evaluated separately.

What the new neuroscience suggests

The earlier rounds drew on brain mechanisms to generate architecture ideas. This round adds recent work on compositional control and hippocampal replay. The useful translation is from an observed function to a falsifiable software hypothesis; naming an agent after a brain region supplies no evidence that the function has been reproduced. These four become ablations of the two main experiments, not separate neuroscience-themed projects.

Shared components need selective activation

A primate study of switching among compositionally related tasks describes shared neural subspaces for reusable task components and suppression of information irrelevant to the active task. It concerns performance among trained tasks; it does not establish how an artificial system acquires new ones.

Candidate experiment

Build two procedures that look similar but require different operations because one variable changes. Compare indiscriminate retrieval with explicit selection, then introduce a new combination of already learned components. Keep memory token budgets equal so a gain cannot be explained by a longer prompt.

Transfer limit: If explicit component selection only helps on the templates used to design it, the representation has not demonstrated broad compositional value. If ordinary in-context examples work equally well, retain the simpler method.

Control can adapt during an action

A human continuous pursuit study describes behavior as changing mixtures of goal-specific controllers, with recorded activity supporting distinctions among latent policy state, mixture changes, and task value. It does not establish that three corresponding software modules are necessary.

Candidate experiment

Give a task an initially reasonable plan, then reveal a contradiction halfway through. Compare a fixed plan, replanning at every step, and replanning triggered by a measurable contradiction. Record recovery success, unnecessary restarts, and extra calls.

Transfer limit: Keep the task objective separate from the current plan and the system's estimate of the environment. That separation is an engineering proposal motivated by the study, not a neural implementation claim.

Replay may support recombination

Intracranial recordings from 28 patients performing structured inference tasks report hippocampal ripple-associated replay and changes in prefrontal representations consistent with assembling familiar elements into inferred configurations. Replay also predicted efficient inference. It does not show that replaying model transcripts causes comparable improvements.

Candidate experiment

After solving component tasks, give each method the same offline compute budget: one reviews full episodes, another reviews compact decision points, and a third creates and checks new combinations in a simulator. Evaluate on withheld combinations and include impossible combinations to detect merely plausible sequences.

Transfer limit: Reject the mechanism if generated rehearsal examples teach invalid compositions or improve only the examples generated during rehearsal. Checks must be external to the model that invented the sequence.

Expected uncertainty can guide preparation

Recordings from 17 patients during predictable and unpredictable stimulus streams associate unpredictability with changes in prestimulus ripples and subsequent cortical responses. This concerns perceptual uncertainty in a clinical population; it does not validate an AI confidence estimator.

Candidate experiment

Compare random checking, confidence-based checking, and checking selected from observed historical error patterns. Then shift the historical pattern so it becomes wrong. A useful policy should recover rather than continue spending on yesterday's uncertainty.

Transfer limit: Raw self-reported confidence is an input to test, not a calibrated probability. The biological result motivates the hypothesis without establishing its effectiveness in software.

The proposed architecture

Start with an inspectable learning loop on a fixed backbone. Record the task, available evidence, chosen operations, tool outcomes, and final judgment. When an episode yields trustworthy feedback, propose a procedural update, test it on separate cases before making it available to future tasks, and preserve the prior version so a failed update can be reversed. The proposal does not require a model's private internal reasoning: observable actions, concise decision summaries, source references, and verified outcomes are sufficient.

ComponentMinimum representationFailure it helps investigate
Task stateObjective, constraints, unresolved questions, relevant environment version.Pursuing an obsolete plan or the wrong objective.
Evidence recordSource identity, location, version, observed result, verification method.Treating repeated claims as independent evidence.
ProcedureOperation, preconditions, expected result, exceptions, provenance.Reusing a successful action outside its valid scope.
ControllerAvailable checks, remaining budget, estimated usefulness.Spending on redundant debate or unnecessary calls.
Update recordProposed change, supporting cases, counterexamples, evaluation result.Promoting an explanation after one lucky success.
Evaluation storeWithheld families, reference outcomes, immutable run records.Optimizing against leaked answers or a changed grader.

A worked lifecycle: a fictional procurement task says an order requires review above a stated threshold, and Noetis learns a procedure that extracts the amount, reads the current threshold, and checks listed exceptions. Later the environment introduces a category-specific exemption. The procedure's history should preserve the old rule and its valid period; a candidate update adds the exemption condition and names the evidence supporting it. Tests include a qualifying purchase, a nearly identical non-qualifying purchase, and an unrelated task that should be unaffected. Only after passing the promotion criteria does the update become active. The benchmark should also include a misleading annotation that looks authoritative but sits outside the task's source hierarchy: a system that accepts every change cannot demonstrate sound revalidation. This is a controlled reasoning environment, not a recommendation about real procurement law.

Experiment one: adaptive verification

Question

Can Noetis select checks that improve final correctness at a fixed budget, especially when candidate answers share a plausible error?

The hypothesis is that a targeted external observation often contributes more than another unconstrained critique. "External" refers to the evidence source, not merely a different model name: two models reading the same flawed summary can still make dependent errors.

The task construction begins with a diagnostic pilot of 30 to 50 independent task families spanning document decisions, executable software tasks, and controlled rule environments, including cases where checking helps and cases already solved correctly. Each family contains a meaningful variation: an appended exception, a swapped role, a changed unit, a stale tool schema, or a plausible answer whose intermediate calculation fails. Entire underlying documents, repositories, and rule templates are reserved for final evaluation, because paraphrasing a training case does not create an independent test family.

System comparedPurpose
Strong single agent with tools and a tuned promptEstablish the simplest serious baseline.
Single agent with additional compute under the same spending capSeparate architecture gains from more reasoning budget.
Current fixed Noetis workflowMeasure the actual contribution of adaptation.
Multiple candidates with a fixed verifierSeparate candidate generation from selective checking.
Adaptive verification controllerTest the proposed mechanism.

Measurements come in order: final task success first, then whether material defects were detected, how often correct answers were wrongly rejected, how often a solvable task was abandoned, and total cost including dollar spend, tool calls, and elapsed time with medians and 95th percentiles. Timeouts and malformed outputs count as outcomes rather than being removed. For multi-candidate tasks, report the fraction of runs containing a correct candidate and the fraction where the selector chose one; this exposes whether the bottleneck is generation or selection. Semantic judgments use a predefined rubric and blinded human review on a stratified sample; a second model judge is useful triage but is not independent ground truth.

The proposed decision rule: advance if the adaptive method shows a practically useful improvement over the strongest comparison at matched spending, or reaches equivalent quality at materially lower cost. The planning targets of a five percentage point improvement in task success or a 20 percent cost reduction are decisions for Encevra, not effects predicted by the literature. Stop or simplify if a fixed verifier matches the adaptive policy, if gains vanish at equal cost, or if the system repeatedly turns correct answers into incorrect ones. Those results would identify a useful limit and save implementation effort.

Experiment two: learning under changing conditions

Question

Can a fixed-backbone system learn a reusable procedure from feedback and preserve its usefulness after relevant rules change, without harming unrelated tasks?

The hypothesis concerns the quality of adaptation, not the volume of stored memory. It requires a system to decide what changed, which prior knowledge remains valid, and which procedure needs revision.

Each environment begins with component tasks that provide limited feedback. The system then encounters new instances and withheld combinations, a relevant rule changes, and it returns to earlier task families whose rules remain unchanged. In one example, a simulated data tool first uses one field convention, then changes a field's meaning: some earlier procedures remain correct, one becomes dangerous, and the environment can verify the actual output, exposing a system that merely updates a remembered label while continuing to compute incorrectly. Multiple independently designed environments keep the mechanism honest: a synthetic environment gives exact answers and controlled changes, while a document or code environment tests whether it survives messier evidence.

ComparisonWhat it controls
No persistent memoryInitial capability and the benefit of accumulated experience.
Raw episode retrieval or a simple notepadThe value of ordinary context retention.
ACE-style playbookStructured experience and reflection.
BASM-style scoped memoryExisting applicability and risk-boundary mechanisms.
Proposed dynamic revalidationLearning and revising conditions after change.

Two evaluation modes prevent accidental leakage. In frozen-transfer mode, learning stops before the withheld tasks and no answers return to memory. In online-adaptation mode, the system receives exactly the feedback the environment permits after each episode and can update only for later episodes. Never score an answer after revealing its reference solution, and keep the final family set unavailable during method selection, including through prompts used to invent synthetic examples.

MetricInterpretation
Improvement with experienceLater success relative to the matched stateless baseline.
First-attempt transferSuccess on a new combination before feedback.
Recovery after a changeEpisodes or cost required to regain useful performance.
Harmful transferErrors caused by applying prior knowledge to an incompatible case.
RetentionPerformance on still-valid earlier tasks after updates.
Useful write rateFraction of promoted procedures that help on separate cases.
Lifetime costLearning, checking, storage operations, and execution across the sequence.

Report curves over time, because a final average can hide a system that learns quickly, forgets badly, and recovers only after repeated corrective feedback. The strongest initial result would show beneficial transfer to withheld combinations, faster recovery after meaningful changes, and acceptable retention at matched lifetime cost. Evidence against the thesis includes a simple notepad winning, improvement only on near-duplicate tasks, gains disappearing when feedback is equalized, or a refusal-heavy policy appearing reliable because it stops doing useful work. If BASM already handles the changes well, the research should identify that result and focus on the remaining failure rather than relabeling the baseline.

How this could develop into deeper model research

Once logged episodes show which observations changed outcomes, Encevra can train a controller to choose checks or retrieve procedures, starting with an interpretable policy or small classifier before a language-model controller. Controller training is a new experiment with its own data split that includes failures, redundant checks, and cases where no intervention was needed; training only on successful traces would bias the system toward over-intervention. Targeted open-model training becomes reasonable if the backbone cannot consistently execute a learned procedure, encode its conditions, or distinguish a decisive counterexample despite strong context. Tiny Recursive Models show small specialized architectures can perform substantial iterative reasoning on particular structured tasks, which motivates narrow controlled experiments, not a claim that a tiny model replaces a general one.

Automated improvement stays bounded before it becomes open-ended. The Darwin Gödel Machine and AlphaEvolve demonstrate useful forms of automated search with external feedback; they do not establish unlimited self-sustaining improvement. Encevra can eventually automate a similarly bounded loop: propose a controller change, run a development evaluation, retain promising variants, and submit the chosen variant to an untouched evaluation, with the scorer and final test access outside the candidate's write permissions and human review reserved for promotion to real product use. The scientifically interesting measure is how often the loop discovers improvements that survive new environments, because an increasing development score can come from adapting to the evaluator. Foundation-model pretraining would be a materially different program, and the current evidence supports a systems-learning program first.

The first 90 days

PeriodWorkRequired result before expanding
Days 1 to 14Freeze a Noetis version, establish strong baselines, build diagnostic families, verify graders.Reproducible runs and an interpretable failure analysis.
Days 15 to 30Run the adaptive-verification pilot and targeted ablations.Evidence of a useful effect, or a decision to use the simpler method.
Days 31 to 55Build procedural learning and faithful scoped-memory comparisons.Demonstrated transfer on development environments without obvious contamination.
Days 56 to 75Freeze the method and run withheld changing environments.Learning and retention results with costs and uncertainty.
Days 76 to 90Reproduce the main result independently and test one bounded product workflow.A publishable report and a justified deployment decision.

The cost figures below are arithmetic illustrations, not current quotes or a commitment to spend; actual per-episode cost is measured in the pilot before any provider or sample-size choice. One episode includes every nested model call and tool operation used to produce its scored outcome. For illustration, 120 independent environment families, five systems, three repeated sequences, and ten scored episodes per sequence produce 18,000 episodes; adding 4,000 development episodes yields 22,000. The repeated sequences do not create 360 independent families.

Cost assumptionLower illustrationHigher illustration
Mean full episode cost$0.10$0.75
22,000 episodes$2,200$16,500
Additional review and infrastructure allowance$300$1,300
Subtotal with 25 percent contingency$3,125$22,250

The illustrations exclude salaries, substantial GPU training, taxes, and specialist review beyond the allowance. A 1,000-episode pilot at the same assumed costs would use $100 to $750 in episode spend before review and infrastructure. If actual episodes cost several dollars, redesign or reduce the study before running it at scale. The main financial control is staged commitment: run a pilot large enough to expose dominant failure modes, then estimate the final sample required for the chosen claim.

On ownership: Noetis owns the shared experimental code and learning mechanisms, Desk exposes document and decision workflows, and Lexplio supplies domain-specific evaluation needs and product feedback. Only authorized cases enter research datasets, and product customer material does not automatically become public training data. The research claim stays distinct from product reliability claims: a controlled simulation result can justify the next experiment, while deployment in consequential work additionally requires relevant domain evaluation and an appropriate review process.

What we would publish, and the decisions after

A useful first release would contain the task-family construction rules, source and environment versions, feedback protocol, grader specification, baseline prompts, costs, and per-family results, including negative results and representative failures, releasing only data and artifacts Encevra has the right to share. The title should state the actual finding: procedure revalidation improving adaptation under changing rules, if supported by results, never a general intelligence breakthrough from a narrow suite.

Observed resultResearch decision
A strong single agent matches Noetis at lower costSimplify orchestration and focus on the remaining failure.
Fixed verification captures most gainsProductize the fixed method while studying where adaptation matters.
Scoped memory works but dynamic revision adds littleCredit the baseline and test harder or more realistic changes.
Gains survive new combinations and rule changesExpand environments and test a learned controller.
Procedure selection succeeds but execution repeatedly failsInvestigate targeted model training.
Gains survive external reproductionPublish the bounded claim and invest in broader transfer.

Encevra's next credible milestone is a reproducible result showing that experience improves future competence under a meaningful change. That would justify a larger research program and provide a concrete reason for others to take the lab's ambition seriously.

The sources

All 26 entries were consulted for this round on 30 September 2026. "Preprint" means the source should not be treated as independently established fact. Access notes identify the material supporting the discussion; they do not imply every appendix was audited. The authors have not evaluated or endorsed Encevra.

1. Towards a Science of Scaling Agent Systems

Kim et al. 2026 · arXiv 2512.08296, version 3, 8 April 2026

Primary empirical study; main results and limitations inspected. Architecture effects are task-dependent, and predictive claims need the paper's statistical qualifications.

Read the source

2. Multi-agent Scaling Across Disjunctive and Compensatory Tasks

Fortuna and Bertalanic 2026 · Preprint, 25 September 2026

Abstract inspected. Recent evidence on candidate coverage, aggregation, and shared error; not a universal law of agent teams.

3. Learning to Orchestrate Agents in Natural Language with the Conductor

Nielsen et al. 2026 · arXiv 2512.04388, version 2, 29 January 2026; ICLR 2026

Methods and main evaluation inspected. Close prior art for learned coordination and recursive workflows.

Read the source

4. Sakana Fugu: A Multi-Agent Orchestration System as a Foundation Model

Sakana AI 2026 · Official announcement, April 2026

Product evidence and author-reported beta comparisons. The table's evaluation provenance limits direct model ranking.

Read the source

5. Recursive Language Models

Zhang et al. 2026 · arXiv 2512.24601, version 3, 11 May 2026

Main method, results, and limitations inspected. Relevant to programmable access to long contexts and variable inference cost.

Read the source

6. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

Agrawal et al. 2026 · arXiv 2507.19457, revised February 2026; ICLR 2026

Paper and official code; abstract and project material inspected. Strong prompt-optimization comparison; the title does not imply universal superiority to reinforcement learning.

Read the source

7. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models

Zhang et al. 2026 · arXiv 2510.04618, version 3, March 2026; ICLR 2026

Main method and evaluation inspected. Prior art for incremental playbook development.

Read the source

8. When Not to Imitate: Boundary-Aware Skill Memory for Reliable Tool-Use LLM Agents

Lin et al. 2026 · EMNLP 2026 Findings, 23 August 2026

Main method and limitations inspected. Closest comparison for conditions and risk boundaries attached to memory.

9. NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents

Liu et al. 2026 · Preprint, 27 September 2026

Method and evaluation protocol inspected. Local policy correction; very recent and not independently reproduced here.

10. AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

Shu et al. 2026 · arXiv 2606.02461, version 3, 26 September 2026

Main protocol and limitations inspected. Separates forms of sequential evaluation and examines memory transfer and interference.

Read the source

11. Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments

Asawa et al. 2026 · Preprint, 4 June 2026

Methods, results, and limitations inspected. Six-domain sequential benchmark with strong simple-context baselines.

12. When Does Continual Learning Require Learning

Harrington et al. 2026 · Preprint, 8 July 2026

Main comparisons and limitations inspected. Evidence is sensitive to model scale and the type of environmental change.

13. When Not to Write Memory: Governing False Promotion from Correlated Agent Traces

Qi, Xu, and Li 2026 · arXiv 2607.02579, submitted 30 June 2026

Abstract inspected. Synthetic and real-trace evidence differ; a useful caution about conservative memory governance and review burden.

Read the source

14. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents

Zhang et al. 2026 · arXiv 2505.22954, revised March 2026

Paper and official project article; abstract and project account inspected. Empirical search over coding agents, with bounded evaluation and oversight.

Read the source

15. AlphaEvolve impact update

Google DeepMind 2026 · Official research update, 7 May 2026

Author-reported applications of evaluator-guided algorithm search. Broader claims require the relevant technical evaluations.

Read the source

16. Building compositional tasks with shared neural subspaces

Tafazoli et al. 2026 · Nature 650, 164 to 172

Peer-reviewed primate study. Author abstract and figure descriptions reviewed.

Read the source

17. Neural basis of compositional control

Chericoni et al. 2026 · Nature, 12 August 2026

Abstract and figure descriptions inspected; full article subscription-restricted. Human continuous-control study, not an artificial-agent architecture test.

18. Human hippocampal ripples coordinate planning sequences and compositional representations in neocortex

He et al. 2026 · Nature Neuroscience 29, 1711 to 1721

Clinical intracranial recordings and structured inference tasks. Author abstract inspected.

Read the source

19. Human hippocampal ripples tune cortical responses based on predicted uncertainty

Frank et al. 2026 · Nature Neuroscience, 25 June 2026

Open-access article. Main findings and experimental context inspected.

20. Nested Learning: The Illusion of Deep Learning Architectures

Behrouz et al. 2025 · arXiv 2512.24695, NeurIPS 2025

Abstract and official research account inspected. Architecture and optimization research beyond external memory storage.

Read the source

21. Less is More: Recursive Reasoning with Tiny Networks

Jolicoeur-Martineau 2025 · arXiv 2510.04871

Paper and official repository inspected. Specialized structured reasoning; general language capability is not established.

Read the source

22. ARC-AGI-3 launch

ARC Prize 2026 · Benchmark introduction, 25 March 2026

Official description of interactive novel environments. Launch scores are historical and are not used as current model rankings here.

Read the source

23. Measuring AI ability to complete long tasks

METR 2026 · Live task-horizon resource, updated May 2026

Method and results inspected. Human-duration horizons are conditional performance measurements, not literal autonomous runtimes.

Read the source

24. Demystifying evals for AI agents

Anthropic 2026 · Engineering guidance, 9 January 2026

Primary practitioner guidance on tasks, trials, graders, and evaluation design.

Read the source

25. PaperBench: Evaluating AI Ability to Replicate AI Research

Starace et al. 2025 · arXiv 2504.11855, April 2025

Abstract and benchmark introduction inspected. Research reproduction is distinct from novel discovery; historical scores are not current rankings.

Read the source

26. CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization

Majumder et al. 2023 · arXiv 2310.12858, October 2023

Abstract inspected. Earlier evidence for persistent textual causal memory in an interactive environment.

Read the source

Continuity with earlier rounds: Brain Research VIII (20 September 2026), Brain Research IX (21 September 2026), the Opportunity Second Look (25 September 2026), and the Ownership Strategy (26 September 2026) establish the prior research and recommendations. Their historical implementation descriptions are not treated as a current code audit.