When scientists think of AI, the chances are they imagine either a chatbot or some undefined thing that combines all their data and gives insight. Unfortunately, neither extremes are true. Today, the chat is the most visible interface to a large language model (LLM). Science does not primarily need fluent conversation it does need a a production scientific system. Programmers have been the first line of humans exposed to AI and their responses have varied from a feeling a total loss of control to handing over everything to AI with abandon. Science cannot not work that way and scientists are right to be sceptical about ceding control. We need claims that can be checked, computations that can be rerun, evidence that can be traced, and decisions that remain accountable to a scientist.
A better mental model is a governed workflow engine with AI inside it. This is essentially wrapping the useful probabilistic code of AI – the agent – in a deterministic framework. The software coordinates evidence, models, databases, and experiments. The scientist still frames the question, defines what counts as acceptable evidence, interprets ambiguity, and decides what to do.
The governing principle is simple:
Agents can own workflows. They should not own truth.
Here, an agent means software in which a model can choose among actions—searching literature, calling a statistical package, querying a database, or asking for review—in pursuit of a goal. The model may propose the next step. It should not be the sole authority that certifies its own result.
This post develops a fourteen-component reference architecture for implementing that division of labor. It is not a new regulatory standard, nor a claim that every pilot needs fourteen separate products. The components are logical responsibilities. Several may live in one platform. Their rigor should be proportional to the system’s context of use: what the system does, who relies on it, and what happens if it is wrong. That risk-based posture is consistent with the NIST AI Risk Management Framework (Tabassi 2023) and the FDA’s draft risk-based credibility framework for AI used in drug-development decisions (U.S. Food and Drug Administration 2025).
The scientific loop
A useful scientific AI system connects the following loop:
- A scientist states a question, objective, and constraints.
- An agent or planner decomposes the work.
- An ontology and scientific context define the domain’s terms and legal relationships.
- Knowledge graphs, document retrieval, and structured databases provide evidence.
- Predictive models, statistics, bioinformatics, chemistry software, and other tools perform computation.
- Deterministic validators and policy reject outputs that violate hard constraints.
- An evidence and provenance graph binds claims to sources and transformations.
- The system presents a recommendation with uncertainty, not an unsupported conclusion.
- A human decides whether to act or run an experiment.
- The result becomes new evidence that updates the research record.
This is also a practical extension of neurosymbolic AI, the subject of my previous post. Neural systems—LLMs and predictive models—handle language, pattern recognition, and approximate inference. Symbolic systems—ontologies, schemas, rules, and state machines—define what terms mean and what must be true. The human scientist governs the symbolic layer and the consequential decisions. Neurosymbolic systems are not automatically correct. Their architectural promise is narrower: symbolic representations and rules provide explicit, inspectable constraints alongside neural learning and inference (Garcez and Lamb 2023). Whether those constraints catch scientifically important errors still has to be evaluated in the intended setting.
The fourteen components below turn that compact loop into an implementable system.
First, three terms that are easy to conflate
Before we dive deeper, there are terms which can be confusing (they were to me) for someone who is just starting out. Let’s take a quick look at them and then go ahead. The terms form the core of the system and are interesting tools in their own right.
If you know the differences between Ontology, Knowledge Graph and Neurosymbolic AI feel free to skip this section.
Ontology: the domain’s (and your team’s) explicit grammar
An ontology is a machine-readable specification of the kinds of things that exist in a domain and the relationships that are allowed among them. A small assay ontology might define Compound, Target, Assay, and Measurement; distinguish an assay from an assay result; and state that a measurement has a numerical value, a unit, and an experimental context.
An ontology is more than a list of preferred words. A controlled vocabulary says which terms to use. A taxonomy adds broader/narrower relationships. An ontology can add formal constraints and logical axioms. OWL 2 is one standard for expressing such semantics (W3C OWL Working Group 2012). In biomedicine, the OBO Foundry coordinates interoperable ontologies such as the Gene Ontology and Chemical Entities of Biological Interest (Smith et al. 2007).
Ontologies do not descend from nature complete and uncontested. They are maintained models of a domain. Scientists and data stewards must decide which distinctions matter, review changes, and preserve version history. Every company and team also has a vocabulary that develops and you want the affordances and limitations those terms imply to be used properly in the workflow.
Knowledge graph: assertions connected by typed relationships
A knowledge graph (KG) represents entities and assertions as a graph: nodes denote things such as compounds, genes, papers, experiments, or samples; edges denote typed relationships such as inhibits, measured_in, derived_from, or contradicts. Unlike a spreadsheet row, a graph can naturally connect the same entity across many sources and follow multi-step paths (Hogan et al. 2021).
The ontology and KG have different jobs:
- the ontology defines the grammar—what a
Compoundis and whetherinhibitsmay connect a compound to a protein; - the knowledge graph contains statements made using that grammar—
compound X inhibits protein Y, supported by assay Z; - provenance records who asserted that statement, from which source, by what method, and when.
A KG should not be confused with a graph neural network. The KG is a representation of knowledge. A graph neural network is a statistical model that may learn from graph-structured data.
Neurosymbolic AI: flexible inference with explicit constraints
Neurosymbolic AI combines statistical or neural components with symbolic representations or reasoning. In this architecture, an LLM might extract a candidate relation from prose; an ontology constrains the entity and relation types; a graph stores the assertion and its evidence; and a deterministic validator checks the proposed graph update. The neural component answers, “What might this text mean?” The symbolic boundary asks, “Does this proposed assertion conform to what our system permits?” A scientist decides whether it is scientifically warranted.
This makes the most of the powerful LLM tech and the established rules and protocols for esxecuting research: expecting a hand-built rule system to understand every form of scientific language, and expecting a language model’s fluency to substitute for definitions, evidence, or validation.
The fourteen components
Now for the main event. The components are not steps, they are more parallel systems that partially stabilize the intent of the researcher each with a partial constraint. This links back to my essays on how to think (or get meaningfully lost) with Places, Spaces and Thickets.
1. Orchestrator and task graph
The orchestrator turns an objective into explicit steps and dependencies. A directed acyclic graph (DAG) is often the right representation: nodes are tasks; edges specify which outputs are required before another task can begin. For example, a literature search and a database query may run in parallel, but the synthesis step needs to wait for both.
A task graph is preferable to one long agent transcript because it makes the workflow inspectable. Each step has declared inputs, outputs, failure states, and ownership. Scientific workflow engines such as Nextflow and Snakemake, together with portability standards such as the Common Workflow Language, already embody much of this discipline (Di Tommaso et al. 2017; Köster and Rahmann 2012; Crusoe et al. 2022). An LLM can plan with or revise a graph without replacing the workflow engine that executes it.
Implement it: begin with a bounded task such as evidence triage, or like my last post on email classification. Draw the DAG before choosing an agent framework. Define success, failure, and escalation for every node. Do not allow the planner to create unbounded subgraphs.
2. Specialized capabilities
“Specialized agent” can mean a separate model, but often it should mean a narrow role with a distinct tool set, prompt, data access policy, and evaluation. Literature review, causal analysis, cheminformatics, and experimental design require different evidence and failure checks. This can also mean a completely non-LLM process such as computer vision based analysis or an omics workflow. A monolithic generalist LLM makes those boundaries difficult to see.
Google’s Co-Scientist separates generation, reflection, ranking, evolution, proximity, and meta-review processes (Gottweis et al. 2026). The Robin approach combines literature and data-analysis agents in an iterative discovery workflow (Ghareeb et al. 2025). These systems demonstrate decomposition, not a general proof that multiple agents are more accurate. Outputs from agents that share a model, instructions, or evidence should not be treated as independent verification. Independent tools, external evidence, and human review still matter.
Implement it: specialize by scientific responsibility, or by actual analysis to be done. Give each capability the minimum tools and context it needs. Evaluate the handoff between capabilities as carefully as each capability itself.
3. Typed tool interfaces
A typed tool limits what is allowed for its inputs and outputs. For example, a dose-calculation tool might require a positive numerical amount, an allowed mass unit, body weight with a mass unit, and a species identifier. It should return a structured object—not a paragraph that another model must reinterpret.
A schema catches malformed fields and impossible enum values before a call reaches a database, instrument, or laboratory information system. Types also make calls loggable, testable, comparable, and cacheable. The ToolUniverse preprint applies this pattern across a large catalog of scientific tools, separating tool discovery from validated invocation (Gao et al. 2025).
Implement it: publish JSON Schema, Pydantic, protobuf, or language-native types for every consequential tool. Validate at both ingress and egress. Represent units explicitly with a library such as Pint or QUDT, rather than embedding units in field names or prose. The LLM should select a tool and propose arguments; conventional software should determine whether the call is valid.
4. Retrieval layer: databases, documents, and graphs
A model’s parameters are not a scientific database. A retrieval layer gives the system access to evidence through complementary modes:
- structured databases for exact identifiers, values, and curated records;
- lexical retrieval for exact terms, accession numbers, and rare names;
- vector retrieval for semantically similar passages whose wording differs;
- knowledge-graph queries for canonical entities, typed relationships, and multi-hop structure.
These modes should not be collapsed into “RAG.” Retrieval-augmented generation usually means supplying retrieved passages to a generative model. That is useful for documents, but it does not replace an exact database query or an ontology-constrained graph traversal. There is no good reason to throw out perfectly good technology, use it alongside the new stuff, pick the right tools for the job.
Implement it: use hybrid lexical and vector search for text, rerank a small candidate set, and preserve passage-level citations; hybrid fusion methods require empirical tuning rather than a universal weighting rule (Bruch, Gai, and Ingber 2023). Resolve entities to stable identifiers before joining sources. Record the query, retrieval date, index version, filters, and returned document IDs. Apply the FAIR principles—findable, accessible, interoperable, reusable—to the data products behind the agent, not just to the final paper (Wilkinson et al. 2016).
5. Persistent scientific state
Conversation history records what was said. Scientific state records what the research program currently believes: hypotheses, evidence, status, open questions, decisions, and planned experiments. I don’t know about you, but I would hate scouring chat transcripts and complex code just to keep track of work that lasts months.
A useful state object might say that hypothesis H17 is proposed, supported by two observational studies, contradicted by one perturbation experiment, and awaiting replication. When new evidence arrives, the state transition should be explicit: proposed → under_test → supported, or proposed → contradicted. Appending a corrective paragraph to a chat transcript is not equivalent.
Implement it: use a database with typed entities and status transitions. Give each record a stable identifier. Link evidence rather than copying it. Record human decisions and supersession. Research Object and RO-Crate practices are useful models for packaging related data, methods, people, and provenance (Belhajjame et al. 2015; Soiland-Reyes et al. 2022).
6. Code sandbox
Agents that write or execute code need an isolated environment. A sandbox limits filesystem, network, credential, memory, CPU, and wall-clock access. Isolation reduces security risk while a defined computational environment improves reproducibility.
A notebook kernel with unrestricted network access and inherited user credentials is not a sandbox. Neither is a container by itself unless its permissions, mounts, secrets, and egress are constrained; NIST’s container-security guidance treats the container runtime, image, host, registry, and orchestration layer as a combined security surface (Souppaya, Morello, and Scarfone 2017).
Implement it: use ephemeral containers or micro-virtual machines; mount inputs read-only; write outputs to a designated artifact directory; deny network access by default; inject short-lived credentials only for an approved task; capture standard output, errors, package locks, and environment metadata. Computational reproducibility requires code, data, dependencies, and execution order—not merely a final notebook (Sandve et al. 2013; Grüning et al. 2018).
7. Deterministic validators
A deterministic validator produces the same verdict for the same input and rule set. It checks properties for which uncertainty is not appropriate: schema conformance, dimensional consistency, identifier format, permission, allowed state transitions, ontology constraints, and physical or mathematical invariants.
Examples include 0 ≤ p ≤ 1, non-negative molecular mass, a required control group, or a rule that a result outside an instrument’s validated range must be flagged. SHACL validates graph data against declared, i.e. expected, shapes (Knublauch and Kontokostas 2017). Unit libraries and ordinary test functions can enforce non-graph constraints.
Validation must occur inside the loop: before a malformed tool call executes, before an unsupported claim is promoted, and before data enter shared state. Asking another LLM whether the first LLM’s answer “looks correct” is critique, not deterministic validation. Neither is asking a friend to look at the outputs – it’s tedious and they won’t.
Implement it: maintain validators as versioned code and schemas. Return specific machine-readable failures. Distinguish hard rejection from warnings that require scientific judgment. Test validators with known-invalid as well as known-valid cases.
8. Evidence and provenance graph
A scientific output should have searchable ancestry. An evidence graph connects a claim to supporting or contradicting evidence. A provenance graph connects an artifact to the activities, software, data, models, and people that produced it. These can be implemented together, but the questions differ: Why should I believe this claim? and How was this result made?
The W3C PROV model represents entities, activities, and agents, with relations such as used, wasGeneratedBy, and wasAttributedTo (Lebo, Sahoo, and McGuinness 2013). For AI-assisted work, a claim should link to retrieved passages or records, tool outputs, transformation steps, model and prompt versions, validator verdicts, and human approval.
Implement it: create provenance as the workflow runs, not during manuscript cleanup. Treat a citation generated from model memory as unverified until the identifier resolves and the cited source supports the claim. Preserve contradictions; do not average away disagreement between studies with different designs.
9. Identity and permissions
Every human, agent, service, and tool should have an identity and a narrowly scoped set of permissions. This is the familiar security principle of least privilege applied to agentic workflows.
The literature agent may read approved indexes but not write to the laboratory information system. An analysis agent may run code on de-identified data but not export row-level records. A workflow may draft a purchase request without authorization to submit it. This goes with point 6, especially if you are working with an agent through the terminal, it usually gets all the permissions you have, one bad command and you risk everything. WHat can also happen is that the text the agent reads ends up instructing it to take unexpected and maybe malicious actions, which is not good.
Implement it: use separate service identities, short-lived credentials, explicit allow-lists, and action-level authorization. Bind each tool call to the initiating user and workflow. Treat retrieved text and uploaded files as untrusted data: instructions inside a paper, web page, or email should not grant new capabilities. Log denied actions as well as successful ones.
10. Human approval gates
A human-in-the-loop label is meaningless unless the system specifies where it stops, what the reviewer sees, and what the approval authorizes. Place synchronous gates where an action is consequential, expensive, externally visible, or difficult to reverse. Low-risk retrieval and draft generation can often proceed with logging and later review.
A gate should present the proposed action, evidence, uncertainty, validator results, alternatives, resource commitment, and what will happen after approval. If reviewers receive more cases than they can inspect, the gate can become a rubber stamp—a familiar human-automation failure in which nominal oversight does not guarantee effective control (Parasuraman and Riley 1997). This happens a lot with AI that keeps asking for permissions, coders very often end up using auto or full permissions mode simply to avoid decision fatigue.
Implement it: define action classes and thresholds from the context of use. Require explicit approve, reject, or modify decisions. Record who approved what and when. Start conservatively, and expand autonomy only after measured performance supports doing so.
11. Observability
Observability is structured evidence about what the system did while it was doing it. For each run, capture the task graph, model calls, retrieved items, tool arguments and results, validator verdicts, state transitions, latency, token or compute use, and errors.
Observability is not the same as provenance. Operational traces help diagnose behavior across many runs; provenance establishes the lineage of a particular scientific artifact. The two should link to one another.
Do not assume that storing hidden chain-of-thought is necessary, available, or a faithful account of the factors that caused an answer (Lanham et al. 2023). Record actionable decision summaries, tool traces, and evidence links: unlike an unconstrained internal monologue, these are externally checkable audit objects.
Implement it: use an event schema and correlation ID across services. Adopt open telemetry conventions where possible. Build views for scientists as well as engineers: “Which sources changed this ranking?” is as important as “Which request timed out?” Protect traces because they may contain sensitive data.
12. Evaluation harnesses
An evaluation harness repeatedly runs defined tests and records results. For scientific AI, evaluation has at least four levels:
- component tests: schema adherence, extraction accuracy, retrieval recall, tool-call success;
- scientific task tests: performance on representative in-domain problems, with temporal and external validation where appropriate;
- human-system tests: whether scientists with AI make better decisions than scientists without it;
- prospective tests: whether recommendations survive new experiments or later evidence.
A random train/test split is often inadequate. Records from the same patient or time period can leak across splits (Kapoor and Narayanan 2023); in ligand-based benchmarks, structural similarity can reward memorization rather than generalization to new chemical series (Wallach and Heifets 2018). Evaluation data should match the context of use and probe plausible shifts.
Measure calibration as well as discrimination when probabilities guide decisions. Predictive accuracy or ranking performance does not by itself guarantee calibrated probabilities (Guo et al. 2017); proper scoring rules such as log score or Brier score reward honest probabilistic forecasts (Gneiting and Raftery 2007).
Implement it: version test sets and rubrics; include negative controls, adversarial cases, and known invariants; compare against simple baselines; stratify results by relevant subgroups; and rerun the harness whenever models, prompts, tools, retrieval corpora, or policies change. A larger LLM is not automatically an improved scientific system.
13. Versioning and reproducibility
A reconstructable run must identify more than the model name. Record:
- source data and retrieval snapshot;
- ontology and knowledge-graph releases;
- code and workflow commit;
- container or environment digest;
- model provider, model identifier, and version/date;
- system and task prompts;
- tool and validator versions;
- parameters, seeds where meaningful, and hardware/runtime metadata;
- approvals and any manual edits.
Classic computational reproducibility aims to rerun the same code on the same data. Hosted generative models complicate bit-for-bit repetition because users may not control an immutable model artifact or the complete inference environment, and repeated generation may vary. The practical target is therefore both reconstructability—knowing exactly what was run—and robust reproducibility—showing that conclusions remain stable across permissible reruns.
Model cards and dataset datasheets provide useful documentation patterns for intended use, limitations, performance, collection, and maintenance (Mitchell et al. 2019; Gebru et al. 2021).
Implement it: store immutable manifests with every result. Hash prompts and artifacts. For nondeterministic components, run repeated trials and define semantic or statistical equivalence criteria rather than promising identical prose.
14. Retry, timeout, and budget controls
An agent that can retry tools or spawn sub-agents has an unbounded worst case unless limits are designed in. Controls should bound:
- retries per step, with backoff and an explicit terminal state;
- wall-clock time per tool and per workflow;
- tokens, accelerator time, API spend, and laboratory resources;
- recursion depth and number of spawned tasks;
- retrieved documents and generated candidates requiring review.
These are scientific controls as well as financial controls. An unbounded search followed by selective reporting can create the same researcher-degrees-of-freedom problem that inflates false-positive findings (Simmons, Nelson, and Simonsohn 2011). It can also flood reviewers with low-value hypotheses and obscure the stopping rule.
Implement it: set budgets at workflow, agent, and tool levels. On exhaustion, stop visibly and preserve partial results. Escalate for reauthorization rather than silently raising the limit. Use observability data to determine whether extra compute changes decisions or merely generates more material.
The components work across the whole scientific cycle
Again, the fourteen components are not sequential boxes. Identity, observability, evaluation, reproducibility, and budgets span the workflow. Validators operate wherever probabilistic output crosses into a consequential boundary. Persistent state connects framing to learning.
This is why the architecture should not be treated as a procurement checklist. A small, read-only literature pilot may implement the responsibilities with a workflow file, a search index, structured outputs, a citation checker, version control, and a final scientist review. A system that orders experiments or contributes to a regulated decision needs much stronger identity, validation, provenance, monitoring, and approval controls.
A practical implementation sequence
Trying to build the complete platform before testing a real scientific task is a reliable way to create infrastructure that scientists, like all living things, will find a way to skip. Instead, start with one consequential decision and build outward based on what your team needs:
- Write the context of use. Name the user, decision, inputs, output, foreseeable error, and consequence.
- Define the human/software contract. What does the scientist decide? What may the software recommend, draft, or execute?
- Choose one bounded workflow. Prefer a reversible, read-only task before an action-taking one.
- Create a task graph and typed tools. Reuse validated scientific software rather than asking an LLM to imitate it.
- Ground the workflow. Connect authoritative databases, document retrieval, identifiers, and the minimum useful ontology.
- Add state and provenance at the start. Retrofitting lineage after a result is produced is unreliable.
- Place deterministic boundaries and human gates. Derive them from failure consequences.
- Build evaluation before expanding autonomy. Compare the human-only process with the human-plus-AI process on outcomes that matter.
- Observe, version, and bound every run. Treat model, prompt, data, ontology, and policy changes as system changes.
- Close the experimental loop. Feed confirmed, contradicted, and inconclusive outcomes back into state and evaluation.
The useful success metric is not the volume of generated hypotheses or minutes saved in drafting. Just like lines of code does not a good programme make. It is scientific decision yield: better-supported decisions and informative experiments per unit of scarce scientific resource. Speed matters when it improves that yield; otherwise it can simply accelerate error.
The scientist remains the center
Human-centered design is sometimes reduced to adding an approval button after an automated pipeline has already framed the problem, selected the evidence, and compressed uncertainty. That is too late. The scientist must remain present in the architecture as:
- principal, setting the research purpose and acceptable risk;
- domain authority, governing definitions, constraints, and evidence standards;
- experimentalist, deciding which uncertainty is worth resolving;
- critic, identifying when a formally valid answer is scientifically nonsensical;
- accountable decision-maker, approving consequential actions;
- learner, updating both scientific understanding and the workflow itself.
Good software makes those responsibilities easier to exercise. It surfaces disagreement instead of hiding it in a synthesis, carries provenance with a claim, routes arithmetic to computation rather than prose generation, and asks for attention at the few points where judgment matters most.
That is the posture I would use for scientific AI: not an autonomous oracle, and not a chatbot bolted onto a database, but a governed instrument built around the working scientist. Put probabilistic models where flexibility is valuable. Put deterministic systems where constraints are knowable. Put evidence under every claim. Keep the human where purpose, interpretation, and accountability belong.