Why Governed AI Agents Matter in Real-Time Clinical Trials

AI governance in clinical trials start with a consequential question: what authority should an AI system have over trial work? The answer determines which tasks it can perform, when people must intervene, and who remains responsible for the outcome.

For AI agents in clinical research, these decisions apply to actions such as routing work, updating records, and initiating follow-up. Accuracy matters, but so do authorization, oversight, and the ability to investigate what happened.

Sponsors need to establish:

• Which actions are within the agent’s approved scope?
• What information and systems may it access?
• When is human review required?
• Can its actions and their outcomes be reconstructed?
• Who is responsible for reviewing and correcting its work?

A 2026 review in eClinicalMedicine helps explain why these controls must reflect the agent’s role. The authors distinguish AI used as a clinical intervention from AI supporting trial operations, where evaluation considers workflow accuracy, decision impact, and safety under real-world conditions. The practical implication is that an agent’s performance needs to be assessed in the workflow where it operates, including the consequences of its actions.[1]

FDA and EMA’s January 2026 principles for good AI practice reinforce this approach through a clearly defined context of use, risk-based validation and oversight, traceable documentation, and ongoing performance review. Together, these principles frame governance as a responsibility that continues throughout the AI lifecycle.[2]

For sponsors, the task is to translate those principles into everyday controls: permissions that limit access, checkpoints that preserve human judgment, and records that make responsibility clear. This article examines how those controls support governed AI execution in real-time clinical trials.

This article is part of our Real-Time Clinical Trials series. For the broader context, read our pillar guide: Real-Time Clinical Trials Are Now an FDA Priority. Maxis AI Has Been Building for This Shift.

Why Do AI Agents Need Governance in Clinical Trials?

AI agents need governance in clinical trials because performing operational work can affect regulated trial processes, records, decisions, and responsibilities. Governance defines what an agent is permitted to do, which actions require human validation, who is responsible for oversight and correction. The controls should reflect the task being performed and the potential consequences of an incorrect or incomplete action.

Governance is not there simply to restrict AI.

It establishes the conditions for appropriate use: a defined purpose, clear operating boundaries, suitable permissions, proportionate human oversight, and records that support review.

FDA and EMA's 2026 principles call for risk-based validation, mitigation, and oversight that are proportionate to an AI system's role and model risk.[2]

As the consequence of an action increases, so does the need for stronger controls around that action.

Why Unrestricted Autonomy Is a Poor Default for Regulated Workflows

Unrestricted autonomy is a poor default because it allows an agent’s actions to extend beyond clear permissions and review points. An incorrect update can trigger further work across systems before anyone intervenes, making the error harder to contain and correct.

This also complicates accountability. Muller and colleagues describe how AI-enabled trials can diffuse responsibility across people and technologies, highlighting the need for explicit responsibility mapping and communication structures.[4]

A governed workflow therefore defines how far the agent may proceed, what requires approval, and who takes over when an exception occurs. Routine, lower-risk steps may proceed without individual review, while higher-impact actions require stronger oversight. The degree of autonomy should reflect the risk and authority assigned to each task.

What Changes When AI Moves From Insight to Action?

When an AI agent changes a record or advances a task, its action can affect the steps that follow. Governance therefore needs to address both the quality of its work and its authority to perform it.

For example, an agent might mark a query as resolved even though it is authorized only to prepare follow-up. The system may accept the update, but the action still exceeds the agent’s authority and could remove the query from further review. Controls must prevent unauthorized closure and preserve evidence of who approved any permitted change.

Recent reviews reinforce the need to assess workflow accuracy and decision impact, alongside validation, change control, transparency, and human oversight.[1][3] The controls should reflect the consequences of the action.

What Makes an AI Agent Governed?

A governed AI agent operates under controls that define its authority, guide human intervention, and preserve evidence of its actions. For AI governance in clinical trials, six controls are especially important.

1. Defined Workflow Scope

Define the agent’s task, intended use, and operating limits before deployment. This includes the workflow it supports, the inputs it uses, the outputs it produces, and the conditions that require escalation. A clear scope establishes where its responsibility begins and ends, consistent with FDA and EMA’s emphasis on a defined context of use.[2]

2. Role-Based Permissions

Give the agent only the access needed for its assigned task. Specify which systems and records it may read, which actions it may execute, and which fields it may modify. Separating read access from permission to act helps enforce workflow limits, complying with ICH E6(R3)’s recommendations on authorized users, roles, and access controls to agent operations.[5]

3. Human Review and Approval Checkpoints

Specify which actions require human review, who approves them, and when the workflow must pause. Reviewers need evidence and authority to amend, reject, or stop an action. EMA’s reflection paper identifies human oversight as a principle of trustworthy AI and recommends considering oversight during planning; these checkpoints are a practical application of that principle.[6]

4. Traceability and Explainability

Record the inputs, actions, system changes, and human reviews needed to reconstruct the work. Provide an understandable basis for consequential outputs so reviewers can assess whether an action was appropriate. Traceability establishes what happened; explainability helps people evaluate its basis. Both are core principles in the 2025 FUTURE-AI consensus guideline for trustworthy healthcare AI.[7]

5. Reproducibility and Controlled Change

Reproducibility means being able to repeat an evaluation under documented conditions and obtain results consistent with predefined acceptance criteria. Preserve model versions, inputs, instructions, and configurations so teams can test this; logs alone do not demonstrate it. Generative outputs may vary in wording, so assess the facts, decisions, and permitted actions that must remain consistent. ICH E6(R3) recommends risk-based system validation and change control, which support reassessment when the workflow or model changes.[5]

6. Accountability

Assign named owners for approving scope, supervising work, investigating errors, and authorizing corrections across sponsor, CRO, site, and vendor boundaries. Muller and colleagues describe how responsibility becomes difficult to fulfill when people lack the necessary knowledge or control. Their proposed response combines responsibility mapping, identification of barriers, and communication channels. Applied to AI agents, owners need access to records and authority to intervene as well as an assigned role.[4]

What Is the Difference Between Human-in-the-Loop and Human-Supervised AI?

The practical difference is when human intervention is required. Terminology varies, and both approaches can operate within the same workflow.

The table below shows how each approach works and where people review or approve the AI’s actions.

Human-in-the-loop AI can therefore be a checkpoint within a broader human-supervised workflow. Both retain human accountability; the appropriate level of intervention depends on the action’s consequences and the judgment it requires.

Why Audit Trails Matter When AI Can Act

When people and AI agents contribute to the same workflow, teams need a clear record of each action, approval, and change. This allows reviewers to reconstruct what happened, investigate errors, and identify who was responsible.

ICH E6(R3) recommends that computerized systems record workflow actions alongside direct data changes, with corrections attributable to the person or system making them. It also addresses the identification of authors, reviewers, and approvers, where appropriate, and the preservation of record history. These recommendations concern computerized systems and trial records generally; applying them to AI-supported workflows means retaining both automated actions and human reviews.[5]

For an agentic workflow, audit trails and linked supporting records should answer five questions:

An attempted action should be distinguishable from a completed one. Recording that an agent submitted an update, for example, does not establish that the receiving system accepted it. Retaining the outcome helps teams verify completion and determine whether further action is needed.

How Governance Enables Scale, Not Just Risk Control

The permissions, review checkpoints, and audit records that support one workflow can provide a foundation for others. Establishing these controls gives teams a consistent starting point as they extend AI use across studies and functions.

Sponsors can standardize several elements:

• Access rules: define how permissions are assigned and restricted.
• Human review: specify which actions need approval and who provides it.
• Exception handling: define when an agent must pause and who takes over.
• Execution records: identify which actions, approvals, and outcomes are retained.
• Validation and ongoing review: Establish how performance is tested and changes are assessed.

Each new workflow still needs assessment against its intended use and risks. Reusing a permission model, for example, does not mean granting every agent the same access.

FDA and EMA’s good AI practice principles emphasize risk-based oversight and lifecycle management, supporting continued assessment as AI systems and their uses evolve.[2]

Governance supports scale by making controls reusable while keeping authority and oversight specific to each workflow.

Governed AI Agents vs. Copilots vs. Autonomous Agents

As organizations extend AI across workflows, they need to define how much authority each system receives. The comparison below focuses on what the system can do and where people remain involved.

These categories can overlap: governance describes the controls around AI, while autonomy describes how independently it can act. Copilots and autonomous agents both need governance appropriate to their role.

For regulated clinical work, these labels alone do not establish suitability. Teams should assess each system’s intended use, permitted actions, validation, oversight, and records available for review. Those decisions determine the controls needed as deployment expands.

How Maxis AI Approaches Governed Execution

Maxis AI applies these controls through its AI Workforce for Clinical Trials, which is designed to support defined clinical workflows under human supervision. Its approach supports greater execution capacity while keeping decisions and accountability with the responsible teams.

Five principles guide this approach:

• Defined boundaries: Each agent works within assigned tasks, with clear limits on the actions it may take.
• Controlled permissions: Access to systems, records, and functions matches the agent’s role.
• Human review checkpoints: Actions requiring clinical, regulatory, or operational judgment are routed to designated reviewers, while permitted routine tasks can proceed.
• Traceable execution: Actions, approvals, workflow changes, and exceptions are recorded to support review and investigation.
• Reproducibility: Versioned context and retained decision records provide a basis for repeat testing. Demonstrating reproducibility also requires checking results against workflow-specific acceptance criteria.

For organizations evaluating AI agents in clinical research, this approach makes the operating model clear: what agents can do, where people intervene, and how completed work is reviewed. These are the foundations of Maxis AI’s governed execution approach.

What Does AI Governance in Clinical Trials Need to Achieve?

Effective AI governance in clinical trials should allow appropriate tasks to proceed while keeping the system’s authority, oversight, and accountability clear. Four outcomes are particularly important:

• Work progresses: Agents complete permitted tasks and escalate issues they cannot resolve.
• Authority stays clear: Each agent acts within its assigned scope, permissions, and approval requirements.
• Actions remain reviewable: Records show what happened, what information supported the action, and whether it succeeded.
• People remain accountable: Named owners oversee the work, address errors, and authorize changes.

Together, these outcomes provide a practical foundation for expanding governed execution across clinical workflows.

Key Takeaways

• Governance should be built into AI execution.
• Agent authority needs clear boundaries.
• Permissions should match workflow responsibilities.
• Human review and approval should reflect the consequences of the action.
• Auditability and accountability must remain intact.

Frequently Asked Questions

Continue Reading: Real-Time Clinical Trials Series

• What FDA's Real-Time Clinical Trial Initiative Means for Clinical Operations Teams
• Real-Time Trial Monitoring vs. Real-Time Trial Execution
• How Sponsors Can Prepare for Real-Time Clinical Trial Execution

References