How AI agents prepare clinical protocol review drafts with ThinkStack

A reviewer starts the interactive path by submitting a protocol and a review request. The extraction agent establishes the document identity and version, retrieves relevant material, and organizes content against the configured checklist. The draft should pair each Complete, Partial, or Missing status with the extracted evidence and its source location.

“Missing” means the agent did not locate the requested content in the reviewed material. The responsible specialist checks whether the content was overlooked, supplied elsewhere, or applicable to the trial. For a pilot, ambiguous or contradictory evidence should remain on an exception list until the reviewer resolves it.

The extraction agent combines a model, review instructions, an attached policy, and knowledge resources.

The demonstration starts by establishing the protocol identity and source document.

The extraction screenshot shows the prototype's 18-category table. The table uses the prototype checklist labels.

The draft surfaces missing and partial content alongside strengths. Its section labels and severity judgments require expert verification.

The gap-analysis agent retrieves reference protocols and compares selected design elements. Its draft identifies what each protocol states and why the difference may deserve review. The reviewer checks whether the comparator is relevant before using that difference to support a finding.

The supplied example selected two melanoma checkpoint-inhibitor protocols, NCT02731729 and NCT03620019, for comparison with the prostate cancer protocol. The disease settings differ. A review lead should set comparator-selection criteria and check whether the trial populations, treatment context, and design support each comparison.

The draft names its comparators so a reviewer can assess their relevance before relying on the comparison.

The statistical comparison shows the agent's proposed findings and severity labels. A specialist must verify the evidence, trial context, and interpretation.

The comparison also reports strengths and recommendations. These are draft assessments to review alongside the comparator limitations.

A public historical protocol documents another study's design. Its presence in the knowledge base establishes neither regulatory acceptance nor a requirement for the study under review. The draft should identify whether a question comes from guidance or from a comparator difference.

The example raises questions about statistical justification, monitoring, and safety reporting. The assigned specialist checks the cited evidence, confirms applicability, and accepts, revises, or rejects each finding. Strengths and comparison limitations belong in the same review packet.

CPAR's coordinating agent has three specialist collaborators. The statistical agent examines sample-size justification and analysis methods; the scientific-rationale agent examines the rationale and endpoints; the safety agent examines reporting language and monitoring provisions. Human specialists validate the findings in their domains.

Shared resources let the coordinating agent bring distinct specialist instructions into the same review task.

ThinkStack supplies the delegation mechanism. The review team defines each specialist's remit, sources, and acceptance criteria. Findings need to remain attributable to the agent and evidence that produced them.

Adding a clinical-pharmacology or biomarker agent requires appropriate sources and representative test cases. The review lead should evaluate whether the added specialist finds useful issues without duplicating findings or increasing unnecessary review work.

ThinkStack's Policy & Guardrails supports content, topic, word, sensitive-information, and grounding rules. Teams select the rules, test their actions, and inspect enforcement records. CPAR's policy is intended to constrain review scope and handle sensitive content and unsupported output; the configured behavior needs testing against the pilot's input and failure cases.

Grounding checks compare output with retrieved material. A passage can pass that check and still be outdated or misapplied. The expert review therefore needs to assess the source and interpretation as well as the policy outcome.

Policy testing exposes configured rules and their behavior before publication.

Calibrate the model judge

CPAR's evaluation setup uses five criteria: section mapping, completeness, gap identification, citations, and regulatory language. ThinkStack documents custom rubrics, versioned expectations, regression runs, and blind human scoring to calibrate the model judge. A review lead can compare automated judgments with specialist assessments before relying on the scores. See the evaluation screen.

The displayed automated scores are 0.9, 0.9, and 1.0. The manual fields are empty, so this screen does not establish expert agreement or independently validated accuracy.

The evaluator-reasoning view makes automated judgments inspectable. Its explanations still need comparison with expert assessments.

Two QA agents also assess the draft outputs. Their judgments should be tested against known failure cases and expert decisions. Agent agreement alone supplies no independent reference answer.

The trace lets the team inspect the retrievals and tool calls behind a disputed finding. After changing the sources, instructions, or evaluator, the team should rerun the same cases and check for new errors as well as improvements.

The CPAR example includes three pipeline stages: extraction, gap analysis, and report generation. ThinkStack's code environment displays the source, dependencies, and deployment configuration. The prepared output is a Key Findings draft for reviewer assessment.

The three-stage pipeline graphic, retained as an illustration of the run sequence. Its status badges do not document a verified batch run.

The pipeline screen shows the implementation and deployment controls for the prototype. It is not evidence of production throughput or unattended batch reliability.

ThinkStack's Agentic ETL documents extraction, validation, and low-confidence review handling. Flow Orchestrator supports fixed node sequences, while Agent Workspaces supports dynamic delegation. CPAR's illustrated pipeline follows a fixed preparation sequence; a team should choose its orchestration route based on the actual handoffs and exception paths.

Scheduled preparation is a possible deployment path. Before enabling it, test the trigger, failed-stage handling, source access, and report destination. Define who owns incomplete runs and prevent an unfinished packet from entering the normal review handoff.