Policy & Guardrails
Configure and test runtime rules. Inspect which rule applied, the text that matched, and the action recorded.
View the capabilityFor compliance, risk, and AI governance leaders
Apply AI policies and review how agents follow them.
Configure how ThinkStack agents handle sensitive information, test the rules, and inspect their actions. Give the people responsible for your controls access to the evidence they need.
The work to improve
Review how an AI workflow applies your controls.
A written policy describes what should happen. The person responsible for a control also needs to see which rule applied, what information the agent used, and what happened when a step failed. ThinkStack provides policy records and execution traces for that review.
01 · Inputs your team can access
Control requirementsPolicies and permitted uses
Test casesRepresentative inputs and responses
Execution recordsTool calls and source lookups
Review recordsPolicy versions and decisions
ThinkStack
Configured agents, tools, and workflow steps
Policy & GuardrailsConfigure and test runtime rules. Inspect which rule applied, the text that matched, and the action recorded.Observability & TracingInspect tool calls, source retrievals, generated responses, errors, and usage costs for each run.Evaluation HarnessScore representative cases against defined rubrics and compare automated scores with human review.Knowledge Base ManagementCurate the reference material available to agents and manage its classification, access, and lifecycle.Enterprise SearchRetrieve relevant passages from connected information within the configured permissions.Document Writing AgentsReview proposed edits as tracked changes, with supporting source passages, author details, and revision history.
Defined access · Configured checks · Accountable reviewers
02 · Outputs for review
Test resultsRules and observed behaviorInvestigation noteRelevant trace evidenceEvidence packageSource material for the control owner’s review
Where to start
Prepare evidence for control owners
Configure each workflow around your policies and assign review responsibilities. These examples cover policy testing, investigation, and evidence preparation.
01Policy configuration and testingSee how a proposed rule behaves before publishing it.
- Start with
- Internal policy requirements and representative prompts, including cases the policy should allow and cases it should block.
- Agent preparation
- Prepare proposed policy changes and test the draft. Show which rules were triggered, the text that matched, the confidence scores, and the actions taken.
- What your team receives
- A proposed policy version and test results for the control owner.
- Human review and approval
- The policy owner reviews the changes, assesses false positives and missed violations, and authorizes publication.
- What to measure
- Time spent reviewing policy changes, false positives, and missed violations.
02AI activity investigationReconstruct the steps that produced a questioned response.
- Start with
- A response, the associated run, and the issue raised by a user or control owner.
- Agent preparation
- Retrieve the relevant source lookups, tool calls, failures, and guardrail decisions. Link the findings to the execution records.
- What your team receives
- An account of the run with the evidence needed to assess the issue.
- Human review and approval
- Investigators decide whether a control failed, assess the consequences, and agree on corrective action.
- What to measure
- Investigation time, completeness of the evidence, and time to verify corrective action.
03Control evidence preparationAssemble the evidence needed for an internal control review.
- Start with
- Approved policies, selected run records, test results, and reviewer decisions.
- Agent preparation
- Organize the available evidence and draft a description of how the configured workflow operates, with links to the sources. Flag missing evidence for the control owner.
- What your team receives
- A review package containing source references and unresolved questions.
- Human review and approval
- The control owner checks the scope, accuracy, and completeness of the evidence and decides whether it is sufficient for the review.
- What to measure
- Preparation hours, missing evidence items, and corrections requested by reviewers.
Review example
Inspect a guardrail decision
A reviewer can inspect the policy version, the matched text, and the action taken, then open the related trace.
- Policy
- The reviewed version of the intake privacy policy
- Input
- A request containing a personal identifier
- Configured action
- Replace the identifier with a placeholder
- Decision record
- The rule, text that matched, action taken, and record identifier
- Related evidence
- The execution trace for the same run
Test representative cases that the policy should allow and cases it should block. Your control owners assess whether the results and remaining limitations are acceptable for the intended use.
Pilot measurement
Measure control behavior and evidence quality.
Agree on the test cases and investigation questions before assessing the workflow. Review both missed violations and unnecessary interventions.
- Policy testing
- False positives and false negatives confirmed by reviewers.
- Investigation
- Time to locate and assess the relevant execution evidence.
- Evidence preparation
- Hours to assemble the package and resolve missing items.
- Change review
- Time to approve a policy version and verify its behavior.
Platform capabilities
The modules behind these workflows
Review the relevant capabilities with your technical team and confirm the configuration for the intended use.
Evaluation Harness
Score representative cases against defined rubrics and compare automated scores with human review.
View the capabilityObservability & Tracing
Inspect tool calls, source retrievals, generated responses, errors, and usage costs for each run.
View the capabilityEnterprise Search
Retrieve relevant passages from connected information within the configured permissions.
View the capabilityDocument Writing Agents
Review proposed edits as tracked changes, with supporting source passages, author details, and revision history.
View the capabilityKnowledge Base Management
Curate the reference material available to agents and manage its classification, access, and lifecycle.
View the capabilityCommon questions
Before you plan the workflow
Does this establish regulatory compliance?
Your organization determines the applicable obligations, control design, and evidence requirements. ThinkStack can support policy enforcement, testing, and evidence preparation within an agreed workflow. Your specialists assess whether that workflow meets the requirements.
What do the records show?
Guardrail records identify the rule, the text that matched it, the action taken, and the record identifier. Execution traces show tool calls, retrieved sources, generated responses, cost, and latency. Confirm which records your review requires.
Who can inspect sensitive information in execution records?
Review access permissions, masking, retention, and export permissions with your security team. The server restricts trace queries to each user’s permitted scope. Confirm that reviewers can access the records they need.
Can we automate monitoring and release decisions?
Scheduled trace analysis, per-agent drift alerts, automatic trace-review queues, and automated release checks based on evaluation results are in development. When planning your deployment, confirm the available controls and required human review steps.