Agent Guardrails
Early customers told us that without strong guardrails, they couldn't approve agents for anything mission-critical. They needed confidence that agents wouldn't expose sensitive data, drift off topic or act beyond their scope. I led the design of AI guardrails for Agent Workbench across three releases, from a first opt-in safety layer to organization-wide policies that developers can extend but never weaken.
- Role
- Lead Product Designer, the only designer on it
- Timeline
- 2025 → 2026
- Team
- Product management and engineering in the AI value stream
- Status
- Released for all agents
- Release notes
- Agent guardrails (beta) ↗
- AI Products
- Agent Experience
- User Experience
Challenge
As customers moved from experiments to production agents, trust became the biggest blocker. Teams handled safety by hand, writing rules into system messages or custom logic, with no consistent approach and no way to prove compliance.
The goal was to let teams ship agents safely, with guardrails on by default, visible in the product and configurable when needed. It also couldn't add to the wiring and testing work customers already complained about.
Two audiences
Admins own the organization's safety baseline; developers need to apply and understand it without becoming policy experts.
No breaking changes
Agents already in production couldn't suddenly start behaving differently when guardrails arrived.
Research
Guardrails came up again and again in early-access interviews and customer conversations. I also mapped how enterprise platforms and agent builders handle safety, and validated the direction with a solution architect and a team lead who build agents for customers. They surfaced needs we hadn't considered, like rules that depend on who is running the agent.
Defaults that match compliance
Safety shouldn't depend on every developer remembering it
Predefined categories, enabled per stage by admins
Force guardrails on every agent
Opt-in isn't enough for regulated teams
A mandatory floor applied to all agents in a stage
Transparency
Know what an agent is allowed to do
Guardrails visible on every agent, with active rules shown
Proof for audits and RFPs
Show that policies were applied
Every violation logged with a consistent, readable message
Company-specific rules
Internal IDs, banned topics, own vocabulary
Custom topics, word filters and data patterns
Key decisions
My first concept, the clickable low-fidelity prototype I validated, let admins create several guardrail profiles and developers pick one for each agent. To deliver value sooner, we split it into releases: one organization-wide policy first, profiles later.
First concept
Low-fidelity prototype, validated with users
Several profiles
Admins define a set of guardrail profiles
Pick one per agent
Developers choose which profile an agent uses
One release
Everything arrives at once
What shipped
Three releases, each live on its own
One floor per stage
Admins set the rules every agent must follow
Stricter per agent
Developers add rules on top, never remove them
Custom policies
Topics, words and data patterns in the third release
| Central policiesEnterprise platforms like ServiceNow, Salesforce and AWS | Per-agent rulesBuilder tools like OpenAI Agent Builder and n8n | Floor plus extensionsChosenOur approach | |
|---|---|---|---|
| Who defines rules | Admins, centrally | Each developer, per agent | Admins set the floor, developers add to it |
| Consistency | High | Depends on every developer | Guaranteed by the floor |
| Fit to the use case | Limited to the policies on offer | High | High, on top of the floor |
| Risk of weakening safety | Low | High | None: rules can only get stricter |
Central policies
Enterprise platforms like ServiceNow, Salesforce and AWS
- Who defines rules
- Admins, centrally
- Consistency
- High
- Fit to the use case
- Limited to the policies on offer
- Risk of weakening safety
- Low
Per-agent rules
Builder tools like OpenAI Agent Builder and n8n
- Who defines rules
- Each developer, per agent
- Consistency
- Depends on every developer
- Fit to the use case
- High
- Risk of weakening safety
- High
Floor plus extensionsChosen
Our approach
- Who defines rules
- Admins set the floor, developers add to it
- Consistency
- Guaranteed by the floor
- Fit to the use case
- High, on top of the floor
- Risk of weakening safety
- None: rules can only get stricter
Combining both models gave admins a guarantee and developers room to tighten rules for their own use case.
A floor that only goes up
Developers can upgrade an action, from log to mask to block, but never downgrade one an admin set. Safety can't be switched off by accident.
Our own categories
Guardrails are grouped in OutSystems terms, not mirrored from the provider behind them, so the platform can evolve without breaking agents.
Nothing breaks on release
Existing agents kept their behavior, and each release carried over what customers had already configured.
Messages that say why
Every violation follows one pattern, like "Personal information detected. Information masked.", so developers and admins see what was detected and what the guardrail did about it.
Solution
Guardrails live in three places: an admin console in ODC Portal that sets policies per stage, a guardrails card on every agent, and the logs where violations show up. I prototyped the latest release in code to test the console's new structure with real data.
ODC Portal
Set the floor
Admins choose the guardrails every agent in a stage must follow
Agent
Tighten it
Developers add stricter rules for their use case
Runtime
Check every call
Inputs and outputs are checked, then blocked, masked or logged
ODC Portal
Explain it
Violations show up in the logs with a readable reason
Agent guardrails
Set per stage by admins, extended per agent by developers
Content filters
Prompt attacks
Attempts to override the agent's instructions
Harmful content
Hate, violence, insults and misconduct
Sensitive information
Personal information
Names, IDs, financial and access data
Custom patterns
Company formats like employee IDs
Denied topics
Topic list
Blocked by meaning, not keywords
Word filters
Custom words and phrases
Exact matches, uploaded in bulk
Profanity filter
A managed, always-updated list
Outcome
First release
Opt-in safety
Admins enable predefined categories, developers opt agents in
Second release
A mandatory floor
Rules apply to every agent in a stage, and can only be tightened
Third release
Custom policies
Denied topics, word filters and company-specific data patterns
All three releases are live, and the same guardrails protect every agent, in Agent Workbench 1.0 and in Agents 2.0. Each release closed a gap customers had named.
Testing the last release brought back an idea from the first concept: reusable guardrail profiles. With one baseline for every agent, admins have to choose between too strict for low-risk agents and too loose for high-risk ones. Profiles would cover most real cases, and became the recommendation for what comes next.
Looking back, splitting the work got safety to customers sooner, but it settled on a single baseline before we'd tested agents with very different levels of risk side by side. Next time I'd test that mix earlier, while the model was still cheap to change.