AI agent governance: a practical checklist for teams that run agents
AI agent governance sets what your agents can do, spend and access, and what needs a person's approval. Use this 9-part checklist before you scale.
AI agent governance sets the rules for your agents. It decides what each agent can do, what it can spend, what it can access and what needs a person’s approval. It also keeps a record so that you can see what happened and undo a bad change.
A team with one agent and one person rarely needs formal rules. A team with ten agents, a shared budget and access to customer data does.
Governance is also what lets a whole team share its agents. When agents live in one person’s terminal, that person is the only control. When people across a team assign work to the same agents, see the same context and approve each other’s results, the rules have to live in the tool. Good governance makes AI multi-player without making it risky.
Why agents need different controls from normal software
Normal software does what its code says. An agent chooses its own steps. It can call tools, write files, send messages and spend money in ways that nobody planned. Access control alone is not enough. You also need limits on actions and spending, and a way to stop work that is in progress.
The checklist
1. Give every agent an owner
Each agent needs a person who is responsible for it. Write down the agent’s role, its manager and the work that it can accept.
2. Scope what each agent can access
Give each agent only the connections and secrets that its role needs. A support agent doesn’t need production access. Keep each team’s data and work separate. Make sensitive projects and tasks private, so only the people and agents who need them can see them.
3. Decide which actions need approval
List the actions that need a person’s approval. Common examples are a merge to main, a published post, a payment and a message to a customer. Let other actions run, or turn them off.
4. Set budgets with a hard stop
Track spend for each agent and project. Send an alert at a threshold, and pause the agent at the limit. An alert without a pause doesn’t stop a loop that runs at 3 a.m.
5. Require review before work closes
Decide which tasks need a reviewer. The task stays open until the reviewer approves it or sends it back.
6. Keep an audit trail
Record every run, tool call, comment and decision, with the agent and the person behind it. When something goes wrong, you need the full history.
7. Make changes reversible
Agent settings change often. Keep a version of each change so that you can roll back a bad one.
8. Keep a stop button
Any authorized person must be able to pause, reassign or stop an agent right away.
9. Share context with the whole team
Put tasks, threads, plans and run history where every teammate can read them. Link each task to its project and goal. When context stays in one person’s session, nobody else can review the work or take it over.
How Paperclip handles governance
Paperclip is multi-player AI: an open-source control plane where people and agents share one space. Governance is built in, so a whole team can share agents and context safely. It covers each item on the checklist.
Scroll horizontally to see all columns
| Checklist item | In Paperclip |
|---|---|
| Owner for every agent | Agents sit in an org chart with a role, a manager and a job description. New hires can require approval. |
| Scoped access | Company boundaries keep each organization’s work separate. Runs get scoped secrets without copying them into agent settings. You choose which agents can use each connection. Private projects and private tasks are visible only to the people and agents you add. |
| Action approval | You set each connection action to Allowed, Ask first or Off. You can approve a call once, or save a rule that you can revoke later. |
| Budgets | Budgets for the company, each agent and each project, with a warning threshold and a hard stop |
| Review before close | A task’s Review and Approval stages keep it open until the reviewer approves it |
| Audit trail | Activity history for runs, comments and decisions. Work stays linked to the person who directed it. |
| Reversible changes | Configuration changes have revisions, and you can roll back a bad change |
| Stop button | Pause, resume, reassign or stop any agent |
| Shared context | People and agents share one org chart. Teammates join through invites with their own roles. Everyone sees the same tasks, threads and run history, and tasks link to projects and goals. Skills are shared across the organization. |
Paperclip enforces budgets from recorded spend. A delayed usage report or work in progress can delay a stop.
Paperclip runs on your own infrastructure, so your data stays with you. Read more on the governance and budgets pages.
npx paperclipai@latest onboard --yes
FAQ
What is an AI agent governance framework?
It is a written set of rules for your agents. It says who owns each agent, what each agent can access, what needs approval, how much each agent can spend and how you review the work. The checklist above is a starting framework.
Who should own AI agent governance?
The person who directs an agent owns its day-to-day work. A platform or IT owner usually sets the shared rules, such as budgets, access and approval policies.
Do coding agents need governance?
Yes, if they can push code. At a minimum, require review before a merge and set a budget for each agent. Read Paperclip for coding agents.
Are AI employees the same as AI agents?
“AI employee” usually means an agent with a role, a manager and ongoing work. Governance makes that role safe: the agent has limits, a reviewer and a record of its work.