AI Agent Security Checklist for Production Deployments

Moving an AI agent from a controlled demonstration into production changes the security problem. A demo can be reset when it behaves unexpectedly. A production agent may read internal documents, act through service accounts, communicate with customers, or update systems of record. The question is no longer whether the model gives an impressive answer. It is whether the complete system remains inside its security boundaries when inputs, data, tools, and conditions are adversarial.

This checklist is designed for security engineers, AI platform teams, application owners, and risk leaders preparing an agent for real use. It complements the OWASP AI Agent Security Cheat Sheet and the OWASP Top 10 for Agentic Applications. It is not a certification and should be adapted to the system's actual impact.

1. Establish ownership before testing

Every production agent needs a named business owner and technical owner. Record its purpose, users, environment, model, orchestration layer, data sources, tools, MCP servers, identities, and approval path. Define who can change prompts, models, retrieval sources, tool definitions, and permissions.

Document unacceptable outcomes in plain language: disclosure of another customer's data, execution of an unapproved transaction, sending information to an unapproved recipient, modification of production code, or retention of sensitive content beyond policy. These outcomes become security-test objectives.

If ownership is unclear, stop. An agent without an accountable owner will eventually accumulate tools, data, and exceptions that nobody can safely review.

2. Draw the real data flow

Map what enters the system and where it goes. Include user messages, system instructions, files, retrieved documents, conversation history, memory, tool metadata, tool results, telemetry, provider logs, and human-review interfaces.

For each flow, classify the data, identify the processor and storage location, document retention, and determine whether the content can be used for provider training. Verify deletion and access controls rather than relying on architecture diagrams alone.

Pay particular attention to content copied into model context. Security teams often protect the source database but overlook traces, evaluation datasets, exported transcripts, and support logs that contain the same information.

3. Separate instructions from untrusted content

An agent may process instructions and data in the same context window. Retrieved documents, websites, emails, tickets, and tool results can contain text that looks like an instruction. Treat those sources as untrusted even when they come from an internal repository; an attacker may influence the upstream content.

Label provenance, minimise unnecessary content, delimit data from policy, and reject the assumption that a prompt alone creates a hard boundary. Test direct and indirect prompt injection using harmless objectives. Observe downstream actions, not just the answer.

4. Test RAG authorisation and retrieval boundaries

Retrieval augmentation can bypass application controls when documents are indexed broadly or access context is lost between the user, retriever, and generator. Verify tenant filters, role filters, source permissions, chunk metadata, cache keys, citations, and post-retrieval filtering.

Create synthetic documents for different users and roles. Ask semantically adjacent questions, reference document titles, manipulate conversation context, and test whether metadata or summaries leak when full text is blocked. The detailed method in How to Test RAG Applications for Data Leakage can serve as a test plan.

5. Isolate memory and sessions

Decide what the agent may remember, for how long, and for whose benefit. Separate short-term conversation state from persistent memory. Namespace memory by user, tenant, role, task, and environment as appropriate.

Test fresh sessions, account switching, shared devices, role changes, logout, deletion, and expired records. Plant non-sensitive canaries and verify that they do not reappear outside the intended scope. Require confirmation before storing sensitive or instruction-like content, and preserve provenance so remembered statements are not mistaken for policy.

6. Inventory tools and MCP servers

Maintain an exact inventory of every capability available to the agent, including tools exposed indirectly through MCP or another agent. For each tool, record server origin, description, schema, side effects, credentials, data reach, network reach, and approval requirements.

Review tool descriptions as security-relevant input. Monitor changes and test MCP tool poisoning. Namespace similar tools, make provenance visible, and remove capabilities that are not required for the agent's current purpose.

7. Constrain tool arguments

Typed schemas are not sufficient when parameters accept arbitrary identifiers, URLs, paths, queries, filters, recipients, or commands. Bind tenant and user identity server-side. Prefer narrow operations such as get_current_user_orders over a general run_query capability.

Test alternate record IDs, wildcards, encoded values, large ranges, redirects, internal addresses, path traversal shapes, unexpected enum values, and values copied from previous results. Verify both rejected and accepted calls with server logs.

8. Apply least privilege to every identity

List every credential the agent can cause to be used. Scope credentials to the user, task, environment, and time window wherever possible. Do not give a customer-facing agent an organisation-wide service token simply because it makes development easier.

Store secrets outside prompts and model-visible context. Rotate and revoke them. Test what happens after user permission changes, credential expiry, agent disablement, and session termination. A hidden credential is not necessarily a well-scoped credential.

9. Put deterministic policy outside the model

Models can help interpret intent, but they should not be the sole authorisation layer. Enforce resource ownership, transaction limits, allowed recipients, file boundaries, and network destinations in deterministic code or policy systems.

The model should not be able to redefine what counts as approved by claiming that an administrator already agreed. Policy decisions need authenticated context and an auditable result.

10. Design meaningful human approvals

Require human confirmation for high-impact, irreversible, or unusual actions. The approval screen should state the action, target, data, recipient, cost, and authority being used. A generic “Continue?” prompt after an agent has hidden key details is not informed approval.

Protect the approval channel from social engineering. Do not let the model fabricate approval status or rewrite the evidence shown to reviewers. Apply step-up authentication where consequence warrants it.

11. Bound autonomy and resource use

Set limits on turns, tool calls, retries, execution time, spend, retrieved records, output size, and concurrent tasks. Use idempotency and transaction boundaries for actions that may repeat. Define what the agent should do when tools disagree, data is missing, or confidence is low.

Test timeouts, partial responses, duplicates, malformed results, and unavailable dependencies. An agent that loops indefinitely or repeats a financial action can create impact without an attacker stealing any data.

12. Validate generated code and commands

If the agent writes or executes code, isolate execution in a disposable sandbox. Restrict filesystem mounts, environment variables, package installation, network egress, and process privileges. Separate proposing a command from approving and executing it.

Use harmless canaries to test whether untrusted repository files, web content, or issue descriptions influence executable output. Never perform destructive tests against an environment that cannot be restored.

13. Log the complete decision-to-action chain

Security teams need to reconstruct the user request, relevant context, retrieved sources, selected tool, arguments, policy decision, result, and final response. Logs should include timestamps, actor identity, agent version, model, prompt/configuration version, server origin, and approval events.

Protect logs as sensitive data. Apply access control, retention, redaction, and tamper resistance. Ensure operational telemetry does not become an uncontrolled copy of customer information.

14. Test multi-turn and cross-layer behaviour

Single prompts are useful for baseline coverage, but many failures require several turns. An attacker may establish false authority, gather information, introduce a target, and only then ask for an action. Test whether boundaries hold as context accumulates.

Combine layers carefully: retrieved content influencing a tool, a tool result entering memory, or one agent delegating to another. Define success criteria before each run and preserve transcripts. AI red teaming should focus on observable impact, not merely creative jailbreak text.

15. Control production testing

Begin in a representative staging environment. For live testing, require written authorisation, an approved target list, time window, rate limits, prohibited actions, harmless test data, kill switch, monitoring, and escalation contacts.

Use lower-intensity checks for recurring validation and reserve deeper campaigns for reviewed engagements. Confirm whether third-party providers permit the planned testing. Production safety is part of the test design, not a disclaimer added afterwards.

16. Make findings reproducible

A finding should include the expected boundary, attack sequence, observed behaviour, relevant tool calls, evidence, impact, confidence, and remediation owner. Separate confirmed behaviour from likely or inconclusive outcomes.

Evaluator or judge scoring can support triage, but consequential findings should be reviewable by an analyst. Preserve enough context for developers to reproduce the issue without rerunning unsafe actions.

17. Retest after every meaningful change

Agent behaviour can change when the model, system prompt, retrieval corpus, memory policy, tool description, permission, orchestration logic, or provider changes. Maintain regression cases for confirmed findings and critical boundaries.

Retesting should verify the actual remediation and detect displaced risk. A prompt change that blocks one phrase may leave the underlying tool permission untouched. A tool restriction may fix the action while sensitive data still appears in the response.

18. Prepare an agent-specific incident plan

Define how to disable an agent, revoke credentials, isolate an MCP server, stop a workflow, preserve evidence, identify affected users, and remove poisoned memory or indexed content. Assign decision-makers before an incident.

Practise scenarios such as leaked transcript data, compromised tool metadata, cross-tenant retrieval, rogue credentials, and repeated unsafe actions. Traditional incident response still applies, but responders need agent-specific telemetry and containment mechanisms.

19. Review governance and vendor dependencies

Track model providers, hosting regions, subprocessors, tool vendors, open-source dependencies, and externally managed MCP servers. Review contracts and data-use settings. Monitor changes that alter retention, training use, availability, or security controls.

Framework mapping can support governance, but do not confuse a mapping with compliance or proof of effective control. The strongest assurance is evidence that the deployed system behaved correctly under relevant tests.

20. Set a release gate based on impact

Before production, require owners to accept residual risk and verify that critical boundaries have passed testing. The release decision should consider the agent's maximum reachable consequence, not its friendly interface.

A read-only public FAQ bot and an agent that can change payment details do not need identical controls. Scale testing depth, approval, monitoring, and evidence to the potential impact.

A practical way to use this checklist

Do not mark every item “complete” in a spreadsheet and move on. Select one production workflow, draw its data and authority paths, identify five unacceptable outcomes, and build tests around them. Preserve what happened, fix the underlying boundary, and retest.

Oxyne's agentic AI security validation platform is designed to assess behaviour visible through supported chat, voice, API, RAG, agent, and MCP interfaces. It does not replace architecture review, source-code review, cloud posture management, or enterprise governance. Those disciplines work together.

The practical standard is straightforward: know what the agent can reach, assume its context can be manipulated, keep authority outside natural-language reasoning, make consequential actions reviewable, and test the complete path repeatedly as the system changes.

Evidence to collect before production approval

A release review becomes much easier when evidence is assembled during engineering rather than requested at the end. Keep the current architecture and data-flow diagram, agent and tool inventory, identity map, data classification, provider settings, threat model, test scope, test results, accepted risks, and incident owner together.

For critical boundaries, preserve direct evidence: a cross-tenant test showing correct denial, a tool log showing server-side identity binding, an approval record tied to exact parameters, a memory-isolation test across users, and a retest showing that a previously confirmed failure no longer reproduces.

Record limitations. If testing was black-box, say so. If voice, asynchronous tasks, a private connector, or a production-only tool was outside scope, do not let the report imply otherwise. Honest scope makes the assurance useful to procurement and risk teams.

Release questions for the accountable owner

Before signing off, the owner should be able to answer:

• What is the highest-impact action this agent can cause?
• Which untrusted sources can influence its context?
• Which controls remain effective if the model follows a malicious instruction?
• Which identity authorises every sensitive data read and write?
• What exactly does a human see before approving an action?
• How quickly can the organisation stop the agent and revoke access?
• Which changes trigger mandatory retesting?
• Who owns remediation and residual risk?

An unanswered question does not always block launch, but it should create a documented decision rather than an invisible assumption.

First-week production monitoring

Increase review during initial deployment. Monitor unusual tool sequences, policy denials, new recipients, high-volume retrieval, repeated retries, memory writes, permission failures, cost spikes, and user reports. Compare actual behaviour with the assumptions in the threat model.

Keep rollout bounded by users, tenants, tools, or transaction limits. Gradual exposure reduces blast radius and gives the team evidence before expanding authority. If the agent's purpose can be achieved read-only at first, delay write capability until the read path is understood.

Production monitoring does not replace pre-release testing, but it can reveal workflows and context that staging did not reproduce. Feed those observations back into the regression suite and update this checklist as the system gains capability.