c5.10 Adversarially test and review agents
Continuously test agents against prompt attacks and review their design and code, so weaknesses are found before attackers exploit them.
Implementation guidance
How to implement on AWS
Use AWS Security Agent to review architecture and pull requests and to run on-demand penetration-testing scenarios. Use AgentCore Evaluations to assess agent quality online and on-demand, and run adversarial/jailbreak test suites before and after deployment.
Acceptance criteria
Agents undergo adversarial testing and security review before release; AgentCore Evaluations run on an ongoing basis; findings are remediated.
Practical implementation
How this control is implemented in each reference architecture:
Agents are adversarially tested and security-reviewed in a non-production account of the OU before promotion, with AgentCore Evaluations running continuously in the Generative AI account.
- Use AWS Security Agent to review architecture and pull requests and to run on-demand penetration-testing scenarios in a development account of the OU.
- Run AgentCore Evaluations plus adversarial/jailbreak test suites before promotion, and record results as the gate to production.
- Pair every negative test (for example an unapproved model, an unapproved tool, or another tenant's identity) with a positive twin in the same run that proves the call would otherwise succeed. Require each denial to name the expected control (policy, guardrail, or IAM), so a test cannot pass for the wrong reason.
- Continue AgentCore Evaluations online in production and remediate findings.Generative AI account AgentCore Evaluations
CloudWatch
How to validate: Agents undergo adversarial testing and security review before release; AgentCore Evaluations run on an ongoing basis; findings are remediated. Every negative test has a positive twin in the same run, and each denial names the expected control.
Avoid: Shipping an agent after a happy-path demo. Agents face adversarial input; test with jailbreak suites and review before and after release.
Adversarial testing, security review, and continuous AgentCore Evaluations run against a pre-production configuration in the account before agents serve production traffic.
- Use AWS Security Agent to review architecture/PRs and run on-demand pen-test scenarios.
- Run AgentCore Evaluations and adversarial/jailbreak suites before promoting to production use.
- Pair every negative test (for example an unapproved model, an unapproved tool, or another tenant's identity) with a positive twin in the same run that proves the call would otherwise succeed. Require each denial to name the expected control (policy, guardrail, or IAM), so a test cannot pass for the wrong reason.
- Keep AgentCore Evaluations running online and remediate findings.In-account logging AgentCore Evaluations CloudWatch
How to validate: Agents undergo adversarial testing and review before release; AgentCore Evaluations run continuously; findings are remediated. Every negative test has a positive twin in the same run, and each denial names the expected control.
Avoid: Testing agents only on expected prompts. Run adversarial suites and keep evaluating after launch.
Threats addressed
- Undiscovered prompt-injection or reasoning-manipulation weaknesses.
- Security regressions introduced as agents evolve.
AWS services
Framework mappings
OWASP GenAI/LLM Top 10 (2026)
- LLM01 Prompt Injection
- LLM03 Excessive Agency
NIST AI Risk Management Framework (AI 100-1)
- MEASURE.1 Appropriate methods and metrics are identified and applied
- MEASURE.2 AI systems are evaluated for trustworthy characteristics
- MEASURE.4 Feedback about efficacy of measurement is gathered
AWS Well-Architected Generative AI Lens (Security)
- GENSEC05-BP01 Implement least privilege and permission boundaries for agentic workflows
OWASP GenAI Top 10 for Agentic Applications (2025)
- ASI10 Rogue Agents
- ASI04 Agentic Supply Chain
Learning resources
References
- AWS SRA for AI — Capability 5, Application security