AI Penetration Testing with AWS Security Agent — Automating Cloud Audits for Custom SaaS 2026

In early May 2026, "Trying Out AI-Powered Reviews and Penetration Testing with AWS Security Agent" ranked on Zenn Trending. Released by AWS, Security Agent is a tool that automates areas previously handled manually by using LLMs to review configurations across major services like IAM, VPC, S3, and RDS, extending even into behaviors equivalent to penetration testing.

In our custom development, many projects follow the flow of building SaaS on AWS → handing off to the client → client operates it, where post-handoff cloud security audits tended to fall into a binary choice: "expensive annual outsourcing vs. doing nothing at all." In this article, we outline an architecture for integrating AWS Security Agent into continuous audits for custom SaaS.

Why AWS cloud security audits were a "blind spot in custom development"

In custom development projects on AWS, common pitfalls discovered after handoff include the following:

These represent "audits on the cloud infrastructure side," pairing with the "source code-side audits" covered in Lessons from the Money Forward GitHub Incident: Source Code Leak Audits You Must Conduct in Custom Development. Defense for client SaaS remains incomplete unless both are in place.

What AWS Security Agent can do (scope relevant to custom development)

We extract the features essential for custom development from the original Zenn article and official AWS documentation.

In particular, "monthly reports on configuration drift" is a feature directly tied to turning operational custom development into subscriptions. Reports that previously could only be obtained through an "annual manual assessment" can now create recurring monthly value in the form of monthly automated generation + interpretation by the custom development team.

Scope design for client SaaS: How far should we rely on AI?

When utilizing AWS Security Agent in custom development, a classic pitfall is misdesigning the scope, causing operations to stall after being buried under a mountain of false positives. Here is a standard scope for stable operation in custom development.

The key is to explicitly maintain a "handled by humans" lane. Even if an AI security agent generates findings with high accuracy, business context, contractual perspectives, and regulatory compliance cannot simply be offloaded to AI. For custom development providers, interpreting AI output and explaining it to clients constitutes the core value.

Absorbing false positives: Mechanisms to implement in custom development

In LLM-based security assessments, absorbing false positives is the crux of operations. The mechanism implemented in custom development consists of the following three layers:

[Security Agent 検出]
   ↓
[1段目: 既知例外DB との突合]   ← 過去に "問題なし" と判断した検出は自動抑制
   ↓
[2段目: 重大度の自動再採点]    ← 業務影響度 + 攻撃成功確率で再採点
   ↓
[3段目: 受託側エンジニアのトリアージ] ← Slack 通知 → 30 分 SLA でトリアージ
   ↓
[顧客報告]                     ← 月次レポートに残す

The key lies in running a "known exceptions DB." Once a finding is judged as "acceptable for business reasons," without a mechanism to automatically suppress it upon recurrence, identical notifications will flood in monthly, triggering alert fatigue. In operational contracts for custom development, maintaining the exception ledger serves as an unglamorous yet critically important task.

This corresponds to the cloud infrastructure layer of the "application- and code-layer AI diagnostics" discussed in AI Security Diagnostics at $0.50/Month: Packaging Architecture and Pricing Strategy for Custom Development. Combining both enables designing both "code and cloud" aspects of client SaaS as a continuous security operation.

Operating penetration-testing equivalents in custom development

While the penetration-testing equivalent feature of AWS Security Agent cannot replace a real penetration test, it proves extraordinarily valuable as "pre-screening + monthly health checks."

In custom development field operations, a two-tiered approach of "AI during routine times, human professionals annually and at major milestones" offers a pragmatic solution. Explaining to clients that an AI security agent alone can cover everything risks an incident later when they discover that a "real manual penetration test was necessary."

Designing monthly operational contracts: Deciding the extent of human involvement

Here are the design axes when structuring monthly operational contracts in custom development powered by AWS Security Agent.

Moving further to the left means operations where the client receives raw AI output as-is, whereas moving to the right layers on engineering effort from the custom development side. Where to position this depends on the scale of the client SaaS and whether the client has personnel available to handle alerts.

Furthermore, carving out an isolated one-off "initial audit only" should be avoided. Without subsequent remediation and ongoing monthly operations, a single-use report merely gathers dust on a shelf without contributing to client security.

Delivery checklist — 12 immediately usable items for custom development

• AWS Security Agent is running in the client account
• Scope (covered targets / out of scope) is documented
• Known exceptions DB is initialized (past decisions handed over)
• Slack notification destinations / severity-based routing are configured
• Monthly report template is approved by client
• Enablement of CloudTrail and GuardDuty is confirmed
• A round of IAM least-privilege tightening has been completed
• Encryption for S3, RDS, and DynamoDB is enabled across all resources
• Workflow for Terraform / CloudFormation diff remediation is operational
• Client has agreed that "Security Agent alone does not replace penetration testing"
• Budget and arrangements for annual manual penetration testing are stated in SOW
• Emergency contact escalation path (24h or business hours) is documented

Conclusion: Cloud audits transition from expensive annual outsourcing to AI monthly subscriptions + human milestones

AWS Security Agent is a tool that fundamentally transforms the economics of cloud audits for client SaaS. Even for small-to-medium SaaS platforms that previously had no choice but to rely on expensive annual outsourcing, running health checks whenever configurations change has become a realistic operational choice.

However, as detailed in this article, the operational weight of an audit shifts substantially depending on AWS account structure, how many exceptions have accumulated, and whether someone on the client side can process alerts. Even within the same "monthly audit," initial-month effort differs drastically between building an exception ledger from scratch versus inheriting an existing one. If you evaluate your AWS environment against the checklist above and find unfulfilled items, or feel vulnerable during the intervals between annual penetration tests, please reach out via our contact form with details of your active account setup. We will assess how much can be shifted to AI before tailoring your operational model.

Sources

• Testing AI-Powered Reviews and Penetration Testing with AWS Security Agent (Zenn)
• AWS Security Agent Official Documentation
• Securing Autonomous AI Agents on Kubernetes（InfoQ）