On May 30, 2026, InfoQ published Arm Open-Sources Metis, an AI Security Framework Outperforming Traditional SAST Tools. Arm open-sourced an AI agent-based security framework named Metis under the Apache 2.0 license. Compared to conventional rule-based SAST tools like Semgrep, CodeQL, and SonarQube, Metis reportedly achieved a 62% reduction in false positives and a 37% increase in true positives on the OWASP Benchmark. On the same day, Hacker News highlighted that EY Canada published a cybersecurity report and most citations were hallucinated, accelerating the widespread realization that “if you use AI, mechanisms to verify the AI's own output are essential.”
From the perspective of supporting application security (AppSec) for mid-sized enterprises through custom development, this means engagements that previously "ended with installing a rule-based SAST tool" have evolved into a phase of "deploying agentic AppSec and providing managed operations, including absorbing false positives and continuous verification." Connecting this with the AI cost structure demonstrated in $0.50/Month Security Diagnostics with AI (GH Media), the infrastructure audit in Custom Audit of an 18-Year-Old Unpatched Nginx Vulnerability (GH Media), and the CI/CD governance in Continuous Audit of GitHub Actions Supply Chains (GH Media), we organize "agentic AppSec audits" as a custom development package.
Why “agent-based AppSec is a turning point”
| Dimension | Rule-based SAST (Semgrep / CodeQL) | Agent-based AppSec (Metis) |
|---|---|---|
| Detection method | Pattern matching + data flow analysis | Code context comprehension + attack scenario generation |
| False positive rate | High (30–60%) | Low (10–20%) |
| True positive rate | Medium (primarily known vulnerabilities) | High (detects novel and unknown logic as well) |
| Language support | Rule reconstruction required for each language | Multilingual support via LLM generalization |
| Business logic vulnerabilities | Weak | Strong (definable via prompts) |
| Remediation suggestions | Links to boilerplate templates | Automated PR review comment generation |
| Cost | OSS or annual subscription | LLM token billing (variable) |
| Audit Logging | Detection results only | Prompt + reasoning path |
In short, agentic AppSec integrates "everything from detection to remediation proposals into a single flow," thereby structurally reducing manual review labor for security teams.
Three structural changes beneficial to custom development projects
Structure 1: From “ending with SAST installation” to “incorporating false-positive absorption into operations”
Mid-market companies have often felt content simply integrating Semgrep or SonarQube into CI, but in reality, 200 to 500 false positives a month lead developers to ignore alerts. In our custom client engagements, we handle end-to-end operational management: deploying agent-based AppSec such as Metis, triaging false positives weekly, and automatically filing true positives into Jira or Linear. This represents the AppSec edition of the architecture shown in $0.50/Month Security Audits with AI (GH Media) that cuts costs to 1/100 using AI.
Structure 2: From “annual vulnerability assessments” to “continuous PR-level verification”
Traditional expensive annual external assessments suffered from the fatal flaw of having a 6- to 12-month lead time between merge and detection. Agent-based AppSec can verify every PR in 30 seconds, determine severity, and return remediation proposals. This is the application code counterpart to the continuous auditing covered in Continuous GitHub Actions Supply Chain Audits (GH Media).
Structure 3: From “security team gatekeeping” to “developer self-service”
Through our client engagements, we offer a model where Metis performs an initial review when a developer opens a PR, allowing the security team to manually inspect only High-severity issues. This realistically enables a 60% reduction in security team workload and an 85% pre-merge detection rate. This is the application-layer equivalent of the infrastructure audits discussed in Client Audits for Vulnerabilities Left Unaddressed in Nginx for 18 Years (GH Media).
The 5 phases of “agent-based AppSec audits” delivered for clients
Phase 1: Current state assessment (2 weeks)
- Inventory of existing SAST / DAST / SCA tools
- Vulnerability and false positive statistics from the past 6 months
- Investigation of development flows (PR / merge / release)
- Security team workload analysis
- Risk scoring + prioritization mapping
Phase 2: Tool selection + design (2 weeks)
- Metis vs. CodeQL vs. Semgrep AI comparison
- LLM vendor selection (OpenAI / Anthropic / Bedrock / in-house)
- Processing boundaries by data sensitivity (OSS / proprietary / isolated environments)
- Detection policy + severity matrix
- False positive triage flow
- KPI design (MTTD / MTTR / detection rate)
Phase 3: Implementation (3–4 weeks)
- CI/CD integration (GitHub Actions / GitLab CI / CircleCI)
- PR review bot implementation
- Automated Jira / Linear ticket creation
- Security dashboards (Grafana / Datadog)
- LLM prompts + evaluation sets
- Incident response runbook
Phase 4: Pilot rollout (3 weeks)
- Commencing operations on 1–2 repositories
- Developer briefing session (30 minutes)
- Design of weekly triage meetings
- Feedback loop for false positives
- KPI measurement + refinement
Phase 5: Monthly operational reviews (ongoing)
- Detection / false positive / remediation completion statistics
- Prompt additions for novel vulnerabilities
- Evaluation of LLM vendor transitions
- Tracking security team workload trends
- Semi-annual threat model updates
Standard technology stack set for custom development
| Layer | Recommended technology | Alternative |
|---|---|---|
| Agent-based SAST | Metis（Arm OSS） | CodeQL AI / Semgrep AI / Snyk Code |
| LLM | Claude Opus 4.X / GPT-5.5 / Bedrock | DeepSeek / Llama 4 |
| CI/CD | GitHub Actions / GitLab CI | CircleCI / Jenkins |
| Tickets | Jira / Linear / GitHub Issues | Notion |
| Dashboard | Grafana / Datadog | New Relic |
| SIEM | Microsoft Sentinel / Splunk | Sumo Logic |
| SCA | Dependabot / Renovate / Snyk | Mend |
| DAST | OWASP ZAP / Burp Enterprise | Acunetix |
Which projects need this and which do not
| Projects requiring this | Projects not requiring this |
|---|---|
| In-house developed applications exist | Off-the-shelf packages only |
| Monthly PR volume of 50 or more | Primarily static websites |
| Audit requirements (ISO 27001 / SOC 2 / PCI DSS) | Not subject to auditing |
| Fatigued by false positives in existing SAST | Operating with zero alerts |
| High OSS dependency ratio | Proprietary closed-source only |
Six clauses to include in client contracts
| Clause | Details | What the client should verify |
|---|---|---|
| Target repositories | Repository names + branch policies | Handling of out-of-scope PRs |
| Detection policy | OWASP Top 10 + internal regulations + industry standards | Business-specific logic |
| Severity assessment | Critical / High / Medium / Low | Escalation thresholds |
| Data handling | Code destination LLM + retention period | Trade secret protection |
| Handover Upon Project Completion | Prompts / evaluation sets / rules | Internal operational continuity |
| Incident operations | 24h / business hours / immediate blocking | SLA |
Client ROI projection (assuming 15 repositories / 800 monthly PRs)
| Item | Existing (rule-based SAST) | After agent-based AppSec implementation | Difference |
|---|---|---|---|
| Monthly false positives | 400 | 100 | -300 |
| Triage workload (monthly) | 120 hours | 30 hours | -90 hours |
| Pre-merge detection rate | 30% | 78% | +48pt |
| Critical incidents (annual) | 2–3 incidents | 0–1 incident | -2 incidents |
| External assessment costs (annual) | 8 million yen | 2M JPY | -¥6 million |
| Annual benefit | — | — | Equivalent to approximately 15 million yen + substantial reduction in post-merge incidents |
Calculated at an hourly rate of 8,000 yen, this yields an annual labor reduction of 8.6 million yen + 6.0 million yen in outsourced assessment savings. Comparing deployment costs (LLM inference costs + setup + operations) against these savings serves as a good benchmark to determine whether the payback period is under 12 months. Since the required effort varies significantly with repository count and PR volume, please adjust projections using your organization's actual metrics.
Five common pitfalls
Pitfall 1: Assuming it is free because it is open source
Even if Metis itself is open source, LLM inference costs and triage labor are incurred separately. Establish monthly budget caps and KPIs from day one.
Pitfall 2: Failing to segment code transmission destinations by sensitivity
Routing all code to cloud LLMs is unacceptable for repositories containing trade secrets or customer data. Route code to Bedrock, Vertex, or isolated LLMs based on sensitivity.
Pitfall 3: Completely replacing existing SAST
The known vulnerability detection of Semgrep and CodeQL remains powerful. Deploy agent-based tools alongside existing SAST as a complement, then transition gradually.
Pitfall 4: Placing blind trust in remediation suggestions
Around 20% to 30% of remediation patches generated by LLMs contain errors. Always pass them through human review and automated testing.
Pitfall 5: Omitting audit logging
If you do not store prompts and responses at detection time, tracing the causes of false positives and improving accuracy becomes impossible. Operate with encryption and time-limited retention.
90-day action plan
| Week | Action |
|---|---|
| Week 1〜2 | Inventory + past vulnerability statistics + risk mapping |
| Week 3〜4 | Tool selection + detection policy + KPI design |
| Week 5〜8 | CI/CD integration + PR bot + dashboards |
| Week 9〜11 | Pilot repository rollout + developer training |
| Week 12 | Company-wide rollout + runbook preparation |
| Week 13 | First monthly review + ROI measurement |
Conclusion — Evolving enterprise security: From “rule-based SAST” to “agent-based AppSec”
The open-sourcing of Arm Metis demonstrates that “the center of gravity in AppSec has shifted from rule-based tools to AI agents.” For providers supporting mid-market enterprise security through custom engagements, “agent-based AppSec audits”—which integrate tool deployment, false-positive absorption, continuous verification, and developer training—represent the new core service offering.
We provide tailored quotes based on repository scale and audit requirements for exploring agent-based AppSec adoption, parallel operation with existing SAST, or false-positive triage design. Please feel free to reach out via the contact form if you face challenges such as “developers ignore SAST alerts because of alert fatigue,” “vulnerabilities continue to be discovered after merging,” or “annual external assessments have far too long a lead time.”
Sources
- Arm Open-Sources Metis, an AI Security Framework Outperforming Traditional SAST Tools（InfoQ 2026-05-30）
- EY Canada published a cybersecurity report and most citations were hallucinated（Hacker News 2026-05-30）
- $0.50/Month Security Audits with AI (GH Media)
- Client Audits for Vulnerabilities Left Unaddressed in Nginx for 18 Years (GH Media)
- Continuous GitHub Actions Supply Chain Audits (GH Media)
