I Built a Multi-Agent AI Pen Tester Because AI Coding Tools Are Shipping Vulnerable Code
Source: Dev.to
The Problem
AI coding assistants are everywhere. Developers are shipping code faster than ever using Claude, Copilot, and Cursor—but they’re also shipping:
- SQL injection
- Hard‑coded secrets
- Broken authentication
- XSS
AI tools optimise for working code, not secure code. Typical output:
- A login form that works but is trivially bypassable with
‘ OR 1=1--. - An API key hard‑coded because it’s the fastest way to make a demo work.
- No input validation unless you explicitly ask for it.
Most solo developers and small teams never hire a penetration tester. A basic pen‑test costs $500–$2,000 and can take weeks to schedule, so the vulnerabilities just ship.
Introducing VulnSwarm
VulnSwarm deploys a swarm of specialised AI agents that mirror a real penetration‑testing team. Instead of one monolithic model, each agent has a distinct role:
| Agent | Role |
|---|---|
| 🔭 Recon Agent | Maps the attack surface, identifies entry points, fingerprints the tech stack, flags the highest‑risk areas. |
| 💥 Exploit Agent | Takes the recon data and determines what’s actually exploitable. Rates each finding by severity, exploitability, and impact (CVSS‑like scores). |
| 🗡️ Red Team Agent | Thinks like an attacker. Chains vulnerabilities into realistic attack paths and finds worst‑case scenarios. |
| 🛡️ Blue Team Agent | The defender. Writes specific, code‑level fixes and prioritises them by effort vs. impact. |
| 📄 Report Agent | Synthesises everything into a professional penetration‑testing report with an overall risk score, severity breakdown, and remediation roadmap. |
The agents debate each other: the Red Team challenges the Exploit analysis, the Blue Team pushes back on severity ratings, and the Report Agent consolidates the nuanced output.
Testing on OWASP Juice Shop
I pointed VulnSwarm at the deliberately vulnerable OWASP Juice Shop app.
Manual test (≈30 seconds)
- Logged in as admin using
' OR 1=1--in the email field. - Accessed the admin panel at
/administration. - Retrieved 21 user email addresses.
- Found an exposed crypto‑wallet seed phrase in customer feedback.
VulnSwarm automatic findings (≈15 minutes on a CPU‑only VPS, llama3.2:3b)
| Severity | Finding | CVSS |
|---|---|---|
| 🔴 Critical | File Upload Endpoints – exploitable to inject malicious code or exfiltrate data. | 9.0 |
| 🔴 Critical | Unvalidated API Endpoints – lack input validation and sanitisation. | 9.0 |
| 🟠 Medium | Missing Content‑Security‑Policy | 5.3 |
| 🟠 Medium | Missing Strict‑Transport‑Security | 5.3 |
| 🟠 Medium | Missing X‑XSS‑Protection | 5.3 |
| 🟠 Medium | Missing Referrer‑Policy | 5.3 |
| 🟠 Medium | Missing Permissions‑Policy | 5.3 |
Larger models (e.g., qwen2.5:14b or Claude) also catch the SQL injection I found manually.
How the Multi‑Agent Architecture Works
Security analysis benefits from multiple perspectives arguing with each other, just like a real security team.
Your Code/App
│
▼
┌──────────┐ ┌───────────┐ ┌──────────┐ ┌─────────┐
│ Recon │───▶│ Exploit │───▶│ Red Team │───▶│ Blue │
│ Agent │ │ Agent │ │ Agent │ │ Team │
└──────────┘ └───────────┘ └──────────┘ └────┬────┘
│
▼
┌──────────┐
│ Report │
│ Agent │
└──────────┘- Recon only sees the raw code/app.
- Exploit only sees recon data (no fixes).
- Red Team only sees exploit findings (no fixes).
- Blue Team only sees attack paths (no recon).
- Report sees everything and produces a developer‑friendly document.
Running VulnSwarm Yourself
VulnSwarm supports Claude, GPT‑4o, Gemini, OpenRouter, and Ollama. To run it free and locally:
# Clone the repo
git clone https://github.com/aaronsood/VulnSwarm.git
cd VulnSwarm
# Install Python dependencies
pip install -r requirements.txtPull a local model (optional)
ollama pull llama3.2:3bStart the scanner
python -m cli.mainSpin up a test target (OWASP Juice Shop)
docker run --rm -p 3000:3000 bkimminich/juice-shopThen point VulnSwarm at http://localhost:3000/.
Web scanning is localhost‑only by default—VulnSwarm won’t touch anything you don’t own.
What It Doesn’t Do (Yet)
- It’s a first pass, not a replacement for a professional security team.
- It won’t discover zero‑days or novel attack chains that require deep business‑logic understanding.
- Smaller models miss findings that larger models catch.
- No CI/CD or GitHub Actions integration (roadmap item).
The roadmap includes deeper model support, CI/CD plugins, and richer reporting.
The Bigger Picture
Using AI to find vulnerabilities is poetic: we let machines that can write code also help us write safer code. VulnSwarm tackles the most common pain point—the 99 % of developers who ship without any security review and have no budget for it—by giving them an affordable, automated “first‑line of defence”.
VulnSwarm
The challenges and responsibilities that AI introduced.
As AI coding tools become the default way software gets written, AI security tooling needs to keep pace.
About the project
- Open source – MIT licensed
- Early stage – contributions are very welcome, especially from those in security or AI tooling
Repository
Built and tested on a Saturday with a CPU‑only VPS, a deliberately hackable web app, and too much coffee.