I scanned 22 popular GitHub repos for security issues - 68% had exposed secrets

Source: Dev.to

Overview

I built a CLI tool called shinobi (available as shinobi-scan on PyPI) that performs a quick, 10‑second security scan on any project directory or GitHub repository. The scan checks for:

• Exposed API keys and secret patterns
• Dangerous defaults (e.g., debug mode, wildcard CORS)
• Vulnerable dependencies
• Missing security basics
• AI‑specific risks

The tool is written entirely in Python, uses argparse for the CLI, regular‑expression pattern matching for secret detection, gitpython for history scanning, and subprocess calls for dependency auditing. It runs 100 % locally, so no data leaves your machine.

pip install shinobi-scan

GitHub repository:

Methodology

I pointed shinobi at 22 popular open‑source projects, including:

• FastAPI
• Flask
• Dify
• Flowise
• LiteLLM
• Lobe‑Chat

Each repository was scanned for the categories listed above. The tool reports threat levels (low, medium, high, critical) based on the severity of the findings.

Findings

• 86 % of the scanned repositories returned high or critical threat levels.
• The most common issue was exposed secret patterns (API key formats found directly in source code).
• The second most frequent problem was dangerous defaults, such as leaving debug=True enabled or using wildcard CORS settings.

Even if many of the exposed keys belong to test fixtures, the presence of real‑looking key patterns across thousands of files indicates that developers who fork or build on these templates inherit significant security debt.

Conclusion

The scan demonstrates that a substantial portion of popular open‑source projects contain easily detectable security weaknesses. Running a local, automated scan like shinobi can help developers identify and remediate these issues before they propagate downstream.

Contact

Feel free to reach out with feedback or questions. I’m available on X (formerly Twitter) throughout the day.