How I Added AI Code Quality Checks to My CI Pipeline in 5 Minutes (And Found 47 Bugs on Day One)

Source: Dev.to

The Problem

My team started using AI coding assistants (Copilot, Cursor, Claude Code) about six months ago. Velocity went up, but a new category of bugs appeared:

// Bug 1: Hallucinated import — package doesn't exist
import { validateEmail } from 'email-validator-pro';  // ❌ Doesn't exist on npm

// Bug 2: Deprecated API — worked fine in 2020
const parsed = url.parse(request.url);  // ⚠️ Deprecated since Node 15

// Bug 3: Security anti‑pattern
const query = `SELECT * FROM users WHERE id = ${userId}`;  // 🔴 SQL injection

These bugs slip past ESLint, Prettier, and even TypeScript. They compile fine but break at runtime. I needed a quality gate that understood AI‑generated code defects specifically.

The Solution: Open Code Review

Open Code Review (OCR) is a free, open‑source CLI that detects AI‑specific code defects:

Best part: L1 mode runs in under 10 seconds without any AI.

5‑Minute Setup

Step 1 – Install (≈ 30 s)

npm install -g @opencodereview/cli

Step 2 – Scan Locally (≈ 1 min)

ocr scan src/ --sla L1

Sample report

╔══════════════════════════════════════════════════════════════╗
║           Open Code Review V4 — Quality Report              ║
╚══════════════════════════════════════════════════════════════╝

  Overall Score: 72/100  🟠 C
  Files Scanned: 48  |  Duration: 6.3s

  🔴 [error] api/handler.ts:45   — Possible hardcoded API key
  🟡 [warn]  utils/request.ts:12 — url.parse() deprecated → WHATWG URL API
  🟡 [warn]  services/auth.ts:67 — Cyclomatic complexity 22 (max: 15)
  ⚪ [info]  types/index.ts:8    — Unused interface (context window artifact)

Step 3 – Add to GitHub Actions (≈ 2 min)

Create .github/workflows/ocr.yml:

name: AI Code Quality Check
on: [pull_request]

jobs:
  code-review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install OCR
        run: npm install -g @opencodereview/cli
      - name: Scan changed files
        uses: raye-deng/open-code-review@v1
        with:
          sla: L1
          threshold: 60
          scan-mode: diff
          github-token: ${{ secrets.GITHUB_TOKEN }}

Step 4 – Generate SARIF Report (≈ 1 min)

For GitHub Code Scanning integration:

      - name: Generate SARIF
        run: ocr scan src/ --sla L1 --format sarif -o ocr-results.sarif
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: ocr-results.sarif

Step 5 – Set Your Quality Threshold (≈ 30 s)

The threshold parameter defines the minimum quality score (0‑100). If the scan score falls below it, the CI check fails.

          threshold: 70  # Adjust based on your team's tolerance

That’s it. Your CI pipeline now catches AI‑generated code defects before they merge.

What I Found on Day One

Running OCR on our main codebase revealed the following:

Total: 47 issues that our existing toolchain (ESLint + TypeScript + SonarQube) completely missed.

L2 Mode – AI‑Powered Deep Analysis

If you need deeper analysis, OCR offers an L2 mode that uses a local AI (Ollama) for:

• Cross‑file coherence checking
• Semantic duplication detection
• AI confidence scoring

.ocrrc.yml example

sla: L2
ai:
  embedding:
    provider: ollama
    model: nomic-embed-text
    baseUrl: http://localhost:11434
  llm:
    provider: ollama
    model: qwen3-coder
    endpoint: http://localhost:11434

L2 takes a bit longer (~30 seconds for medium projects) but catches subtle issues that pattern‑matching alone can’t.

How It Compares

OCR complements your existing tools — it doesn’t replace them. Keep ESLint for style; add OCR for AI‑specific defects.

GitLab CI Integration

Not on GitHub? No problem:

code-review:
  script:
    - npx @opencodereview/cli scan src/ --sla L1 --threshold 60 --format json --output ocr-report.json
  artifacts:
    reports:
      codequality: ocr-report.json

Tips from Production Use

• Start with L1. It’s fast enough for every PR. Add L2 later for critical paths.
• Use scan-mode: diff in CI to scan only changed files — keeps PR checks fast.
• Don’t set the threshold too high initially. Begin with 50‑60, then raise it as the team gets accustomed.
• Commit the .ocrrc.yml file to version control so the same rules run locally and in CI.
• Combine with existing linters (ESLint, SonarQube) for a comprehensive safety net.

Enjoy a safer, AI‑aware development workflow! 🚀

Wrapping Up

AI coding assistants are incredible productivity boosters, but they also introduce a new category of bugs that traditional tools weren’t designed to catch.

Open Code Review fills that gap. It’s free, open‑source, runs locally, and takes only 5 minutes to set up.

Give it a try

npm install -g @opencodereview/cli
ocr scan src/ --sla L1

GitHub · Portal · NPM

What AI‑generated code bugs have you encountered? I’d love to hear about your experience in the comments.