My AI Side Project Would Fail an EU AI Act Audit — Here's How I Fixed It

Source: Dev.to

I’ve been building a small AI app — a LangChain‑powered tool that summarizes legal documents. Nothing fancy, just a side project I was tinkering with on weekends.

Then someone in a GitHub discussion asked me: “Is your app EU AI Act compliant?”
My honest answer was “I’m too small for that to matter.” Turns out, I was wrong.

The EU AI Act doesn’t care about your company size

The regulation that started applying in February 2025 (with full enforcement coming August 2026) covers any AI system made available in the EU market. That includes:

• Your SaaS with 10 users
• Your open‑source tool on GitHub
• Your internal tool if it processes EU‑citizen data

There is no “small developer exemption” for the core transparency requirements. If you’re using an AI model in production, you have obligations.

I know because I read the actual regulation text (all 144 pages — don’t recommend it on a Friday evening).

I scanned my own project

I pointed a compliance scanner at my own repo. Here’s what my project looked like:

├── app.py              # LangChain pipeline
├── requirements.txt    # langchain, openai
├── prompts/
│   └── summarizer.py   # System prompts
└── README.md           # "A tool that summarizes stuff"

The scan detected:

• LangChain framework (via requirements.txt)
• OpenAI API usage (via imports in app.py)
  • Risk category: Limited (text‑generation system)

And here’s what I was missing — three things that took me a total of 35 minutes to fix.

1. No transparency disclosure

Article 50 of the EU AI Act requires that users know when they’re interacting with AI‑generated content. My app returned summaries with zero indication they came from a machine.

Before

def summarize(document: str) -> str:
    return chain.invoke({"document": document})

After

from datetime import datetime

def summarize(document: str) -> dict:
    result = chain.invoke({"document": document})
    return {
        "summary": result,
        "ai_disclosure": (
            "This summary was generated by an AI system "
            "(OpenAI GPT‑4 via LangChain)."
        ),
        "model": "gpt-4",
        "generated_at": datetime.utcnow().isoformat(),
    }

Time spent: 5 minutes. The API response now tells consumers exactly what generated the content.

2. No technical documentation

Even for limited‑risk systems, having documentation is the difference between “we take compliance seriously” and “we’ll figure it out when the auditor shows up.”

I had a three‑line README. Here’s what I added as AI_COMPLIANCE.md:

# AI Compliance Documentation

## System Overview
- **Purpose**: Summarize legal documents using LangChain + OpenAI GPT‑4.
- **Architecture**: Simple FastAPI endpoint that receives a document, passes it to a LangChain chain, and returns a JSON payload with the summary and disclosure metadata.

## Data Flow
1. User uploads a document (plain text or PDF).
2. The document is sent to the backend over HTTPS.
3. The backend calls the OpenAI API.
4. The response is wrapped with disclosure fields and returned to the user.

## Risk Assessment
- **Risk level**: Limited (text generation).
- **Mitigations**:
  - Transparency disclosure (see `app.py`).
  - No storage of raw user data beyond the request lifecycle.
  - Rate limiting to prevent abuse.

## Transparency Measures
- Every response includes `ai_disclosure`, `model`, and `generated_at` fields.
- UI displays a banner: “This summary was generated by AI.”

## Maintenance & Updates
- Dependencies are pinned in `requirements.txt`.
- Security patches are applied within 7 days of release.
- Compliance scan is run weekly via CI.

## Contact
- **Developer**: Arkforge ([@arkforge‑ceo](https://dev.to/arkforge-ceo))
- **Email**: security@example.com

3. (Placeholder for the third fix)

You can continue adding the remaining two items in the same structured format.

AI System Documentation

• Purpose: Summarize legal documents for quick review
• Model: OpenAI GPT‑4 via LangChain
• Training data: None (uses pre‑trained model via API)
• Risk category: Limited (AI‑generated text, Article 50)
• Transparency: All outputs include AI disclosure
• Limitations: May miss nuances in complex legal language. Not suitable for legal advice.
• Human oversight: Summaries are review assistance only
• Data retention: No user data stored beyond session

Auditing Input/Output

import logging
import uuid

logger = logging.getLogger("ai_audit")

def summarize_with_audit(document: str) -> dict:
    request_id = uuid.uuid4().hex[:8]
    logger.info(f"[{request_id}] Input length: {len(document)} chars")

    result = chain.invoke({"document": document})

    logger.info(f"[{request_id}] Output length: {len(result)} chars")
    return {
        "summary": result,
        "request_id": request_id,
        "ai_disclosure": "Generated by AI (OpenAI GPT‑4)"
    }

Time spent: 10 minutes – now every request has a trace.

What Surprised Me

• The fixes were easy – total effort ~35 minutes.
• The hard part was awareness: knowing what needed to be done.
• My project turned out to be limited‑risk, not high‑risk. Most developer tools fall into this category, with obligations mainly around transparency and documentation.

Automated Compliance Check (GitHub Action)

# .github/workflows/ai-compliance.yml
name: EU AI Act Compliance Check

on: [push, pull_request]

jobs:
  compliance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Detect AI frameworks
        run: |
          echo "=== Scanning for AI frameworks ==="
          FOUND=$(grep -rl "import openai\|from langchain\|import anthropic\|import transformers" \
            --include="*.py" . 2>/dev/null | wc -l)
          echo "Files with AI imports: $FOUND"
          if [ "$FOUND" -gt 0 ]; then
            echo "AI frameworks detected — checking compliance..."
          fi

      - name: Verify transparency disclosure
        run: |
          DISCLOSURES=$(grep -rl "ai_disclosure\|AI-generated\|generated by AI" \
            --include="*.py" . 2>/dev/null | wc -l)
          if [ "$DISCLOSURES" -eq 0 ]; then
            echo "::warning::No AI transparency disclosure found in code"
          else
            echo "Transparency disclosures found in $DISCLOSURES files"
          fi

      - name: Check compliance documentation
        run: |
          if [ ! -f "AI_COMPLIANCE.md" ]; then
            echo "::warning::No AI_COMPLIANCE.md found"
            echo "Consider adding AI system documentation"
          else
            echo "AI compliance documentation found"
          fi

Time spent: 10 minutes – this catches the obvious compliance items.
For deeper scans (detecting 16 AI frameworks and mapping them to specific EU AI Act obligations), I use the free MCP compliance scanner.

Advice to My Past Self

• Add AI_COMPLIANCE.md on day 1 – takes ~20 minutes and forces you to think about the system you’re building.
• Tag every AI output – an ai_disclosure field in your response schema satisfies the easiest obligation.
• Know your risk category – most side projects are “limited” or “minimal,” meaning lighter obligations.
• Log request IDs – essential for both compliance and debugging LLM outputs.

The EU AI Act isn’t meant to kill indie projects; it’s about ensuring people know when AI is involved in decisions that affect them. With the enforcement deadline in August 2026, starting now prevents a scramble later.