The Hidden Trust Problem in AI-Generated Documentation

Source: Dev.to

The Hidden Trust Problem in AI‑Generated Documentation

The first time AI generated documentation for my project, it looked perfect: clear structure, confident tone, professional language.
That was exactly the problem.

A week later, when I tried to review it, I couldn’t answer a basic question:

Which parts of this document came from my requirements, and which parts did the AI make up?

Everything was written with equal confidence. There was no way to tell where I should trust the content—and where I needed to verify it.

When AI creates documentation, it doesn’t distinguish between:

• facts you explicitly provided
• information inferred from existing documents
• assumptions made to fill gaps
• general industry conventions

All of them look the same on the page. At first, that feels convenient. Later, it becomes dangerous because you can no longer tell what is actually true versus what merely sounds reasonable.

Tagging Every Statement with Its Source

The fix is simple in concept, but powerful in practice: require AI to tag every statement with its source. Each claim must declare where it came from.

Tag Definitions

Example Rewrite

[explicit] The API uses REST architecture with JSON responses.
[inferred] Authentication requires Bearer tokens.
   └─ "All endpoints require authentication" (REQUIREMENTS.md L.23)
[assumed] Rate limiting is set to 100 requests per minute.
[general] Error responses follow RFC 7807 format.

Now the review effort is obvious: I know exactly where to focus. The [inferred] tag turned out to be the most dangerous one because AI is very good at post‑hoc rationalization—it can reach a conclusion first, then search for text that sounds supportive.

Enforcing Verifiable Inferences

Rule: every [inferred] statement must include a verbatim quotation from its source.

[inferred] Retry policy allows 3 attempts
   └─ "External API calls should retry up to 3 times" (API_DESIGN.md L.28)

If the quote doesn’t support the conclusion, the problem is immediately visible. Without the quote, I’d have to hunt through documents myself; with it, verification takes seconds.

When Tags Are Required

Not every document needs tags. The rule is simple: tag documents that others will rely on as truth.

Source code already has a verification mechanism (tests). Documentation doesn’t. Source tags provide the missing verification metadata.

Handling Terminology Drift

Source attribution alone breaks down when terminology drifts.

# API_DESIGN.md
[explicit] Retry policy allows 3 attempts

# SERVICE_SPEC.md
[inferred] Re‑execution strategy permits 3 tries

# TEST_SPEC.md
[inferred] Fault recovery mechanism uses 3 retries

Three different names for the same concept make greps and impact analyses fail, eroding trust.

Conversational Unification

Instead of a static glossary, unify terminology during the AI‑human conversation:

Human: “The re‑execution count should increase to 5.”
AI: “You mentioned ‘re‑execution’. Is this the same as ‘retry policy (#retry‑policy)’ in existing docs?”
Human: “Yes.”
AI: “Understood. Updating retry policy to 5 attempts.”

Drift is caught at the point of entry, not after the damage spreads. This works well for single‑user workflows but breaks with multiple users, where parallel truths can emerge. Solving that requires shared infrastructure: synchronized glossaries, versioned terminology, or serialized workflows—a different class of problem.

Practical Guidance

• New projects: involve AI from day one; tags and unified terminology stay clean.
• Legacy systems: use question‑driven integration, then enforce tagging rules moving forward.
• Boundaries: new work follows the protocol; legacy stays untouched until it’s modified.

Source attribution doesn’t make AI perfect—it doesn’t prevent mistakes. What it does is make mistakes visible. When you can see where AI was certain versus where it guessed, you know where to apply human judgment. That visibility is the foundation of trust in AI‑collaborative development.

---

This article is part of the Beyond Prompt Engineering series, exploring systematic—not accidental—ways to work with AI.