Serve Markdown to AI Agents (10x Smaller Payloads)

Source: Dev.to

Overview

Guillermo Rauch shared that Vercel’s changelog now serves Markdown when agents request it—using the same URL but a different Accept header.
The key insight isn’t just the size reduction; it shows that an entire infrastructure layer (CSS, JS, frameworks) can become optional for a growing class of consumers.

• Browsers send Accept: text/html.
• Agents can send Accept: text/markdown.

Both receive the same URL but get different representations.

Implementation in Hugo

Add a second output format for Markdown in your Hugo configuration:

[outputs]
page = ["HTML", "MARKDOWN"]

[outputFormats.MARKDOWN]
baseName = "index"
mediaType = "text/markdown"
isPlainText = true

Middleware (Vercel Edge)

Deploy edge middleware that rewrites requests when the Accept header includes text/markdown:

export const config = { matcher: ['/', '/posts/:path*'] }

export default async function middleware(request) {
  if (request.headers.get('accept')?.includes('text/markdown')) {
    const url = new URL(request.url)
    url.pathname = url.pathname.replace(/\/?$/, '/index.md')
    return fetch(url)
  }
}

Testing the Endpoint

curl -H 'Accept: text/markdown' https://evoleinik.com/posts/markdown-for-agents/

The response size drops from ~20 KB (HTML) to ~2 KB (Markdown)—about a 10× reduction. Vercel’s changelog sees a 250× reduction, but even a 10× saving adds up.

Benefits

• Reduced payloads lower bandwidth and token usage for AI agents.
• Simpler content provides cleaner context for downstream processing.
• Static sites (e.g., Hugo) can serve Markdown trivially since it’s the source format.

Considerations

• For static generators, maintaining two output formats is straightforward.
• For dynamic sites or SPAs, generating Markdown server‑side or maintaining parallel content can be more complex.
• Agent traffic is increasing, making lightweight, structured content increasingly valuable.

The visual web was built for human browsers; the agent web can dispense with decorative assets and focus on concise, machine‑readable content.