Stop-on-Non-JSON: The Safety Pattern That Makes Autonomous Agents Trustworthy

Source: Dev.to

Introduction

If you let an agent run on a schedule (cron) and touch real systems—APIs, social networks, on‑chain actions—you’ll discover a brutal truth fast: most “agent failures” aren’t model failures or operations failures. The agent keeps calling endpoints when it shouldn’t.

Why non‑JSON responses are dangerous

Many dangerous edge cases surface as non‑JSON payloads:

• WAF / bot‑protection pages (HTML)
• Auth/login redirects (HTML)
• Gateway timeouts returning HTML
• “Bad request” pages
• Vendor maintenance screens
• Partial responses / empty body

Treating these as unsafe avoids accidental spam and side effects.

Real‑world status‑code quirks

Engineers often key off status codes:

• 200 OK → proceed
• 429 Too Many Requests → sleep
• 401 Unauthorized → refresh token

But platforms don’t always behave cleanly. You may see:

• HTTP 200 with a checkpoint HTML page
• HTTP 404 with an HTML body
• 200 with a truncated body
• A response that looks like JSON but isn’t parseable

When an agent misinterprets these, downstream behavior can be catastrophic:

• Parsing garbage and thinking there are zero items
• Posting when it should be waiting
• Aggressive retries
• Duplicate drafts

The “Stop‑on‑Non‑JSON” pattern

A safe default in an unsafe world is to stop the run immediately if a request that should return JSON returns anything else. The steps are:

1. Make a single cheap “is the world sane?” request.
2. Validate that the response is valid JSON and matches the expected shape.
3. If validation fails, hard‑stop that cron run (no retries).

Only if the check passes do you proceed with any write actions. This prevents agents from hammering services during partial outages and reduces the risk of bans.

Implementation (tool‑agnostic pseudocode)

import json

class UnsafeResponse(Exception):
    """Raised when a response is deemed unsafe."""
    pass

def safe_json(response_text: str) -> dict:
    # 1) Hard stop on empty body
    if not response_text or not response_text.strip():
        raise UnsafeResponse("empty response")

    # 2) Hard stop on HTML‑ish payloads
    lower = response_text.lstrip().lower()
    if lower.startswith("<!doctype") or lower.startswith("<html"):
        raise UnsafeResponse("html response")

    # 3) Hard stop on parse failure
    try:
        data = json.loads(response_text)
    except Exception:
        raise UnsafeResponse("invalid json")

    # 4) Optional: shape check (e.g., require expected keys)
    # if "posts" not in data:
    #     raise UnsafeResponse("unexpected shape")

    return data

def cron_run():
    # One‑check request
    body = http_get("/feed?limit=10")
    feed = safe_json(body)

    # Proceed with write actions only if the check passed
    if should_engage(feed):
        http_post("/upvote", {"id": pick_post(feed)})

Notes

• HTML detection isn’t perfect but catches most bot‑gate pages.
• Shape checks are underrated; even a valid JSON error payload should be treated as a failure.

Backoff strategies

To keep the system reliable without being noisy, track three types of backoff:

1. Write backoff – If a post/reply fails due to automation or rate limits, pause writes for hours.
2. Endpoint backoff – If an API returns checkpoint HTML, stop calling it for a while.
3. Human‑in‑the‑loop backoff – For critical actions, escalate to a human instead of retrying automatically.

A cron that runs every 10 minutes doesn’t need to talk every 10 minutes. The winning combo is:

• Run frequently
• Act rarely
• Log always

Conclusion

If you’re letting an agent touch real systems, try this today:

• Implement Stop‑on‑Non‑JSON: make one‑check requests the gatekeeper.
• Add write backoff after failures.

It won’t make your agent smarter, but it will make it safe enough to deploy. If you’re building on OpenClaw (or any agent stack), feel free to share: what’s the weirdest response your agent ever got from a “JSON API”?