The Wrong Way to Use AI for Debugging (And the Mental Model That Actually Works)

Source: Dev.to

Staging Incident – A AI‑Assisted Investigation

TL;DR – Three senior engineers spent hours chasing dead ends. I solved the mystery in ~20 minutes with a single, open‑ended prompt to Claude. The breakthrough came from letting the AI change its own investigative approach instead of constraining it with specific “check X” commands.

The symptom

Slack message
Staging is broken. We're investigating.

Running the migration tool on the staging database produced:

alembic current
ERROR: Can't locate revision identified by 'ba29cdc8739d'
FAILED: Can't locate revision identified by 'ba29cdc8739d'

For context: Alembic is a Python database‑migration framework. Each migration file has a unique revision ID. The staging DB thought it was on a revision that no longer existed in the codebase – a migration from a branch that never got merged.

What the senior team did

Takeaway – When an AI gives a confident but wrong answer, investigators can become anchored to that line of thought (the “Jagged Frontier” effect described by Harvard/BCG).

After several hours the team could state:

“We know what broke, but we can’t figure out who did it or when.”

My (11‑month‑old) approach

I have limited knowledge of CloudTrail, ECS, etc. My prompt was deliberately minimal:

Prompt – “Staging is broken, can you look into it? Pull the latest release branch and check.”

No specific instructions, no “check CloudTrail”. Claude:

1. Confirmed the missing revision didn’t exist on any main branch.
2. Traced the revision to a single feature branch with a single developer’s commits.
3. Hit the same walls as the senior team (SSH failure, expired SSO token, no relevant RDS logs).

I then asked a single follow‑up:

Prompt – “Is there a smarter way to find this?”

Claude switched tactics.

The breakthrough

Claude examined the migration source code and noticed the INSERT used NOW() for the created_at column. That meant the timestamp stored in the DB was the exact moment the migration ran.

SELECT rule_type, created_at
FROM firewall_rules
WHERE rule_type IN ('PolicyTypeA', 'PolicyTypeB');

Result:

The timestamp (15:40 PDT on March 31) matched a git commit on the feature branch:

2026-03-31 15:41:20 -0700 — fix: rename migrations and split enum ALTER from seed INSERT

The migration ran 55 seconds after that commit.

# Who committed on that branch?
git log origin/develop..origin/feature/the-branch --format="%an" | sort -u

Only one author appeared.

Why CloudTrail showed nothing: the developer likely had the DB password cached locally (.env), so no new GetSecretValue API call was made.
Why SSH logs were empty: the migration was executed from a local machine directly against the DB, not via the staging server.

The database itself held the answer – nobody thought to query it.

The mental‑model shift

Key insight – The human sets the goal; the AI chooses the investigative path. When the path is wrong, the human must supply a new direction. If the human instead anchors the AI to a specific, familiar technique (e.g., “check CloudTrail”), the investigation can get stuck.

Takeaways

1. Open‑ended prompts let the AI explore alternative evidence sources.
2. Specific, narrow prompts can lock the AI (and the team) into a dead‑end path.
3. The database (or system) itself often contains forensic clues (timestamps, logs, audit columns) that are easy to miss.
4. The pattern I used is not a formal framework – it’s an observation from a single incident. It may not generalize, but it highlights a useful mental model for AI‑augmented debugging.

TL;DR

• Senior engineers spent hours chasing the wrong lead because they prescribed the AI’s investigative steps.
• A junior engineer gave Claude a high‑level goal and a single “smarter way?” nudge.
• Claude pivoted to database‑level evidence (migration timestamps) and identified the culprit in minutes.

When using AI for incident response, let the AI decide how to search, and only intervene when you need it to change direction.

Reflections on the Debugging Process

The senior engineers did valuable work. They systematically eliminated possibilities — wrong branch deployment, unauthorized access, CI pipeline issues. Their elimination narrowed the search space. I benefited from knowing what wasn’t the cause.

“Not knowing” isn’t universally an advantage. In this specific case, I couldn’t over‑specify because I didn’t know the domain. But ignorance during incident response, when writing a fix, or in a post‑mortem — that’s a liability, not a feature.

The narrow lesson is: when you have no strong hypothesis, keeping the AI’s search space open is better than committing to a weak one.

The AI still did the technical work. Claude correlated timestamps, read migration source code, and ran git commands. That’s precise, analytical work. The “open‑ended” approach simply meant I didn’t prevent it from choosing which precise work to do.

Practical Tip: When Stuck Debugging with AI

If you’re stuck and your current approach isn’t working, try this before you give up:

1. Stop telling the AI where to look.
   Instead of: “Check the logs for X” → Try: “We’ve checked A, B, C and found nothing. What other evidence might exist in the system?”

2. Shift from concrete commands to problem framing.
   Instead of: “Grep for this function” → Try: “This endpoint is broken, investigate it.”

It won’t always work, but it costs you one sentence to try, and sometimes it unlocks an angle you wouldn’t have thought of — because you literally couldn’t have thought of it.

Breakthrough in my case: one sentence – “Is there a smarter way?”

That’s not a philosophy. It’s just a useful question.

Outcome

The staging incident was resolved. The evidence was shared. The developer confirmed it was their machine.

Forensics Nugget

If a migration uses NOW() in the INSERT, the created_at timestamp is your crime‑scene timestamp. Compare it with the git log and you might find your 55‑second smoking gun.