AI agents are useful debugging partners, but they can become overconfident when a symptom resembles a familiar failure. Evidence-first runbooks slow that jump. They ask the assistant to collect facts, name uncertainty, and verify each claim before recommending a state-changing fix.

Evidence-first debugging means the first deliverable is not a fix. It is a compact evidence table: what was observed, where it came from, how fresh it is, and what it does or does not prove.

Why evidence should come before diagnosis

  • Symptoms overlap: stale cache, bad input, expired credentials, and deployment drift can produce similar errors.
  • Tool output ages: a check that was true yesterday may not explain the current incident.
  • Partial access misleads: an agent may see logs but not configuration, or documentation but not live state.
  • Fixes have blast radius: restarts, migrations, deletes, and publishes should wait until the evidence supports them.

A small evidence table

Keep the table short enough to fit in an incident note or pull request comment. It should distinguish direct evidence from inference.

Evidence table
Observation: <fact from command, log, test, page, or user report>
Source: <where it came from, generalized if public>
Freshness: <timestamp or "current run">
Supports: <which hypothesis this makes more likely>
Does not prove: <important limits or missing context>
Next safe check: <read-only verification step>

Checklist for evidence-first agent debugging

  • Start with read-only checks whenever possible.
  • Separate observations from hypotheses in the runbook output.
  • Mark stale, missing, redacted, or unavailable evidence explicitly.
  • Prefer the smallest check that can rule a hypothesis in or out.
  • Do not expose credentials, private paths, private identifiers, private network details, or raw sensitive logs.
  • Only promote to a state-changing action when the evidence, scope, rollback, and verification plan are clear.

When the agent should stop

Evidence-first debugging is also a safe delegation boundary. The agent should pause when evidence conflicts, a required source is unavailable, the proposed fix touches data that cannot be restored, or the next step requires authority it has not been given.

A good stop report is not a failure. It gives the human a narrow question, the evidence already gathered, and the safest next verification step.

Copyable prompt for safe delegation

Debug this issue evidence-first. Begin in read-only mode. Create a compact table with observations, sources, freshness, supported hypotheses, limits, and next safe checks. Do not edit, restart, deploy, delete, publish, or reveal secrets. If a state-changing action looks necessary, stop and provide the evidence, expected blast radius, rollback path, and verification plan.

Rule of thumb: if an AI agent cannot show the evidence that connects a symptom to a proposed fix, it is not ready to make the fix.