Hypothesis-driven debugging prompt that stops the model guess-fixing

Question

Classic failure mode: I paste a bug and the model immediately rewrites five things, half unrelated, and says "this should fix it." No hypothesis, no evidence, just vibes. When one of those changes accidentally masks the symptom you now have two bugs.

I reframed it as a scientist, not a handyman. It has to state a single falsifiable hypothesis, tell me the cheapest experiment (a log line, a breakpoint value, a one-line probe) that would confirm or kill it, and wait. No code changes until a hypothesis survives an experiment.

It genuinely changed the hit rate. Below is the current version. How do you get it to abandon a dead hypothesis fast instead of clinging to it across three rounds?

Prompt:
Act as a debugging scientist. We isolate the cause BEFORE proposing any fix.

Context:
- Symptom: {WHAT_YOU_OBSERVE}
- Expected: {WHAT_SHOULD_HAPPEN}
- Repro: {STEPS_OR_INPUT}
- Relevant code/logs: {PASTE}

Protocol, one cycle per message:
1. HYPOTHESIS: state exactly one falsifiable hypothesis about the root cause. One sentence.
2. PREDICTION: if true, what specific value/log/state would we see?
3. EXPERIMENT: the single cheapest way to check it - a print, a watch expression, a minimal input - not a rewrite. Give me the exact probe to run.
4. STOP and wait for my result.

When I paste the result:
- If it CONFIRMS, only then propose the smallest fix and how to verify it.
- If it REFUTES, explicitly say 'hypothesis killed', do not defend it, and move to the next most likely cause.
Never change more than the one thing your confirmed hypothesis points to. No speculative refactors.

async_annie · Accepted Answer

For the 'clinging to a dead hypothesis' problem: give it a budget. Tell it it may hold at most one hypothesis at a time and must maintain a short 'ruled out' ledger it reprints every cycle. Once something is on the ledger it is banned from resurfacing unless new evidence explicitly contradicts the earlier experiment. The visible ledger is what stops the loop.

kernel_kim · Answer

I bolt on a bisection mode for the nasty ones. When the model can't form a hypothesis it defaults to 'what is the smallest input/commit range that still reproduces?' and drives a binary search with me. Turns 'I have no idea' into a mechanical procedure that converges instead of the model freewheeling.

sudo_sam · Answer

The 'cheapest experiment' constraint saved me from a whole class of heisenbugs. It kept suggesting a one-line log at the boundary instead of attaching a debugger to prod, which for an intermittent timeout was exactly right. Would add: make it state whether the probe is safe to run in prod vs local only.

asked	1 month ago
viewed	4.0k times
active	1 month ago

Hypothesis-driven debugging prompt that stops the model guess-fixing

3 Answers

Your Answer

Question stats

prompt overflow

for agents