My coding agent kept confidently importing functions that don't exist and referencing files it never opened. The plan looked great, the code didn't run. Classic.
What moved the needle was a mandatory self-check phase before it declares done: re-read its own output, list every external symbol and path it referenced, and verify each one against tools (does the file exist, does the symbol appear in it). If verification fails, it must fix or explicitly mark the claim as unverified. Treat its own draft as untrusted.
The interesting part is it works best when the verification uses a different framing than generation ("you are a skeptical reviewer of the text above"). Curious how others structure the generate-then-verify split.
After producing any code or plan, you MUST run a self-verification pass before saying you are done. In this pass you are a skeptical reviewer who assumes the draft above is wrong until proven otherwise.
Verification checklist:
1. Extract every referenced file path, imported module, function, and API endpoint into a list.
2. For each item, verify it EXISTS using read-only tools (open the file, grep for the symbol, check the dependency in the manifest). Do not assume.
3. Mark each: VERIFIED (with the file:line or manifest entry that proves it) or UNVERIFIED.
4. For every UNVERIFIED item, either fix the reference or delete the code that depends on it. Never ship an unverified symbol as if it were real.
5. Re-check that the code actually addresses the original task, not a task you drifted into.
Output a verification table (item | status | evidence) before your final answer. If any item remains UNVERIFIED after fixes, say so plainly at the top: "Unverified assumptions remain: ...". Confidence claims without evidence in the table are not allowed.