22

Had a flaky test that failed maybe 1 in 30 runs. Every model I tried "fixed" it by adding an await asyncio.sleep(0.1) somewhere, which is not a fix, it is a bribe. The real bug was two coroutines mutating a shared dict between awaits.

What worked was banning timing-based fixes outright and forcing it to reason about interleavings: identify every await point in the critical section, then construct a specific interleaving that produces the bad state. Once it had to name the schedule, it found the shared-state mutation instead of masking it.

Posting the prompt because concurrency is where models flail the most. How do you push it toward the minimal correct synchronization instead of slapping one big lock on everything?

THE PROMPT
We have an intermittent concurrency bug. Time-based 'fixes' are BANNED: no added sleeps, no retries, no timeout bumps, no 'just await it twice'. Those hide races, they do not fix them.

Analyze like this:
1. Identify the shared mutable state touched by more than one coroutine/task/thread.
2. Mark every yield point (await / lock release / IO boundary) inside the critical sections.
3. Construct ONE concrete interleaving: 'Task A runs to line X and suspends at the await, Task B runs lines Y-Z observing stale state, then A resumes' - that produces the observed bad result. Walk the shared state values step by step.
4. Only after you can exhibit the failing schedule, propose the MINIMAL synchronization that makes that interleaving impossible: prefer making the critical section not span an await, or a single narrow lock, over a coarse global lock. Justify why nothing smaller works.
5. Give me a test that deterministically reproduces the race (e.g. by controlling scheduling / injecting the interleaving), not one that just runs it 1000 times and hopes.

Code:
{PASTE}
Observed bad behavior: {SYMPTOM}
'time-based fixes are BANNED' is the whole post. a sleep that makes the flake go away is a landmine you buried for future-you.the_debugger 1 month ago
add a comment

1 Answer

19

Step 3 (exhibit the schedule) is exactly how you separate a real fix from a bribe. For step 4 minimality, I make it enumerate the options explicitly: (a) shrink critical section below any await, (b) narrow lock, (c) immutable/copy-on-write, (d) single-owner actor. It has to pick the earliest one in that list that works and argue why the cheaper ones don't. Stops it reaching for a global mutex reflexively.

THE PROMPT
Add: 'Consider fixes in this order and choose the first that works: (a) remove await from the critical section, (b) narrow lock, (c) copy-on-write/immutable, (d) single-owner task. Justify why each earlier option is insufficient.'

Your Answer