9

I do strict TDD and I wanted the model to do it with me, not for me. Default behavior with GPT is: ask for a feature, get the implementation plus some tests it wrote to match the code it just wrote. Those tests are worthless because they assert whatever the code happens to do.

What finally worked was splitting it into two hard-gated turns. Turn one it may ONLY produce a failing test and must literally show me the failure it expects. It is forbidden from writing any implementation. I run it red, then unlock turn two.

This catches so many spec ambiguities up front. Curious how others stop the model from sneaking a stub implementation into the 'test only' turn.

THE PROMPT
We are doing test-driven development. Follow the red-green-refactor loop strictly. This turn is RED only.

Rules for this turn:
1. Output ONLY one new test for the next smallest slice of: {FEATURE_DESCRIPTION}.
2. The test MUST fail for the RIGHT reason. State the exact assertion error or exception you expect when I run it against the current code (which does not implement this yet).
3. Do NOT write, stub, or scaffold any implementation. No new production functions, no `pass`, no `NotImplementedError` placeholders in prod code. If you need a symbol that does not exist, that missing symbol IS the expected failure.
4. Name the test as a behavioral sentence and add one comment explaining the business rule it encodes.
5. End by asking me to run it and paste the actual failure so we can confirm it is red before you are allowed to make it green.

Stack: {LANGUAGE_AND_TEST_FRAMEWORK}.
1the two-turn gate is the trick. one turn and it always cheats by writing the impl 'to be helpful'.shipfast 2 months ago
add a comment

1 Answer

7

The 'fail for the RIGHT reason' clause is doing a lot of heavy lifting and people skip it. Without it you get a test that fails on an import error and the model calls that red, then 'green' just means the import resolved. I extend it: make it predict the failure category (assertion vs error vs collection error) and reject its own test if a mere syntax/import problem is what turns it red.

Your Answer