Ask any model for tests and you get three example-based cases: a normal input, an empty input, maybe a null. Fine, but they miss the weird stuff. I wanted property-based tests that assert invariants across generated inputs, which is where the real bugs hide.
With a local Qwen model I had to be very explicit or it would just wrap my happy-path examples in a loop and call it "property-based." The prompt that worked makes it first articulate the invariants in plain English (round-trip, idempotence, ordering, conservation), then map each to a hypothesis/fast-check strategy, and only then write generators.
It found a genuine off-by-one in a pagination helper via a round-trip property. Curious how others get good input generators without the model over-constraining them into triviality.
Design PROPERTY-BASED tests, not example tests. Example-only cases are rejected.
STEP 1 - Invariants in English. For the function below, list the properties that must hold for ALL valid inputs. Consider these classes explicitly and use the ones that apply:
- Round-trip: decode(encode(x)) == x
- Idempotence: f(f(x)) == f(x)
- Invariance/conservation: some quantity is preserved (length, sum, set of elements)
- Ordering/monotonicity
- Oracle: agreement with a slow-but-obvious reference implementation
- Metamorphic: relation between f(x) and f(transform(x))
STEP 2 - Strategies. For each property, define the input-generation strategy (types, ranges, edge distributions) using {PROPERTY_TESTING_LIB, e.g. Hypothesis / fast-check}. Generators must be WIDE: include empty, huge, unicode, negative/zero, duplicates, and near-boundary values. Do not narrow a generator just to make a property pass.
STEP 3 - Tests + shrinking note. Write the tests, and for each property state what a minimal failing example would look like so I can sanity-check the shrinker.
Function under test:
{PASTE}