18

Reporting handed me an events table with duplicate rows per entity (late-arriving data, at-least-once delivery) and I needed exactly the latest row per key. The naive GROUP BY + MAX(updated_at) approach fell apart when two rows shared the same timestamp, and a DISTINCT silently kept the wrong columns.

My prompt now forces ROW_NUMBER() with an explicit, total-ordering tiebreaker and requires the model to justify why the result is exactly one row per key. Making it prove the ordering is total (no ties possible) is what stopped the flaky output.

Anyone have a cleaner pattern than QUALIFY for warehouses that don't support it? I fall back to a CTE + WHERE rn = 1.

THE PROMPT
Write a SQL query (dialect: {DIALECT}) that returns exactly one row per {KEY_COLS} from table `{TABLE}`: the latest record by {ORDER_COL}.

Requirements:
- Use `ROW_NUMBER() OVER (PARTITION BY {KEY_COLS} ORDER BY ...)`. The ORDER BY must define a TOTAL order so no two rows can tie: primary key `{ORDER_COL} DESC`, then a deterministic tiebreaker (e.g. `id DESC`). Explain why ties are impossible.
- Do NOT use `SELECT DISTINCT` or `GROUP BY + MAX` for this (explain in one line why they're wrong here: DISTINCT can't pick a full row, MAX ignores tiebreakers).
- Structure as a CTE (`ranked`) then `WHERE rn = 1` so it runs on warehouses without `QUALIFY`; if {DIALECT} supports QUALIFY, also show that shorter form.
- Filter out soft-deleted rows if `{SOFT_DELETE_COL}` applies.
- Add a `-- SELF-CHECK` query that asserts the result is truly one-row-per-key: e.g. `SELECT {KEY_COLS}, COUNT(*) c FROM result GROUP BY {KEY_COLS} HAVING c > 1` must return zero rows. State the expected result.
- Note which columns should be indexed to make the window efficient.

Be explicit about NULL handling in the ORDER BY (NULLS FIRST/LAST) and why you chose it.
1The self-check HAVING COUNT(*) > 1 returning zero rows is such a simple guardrail and I'd never thought to bake it into the prompt itself. Instant confidence in the result.data_dan 3 months ago
add a comment

2 Answers

8

For the QUALIFY-less fallback, the CTE + WHERE rn = 1 is right, but watch that some optimizers won't push filters into the window CTE, so also try the correlated NOT EXISTS (a newer row) form and EXPLAIN both. On Postgres the window version usually wins; on some MySQL versions the anti-join is faster. Worth having the model emit both and letting you benchmark.

5

The 'prove the ordering is total' requirement is the part I steal. On big tables I also make the model consider whether a covering index on (key_cols, order_col DESC, tiebreaker) lets the engine do an index-only scan and skip the sort entirely, which is the difference between a 200ms query and a 40s one. Your indexing note hints at it but I'd make it mandatory to name the exact index.

THE PROMPT
Require: propose one covering index `(<key_cols>, <order_col> DESC, <tiebreaker> DESC)` and explain whether it enables an index-only / pre-sorted scan for the window function in {DIALECT}.

Your Answer