I wanted a small tool to slice gigabyte-sized JSON-lines logs by level and time range and print a summary, but the first version the model gave me did open(f).read() and json.loads on the whole thing, which OOM'd on anything real.
The prompt now mandates streaming line-by-line, tolerating malformed lines, and supporting both a file and stdin so it drops into a pipe. Requiring it to handle a truncated final line and non-JSON garbage without crashing made it robust against real logs, which are always messier than the sample.
How do people prompt for time-range filtering when logs mix timezones? I force UTC normalization and treat naive timestamps as a configurable default.
Build a Python CLI `logslice` that reads JSON-lines logs (one JSON object per line) from a FILE argument or stdin, filters and summarizes them, and never loads the whole file into memory.
Requirements:
- Stream line by line (iterate the file object); memory usage must be O(1) in the number of lines. Works on multi-GB files and on `tail -f`-style pipes.
- Read from stdin when no file is given or the arg is `-`, so it composes in pipelines.
- Tolerate messy input: skip and count malformed/non-JSON lines and a truncated final line; never crash on bad data. A `--strict` flag makes malformed lines a hard error instead.
- Filters: `--level {debug,info,warn,error}` (>= threshold), `--since`/`--until` ISO timestamps, `--grep REGEX` on the message field. Normalize all timestamps to UTC; treat naive timestamps as `{DEFAULT_TZ}`.
- Output modes: default prints matching lines; `--summary` prints counts by level, top {N} messages, and time span, without printing every line.
- Correct exit codes (0 = matches found or summary ok, 1 = runtime error, 2 = bad args) and grep-like behavior for pipelines. Flush stdout so `| head` doesn't cause a BrokenPipe traceback (handle SIGPIPE).
- Keep parse/filter/summarize as separate pure functions for unit tests.
List the failure modes you handle (empty file, all-malformed, huge single line, mixed timezones) and confirm each is covered.