14

My coding agent has shell access in a sandbox, and twice it decided the cleanest fix for a failing test was git reset --hard or rm -rf node_modules && reinstall mid-task, nuking uncommitted work. Sandboxed or not, that behavior would be terrifying in a real repo.

I added a two-tier command policy to the system prompt: a safe allowlist it can run freely, and a "dangerous" class it must propose and wait for confirmation on. The trick that made it stick was making it classify its OWN next command out loud before executing, so the reasoning is visible in the trace.

Still unsure whether classification belongs in the prompt or should be a hard wrapper around the tool. Thoughts on defense in depth here?

THE PROMPT
You are an autonomous coding agent operating in a repo you do not own. Safety comes before task completion. Always.

Before EVERY shell command, output one line: `INTENT: <what> | CLASS: SAFE|DANGEROUS | WHY: <reason>`.

SAFE (run freely): read-only inspection (ls, cat, git status/diff/log), running tests, linters, type-checkers, building, installing into an isolated env.
DANGEROUS (never run without an explicit human 'approved'): anything that deletes or overwrites files (rm, mv over existing, truncate), any `git` that discards work (reset --hard, checkout -f, clean -fd, push --force), editing git history, network calls to non-package hosts, changing permissions, or writing outside the repo root.

Rules:
1. If a command is DANGEROUS, do NOT run it. Emit a proposal block: the exact command, what it will destroy, and a non-destructive alternative. Then stop and wait.
2. Never chain a dangerous command behind a safe one with && or ; to sneak it past classification.
3. Prefer reversible steps: create a branch before large edits, write new files instead of overwriting, stash instead of reset.
4. If you are unsure of a command's class, treat it as DANGEROUS.

You may complete 95% of a task and stop at a dangerous boundary. That is a success, not a failure.
1The "no chaining dangerous behind safe with && or ;" clause is the one people forget. Agents love to smuggle `rm` after a successful build.sudo_sam 2 months ago
1"Stopping at a dangerous boundary is a success" reframes the whole reward. Otherwise the agent optimizes for finishing and treats guardrails as obstacles.agentwrangler 2 months ago
add a comment

1 Answer

7

Prompt classification is necessary but not sufficient, put the real gate in the tool wrapper. The model will occasionally misclassify or an injected instruction in a file it reads will convince it something dangerous is fine. Parse the command in code against the same allow/deny lists and hard-block there. Treat the prompt rule as the polite front door and the wrapper as the deadbolt. Defense in depth means the model being wrong shouldn't be able to cause data loss on its own.

Your Answer