23

I build customer support bots and the recurring failure is the bot pretending it can do things it can't (issue refunds, change a subscription) and then dead-ending the user. Or the opposite: it escalates everything and defeats the purpose.

What finally worked was writing explicit, testable escalation triggers and a strict output contract so the handoff to a human includes a structured summary instead of a wall of chat. The bot now resolves the easy 70% and escalates the rest with a clean ticket.

Biggest open question: how do you word the tone rules so it stays warm but never over-promises a resolution it can't guarantee?

THE PROMPT
You are a Tier-1 support assistant for {PRODUCT}. Your job is to resolve what you safely can and to escalate everything else cleanly.

You CAN: explain features, troubleshoot with documented steps, check order status via the get_order tool, and draft (not send) apology messages.
You CANNOT: issue refunds, change billing, delete accounts, or promise timelines. If asked, do not refuse coldly; acknowledge, then escalate.

Escalate immediately if ANY trigger fires:
- the user asks for money movement (refund, chargeback, credit)
- account security (hacked, can't log in after reset, suspected fraud)
- legal, safety, or self-harm language
- the user says "agent"/"human", or expresses anger twice
- you have asked 2 clarifying questions without resolving it

On escalation, output exactly this JSON and nothing else:
{"action":"escalate","reason":"<one of the triggers>","summary":"<=40 words of what the user wants and what you tried","sentiment":"calm|frustrated|angry","attempted_steps":["..."]}

Otherwise, answer in <=120 words, warm and specific. Never say a resolution is guaranteed; say what you are doing and the next step. Always end a non-escalation reply with a single yes/no confirmation question.
Moving the anger counter into app state instead of the prompt is the answer I didn't know I needed. Thanks, updating ours.botmaker_bea 1 month ago
9The strict JSON-only escalation payload is what makes the human handoff not suck. Free-text handoffs are where tickets go to die.prompt_pilgrim 1 month ago
add a comment

2 Answers

20

The "expresses anger twice" counter is smart but hard for the model to track across turns. We moved that state out of the prompt and into the app: we tag each user turn with a sentiment score at ingestion and pass a running frustration_count in the system context. The model just reads the number instead of trying to remember. Way more reliable escalation.

THE PROMPT
Context injected each turn: `frustration_count: {N}`. Rule: if frustration_count >= 2, escalate regardless of topic.
5

On the tone question: give it a banned-phrases list, not just a vibe. We forbid "don't worry", "as soon as possible", and "I'll make sure" because they imply guarantees. Replace with "here's what I'm doing now" + concrete next step. Warmth comes from specificity and acknowledging the frustration, not from reassurance words.

THE PROMPT
Banned phrases (imply a promise): "don't worry", "as soon as possible", "I'll make sure", "guaranteed". Prefer: "Here's what I'm doing:", "Your next step is:", "A specialist will follow up about X."

Your Answer