Propose, Probe, Compose Prompt Refiner

Act as a prompt engineer who improves instructions by comparing real outputs, not by guessing.

Task: Refine the working instructions below for the stated job, without needing a correct answer to check against.

Instructions to improve: You are a support assistant for a small online bakery. Answer customer emails about orders. Be friendly and helpful. Keep replies short.
Job and sample input: Customer email: 'My birthday cake order #4821 was supposed to arrive yesterday and it still hasn't. The tracking page just says processing. I need it for Saturday. What are you going to do about this?'
What a good result looks like: The reply apologises once without grovelling, states what will happen next with a clear timeframe, offers a concrete remedy such as a refund or reshipment, avoids inventing facts about the order, and stays under 120 words.

Steps:
1. Propose three distinct edits to the instructions, each aimed at a different weakness (for example clarity, missing constraints, or output format).
2. Probe each edit: write out the response the edited instructions would most likely produce for the sample input, and do the same for the original.
3. Contrast the four responses against the quality criteria. Note which edit changed behaviour, in what way, and whether it helped or hurt.
4. Compose one final set of instructions that keeps only the edits with clear evidence of helping.

Output:
A short table of the three edits with a one-line verdict each, followed by the final instructions in a single block, then two lessons learned that could be reused for similar jobs.

## Edits and verdicts | # | Edit (weakness targeted) | Effect on the sample reply | Verdict | |---|---|---|---| | A | **Structure:** apologise once, then say what happens next with a timeframe, then give a remedy, then a one-line close (output format) | Reply now follows the right order and has a clear next step. The timeframe is sometimes vague, and it may invent a cause. | **Keep.** It fixes ordering and the missing next step. | | B | **Constraint:** use only facts from the email or order data, never invent tracking, courier, cause or dates, and keep it under 110 words (missing constraints) | Fabrication disappears and length is controlled. Alone, it makes the reply over-hedged ("I'll update you when I can") with no remedy. | **Keep.** It is essential but insufficient alone. | | C | **Remedy rules:** offer a refund or priority reshipment on late orders, commit only to our own follow-up time, and don't promise a delivery date (ambiguity about what "helpful" means) | Adds a concrete remedy and a firm follow-up time. Without the "no delivery promise" clause it tends to promise Saturday, which is an invention. | **Keep, with the guard clause.** | ## Probe outputs These are predicted outputs, not live runs. Run each prompt 3–5 times on the real model before trusting them. **Original:** "Hi there! So sorry to hear your cake is late, and we know how important birthdays are! 🎂 I'll look into this right away and get back to you soon. Thanks so much for your patience!" - Fails: no timeframe ("soon"), no remedy, and it doesn't answer "what are you going to do?". Apology and cheer are heavy. About 40 words. **Edit A only:** "I'm sorry your cake hasn't arrived. Here's what happens next: I'm checking with our kitchen and the courier, and I'll email you by end of day. If it can't make Saturday, we can sort out a refund or a new cake." - Better order and a timeframe. But "kitchen and courier" is assumed, "end of day" is invented policy, and the remedy is conditional. **Edit B only:** "I'm sorry about the delay with order #4821. I can see the status is still 'processing' and I'm passing this to our team to check. I'll update you as soon as I have more information." - No fabrication, short. But no firm timeframe and no remedy, so it fails two criteria and feels evasive given the Saturday deadline. **Edit C only:** "Sorry about this. I'll email you within 2 hours with an update. You can choose a full refund or a priority reshipment, and your cake will arrive by Saturday." - Concrete timeframe and remedy. But "will arrive by Saturday" is an unverifiable promise, and it may also claim a cause. **Combined (A+B+C with guard):** "I'm sorry, order #4821 should have reached you yesterday, and 'processing' isn't a helpful status when you need it Saturday. I'm checking with our team now and will email you within 2 hours with a confirmed update. If we can't confirm delivery in time for Saturday, you can choose a full refund or a priority reshipment. Just reply with which you'd prefer." (about 65 words) - Meets every criterion: one apology, a timeframe, a remedy, no invented facts, and it addresses the deadline without promising it. ## Final instructions ``` You are a support assistant for a small online bakery, replying to customer emails about orders. Tone: warm, calm and plain. No emojis, no stacked exclamation marks. Structure every reply as: 1. One sentence acknowledging the problem and apologising once. Do not apologise again. 2. What happens next, with a specific timeframe: we will email a confirmed update within [2 business hours]. 3. A concrete remedy. For late or missing orders, offer the customer a choice of a full refund or a priority reshipment. If the customer mentions a deadline, say we will confirm whether it can be met. Never promise a delivery date. 4. One closing line telling them what to reply with. Rules: - Use only facts given in the email or in order data supplied to you. Never invent tracking details, couriers, causes of delay, or dates. - If the status is unclear, say what you can see (e.g. "processing") and that you are checking. - Answer the customer's direct question. - Maximum 110 words, plain text, no headers or bullet points. ``` Replace the bracketed value and the refund/reshipment options with what the bakery can actually honour. The prompt will repeat whatever commitments it is given. ## Lessons learned 1. **Constraints and remedies must be paired.** "Don't invent facts" alone produces vague, evasive replies. Pair every prohibition with the concrete thing the model *is* allowed to commit to, such as a follow-up time or a refund choice. 2. **A helpful instruction can create a new failure.** Telling the model to be concrete and reassuring pushed it toward promising unverifiable outcomes (Saturday delivery). Whenever you add an instruction that encourages commitment, add a guard clause for what it must not commit to.