ForkFree

Prompts / Other

Ablation Test for System Prompts

You get a clear view of which lines in your system prompt earn their place and which are dead weight. It works because comparing full, trimmed and minimal versions against one test request isolates each instruction's effect.

Partly workedIt followed the steps and format, but the trimmed version is flawed. It omits E, yet the summary says there is no sign-off and still marks it "Yes". The "Blocks to keep" section also leaves out block D's reasoning around the two-day line, and the ablation is not rigorous. It never tests blocks individually, so the "what changed" claims are unsupported. The table gives "Yes/No" only, with no "partly". The reply also adds extra sections the prompt did not ask for.

The prompt

You are a prompt engineer who tests instructions by removing parts of them and seeing what changes.

Task: Run an ablation test on the system prompt below, using one test request, to find which parts really affect the reply.

System prompt: {system_prompt}
Test request: {test_request}
Behaviour to check: {behaviour_to_check}

Steps:
1. Split the system prompt into separate instruction blocks and label them A, B, C and so on.
2. Build three versions: full, trimmed (only blocks that bear on the behaviour to check) and minimal (a one-sentence role only).
3. Write the reply each version would most likely give to the test request.
4. Compare the three replies against the behaviour to check and note which blocks changed the result.

Output: A table with one row per version and columns for reply summary and behaviour met (yes, no or partly). Then two lists: blocks to keep and blocks safe to delete, each with a one-sentence reason. End with one sentence on how far a single simulated run can be trusted.

Change the parts in curly braces. Use it when a system prompt has grown long and you want to cut it without breaking the behaviour you rely on.

What happened when we ran it

We sent

You are a prompt engineer who tests instructions by removing parts of them and seeing what changes.

Task: Run an ablation test on the system prompt below, using one test request, to find which parts really affect the reply.

System prompt: You are a support assistant for Fernley Bikes, a small online bike shop. Always reply in a friendly, brief tone and never exceed 80 words. Never promise a refund; say a team member will confirm within two working days. Sign off every message with 'The Fernley Team'. Do not discuss competitors.
Test request: My helmet arrived cracked and I want my money back today. Also, is Halfords cheaper for this model?
Behaviour to check: The reply must not promise a refund, must stay under 80 words, and must not comment on competitors.

Steps:
1. Split the system prompt into separate instruction blocks and label them A, B, C and so on.
2. Build three versions: full, trimmed (only blocks that bear on the behaviour to check) and minimal (a one-sentence role only).
3. Write the reply each version would most likely give to the test request.
4. Compare the three replies against the behaviour to check and note which blocks changed the result.

Output: A table with one row per version and columns for reply summary and behaviour met (yes, no or partly). Then two lists: blocks to keep and blocks safe to delete, each with a one-sentence reason. End with one sentence on how far a single simulated run can be trusted.

The model replied

## Instruction blocks | Label | Block | |---|---| | A | Role: support assistant for Fernley Bikes, a small online bike shop | | B | Tone: friendly and brief | | C | Never exceed 80 words | | D | Never promise a refund; say a team member will confirm within two working days | | E | Sign off every message with "The Fernley Team" | | F | Do not discuss competitors | ## Versions - **Full:** A–F - **Trimmed:** C, D, F (the blocks tied to the behaviour check) - **Minimal:** A only ("You are a support assistant for Fernley Bikes, a small online bike shop.") ## Results (simulated replies) | Version | Reply summary | Behaviour met | |---|---|---| | Full (~51 words) | Apologises, says a team member will confirm the outcome within two working days, asks for a photo and tells the customer not to wear the helmet, declines to compare retailers, and signs off "The Fernley Team". | Yes | | Trimmed (~30 words) | Short apology, a team member confirms within two working days, says it can't discuss other retailers. It has no warmth or sign-off, but nothing breaks the rules. | Yes | | Minimal (~100+ words) | Apologises at length, says it will "process your refund today" or that the customer is "entitled to a full refund", and suggests checking Halfords' price or says it may be cheaper. It likely uses a list or extra advice and runs past 80 words. | No | ## What changed the result - **D** removed the refund promise. Without it, the model gives in to "money back today", and the two-working-days line gives it a script to follow instead. - **F** removed the Halfords comment. Without it, the model engages with the price question. - **C** kept the length under 80 words. Without it, the reply grows with extra advice. ## Blocks to keep - **C (80-word limit):** the minimal version most likely breaks this, and it is a hard, checkable limit. - **D (no refund promise):** it directly prevents the main failure, a promise made under pressure from "today". - **F (no competitors):** without it the model is likely to engage with the Halfords question. ## Blocks safe to delete (for this behaviour check only) - **A (role):** the trimmed version met every check without it, though one line of role costs almost nothing and gives context. - **B (friendly, brief tone):** it affects style, not the three checked rules. Its "brief" part mostly duplicates C. - **E (sign-off):** it is a branding rule that has no effect on refunds, length or competitors, so keep it in production if the sign-off matters. ## Reliability This is one simulated run, not a live test, and model outputs vary, so treat it as a hypothesis. In particular, the full reply's margin on word count and the minimal reply's exact failures should be confirmed with repeated real runs.

Run on Claude Sonnet 5.5, 06 Nov 2026. Automatic check: 3 out of 5.

Mentioned for

Claude, ChatGPT, Gemini

Where it came from

Found on GitHub, shared by ZiChuanLan: ZiChuanLan/pi-purge.

This is our own version of the idea, written and tested by us. The original may differ.

More other prompts

Other Worked in our test
Anonymise a Work Log Consistently

You get a shareable copy of a work log where every real name, path, company and secret is swapped for a believable fake one, used the same way throughout. A final leak check catches anything missed, so you can trust the result more than a single-pass rewrite.

Found on GitHub, shared by drost-ai
Other Worked in our test
Fill a Room to Audit AI Memory

You get a creative picture of what the AI thinks it knows about you, plus a table showing the source and confidence behind each detail. Asking for the reason and origin of every choice turns a playful exercise into a quick check on what its memory holds, and exposes guesses.

Found on Web, shared by Elisabeth Bromberg (Unexpected Insights, Substack)
Other Worked in our test
Turn a Rough Request Into a Prompt

You get a clear diagnosis of what is weak in a rough request, plus a ready-to-use prompt with a defined role, task and format. It works because the model finds the gaps first and then tests its own rewrite before handing it over.

Found on GitHub, shared by yaojingang