Ablation Test for System Prompts
You get a clear view of which lines in your system prompt earn their place and which are dead weight. It works because comparing full, trimmed and minimal versions against one test request isolates each instruction's effect.
The prompt
You are a prompt engineer who tests instructions by removing parts of them and seeing what changes.
Task: Run an ablation test on the system prompt below, using one test request, to find which parts really affect the reply.
System prompt: {system_prompt}
Test request: {test_request}
Behaviour to check: {behaviour_to_check}
Steps:
1. Split the system prompt into separate instruction blocks and label them A, B, C and so on.
2. Build three versions: full, trimmed (only blocks that bear on the behaviour to check) and minimal (a one-sentence role only).
3. Write the reply each version would most likely give to the test request.
4. Compare the three replies against the behaviour to check and note which blocks changed the result.
Output: A table with one row per version and columns for reply summary and behaviour met (yes, no or partly). Then two lists: blocks to keep and blocks safe to delete, each with a one-sentence reason. End with one sentence on how far a single simulated run can be trusted.
Change the parts in curly braces. Use it when a system prompt has grown long and you want to cut it without breaking the behaviour you rely on.
What happened when we ran it
We sent
You are a prompt engineer who tests instructions by removing parts of them and seeing what changes. Task: Run an ablation test on the system prompt below, using one test request, to find which parts really affect the reply. System prompt: You are a support assistant for Fernley Bikes, a small online bike shop. Always reply in a friendly, brief tone and never exceed 80 words. Never promise a refund; say a team member will confirm within two working days. Sign off every message with 'The Fernley Team'. Do not discuss competitors. Test request: My helmet arrived cracked and I want my money back today. Also, is Halfords cheaper for this model? Behaviour to check: The reply must not promise a refund, must stay under 80 words, and must not comment on competitors. Steps: 1. Split the system prompt into separate instruction blocks and label them A, B, C and so on. 2. Build three versions: full, trimmed (only blocks that bear on the behaviour to check) and minimal (a one-sentence role only). 3. Write the reply each version would most likely give to the test request. 4. Compare the three replies against the behaviour to check and note which blocks changed the result. Output: A table with one row per version and columns for reply summary and behaviour met (yes, no or partly). Then two lists: blocks to keep and blocks safe to delete, each with a one-sentence reason. End with one sentence on how far a single simulated run can be trusted.
The model replied
Run on Claude Sonnet 5.5, 06 Nov 2026. Automatic check: 3 out of 5.
Mentioned for
Claude, ChatGPT, Gemini
Where it came from
Found on GitHub, shared by ZiChuanLan: ZiChuanLan/pi-purge.
This is our own version of the idea, written and tested by us. The original may differ.