Reading Time: 4 mins
The best way to internalize prompt engineering best practices is studying real world examples. In this post, I’ll walk through case studies illustrating the optimization process and quantifiable improvements achieved.
As an AI consultant, I maintain extensive records of prompt tuning engagements. These examples showcase the art and science of prompting in action across diverse domains. Let’s explore and learn from prompt engineering wins in the wild.
First, why are concrete examples so helpful for mastering prompt optimization? Some key benefits:
Nothing substitutes for prompts refined under actual operating constraints.
Let’s analyze a project improving prompts for customer support conversations.
Initial prompts: Basic customer queries provided without context.
My order still hasn’t arrived 3 weeks after I placed it. What’s going on?
Metrics: 29% satisfaction rating, 11% coherence, slow response times
Iteration 1: Added prefix activating helpfulness, emphasis on patience and details.
I am Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. Please respond to this customer politely and provide as much detail as possible to thoroughly answer their question:
Satisfaction rating increased to 59%, coherence to 83%, and response time decreased 22%.
Iteration 2: Refined with more conversational tone, empathy, and solution focus.
I’m Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. I know delayed orders can be frustrating. Please respond very politely to this upset customer by first expressing understanding of their situation. Then provide additional details on what may have caused the delay, and actionable solutions for resolving the issue:
Satisfaction rating increased to 78%, coherence 96%, response time decreased another 5%.
This showcases iteratively honing prompts for a target vertical.
Here’s an example optimizing prompts for summarizing research papers:
Initial prompt: Minimal guidance provided.
Summarize the key points of this research paper.
Metrics: 22% accuracy, 38% coherence, 97% plagiarism
Iteration 1: Added formatting constraints and example structure.
In your own words, write a 5 sentence abstract-style summary of this paper’s key points, contributions, and results.
Accuracy increased to 33%, coherence 63%, plagiarism down to 72%
Iteration 2: Enhanced with concision prompt and anti-plagiarism technique.
Briefly summarize in exactly 5 sentences the core claims and contributions of this paper in your own words as if explaining to a general audience. Do not directly copy full sentences from the paper.
Accuracy scored 44%, coherence 78%, plagiarism reduced to 54%.
Here is a prompt optimization example for an enterprise search assistant:
Initial prompt: Minimal guidance on retrieving relevant company information.
Find me data on last quarter’s sales numbers.
Metrics: 23% relevance, 58% of results from unreliable sources
Iteration 1: Added instructions to leverage the company knowledge graph and data warehouse.
Using Acme’s knowledge graph and data warehouse, please find highly relevant data on last quarter’s sales numbers, focusing on official verified sources and metrics.
Relevance increased to 62%, unreliable sources down to 42%
Iteration 2: Further improved relevance by requesting specific key figures.
Leveraging Acme’s knowledge graph and sales data warehouse, please retrieve highly relevant verified figures for last quarter including: total revenue, units sold, revenue by product line, and year-over-year growth. Only use official company data sources.
Relevance scored 87%, unreliable sources reduced to 22%.
Some key learnings that emerge across optimization case studies:
Reviewing real examples provides prompt engineering clarity. I hope these case studies provide ideas and reinforcement for leveling up your own prompts. Please let me know if you would like to discuss any other prompt optimization examples and lessons learned!
ENROLL NOW FOR FREE DEMO CLASS
**We Don’t Spam
