When testing prompt revisions, manual inspection doesn't scale. Use a stronger model (e.g. Claude 3.5 Sonnet or GPT-4o) as an impartial evaluation judge:
markdown
Evaluate the following candidate response on a scale of 1 to 5 across 3 criteria:
1. Factual Accuracy: Are all claims strictly supported by the reference material?
2. Instruction Adherence: Did it follow all negative formatting rules?
3. Conciseness: Is there unnecessary boilerplate or verbose repetition?
Output strictly valid JSON:
{
"scores": { "accuracy": 5, "adherence": 4, "conciseness": 5 },
"total_score": 14,
"critical_defect": null,
"rationale": "Followed all constraints, minor markdown whitespace gap."
}Running this evaluation across 50 test inputs gives you quantifiable benchmark scores before deploying prompt changes to production.