User Experience Metrics
Master UX metrics to measure and optimize AI agent performance from the user perspective
Your Progress
0 / 5 completedResponse Quality Metrics
Quality metrics assess whether responses are accurate, relevant, complete, clear, and helpful. While technical metrics measure model performance, quality metrics capture user perception of value. A technically accurate response that users find confusing or incomplete scores low on quality.
Interactive: Quality Score Calculator
Adjust quality dimensions to see how they affect overall response quality. Different use cases prioritize different dimensions:
Quality Dimensions Explained
- •Accuracy: Factually correct information. No hallucinations or errors. Verify with ground truth.
- •Relevance: Directly addresses user query. No tangents or off-topic content. Context-appropriate.
- •Completeness: Provides all necessary information. No critical gaps. Users don't need follow-ups.
- •Clarity: Easy to understand. No jargon or confusing explanations. Well-structured.
- •Helpfulness: Actually solves user problem. Actionable guidance. Empathetic tone.
How to Measure Quality
Sample 100-200 responses weekly. Raters score each dimension 1-5. Expensive but accurate.
Use GPT-4 to evaluate responses on quality dimensions. Scale to 100% coverage. Correlate with human ratings.
Track which quality dimensions correlate with thumbs up/down. Optimize high-impact dimensions first.
Quality and satisfaction are highly correlated (r = 0.8-0.9). Improving quality from 75% to 85% typically increases satisfaction by 10-15 points. Focus on dimensions users care most about— for support agents, helpfulness and clarity matter more than completeness. For research agents, accuracy and completeness are critical.