Usability Tester

Overview

Validates that users can successfully complete core tasks through systematic observation and expert evaluation. Covers moderated and unmoderated testing, heuristic evaluation, accessibility checks, and issue severity scoring. Not a substitute for analytics or A/B testing -- those measure what happens, usability testing reveals why.

When to use: Testing user flows, validating designs, identifying friction points, running heuristic evaluations, ensuring users can complete core tasks, planning and executing usability test sessions.

When NOT to use: Analytics or A/B test setup, visual design critique without task-based evaluation, automated UI testing (use a testing framework), performance benchmarking.

Quick Reference

Core Metrics

Issue Severity

Severity equals Impact (1-3) multiplied by Frequency (1-3). Critical (8-9): fix before release. High (6-7): fix before release. Medium (4-5): next release. Low (1-3): backlog.

Common Mistakes

Delegation

• Test plan and scenario creation: Use Task agent to draft objectives, research questions, and realistic task scenarios for each persona
• Heuristic evaluation: Use Task agent to walk through interfaces against Nielsen's 10 heuristics and document violations with severity ratings
• Metrics analysis and severity scoring: Use Task agent to calculate SUS scores, task success rates, and severity ratings from raw test data
• Test report generation: Use Task agent to compile findings into structured reports with prioritized recommendations
• Test infrastructure planning: Use Plan agent to select testing methods, define recruitment criteria, and schedule test waves

If the quality-auditor skill is available, delegate full codebase quality audits and scoring to it.
Otherwise, recommend: pnpm dlx skills add oakoss/agent-skills -s quality-auditor -a claude-code -y

References

• Test Planning -- objectives, research questions, recruitment, task scenario templates, screening criteria
• Conducting Tests -- think-aloud protocol, facilitation rules, post-task questions, session structure
• Testing Methods -- unmoderated, guerrilla, first-click, cognitive walkthrough, method selection by lifecycle stage
• Heuristic Evaluation -- Nielsen's 10 heuristics, evaluation process, severity rating, combining with user testing
• Metrics and Severity -- success rate, time on task, errors, satisfaction, SUS scoring, severity formula
• Reporting -- test report template, key insights format, stakeholder presentation, recommended actions
• Accessibility Testing -- inclusive recruitment, assistive technology testing, WCAG alignment, accessibility heuristics
• Remote and Tools -- remote vs in-person comparison, testing tools, test frequency, checklists