Episode description

UX benchmarking may seem like a lot of work, but Kate Moran is here to show you how to do it effectively. She's VP of Research & Content at Neilsen Norman Group and leads UX teams to better benchmarking, teaches newbies how to get started, and explains this complicated subject with clarity. She joined Erin and JH on our very first live episode to explain how UX benchmarking can help teams show the ROI of their work. 

She walked through how benchmarking can help get stakeholders on board, how to choose the right metrics early on, and most importantly, how to translate that to real ROI. 

Our very first live podcast was a great learning experience and a ton of fun! We really enjoyed the interactive aspect, and our audience asked a lot of thoughtful questions. 

Highlights

• [2:01] Kate explains what UX benchmarking is
• [3:37] How to choose benchmarking metrics
• [12:01] The difference between summative and formative studies, and why you need to distinguish between them.
• [17:21] Why context matters when evaluating benchmarking metrics
• [21:28] How to translate benchmarking results to ROI
• [29:11] Kate talks about case studies from NNg's ROI for Usability report
• [35:34] Q&A - How do you limit bias in unmoderated studies with non-users and users?
• [38:16] Q&A - How do you measure time spent on a task? Stopwatches aren't great.
• [39:59] Q&A - How do session replay tools fit into this?
• [41:10] Q&A - What happens when your stakeholders have different metrics for success?
• [44:57] Q&A - If a participant thinks they completed a task successfully, is that a success?
• [46:42] Q&A - How do you benchmark for emotional aspects, like how fun a product is?
• [49:07] Parting words of wisdom

Kate's recommended resources 

• NN/g full-day seminar: Measuring UX & ROI
• NN/g articles about benchmarking, ROI, and quantitative research
• Jeff Sauro’s articles about quantitative UX research (he also has very useful confidence interval and statistical significance calculators): measuringu.com 
• Jeff Sauro’s book about benchmarking: Benchmarking the User Experience
• Two great books for getting started with statistics: Straightforward Statistics and The Cartoon Guide to Statistics