It Depends: Understanding Psychometrics for Testing + Assessment, by Dr. Amanda Dainis + Tim Burnett

It Depends: Understanding Psychometrics for Testing + Assessment•www.testcommunity.network

This podcast makes the complex world of psychometrics easy to understand. Tim Burnett and Dr. Amanda Dainis mix their deep knowledge with a bit of fun, aiming to make the topic enjoyable for everyone. They explore how tests are made and their real-life uses, inviting listeners to join the conversation. Whether you're new to psychometrics or already know a lot, this podcast has something for you. A Podcasting Member of the Test Community.Network

Last refreshed: July 25th, 2025 at 2:28 PM ⓘ

Follow this podcast in the Metacast mobile app to refresh it and see new episodes.

Follow on

Apple Podcasts

Spotify

RSS

Podcasts are better in Metacast mobile app

Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episodes

It Depends Episode 10: Criterion, Norm, CAT & LOFT

This episode of the It Depends Podcast examines criterion referenced testing, norm referenced testing, computer adaptive testing (CAT), and linear on-the-fly testing (LOFT). Amanda and Tim explore the key differences between these assessment types, their applications, and considerations for implementation. Key questions answered: What is the difference between criterion and norm referenced testing? How do CAT and LOFT tests work? When should organizations consider using CAT or LOFT? Topic Keywor...

Nov 29, 2024•59 min•Season 1Ep. 10

Ep9 - Test Centre V Online Proctoring

This episode of the It Depends Podcast compares online proctoring and test centres for assessment delivery. Amanda and Tim explore the advantages and challenges of both methods, discussing candidate experience, technology reliability, security concerns, and cost factors. They debate the misconceptions surrounding each approach and emphasise the importance of considering an organisation's specific needs and budget constraints. The conversation also touches on future trends, including the impact o...

Sep 20, 2024•50 min

Ep8 - Testing on a Budget

This episode of the It Depends Podcast explores testing on a budget, from test development to secure delivery. Amanda and Tim discuss testing on a budget, exploring various ways organisations can implement and maintain testing programs without necessarily investing in expensive, large-scale platforms. Key questions answered: Are big vendor platforms always the best option for your organisation if you're testing on a budget? Is it possible to do item banking on a budget? Can you do secure test de...

Aug 14, 2024•38 min

Ep 7 - Performance Based Testing

This episode of the It Depends Podcast explores performance-based testing in professional certification and licensure. Amanda and Tim discuss the benefits and challenges of performance tests compared to traditional knowledge-based exams, covering various assessment types, implementation considerations, costs, and future trends in performance-based assessment. Key questions answered: What is performance-based testing? What are some advantages of performance tests What challenges come with perform...

Jul 16, 2024•39 min

Episode 6 - Exam Maintenance

Top Episode Takeaways: 📈 Test maintenance involves regularly monitoring test and item performance statistics to ensure content remains current and secure, and should be conducted annually at minimum or as frequently as monthly for large programs. 🔍 The more frequently a test is administered, the more often maintenance should be performed, including reviewing key test-level statistics, item-level metrics, and qualitative candidate feedback to proactively identify issues. 🛠️ Typical maintenance...

Jun 20, 2024•39 min

Ep5: Test Blueprints

Top Episode Takeaways: 📐 Test blueprints provide the overall design and structure for an assessment. They define what content areas the test will cover and what proportion of the test each content area should comprise. Having a blueprint helps ensure the test adequately measures the intended knowledge and skills in a balanced way that reflects their importance. 🎯 Blueprints should be created for most types of assessments, including certification/licensure exams, educational tests, and even wor...

May 12, 2024•28 min

Ep4: Exploring Item Types

Top Episode Takeaways: 📝 Multiple choice questions are the most common item type, but they can be done well or poorly. A good multiple choice question takes effort to create but can measure a wide range of knowledge and skills. 🎥 Technology-enhanced items (e.g., audio/video clips, drag-and-drop) can make assessments more engaging, but they should be used purposefully to measure relevant skills, not just for the "wow" factor. 🚢 Performance-based assessments, such as simulations, are the gold s...

Apr 18, 2024•38 min

Ep3: Cut Scores Explained

Top Episode Takeaways: ✂️ Cut scores are important decision points in criterion-referenced tests that categorize test takers as passing or failing. They should be set thoughtfully using a defensible process, not arbitrarily (e.g. always using 70%). Setting the cut score too high or low can have significant consequences. 📏 The Angoff method is a common way to set cut scores. It involves having subject matter experts review test questions and estimate what percentage of minimally competent test t...

Mar 22, 2024•38 min

Special Episode 1 - Vendor Shopping

Top Episode Takeaways: 🤝 Tactics for negotiation - Ask vendors about integration costs upfront, and use your accreditation status as leverage or potential for publicity. Sometimes, size does matter. Too big and you're one among many, too small and they can't keep up. 📁 Importance of owning your data and access - Ensure you can export items, stats, and raw response data. This relationship may not last forever, your business may change, and the provider might be purchased, what happens then? Be ...

Feb 20, 2024•39 min

Ep2 - JTAs (Job Tasks Analysis), Unravelling the Essentials of Effective Test Development

Top Episode Takeaways: 🧠 Importance of Job Task Analysis (JTA) in Psychometrics : The podcast emphasizes the critical role of JTA in creating and updating certification exams. JTA helps ensure that exam questions are relevant and representative of the actual knowledge, skills, and abilities required for the job. 🤝 Role of Subject Matter Experts (SMEs) : SMEs are vital in the JTA process. Their input helps form a comprehensive understanding of the job role. The podcast highlights the challenges...

Jan 30, 2024•33 min

Ep1 - Psychometrics for the Non-Psychos: A Friendly Introductory Journey

Top Episode Takeaways: 🤝 Meet the Experts: Discover the unique backgrounds and expertise of hosts Tim Burnett and Dr. Amanda Dainis. 📊 Behind the Tests: Uncover the intricate process of test development and what makes a good test. 🌟 Breaking Stereotypes: Learn how psychometrics is more than just dry statistics – it's a field filled with creativity and real-world impact. Get the full transcript here: https://www.testcommunity.network/podcast-it-depends Learn about: Psychometrics, Test Developm...

Dec 14, 2023•30 min

Hosted on Spotify for Creators (Anchor)

For the best experience, listen in Metacast app for iOS or Android