25: Misunderstanding p-values

P-values are universal, but do we really know what they mean? In this episode, Dan and James discuss a recent paper describing the failure to correctly interpret p-values in a sample of academic psychologists.

Some of the topics discussed:

• Common p-value misconceptions
• James tests Dan on his p-value knowledge
• p-values vs. effect size
• The problem of sample size with p-value interpretation
• The Facebook mood manipulation study
• Data peeking
• Equivalent p-values do not represent equivalent results
• Meta-analytical thinking
• Using significance as a categorical factor
• Statistical vs. clinical significance
• Clinical trial registration and 'secondary outcome creep'
• Dan and James answer listener questions
• Science communicator vs. scientist
• Grant titles and the 'Pub test'
• NASA and social media

Links

• The article

http://journal.frontiersin.org/article/10.3389/fpsyg.2016.01247/full

• Geoff Cumming's book (we got the name completely wrong - sorry Geoff!)

http://www.amazon.com/Understanding-The-New-Statistics-Meta-Analysis-ebook/dp/B007M9D76G/ref=pd_sim_kstore_1?ie=UTF8&refRID=1QWKES82EP85DBAEKNT1

The story on research passing the 'pub' test

https://theconversation.com/if-youre-going-to-ridicule-research-do-your-homework-64238

Real scientists

http://realscientists.org

Facebook page

https://www.facebook.com/everythinghertzpodcast/

Twitter account

https://www.twitter.com

Support Everything Hertz