Multiple Measures: P-Hacking Your Way To Fame | Part 3 of 6

This video is part of a series of videos on p-hacking, what it is, and why it’s so dangerous for science. I give a pretty detailed high-level explanation of p-hacking in the first video in this series, so if you’re not familiar with the idea, please have a look there first. I’ll put a link to that video below. But in a few seconds, researchers are motivated to get what’s called a p-value to be below a threshold of .05. If they do that, their findings are considered meaningful and they can typically publish their results. If they don’t, well all their work is largely wasted. And to get those p-values below .05, there are some very dubious and unethical approaches they can take. In this video, we’ll dig in to one of those unethical approaches that researchers can use to p-hack their data...by collecting multiple measures and only reporting those that “work.”
Welcome to Data Demystified. I’m Jeff Galak and today we’ll dig deeper into p-hacking so that you can understand how to spot it when you see research results and avoid it when you do the research yourself. The goal here is to build intuition, so we’ll avoid heavy duty math and statistics, and focus on what you really need to know.
Calculation for multiple comparisons: 1 - ((1 - .05)^5)
P-Hacking Series Playlist: • P-Hacking
Video 1: P-Hacking Explained (Released on 6/24/2021): • P-Hacking Your Way To ...
Video 2: Dropping Conditions that "Work" (Released on 7/1/2021): • Dropping Conditions th...
Video 3: Multiple Measures Misuse (Released on 7/8/2021): • Multiple Measures: P-H...
Video 4: Covariate Misuse (Released on 7/15/2021): • Covariate Misuse: P-Ha...
Video 5: Selective Stopping Rules (Released on 7/22/2021): • Selective Stopping Rul...
Video 6: P-Curve (Released on 7/29/2021): • P-Curve : P-Hacking Yo...
Link to video about statistical significance testing: • Statistical Significan...
Link to video about randomized experiments and causation: • Correlation Doesn't Me...
Link to video about False Positives: • False Positives vs. Fa...
Link to academic paper by Simmons, Nelson and Simonhson: journals.sagep...
Link to academic paper by John, Lowenstein, and Prelec: www.cmu.edu/di...
Follow me at:
LinkedIn: / jeff-galak-768a193a
Patreon: / datademystified
Equipment Used for Filming:
Nikon D7100: amzn.to/320N1FZ
Softlight: amzn.to/2ZaXz3o
Yeti Microphone: amzn.to/2ZTXznB
iPad for Teleprompter: amzn.to/2ZSUkNh
Camtasia for Video Editing: amzn.to/2ZRPeAV