RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable
Back in 2020, a team of scientists, including some prominent figures among champions of open science, put out a preprint titled "High Replicability of Newly-Discovered Social-behavioral Findings is Achievable". Simply put, the authors offered evidence that methodological innovations such as pre-registration and high power cause increased replication rates.
Fast forward four years, and comments such as...
- Causal claims about scientific rigor require rigorous causal evidence.
- Of course its preregistered. Just give me a sec
- Here are some ways of making your study replicable. (No, the first steps are not preregistration or increasing the sample size!)
- More on possibly rigor-enhancing practices in quantitative psychology research
- The feel-good open science story versus the preregistration (who do you think wins?)
... The manuscript is now retracted at Nature Human Behavior, with an updated title: "RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable". This has been a wild adventure, and closure is yet to come because the retraction note currently errors with a 404: https://doi.org/10.1038/s41562-024-01997-3
This whole story is giving me heartburn. So what to do? This isn't to say that things like pre-registration don't have value--we can learn a lot from them, especially as authors!--but perhaps that the quick wins in replicability were already well-known and not very controversial (Gelman):
- Make it clear what you’re actually doing. Describe manipulations, exposures, and measurements fully and clearly.
- Increase your effect size, e.g., do a more effective treatment.
- Focus your study on the people and scenarios where effects are likely to be largest.
- Improve your outcome measurement.
- Improve pre-treatment measurements.