Background: “P-hacking” occurs when researchers preferentially select data or statistical analyses until nonsignificant results become significant. We wanted to evaluate if the phenomenon of p-hacking was evident in orthopaedic literature.
Methods: We text-mined through all articles published in three top orthopaedic journals in 2015. For anonymity, we cipher-coded the three journals. We included all studies that reported a single p value to answer their main hypothesis. These p values were then charted and frequency graphs were generated to illustrate any evidence of p-hacking. Binomial tests were employed to look for evidence of evidential value and significance of p-hacking.
Results: Frequency plots for all three journals revealed evidence of p-hacking. Binomial tests for all three journals were significant for evidence of evidential value (p < 0.0001 for all). However, the binomial test for p-hacking was significant only for one journal (p = 0.0092).
Conclusions: P-hacking is an evolving phenomenon that threatens to jeopardize the evidence-based practice of medicine. Although our results show that there is good evidential value for orthopaedic literature published in our top journals, there is some evidence of p-hacking of which authors and readers should be wary.
Investigation performed at the Department of Orthopaedic Surgery, Singapore General Hospital, Singapore
Disclosure: There was no source of external funding for this study. The Disclosure of Potential Conflicts of Interest forms are provided with the online version of this article.
- Copyright © 2016 by The Journal of Bone and Joint Surgery, Incorporated
Enter your JBJS login information below.
Please note that your username is the email address you provided when you registered.