Commentary and Perspective   |    
A Calcaneal Fracture Study Illustrates a Need for Better Statistical Methods for Orthopaedic OutcomesCommentary on an article by Per-Henrik Ågren, MD, et al.: “Operative Versus Nonoperative Treatment of Displaced Intra-Articular Calcaneal Fractures. A Prospective, Randomized, Controlled Multicenter Trial”
Alastair Younger, MB, ChB, MSc, ChM, FRCSC1
1 University of British Columbia, Vancouver, British Columbia, Canada
View Disclosures and Other Information
  • Disclosure statement for author(s): PDF

The author did not receive payments or services, either directly or indirectly (i.e., via his institution), from a third party in support of any aspect of this work. He, or his institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. The author has not had any other relationships, or engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.

Copyright © 2013 by The Journal of Bone and Joint Surgery, Inc.
J Bone Joint Surg Am, 2013 Aug 07;95(15):e111 1-2. doi: 10.2106/JBJS.M.00687
5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case
I can feel the excitement of these authors when they started designing their study in the early 1990s: We had all been told that randomized controlled trials (RCTs) were the answer, and they were going to contribute with a well-designed study.
Roll on twenty years and I can feel their disappointment. Fortunately, their hard work has been rewarded with publication in JBJS. However, the results they report are no more illuminating than those in previous papers—with words like “underpowered” being used to diminish the importance and impact of the study.
This is unfair to the authors and unfair to the orthopaedic community. These RCT models arose from drug trials and may not be the best model in orthopaedics. The concept of a clinically significant difference might make some sense in a placebo-controlled study on a blood pressure medication, although I would contend that blood pressure is not the true outcome in that case—the true outcome is mortality reduction. Fortunately, orthopaedic outcomes research has been geared toward true outcomes and not been accepting of surrogate measures such as radiographic measurements.
So, if power analysis is of little use for drug studies and no use for clinical studies, the slavish application of power analysis is wrong. An effects size table may be a better way to determine the proper sample size. I believe that the authors consequently followed the correct approach by not expanding the study.
The study was further limited because the validity and responsiveness of some of the outcome instruments have not been documented. Loss of patients due to exclusion criteria further frustrates enrollment; patients are not clones, and many trauma patients have other diseases or life factors that make them hard to recruit. Finally, trauma studies lack sensitivity to measure change, as a preoperative score cannot be obtained.
Outcomes in orthopaedics also require a long time to obtain. In the drug studies that formed the model for orthopaedic statistical methods, the outcomes were measured over days or weeks, making patients easy to trace. Outcomes in orthopaedics require years to measure and obtain, and these studies are at real risk of investigator and participant fatigue. The investigators may retire or die before the study is completed. The need to publish in order to attain promotion may be long past by the end of this time period. The dedication and perseverance of authors such as those in this group must be recognized and applauded.
What struck me as I read through this paper was that so many of the outcome measures outlined pointed in favor of surgical treatment. At the one-year follow-up, three of four primary outcomes and three of four secondary outcomes favored surgery. At the latest follow-up, all four primary outcomes and three of four secondary outcomes favored surgery, with the last measure being a draw.
Furthermore, surgical procedures and the resulting outcome scores have improved since the study was started. I believe that the results therefore support operative care of calcaneal fractures. I will continue to offer surgical treatment to patients, provided that my assessment of the risks and benefits for that patient favors surgery. I would encourage readers of JBJS to do the same in the best interests of patient care, and to listen to their patients after surgical and nonsurgical treatment to add the art of medicine to the science offered in these articles until the science improves.
The authors also observed a higher complication rate with surgical treatment. For the future, we need to improve the results of surgery and reduce the complication rate. Reducing the invasiveness of surgery and reducing wound complications would go a long way toward this goal. Improvement of the quality of reduction would also help in improving outcomes.
We also need some really smart PhD statisticians in outcomes research to redefine orthopaedic outcomes research from the ground up so that future authors do not dedicate the time and effort to this type of outcomes research and end up with equivocal results. Authors considering embarking on similar studies should seek lots of high-quality statistical advice. This will ensure that study design and outcome measurements are optimal to allow the conclusions to be reached correctly twenty years from now and avoid a type-II error—concluding that there is no difference between the groups when there really is.

Submit a comment


Accreditation Statement
These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.
CME Activities Associated with This Article
Submit a Comment
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe

Philip R Hardy, MD
Posted on August 09, 2013
Here, here
Jacksonville Orthopaedic Institute, Jacksonville, FL, USA

At last, a voice of reason in a world which is otherwise obsessed with statistical analysis & evidence based medicine. Appreciating the limitations of such methods may be realized by noting that according to the methods of evidence based medicine, there is no evidence that it is faster to fly across country than to walk, because there have been no double blinded randomized controlled trials.

Related Content
The Journal of Bone & Joint Surgery
JBJS Case Connector
Topic Collections
Related Audio and Videos
PubMed Articles
Clinical Trials
Readers of This Also Read...
S. Carolina - Department of Orthopaedic Surgery Medical Univerity of South Carlonina
New York - Icahn School of Medicine at Mount Sinai
Connecticut - Yale University School of Medicine