0
Scientific Articles   |    
Intention-to-Treat Analysis and Accounting for Missing Data in Orthopaedic Randomized Clinical Trials
Amir Herman, MD, MSc1; Itamar Busheri Botser, MD1; Shay Tenenbaum, MD1; Ahron Chechick, MD1
1 Department of Orthopedic Surgery, Chaim Sheba Medical Center, Ramat-Gan 52621, Israel. E-mail address for A. Herman: amirherm@gmail.com
View Disclosures and Other Information
Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.
Investigation performed at the Department of Orthopedic Surgery, Chaim Sheba Medical Center, Ramat-Gan, Israel

The Journal of Bone and Joint Surgery, Inc.
J Bone Joint Surg Am, 2009 Sep 01;91(9):2137-2143. doi: 10.2106/JBJS.H.01481
5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

Abstract

Background: The intention-to-treat principle implies that all patients who are randomized in a clinical trial should be analyzed according to their original allocation. This means that patients crossing over to another treatment group and patients lost to follow-up should be included in the analysis as a part of their original group. This principle is important for preserving the randomization scheme, which is the basis for correct inference in any randomized trial. In this study, we examined the use of the intention-to-treat principle in recently published orthopaedic clinical trials.

Methods: We surveyed eight leading orthopaedic journals for randomized clinical trials published between January 2005 and August 2008. We determined whether the intention-to-treat principle was implemented and, if so, how it was used in each trial. Specifically, we ascertained which methods were used to account for missing data.

Results: Our search yielded 274 randomized clinical trials, and the intention-to-treat principle was used in ninety-six (35%) of them. There were significant differences among the journals with regard to the use of the intention-to-treat principle. The relative number of trials in which the principle was used increased each year. The authors adhered to the strict definition of the intention-to-treat principle in forty-five of the ninety-six studies in which it was claimed that this principle had been used. In forty-four randomized trials, patients who had been lost to follow-up were excluded from the final analysis; this practice was most notable in studies of surgical interventions. The most popular method of adjusting for missing data was the "last observation carried forward" technique.

Conclusions: In most of the randomized clinical trials published in the orthopaedic literature, the investigators did not adhere to the stringent use of the intention-to-treat principle, with the most conspicuous problem being a lack of accounting for patients lost to follow-up. This omission might introduce bias to orthopaedic randomized clinical trials and their analysis.

Figures in this Article
    Sign In to Your Personal ProfileSign In To Access Full Content
    Not a Subscriber?
    Get online access for 30 days for $35
    New to JBJS?
    Sign up for a full subscription to both the print and online editions
    Register for a FREE limited account to get full access to all CME activities, to comment on public articles, or to sign up for alerts.
    Register for a FREE limited account to get full access to all CME activities
    Have a subscription to the print edition?
    Current subscribers to The Journal of Bone & Joint Surgery in either the print or quarterly DVD formats receive free online access to JBJS.org.
    Forgot your password?
    Enter your username and email address. We'll send you a reminder to the email address on record.

     
    Forgot your username or need assistance? Please contact customer service at subs@jbjs.org. If your access is provided
    by your institution, please contact you librarian or administrator for username and password information. Institutional
    administrators, to reset your institution's master username or password, please contact subs@jbjs.org

    References

    Accreditation Statement
    These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.
    CME Activities Associated with This Article
    Submit a Comment
    Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
    Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

    * = Required Field
    (if multiple authors, separate names by comma)
    Example: John Doe





    Michael D. McKee, MD, FRCSC
    Posted on January 11, 2010
    Response
    St. Michael's Hospital, University of Toronto, Ontario, Canada

    EDITOR’S NOTE: The authors were invited to respond to the letter, but to date, have not done so.

    To the Editor:

    We appreciate the opportunity to respond to the interesting article by Herman et al. entitled, “Intention-to-treat analysis and accounting for missing data in orthopaedic randomized clinical trials” (1), in which they use as an example data from a trial recently published by our group, the Canadian Orthopaedic Trauma Society (2). In general, we agree with the authors’ contention that more rigorous and consistent analysis of data from randomized clinical trials is needed. However, as a group dedicated to performing and reporting such studies, we would like to point out a few factors which may not be readily apparent to those who have limited experience in actually conducting or publishing a randomized clinical trial in fracture care.

    A practical reality of randomized trials in trauma care is the young, transient nature of the population involved. We are aware of a randomized trial of clavicle fracture treatment similar to ours, with comparable findings, that was presented but never published due primarily to the number of patients lost to follow-up: the important data from this study has thus never been available. We believe that our follow-up rate of 84% (111 of 132 patients) is a testimony to the experience, diligence and effort of the research coordinators involved in the study. It compares favorably with any other North American trauma study and probably approaches what is realistically possible in trials of this nature. We have also noticed, in our prior randomized clinical trials in fracture care, that there is a higher rate of patients lost to follow-up in those randomized to non-operative care (consistent with Figure 2 in the article by Herman et al.). Human nature being what it is, these patients simply do not have a similar level of commitment and rate of return that patients who have received an operation do. Also, they may seek care elsewhere if they perceive that they have been “denied” an optimal treatment due to randomization.

    We believe that it is a mistake to assume that such patients all do well, and would like to comment specifically on Table III of their paper, which compares our original analysis of our data with their re-analysis using the “Last Observation Carried Forward” method. This method uses the last observed value as a replacement for missing final observations and Table III assumes that the 16 non-operative patients lost to follow-up in our study had no major complications. In fact, this is not the case. Since the orthopaedic community involved in the care of such individuals is relatively small in Canada, we know from personal communications that the two non-operative patients lost to follow-up from our center (St. Michael’s Hospital) subsequently underwent operative reconstruction (one for nonunion, and one for a symptomatic malunion). We are aware of at least two similar patients at other centers from our study. Thus, in the specific example Herman et al. use in Table III the “Last Observation Carried Forward” analysis would, practically, produce erroneous conclusions. We suspect that this might be the case in other similar situations.

    Additionally, we would like to point out that only one patient (of 132) in our study did not receive the treatment they were randomized to. One patient was randomized to surgery and subsequently refused operative intervention; he was, consistent with the “Intention-to-Treat” principle, assessed as part of the “Surgical” group. This patient developed a nonunion, a complication attributed to the “Surgical” group which had a negative overall effect on the strength of our conclusions.

    In summary, we would suggest that there must be a balance between the practical reality of conducting and reporting randomized trials in fracture care and purely theoretical or statistical concerns that might interfere with the essence of the conclusions reached. We believe that it is incorrect to assume that patients in a randomized trial who are lost to follow-up universally do well. In particular, we feel strongly that our conclusions regarding the superiority of primary operative intervention for completely displaced midshaft fractures of the clavicle, are valid. We look forward to future publications from Herman et al. that add practical results to their obvious fund of theoretical expertise.

    The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

    Please refer to http://www.ejbjs.org/cgi/content/full/89/1/1 for the original Disclosure Statement from the 2007 JBJS article by the Canadian Orthopaedic Trauma Society.

    References

    1. Herman A. Botser IB, Tenenbaum S, Chechick A. Intention-to-treat analysis and accounting for missing data in orthopaedic randomized clinical trials. J Bone Joint Surg Am. 2009;91:2137-43.

    2. Canadian Orthopaedic Trauma Society. Nonoperative treatment compared with plate fixation of displaced midshaft clavicular fractures. A multicenter, randomized clinical trial. J Bone Joint Surg Am. 2007;89:1-10.

    Karl E. Peace, PhD, MS
    Posted on December 29, 2009
    Re: Make a Plan and Follow it
    Jiann-Ping Hsu College of Public Health, Georgia Southern University, Statesboro, Georgia

    To the Editor:

    I agree with Dr. Ring's statement "... that, in order to limit bias, it is critical that clinical scientists decide before they begin the experiment which method of intention-to-treat analysis they will use and then not veer from that plan." Not only is it critical in the reduction of bias, but it is entirely consistent with the scientific method, and hence critical for the experiment to produce a valid inference. In a lecture thirty-five years ago, Clyde Kramer described the PARC (plan after research completed) method, which regrettably still has many advocates.

    The author did not receive any outside funding or grants in support of his research for or preparation of this work. Neither he nor a member of his immediate family received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the author, or a member of his immediate family, is affiliated or associated.

    David Ring, MD
    Posted on November 27, 2009
    Make a Plan and Follow it
    Massachusetts General Hospital, Boston, Massachusetts

    To the Editor:

    I learned a great deal from the paper by Herman and colleagues (1). It is eye opening that specific techniques of intention-to-treat analysis can result in different interpretations of the data. I want to place emphasis on the fact that, in order to limit bias, it is critical that clinical scientists decide before they begin the experiment which method of intention-to-treat analysis they will use and then not veer from that plan.

    The author did not receive any outside funding or grants in support of his research for or preparation of this work. Neither he nor a member of his immediate family received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the author, or a member of his immediate family, is affiliated or associated.

    Reference

    1. Herman A, Botser IB, Tenenbaum S, Chechick A. Intention-to-treat analysis and accounting for missing data in orthopaedic randomized clinical trials. J Bone Joint Surg Am. 2009;91:2137-43.

    Related Content
    The Journal of Bone & Joint Surgery
    JBJS Case Connector
    Topic Collections
    Related Audio and Videos
    PubMed Articles
    Clinical Trials
    Readers of This Also Read...
    JBJS Jobs
    12/31/2013
    S. Carolina - Department of Orthopaedic Surgery Medical Univerity of South Carlonina
    12/04/2013
    New York - Icahn School of Medicine at Mount Sinai
    04/02/2014
    W. Virginia - Charleston Area Medical Center
    02/28/2014
    District of Columbia (DC) - Children's National Medical Center