0
Scientific Articles   |    
Statistical Consideration for Bilateral Cases in Orthopaedic Research
Moon Seok Park, MD1; Sung Ju Kim, MS2; Chin Youb Chung, MD1; In Ho Choi, MD3; Sang Hyeong Lee, MD1; Kyoung Min Lee, MD1
1 Department of Orthopaedic Surgery, Seoul National University Bundang Hospital, 300 Gumi-Dong, Bundang-Gu, Sungnam, Kyungki 463-707, South Korea. E-mail address for K.M. Lee: oasis100@empal.com
2 Health Economics and Outcome Research, Market Access, Novartis Korea, 84-11 Namdaemun 5th Street, Chung-Gu, Seoul 100-753, South Korea
3 Department of Orthopaedic Surgery, Seoul National University Children's Hospital, 28 Yongun-Dong, Chongno-Gu, Seoul 110-744, South Korea
View Disclosures and Other Information
Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity.

Investigation performed at Seoul National University Bundang Hospital, Sungnam, Kyungki, South Korea

Copyright ©2010 American Society for Journal of Bone and Joint Surgery, Inc.
J Bone Joint Surg Am, 2010 Jul 21;92(8):1732-1737. doi: 10.2106/JBJS.I.00724
5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

Abstract

Background: 

Statistical independence means that one observation is not affected by another; however, the principle of statistical independence is violated if left and right-side measures within a subject are considered to be independent, because they are usually correlated and can affect each other. The purpose of the present study was to analyze the violation of statistical independence in recent orthopaedic research papers and to demonstrate the effect of statistical analysis that considered the data dependency within a subject.

Methods: 

First, all original articles that had been published in The Journal of Bone and Joint Surgery (American Volume) over a two-year period were evaluated. The analysis was designed to identify articles that included bilateral cases and possible violations of statistical independence. Second, a demonstrative logistic regression without consideration of statistical independence was performed and was compared with a statistical analysis that considered data dependency within a subject. Radiographs of 1200 hips in 600 patients were used to examine the differences in terms of odds ratios (with 95% confidence intervals) of the risk factors for hip osteoarthritis.

Results: 

Four hundred and eighty-six original articles were reviewed, and 151 articles (including forty-one articles involving the hip, thirty-four involving the knee, twenty-one involving the foot or ankle, nineteen involving the shoulder, ten involving the hand or wrist, nine involving the elbow, and seventeen involving other structures) were considered to include bilateral cases. Of the 486 articles that were reviewed, 120 articles (25%) (including thirty-six articles involving the hip, twenty-six involving the knee, fifteen involving the foot or ankle, fourteen involving the shoulder, seven involving the elbow, six involving the hand or wrist, and sixteen involving other structures) were found to have possibly violated statistical independence. Demonstrative statistical analysis showed that logistic regression was not robust to the violation of statistical independence. The 95% confidence intervals of the odds ratios for the risk factors showed narrower ranges (1.13 to 2.68 times) when data dependency within a subject was not considered.

Conclusions: 

Researchers need to consider statistical independence when performing statistical analysis, particularly in studies involving bilateral cases. If data dependency within a subject is not considered, studies involving bilateral cases can bias results, depending on the context of those studies.

Figures in this Article
    Sign In to Your Personal ProfileSign In To Access Full Content
    Not a Subscriber?
    Get online access for 30 days for $35
    New to JBJS?
    Sign up for a full subscription to both the print and online editions
    Register for a FREE limited account to get full access to all CME activities, to comment on public articles, or to sign up for alerts.
    Register for a FREE limited account to get full access to all CME activities
    Have a subscription to the print edition?
    Current subscribers to The Journal of Bone & Joint Surgery in either the print or quarterly DVD formats receive free online access to JBJS.org.
    Forgot your password?
    Enter your username and email address. We'll send you a reminder to the email address on record.

     
    Forgot your username or need assistance? Please contact customer service at subs@jbjs.org. If your access is provided
    by your institution, please contact you librarian or administrator for username and password information. Institutional
    administrators, to reset your institution's master username or password, please contact subs@jbjs.org

    References

    Accreditation Statement
    These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.
    CME Activities Associated with This Article
    Submit a Comment
    Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
    Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

    * = Required Field
    (if multiple authors, separate names by comma)
    Example: John Doe





    Kyoung Min Lee, MD
    Posted on August 26, 2010
    Dr. Lee and colleagues respond to Drs. Chen and Lin
    Seoul National University Bundang Hospital, 300 Gumi-Dong, Bundang-Gu, Sungnam, Kyungki, South Korea

    The aim of our previous study was to assess the possible violation of statistical independence in recent orthopaedic studies and demonstrate the distortion of statistical results by not considering the correlation between the right and left side within a subject. In the second part, two statistical methods, the generalized estimating equation (GEE) and multilevel model, were used to compare with logistic regression. However, the study did not compare the GEE with the multilevel model or provide guidelines for the use of these two methods, which we believe were beyond the scope of the study. We appreciate Dr. Chen and Dr. Lin for the thorough review of our data. They raised two major concerns in the demonstration of the GEE and multilevel model. Our responses are as follows.

    First, the lack of a description of the correlation matrices made it difficult to rebuild the GEE model. In our study, an exchangeable structure for the working correlation was used for the GEE approach based on the Akaike Information Criterion (AIC) (1). The GLMM (generalized linear mixed model) was used for multilevel modeling (2).

    Second, they commented that, in binary data as in our study, the odds ratio from the multilevel model (random effect model) is larger than that from the GEE model (marginal model) and logistic regression (3). This is "usually" true in most cases. Allison explained this phenomenon in more detail and called it "heterogeneity shrinkage" because the difference between the marginal and random effects increases with increasing heterogeneity between subjects (4). However, it could be different in some situations as in the results of previous reports (5,6). It was also claimed that, in binary data, the multilevel model can generate potentially biased estimates of the covariance parameters due to the absence of a true objective function (GLMM estimate the approximate values by the Quasi-Newton method) (7). In our study, some of the odds ratio (age and BMI in 1200 hips; age and gender in 600 hips; age and gender in 200 hips; and age and gender in 100 hips, table II) were larger in the GEE. We suspect that the low total event rate (prevalence of hip osteoarthritis) and skewed nature of independent variables occasionally caused larger values from the GEE than those from the multilevel model. However, it is not easy to confirm with the current knowledge and further studies will be needed to answer the question about this skewed phenomenon.

    Dr. Chen and Dr. Lin criticized the use of GEE and multilevel models, and successfully pointed out the limitations of these models in our data. As he mentioned, in binary data, the GEE appears more intuitive than multilevel models in that it changes only the confidential interval without changing the point of estimate (odds ratio). However, despite some limitations, we believe that both methods better approximate the orthopaedic study results to the truth than logistic regression does.

    References

    1. Akaike H. A new look at the statistical model identification. IEEE Trans Automatic Control. 1974;19:716-23.

    2. Breslow NE, Clayton DG. Approximate inference in generalized linear mixed models. J Am Stat Assoc. 1993;88:9-25.

    3. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988;44:1049-60.

    4. Allison P. Logistic regression using the SAS® system: theory and application. Cary, NC: SAS Institute, Inc. and John Wiley and Sons; 1999.

    5. Kim HY, Preisser JS, Rozier RG, Valiyaparambil JV. Multilevel analysis of group-randomized trials with binary outcomes. Community Dent Oral Epidemiol. 2006;34:241-51.

    6. Shaffer ML, Kunselman AR, Watterberg KL. Analysis of neonatal clinical trials with twin births. BMC Med Res Methodol. 2009;9:12.

    7. Schabenberger O. Introducing the GLIMMIX procedure for generalized linear mixed models. In: Proceedings of the Thirtieth Annual SAS® Users Group International Conference; 2005 Apr 10-13. Cary, NC: SAS Institute Inc; 2005. p 1-20. Paper no 196-30.

    Chao-Ping Chen
    Posted on July 31, 2010
    GEE and Multilevel Models
    Department of Orthopaedics, Taichung Veterans General Hospital, Taichung, Taiwan

    To the Editor:

    We read the article entitled, "Statistical Consideration for Bilateral Cases in Orthopaedic Research" by Dr. Park and colleagues with interest (2010;92:1732-7). The authors urged researchers to consider statistical independence when performing statistical analysis with illustrated examples of a logistic regression model, generalized estimating equation (GEE) model and multilevel model. Most clinicians know the correlations between right and left, but they have to turn to statisticians for complex statistical analyses. To our knowledge, there are no standardized guidelines concerning reporting the results of GEE model and multilevel modeling. On the contrary, there are well established formats concerning reporting and interpreting the results of univariate, multiple linear regression, and logistic regression analyses (1). The demands and cost of journal space dictate summarized presentations of analyses (2). While reviewing the summary results of GEE and multilevel models, it is very difficult to rebuild the whole model, recalculate and confirm the results without the knowledge of correlation matrices being used. Katz (2) suggested the authors provide a thorough analysis report on the web to solve the dilemma.

    GEE models increase the standard errors (and therefore the confidence intervals) of the point estimates but do not change the point estimates (2). Multilevel models change the point estimates and increase the standard errors (2). In the case of Gaussian data, interpretation of the condition coefficient is the same in GEE and multilevel models (3); however, in the case of binary data, the condition coefficient from a GEE model is smaller in absolute value than that from a multilevel model (3-4). However, the authors reported otherwise except in all 600 persons (1200 joints) in Table II. Interpretation differs between the two approaches. In the GEE model, the condition coefficient is the between-person difference; in the multilevel model, the condition coefficient is the within-person change (3). Though the literature appears to favor the multilevel model approach, some argue GEE models provide a more useful approximation of the truth (5).

    The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity.

    References

    1. Lang TA, Secic M. How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers. 2nd ed. Philadelphia: American College of Physicians; 2006.

    2. Katz MH. Correlated observations. Multivariable analysis: a practical guide for clinicians. 2nd ed. Cambridge: Cambridge University Press; 2006. p 158-78.

    3. Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004;94:423-32.

    4. Normand SL. Some old and some new statistical tools for outcomes research. Circulation. 2008;118:872-84.

    5. Hubbard AE, Ahern J, Fleischer NL, Van der Laan M, Lippman SA, Jewell N, Bruckner T, Satariano WA. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology. 2010;21:467-74.

    Related Content
    The Journal of Bone & Joint Surgery
    JBJS Case Connector
    Topic Collections
    Hip
    Related Audio and Videos
    Clinical Trials
    Readers of This Also Read...
    JBJS Jobs
    10/04/2013
    California - Mercy Medical Group
    01/08/2014
    Pennsylvania - Penn State Milton S. Hershey Medical Center